This page contains papers and related content linked to the loss landscape challenge. If you have a paper or article that you would like to share, email ideami@ideami.com.

  • Why Flatness Correlates With Generalization For Deep Neural Networks
    huofeng Zhang, Isaac Reid, Guillermo Valle Pérez, Ard Louis
    Submitted on 10 March, 2021;

  • Quantum Earth Mover’s Distance: A New Approach to Learning Quantum Data
    Bobak Toussi Kiani, Giacomo De Palma, Milad Marvian, Zi-Wen Liu, Seth Lloyd
    Submitted on 08 Jan, 2021;
    arXiv: 2101.03037

  • Towards a Better Global Loss Landscape of GANs
    Ruoyu Sun, Tiantian Fang, Alex Schwing
    Submitted on 10 Nov, 2020;
    arXiv: 2011.04926

  • Approximation and convergence of GANs training: an SDE approach
    Haoyang Cao, Xin Guo
    Submitted on 2 Jun, 2020;
    arXiv: 2006.02047

  • Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness
    Pu Zhao, Pin-Yu Chen, Payel Das, Karthikeyan Natesan Ramamurthy, Xue Lin
    Submitted on 30 Apr, 2020;
    arXiv: 2005.00060

  • Revisiting Loss Landscape for Adversarial Robustness
    Dongxian Wu, Yisen Wang, Shu-tao Xia
    Submitted on 13 Apr, 2020;
    arXiv: 2004.05884

  • Understanding Global Loss Landscape of One-hidden-layer ReLU Neural Networks
    Bo Liu
    Submitted on 12 Feb, 2020;
    arXiv: 2002.04763

  • Deep Ensembles: A Loss Landscape Perspective
    Stanislav Fort, Huiyi Hu, Balaji Lakshminarayanan
    Submitted on 5 Dic, 2019;
    arXiv: 1912.02757

  • Emergent properties of the local geometry of neural loss landscapes
    Authors: Stanislav Fort, Surya Ganguli
    Submitted on 14 Oct, 2019;
  • Loss Landscape Sightseeing with Multi-Point Optimization
    Authors: Ivan Skorokhodov, Mikhail Burtsev
    Submitted on 9 Oct, 2019;
  • How noise affects the Hessian spectrum in overparameterized neural networks
    Authors: Mingwei Wei, David J Schwab
    Submitted on 1 Oct, 2019;



  • The Difficulty of Training Sparse Neural Networks
    Authors: Utku Evci, Fabian Pedregosa, Aidan Gomez, Erich Elsen
    Submitted on 25 June, 2019;


  • Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets
    Authors: Rohith Kuditipudi, Xiang Wang, Holden Lee, Yi Zhang, Zhiyuan Li, Wei Hu, Sanjeev Arora, Rong Ge
    Submitted on 14 June, 2019;


  • Large Scale Structure of Neural Network Loss Landscapes
    Stanislav Fort, Stanislaw Jastrzebski
    Submitted on 11 Jun 2019



  • The Effect of Network Depth on the Optimization Landscape
    Behrooz Ghorbani, Ying Xiao, Shankar Krishnan
    Submitted 28 May 2019


  • Visualizing Loss Landscape of Deep Neural Networks…..but can we Trust them?
    Jae Duk Seo
    May 5, 2019


  • Negative eigenvalues of the Hessian in deep neural networks
    Guillaume Alain, Nicolas Le Roux, Pierre-Antoine Manzagol
    Submitted on 6 Feb 2019


  • Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning
    Charles H. Martin, Michael W. Mahoney
    Submitted on 2 Oct 2018


  • On the loss landscape of a class of deep neural networks with no bad local valleys
    Quynh Nguyen, Mahesh Chandra Mukkamala, Matthias Hein
    Submitted on 27 Sep 2018


  • The Goldilocks zone: Towards better understanding of neural network loss landscapes
    Stanislav Fort, Adam Scherlis
    Submitted on 6 Jul 2018


  • The loss landscape of overparameterized neural networks
    Y Cooper
    Submitted on 26 Apr 2018


  • Measuring the Intrinsic Dimension of Objective Landscapes
    Chunyuan Li, Heerad Farkhoor, Rosanne Liu, Jason Yosinski
    Submitted on 24 Apr 2018


  • A Mean Field View of the Landscape of Two-Layers Neural Networks
    Song Mei, Andrea Montanari, Phan-Minh Nguyen
    Submitted on 18 Apr 2018


  • The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
    Jonathan Frankle, Michael Carbin
    Submitted on 9 Mar 2018


  • Essentially No Barriers in Neural Network Energy Landscape
    Felix Draxler, Kambis Veschgini, Manfred Salmhofer, Fred A. Hamprecht
    Submitted on 2 Mar 2018


  • Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs
    Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry Vetrov, Andrew Gordon Wilson
    Neural Information Processing Systems 2018
    Submitted on 27 Feb 2018


  • Visualizing the Loss Landscape of Neural Nets (A favourite of mine)  *********
    Li, Hao and Xu, Zheng and Taylor, Gavin and Studer, Christoph and Goldstein, Tom
    Neural Information Processing Systems 2018
    Submitted on 28 Dec 2017


  • Sharp Minima Can Generalize For Deep Nets
    Laurent Dinh, Razvan Pascanu, Samy Bengio, Yoshua Bengio
    Submitted on 28 March 2017


Go deep

What lies within the landscape?