Loss

Knowledge

This page contains papers and related content linked to the loss landscape challenge. If you have a paper or article that you would like to share, email ideami@ideami.com.

  • Why Flatness Correlates With Generalization For Deep Neural Networks
    huofeng Zhang, Isaac Reid, Guillermo Valle Pérez, Ard Louis
    Submitted on 10 March, 2021;
    arXiv:2103.06219

  • Quantum Earth Mover’s Distance: A New Approach to Learning Quantum Data
    Bobak Toussi Kiani, Giacomo De Palma, Milad Marvian, Zi-Wen Liu, Seth Lloyd
    Submitted on 08 Jan, 2021;
    arXiv: 2101.03037

  • Towards a Better Global Loss Landscape of GANs
    Ruoyu Sun, Tiantian Fang, Alex Schwing
    Submitted on 10 Nov, 2020;
    arXiv: 2011.04926

  • Approximation and convergence of GANs training: an SDE approach
    Haoyang Cao, Xin Guo
    Submitted on 2 Jun, 2020;
    arXiv: 2006.02047

  • Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness
    Pu Zhao, Pin-Yu Chen, Payel Das, Karthikeyan Natesan Ramamurthy, Xue Lin
    Submitted on 30 Apr, 2020;
    arXiv: 2005.00060

  • Revisiting Loss Landscape for Adversarial Robustness
    Dongxian Wu, Yisen Wang, Shu-tao Xia
    Submitted on 13 Apr, 2020;
    arXiv: 2004.05884

  • Understanding Global Loss Landscape of One-hidden-layer ReLU Neural Networks
    Bo Liu
    Submitted on 12 Feb, 2020;
    arXiv: 2002.04763

  • Deep Ensembles: A Loss Landscape Perspective
    Stanislav Fort, Huiyi Hu, Balaji Lakshminarayanan
    Submitted on 5 Dic, 2019;
    arXiv: 1912.02757

  • Emergent properties of the local geometry of neural loss landscapes
    Authors: Stanislav Fort, Surya Ganguli
    Submitted on 14 Oct, 2019;
    arXiv:1910.05929
  • Loss Landscape Sightseeing with Multi-Point Optimization
    Authors: Ivan Skorokhodov, Mikhail Burtsev
    Submitted on 9 Oct, 2019;
    arXiv:1910.03867
  • How noise affects the Hessian spectrum in overparameterized neural networks
    Authors: Mingwei Wei, David J Schwab
    Submitted on 1 Oct, 2019;
    arXiv:1910.00195

 

 

  • The Difficulty of Training Sparse Neural Networks
    Authors: Utku Evci, Fabian Pedregosa, Aidan Gomez, Erich Elsen
    Submitted on 25 June, 2019;
    arXiv:1906.10732

 

  • Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets
    Authors: Rohith Kuditipudi, Xiang Wang, Holden Lee, Yi Zhang, Zhiyuan Li, Wei Hu, Sanjeev Arora, Rong Ge
    Submitted on 14 June, 2019;
    arXiv:1906.06247

 

  • Large Scale Structure of Neural Network Loss Landscapes
    Stanislav Fort, Stanislaw Jastrzebski
    Submitted on 11 Jun 2019
    arXiv:1906.04724

 

 

  • The Effect of Network Depth on the Optimization Landscape
    Behrooz Ghorbani, Ying Xiao, Shankar Krishnan
    Submitted 28 May 2019
    Link

 

  • Visualizing Loss Landscape of Deep Neural Networks…..but can we Trust them?
    Jae Duk Seo
    May 5, 2019
    Link

 

  • Negative eigenvalues of the Hessian in deep neural networks
    Guillaume Alain, Nicolas Le Roux, Pierre-Antoine Manzagol
    Submitted on 6 Feb 2019
    arXiv:1902.02366

 

  • Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning
    Charles H. Martin, Michael W. Mahoney
    Submitted on 2 Oct 2018
    arXiv:1810.01075

 

  • On the loss landscape of a class of deep neural networks with no bad local valleys
    Quynh Nguyen, Mahesh Chandra Mukkamala, Matthias Hein
    Submitted on 27 Sep 2018
    arXiv:1809.10749

 

  • The Goldilocks zone: Towards better understanding of neural network loss landscapes
    Stanislav Fort, Adam Scherlis
    Submitted on 6 Jul 2018
    arXiv:1807.02581

 

  • The loss landscape of overparameterized neural networks
    Y Cooper
    Submitted on 26 Apr 2018
    arXiv:1804.10200

 

  • Measuring the Intrinsic Dimension of Objective Landscapes
    Chunyuan Li, Heerad Farkhoor, Rosanne Liu, Jason Yosinski
    Submitted on 24 Apr 2018
    arXiv:1804.08838

 

  • A Mean Field View of the Landscape of Two-Layers Neural Networks
    Song Mei, Andrea Montanari, Phan-Minh Nguyen
    Submitted on 18 Apr 2018
    arXiv:1804.06561

 

  • The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
    Jonathan Frankle, Michael Carbin
    Submitted on 9 Mar 2018
    arXiv:1803.03635

 

  • Essentially No Barriers in Neural Network Energy Landscape
    Felix Draxler, Kambis Veschgini, Manfred Salmhofer, Fred A. Hamprecht
    Submitted on 2 Mar 2018
    arXiv:1803.00885

 

  • Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs
    Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry Vetrov, Andrew Gordon Wilson
    Neural Information Processing Systems 2018
    Submitted on 27 Feb 2018
    arXiv:1802.10026

 

  • Visualizing the Loss Landscape of Neural Nets (A favourite of mine)  *********
    Li, Hao and Xu, Zheng and Taylor, Gavin and Studer, Christoph and Goldstein, Tom
    Neural Information Processing Systems 2018
    Submitted on 28 Dec 2017
    arXiv:1712.09913

 

  • Sharp Minima Can Generalize For Deep Nets
    Laurent Dinh, Razvan Pascanu, Samy Bengio, Yoshua Bengio
    Submitted on 28 March 2017
    arXiv:1703.04933

xyz

Go deep

What lies within the landscape?