This page contains papers and related content linked to the loss landscape challenge. If you have a paper or article that you would like to share, email ideami@ideami.com.
- An Optimisation Framework for the Well-Conditioned Training of Physics-Informed Neural Networks Joseph Webb, Sadok Jerad, Coralia Cartis — 2 July 2026 arXiv:2607.02194
- Revisiting the Volume Hypothesis Ari Pakman, Lior Kreimer, Yakir Berchenko — 30 June 2026 arXiv:2606.31282
- Scalar Representations of Neural Network Training Dynamics Pedro Jiménez-González, Miguel C. Soriano, Lucas Lacasa — 29 June 2026 arXiv:2606.30384
- Exploiting Local Flatness for Efficient Out-of-Distribution Detection Seonghwan Park, Hyunji Jung, Dongyeop Lee, Namhoon Lee — 29 June 2026 arXiv:2606.29952
- Closed-Form Steepest Descent Direction toward Flat Minima: Reducing Upper Bounds on the Loss Hessian Eigenspectrum in Neural Networks Yuto Omae, Kazuki Sakai, Yohei Kakimoto, Makoto Sasaki, Yusuke Sakai, Hirotaka Takahashi — 26 June 2026 arXiv:2606.28662
- Neural Network Quantization by Learning Low-Loss Subspaces Vladimir Protsenko, Mikhalina Kharkevich, Alexander Vashchilko, Vladimir Kryzhanovskiy — 23 June 2026 arXiv:2606.25087
- Curvature-Guided Mixing for MLLM Adaptation Jinglong Yang, Jiaxuan He, Wenjian Huang, Zhan Zhuang, Jianguo Zhang — 23 June 2026 arXiv:2606.24963
- Flatness Preserves Instruction Following in Vision-Language-Action Models Haochen Zhang, Yonatan Bisk — 22 June 2026 arXiv:2606.23641
- Exposing the Illusion of Erasure in Knowledge Editing for LLMs Advik Raj Basani, Anshuman Chhabra — 22 June 2026 arXiv:2606.23276
- Bypassing Minimization Bias: A Shift-Invariant Variance Estimator for Off-Equilibrium Local Learning Coefficients Yingjia Cai — 21 June 2026 arXiv:2606.22389
- Loss Landscape Poisoning: Targeted Extraction of Unseen Training Data from LLMs Md Abdullah Al Mamun, Ngoc Phu Doan, Pedram Zaree, Nael Abu-Ghazaleh, Ihsen Alouani — 15 June 2026 arXiv:2606.17110
- A Bifurcation Theory Framework for Gradient Descent on the Edge of Stability Eric Gan — 13 June 2026 arXiv:2606.15551
- Lost in the Non-convex Loss Landscape: How to Fine-tune the Large Time Series Model? Xu Zhang, Peang Wang, Wei Wang — 7 June 2026 arXiv:2606.08578
- Beyond Structural Symmetries: Linear Mode Connectivity via Neuron Identifiability Vincent Bürgin, Daniel Herbst, Ya-Wei Eileen Lin, Stefanie Jegelka — 3 June 2026 arXiv:2606.04754
- When Both Layers Learn: Training Dynamics of Representing Linear Models via ReLU Networks Berk Tinaz, Changzhi Xie, Mahdi Soltanolkotabi — 3 June 2026 arXiv:2606.04476
- A Geometric Characterization of the Stationary Plateau for Two-Layer Neural Networks Tian Ding, Dawei Li, Ruoyu Sun — 2 June 2026 arXiv:2606.04327
- Quadratic integrate-and-fire neurons exhibit less fragmented loss landscapes and outperform leaky integrate-and-fire neurons in spike-based gradient descent Carlo Wenig, Raoul-Martin Memmesheimer, Christian Klos — 2 June 2026 arXiv:2606.03935
- Adaptive Sharpness-Aware Minimization with a Polyak-type Step size: A Theory-Grounded Scheduler Dimitris Oikonomou, Nicolas Loizou — 1 June 2026 arXiv:2606.01827
- Taming the Loss Landscape of PINNs with Noisy Feynman-Kac Supervision: Operator Preconditioning and Non-Asymptotic Error Bounds Nathanael Tepakbong, Hanyu Hu, Chengyu Liu, Xiang Zhou — 30 May 2026 arXiv:2606.00643
- Exploiting weight-space symmetries for approximating curvature Artem Artemev, Rui Xia, Benjamin M. Boyd, Youjing Yu, Felix Dangel, Guillaume Hennequin, Alberto Bernacchia — 29 May 2026 arXiv:2606.00442
- Why Flatness Correlates With Generalization For Deep Neural Networks Shuofeng Zhang, Isaac Reid, Guillermo Valle Pérez, Ard Louis — 10 March 2021 arXiv:2103.06219
- Quantum Earth Mover's Distance: A New Approach to Learning Quantum Data Bobak Toussi Kiani, Giacomo De Palma, Milad Marvian, Zi-Wen Liu, Seth Lloyd — 08 Jan 2021 arXiv:2101.03037
- Towards a Better Global Loss Landscape of GANs Ruoyu Sun, Tiantian Fang, Alex Schwing — 10 Nov 2020 arXiv:2011.04926
- Approximation and convergence of GANs training: an SDE approach Haoyang Cao, Xin Guo — 2 Jun 2020 arXiv:2006.02047
- Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness Pu Zhao, Pin-Yu Chen, Payel Das, Karthikeyan Natesan Ramamurthy, Xue Lin — 30 Apr 2020 arXiv:2005.00060
- Revisiting Loss Landscape for Adversarial Robustness Dongxian Wu, Yisen Wang, Shu-tao Xia — 13 Apr 2020 arXiv:2004.05884
- Understanding Global Loss Landscape of One-hidden-layer ReLU Neural Networks Bo Liu — 12 Feb 2020 arXiv:2002.04763
- Deep Ensembles: A Loss Landscape Perspective Stanislav Fort, Huiyi Hu, Balaji Lakshminarayanan — 5 Dec 2019 arXiv:1912.02757
- Emergent properties of the local geometry of neural loss landscapes Stanislav Fort, Surya Ganguli — 14 Oct 2019 arXiv:1910.05929
- Loss Landscape Sightseeing with Multi-Point Optimization Ivan Skorokhodov, Mikhail Burtsev — 9 Oct 2019 arXiv:1910.03867
- How noise affects the Hessian spectrum in overparameterized neural networks Mingwei Wei, David J Schwab — 1 Oct 2019 arXiv:1910.00195
- Visualizing and Understanding the Effectiveness of BERT Yaru Hao, Li Dong, Furu Wei, Ke Xu — 15 Aug 2019 arXiv:1908.05620
- The Difficulty of Training Sparse Neural Networks Utku Evci, Fabian Pedregosa, Aidan Gomez, Erich Elsen — 25 Jun 2019 arXiv:1906.10732
- Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets Rohith Kuditipudi, Xiang Wang, Holden Lee, Yi Zhang, Zhiyuan Li, Wei Hu, Sanjeev Arora, Rong Ge — 14 Jun 2019 arXiv:1906.06247
- Large Scale Structure of Neural Network Loss Landscapes Stanislav Fort, Stanislaw Jastrzebski — 11 Jun 2019 arXiv:1906.04724
- Understanding Generalization through Visualizations W. Ronny Huang, Zeyad Emam, Micah Goldblum, Liam Fowl, Justin K. Terry, Furong Huang, Tom Goldstein — 7 Jun 2019 arXiv:1906.03291
- The Effect of Network Depth on the Optimization Landscape Behrooz Ghorbani, Ying Xiao, Shankar Krishnan — 28 May 2019 Link
- Visualizing Loss Landscape of Deep Neural Networks…..but can we Trust them? Jae Duk Seo — 5 May 2019 Link
- Negative eigenvalues of the Hessian in deep neural networks Guillaume Alain, Nicolas Le Roux, Pierre-Antoine Manzagol — 6 Feb 2019 arXiv:1902.02366
- Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory Charles H. Martin, Michael W. Mahoney — 2 Oct 2018 arXiv:1810.01075
- On the loss landscape of a class of deep neural networks with no bad local valleys Quynh Nguyen, Mahesh Chandra Mukkamala, Matthias Hein — 27 Sep 2018 arXiv:1809.10749
- The Goldilocks zone: Towards better understanding of neural network loss landscapes Stanislav Fort, Adam Scherlis — 6 Jul 2018 arXiv:1807.02581
- The loss landscape of overparameterized neural networks Y Cooper — 26 Apr 2018 arXiv:1804.10200
- Measuring the Intrinsic Dimension of Objective Landscapes Chunyuan Li, Heerad Farkhoor, Rosanne Liu, Jason Yosinski — 24 Apr 2018 arXiv:1804.08838
- A Mean Field View of the Landscape of Two-Layers Neural Networks Song Mei, Andrea Montanari, Phan-Minh Nguyen — 18 Apr 2018 arXiv:1804.06561
- The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle, Michael Carbin — 9 Mar 2018 arXiv:1803.03635
- Essentially No Barriers in Neural Network Energy Landscape Felix Draxler, Kambis Veschgini, Manfred Salmhofer, Fred A. Hamprecht — 2 Mar 2018 arXiv:1803.00885
- Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry Vetrov, Andrew Gordon Wilson (NeurIPS 2018) — 27 Feb 2018 arXiv:1802.10026
- Visualizing the Loss Landscape of Neural Nets Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, Tom Goldstein (NeurIPS 2018) — 28 Dec 2017 arXiv:1712.09913
- Sharp Minima Can Generalize For Deep Nets Laurent Dinh, Razvan Pascanu, Samy Bengio, Yoshua Bengio — 28 Mar 2017 arXiv:1703.04933