No AI summary available for this article.
Why It Matters
Neural-network optimization in 2025-2026 is no longer well described as a succession of new Adam variants.
Provenance
Discovered via ArXiv and published by ArXiv.
Key Claims
Original description
Neural-network optimization in 2025-2026 is no longer well described as a succession of new Adam variants. The design space has expanded from coordinates to matrices and layers, from fixed training horizons to policies over time, and from mathematical update rules to state representations that must survive sharding and low-precision computation. This survey organizes recent optimizers and training optimization methods along four largely independent axes: temporal estimation, update geometry, horizon management, and representation and systems. It connects the spectral normalization of Muon, the...
Discovered via ArXiv
Research papers and preprints from arXiv.
Publisher: arxiv.org
ID: http://arxiv.org/abs/2608.28557v1 · Indexed 1 day ago