Kavli Affiliate: Yi Zhou | First 5 Authors: Yufeng Yang, Yufeng Yang, , , | Summary: Recent studies have shown that many nonconvex machine learning problems satisfy a generalized-smooth condition that extends beyond traditional smooth nonconvex optimization. However, the existing algorithms are not fully adapted to such generalized-smooth nonconvex geometry and encounter significant technical limitations […]
Continue.. Adaptive Gradient Normalization and Independent Sampling for (Stochastic) Generalized-Smooth Optimization