Machine learningML & AI Methods
Scaling Law with Learning Rate Annealing
The study introduces a scaling law for predicting the loss of neural language model training at any step and learning rate, aiding researchers in choosing optimal learning rate schedules.
Featured in No. 62 on 21 Aug 2024 · 1 day after release · 31 citations today · published in Neural Information Processing Systems
- Released
- 20 Aug 2024
- First featured
- No. 62 · 21 Aug 2024
- Citations (Semantic Scholar)
- 31
- Influential citations
- 4
- Published in
- Neural Information Processing Systems
- Shares when featured
- 161
- Identifier
- arXiv:2408.11029
Citations and venue from Semantic Scholar (ODC-BY), refreshed weekly. Summary: Quant Letter (CC BY 4.0).