Machine learningML & AI Methods

An exactly solvable model for emergence and scaling laws in the multitask sparse parity problem

A new framework represents each new ability in deep learning models as a basis function, providing analytic expressions for the emergence of new skills and scaling laws of the loss with various factors.