---
title: On the hardness of learning under symmetries
url: https://www.ml-quant.com/papers/arxiv/2401.01869/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-09-26
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: arXiv:2401.01869
source_url: http://arxiv.org/abs/2401.01869
featured: 2024-01-09
citations: 18
topic: ML & AI Methods
---


# On the hardness of learning under symmetries

The process of learning neural networks through gradient descent is complex, despite the advantages of integrating known symmetries, as indicated by lower bounds for various network types.

- Source: http://arxiv.org/abs/2401.01869
- Identifier: arXiv:2401.01869
- Released: 2024-01-03
- First featured: Quant Letter No. 32 (2024-01-09): https://www.ml-quant.com/issues/2024-01-09/
- Citations (Semantic Scholar): 18
- Published in: International Conference on Learning Representations
- Topic: ML & AI Methods

## Related

- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://www.ml-quant.com/papers/arxiv/2312.00752/): Sequence Modeling: Mamba, a neural network architecture that doesn't use attention or MLP blocks, provides faster inference and better performance in language, audio, and genomics than Transformers.
- [Graph Mamba: Towards Learning on Graphs with State Space Models](https://www.ml-quant.com/papers/arxiv/2402.08678/): Graph Mamba Networks, a new type of Graph Neural Networks, have been introduced, which achieve excellent performance in various benchmark datasets despite lower computational cost.
- [Edge Directionality Improves Learning on Heterophilic Graphs](https://www.ml-quant.com/papers/arxiv/2305.10498/): The study presents Directed Graph Neural Network (Dir-GNN), a new deep learning framework for directed graphs that surpasses traditional models in heterophilic benchmarks.
- [Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks](https://www.ml-quant.com/papers/arxiv/2310.02244/): Deep Residual Network Feature Learning: The research explores depthwise parametrizations in deep residual networks, pinpointing Depth-$\mu$P as the best parametrization for maximizing feature learning and diversity, but notes its limitations in deeper networks.
- [Modular Duality in Deep Learning](https://www.ml-quant.com/papers/arxiv/2410.21265/): The article presents a new theory of modular dualization for general neural networks, providing a theoretical basis for fast and scalable training algorithms, potentially leading to a new generation of optimizers for neural architectures.
- [SparseProp: Efficient Event-Based Simulation and Training of Sparse Recurrent Spiking Neural Networks](https://www.ml-quant.com/papers/arxiv/2312.17216/): SparseProp, an event-based algorithm for simulating and training large-scale spiking neural networks, is introduced, offering reduced computational cost and efficient training.