Machine learningLLMs & Text
In-Context Pretraining: Language Modeling Beyond Document Boundaries
Modeling Beyond Documents: The paper introduces In-Context Pretraining, a novel method that improves language models' performance by pretraining them on related documents, enhancing their contextual reasoning skills.
Featured in No. 22 on 18 Oct 2023 · 2 days after release · 101 citations today · published in International Conference on Learning Representations
- Released
- 16 Oct 2023
- First featured
- No. 22 · 18 Oct 2023
- Citations (Semantic Scholar)
- 101
- Influential citations
- 14
- Published in
- International Conference on Learning Representations
- Shares when featured
- 19
- Identifier
- arXiv:2310.10638
Citations and venue from Semantic Scholar (ODC-BY), refreshed weekly. Summary: Quant Letter (CC BY 4.0).