---
title: Language Model Training on Edit Sequences Enhances Code Synthesis
url: https://www.ml-quant.com/papers/web/9b37a775db/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-09-26
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: arxiv.org
source_url: https://arxiv.org/abs//2410.02749
featured: 2024-10-09
citations: unknown
topic: LLMs & Text
---


# Language Model Training on Edit Sequences Enhances Code Synthesis

The paper presents LintSeq, a synthetic data generation algorithm that refactors code into a sequence of edits, resulting in more diverse programs and improved code synthesis performance.

- Source: https://arxiv.org/abs//2410.02749
- Identifier: arxiv.org
- Released: 2024-10-03
- First featured: Quant Letter No. 69 (2024-10-09): https://www.ml-quant.com/issues/2024-10-09/
- Citations (Semantic Scholar): not tracked
- Published in: not yet
- Topic: LLMs & Text

## Related

- [Scaling Synthetic Data Creation with 1,000,000,000 Personas](https://www.ml-quant.com/papers/arxiv/2406.20094/): A new method for creating synthetic data uses 1 billion diverse personas, potentially transforming large language model research and development.
- [Synthetic continued pretraining](https://www.ml-quant.com/papers/arxiv/2409.07431/): Researchers suggest using EntiGraph, a synthetic data augmentation algorithm, for synthetic continued pretraining to help language models answer questions and follow instructions related to source documents more effectively.
- [Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources](https://www.ml-quant.com/papers/arxiv/2409.08239/): Synthetic Data for LLMs: Source2Synth, a new method for teaching Large Language Models new skills without human annotations, has improved multi-hop and tabular question answering by 22.57% and 25.51% respectively.
- [Understanding Synthetic Context Extension via Retrieval Heads](https://www.ml-quant.com/papers/arxiv/2410.22316/): The paper finds that fine-tuning long-context language models with synthetic data improves performance in retrieval and reasoning tasks, with attention heads predicting performance.
- [Better Alignment with Instruction Back-and-Forth Translation](https://www.ml-quant.com/papers/arxiv/2408.04614/): A new method, instruction back-and-forth translation, is introduced for creating high-quality synthetic data to improve large language models, outperforming other datasets on AlpacaEval.
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://www.ml-quant.com/papers/arxiv/2402.03300/): Advancing Math Reasoning in Language Models: DeepSeekMath7B is a new language model that uses web data and Group Relative Policy Optimization for advanced mathematical reasoning, scoring high on the MATH benchmark.