---
title: LLARVA: Vision-Action Instruction Tuning Enhances Robot Learning
url: https://www.ml-quant.com/papers/arxiv/2406.11815/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-09-26
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: arXiv:2406.11815
source_url: https://arxiv.org/abs/2406.11815
featured: 2024-06-20
citations: 96
topic: ML & AI Methods
---


# LLARVA: Vision-Action Instruction Tuning Enhances Robot Learning

Robot Learning Enhancement: The LLARVA model, trained with a new method, successfully unifies various robotic learning tasks and performs well in different robot environments.

- Source: https://arxiv.org/abs/2406.11815
- Identifier: arXiv:2406.11815
- Released: 2024-06-17
- First featured: Quant Letter No. 54 (2024-06-20): https://www.ml-quant.com/issues/2024-06-20/
- Citations (Semantic Scholar): 96
- Published in: Conference on Robot Learning
- Topic: ML & AI Methods

## Related

- [Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Control](https://www.ml-quant.com/papers/arxiv/2409.08861/): The study presents Adjoint Matching, a new algorithm that enhances dynamical generative models by refining reward fine-tuning, leading to improved consistency, realism, and adaptability to unseen human preference reward models.
- [Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRA](https://www.ml-quant.com/papers/arxiv/2304.06027/): CLoRA, a new method, prevents catastrophic forgetting in text-to-image models when introducing new concepts, achieving top performance in continual learning settings for image classification.
- [Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers](https://www.ml-quant.com/papers/arxiv/2409.20537/): The paper presents Heterogeneous Pre-trained Transformers (HPT), a method for training robotic models across various tasks, improving the performance of fine-tuned policies by over 20% on unseen tasks in both simulated and real-world environments.
- [Fine-tuning can cripple your foundation model; preserving features may be the solution](https://www.ml-quant.com/papers/arxiv/2308.13320/): Concept forgetting in AI models can be significantly reduced by a new fine-tuning method called LDIFS, which helps retain pre-trained knowledge while working on different tasks.
- [DITTO: Diffusion Inference-Time T-Optimization for Music Generation](https://www.ml-quant.com/papers/arxiv/2401.12179/): Music Optimization: The DITTO framework allows control of pre-trained text-to-music diffusion models at inference-time, enabling music generation without fine-tuning.
- [ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuning](https://www.ml-quant.com/papers/arxiv/2411.05003/): ReCapture is a method for creating new videos with unique camera trajectories from a single video, allowing for the regeneration of the video from different angles and cinematic camera motion.