---
title: From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queries
url: https://www.ml-quant.com/papers/arxiv/2406.12824/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-09-26
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: arXiv:2406.12824
source_url: https://arxiv.org/abs/2406.12824
featured: 2024-06-20
citations: 20
topic: LLMs & Text
---


# From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queries

Retrieval Augmented Generation (RAG) enhances language models' reasoning abilities using external context, but models tend to rely heavily on this context and less on their parametric memory.

- Source: https://arxiv.org/abs/2406.12824
- Identifier: arXiv:2406.12824
- Released: 2024-06-18
- First featured: Quant Letter No. 54 (2024-06-20): https://www.ml-quant.com/issues/2024-06-20/
- Citations (Semantic Scholar): 20
- Published in: Proceedings of the Nineteenth ACM International Conference on Web Search and Data Mining
- Topic: LLMs & Text

## Related

- [RAFT: Adapting Language Model to Domain Specific RAG](https://www.ml-quant.com/papers/arxiv/2403.10131/): Retrieval Augmented FineTuning (RAFT) is a new training method that enhances large language models' ability to answer domain-specific questions by training them to ignore irrelevant documents and cite relevant ones.
- [HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extraction](https://www.ml-quant.com/papers/arxiv/2408.04948/): Q&A Systems for Financial Data: HybridRAG, a new method combining Knowledge Graphs and VectorRAG techniques, improves question-answer systems for extracting information from financial documents, offering better accuracy and answer generation.
- [ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilities](https://www.ml-quant.com/papers/arxiv/2407.14482/): Bridging the Gap: ChatQA 2 is a model that improves long-context understanding and retrieval-augmented generation, matching the accuracy of top proprietary models.
- [Language Modeling with Editable External Knowledge](https://www.ml-quant.com/papers/arxiv/2406.11830/): The paper presents ERASE, a method that improves the performance of retrieval-augmented generation models by incrementally modifying the knowledge base when new documents are added.
- [FACTS About Building Retrieval Augmented Generation-based Chatbots](https://www.ml-quant.com/papers/arxiv/2407.07858/): The article introduces the FACTS framework for developing Retrieval Augmented Generation (RAG)-based chatbots, and presents empirical results on the balance between accuracy and latency in large and small LLMs.
- [FlashBack: Efficient Retrieval-Augmented Language Modeling for Fast Inference](https://www.ml-quant.com/papers/arxiv/2405.04065/): Efficient LM: The paper introduces FlashBack, a Retrieval-Augmented Language Modeling system that enhances inference efficiency by adding retrieved documents to the context, leading to quicker inference speed and lower costs.