Researchers propose an Entropy-Driven Adaptive Router (EDAR) that dynamically chooses between retrieval-augmented generation and full long-context processing at inference time. The framework leverages predictive entropy from token-level probability distributions to decide whether retrieved chunks are sufficient or if the model must process the entire input. This hybrid approach aims to mitigate the high costs and middle-information loss of pure long-context models, while addressing the retrieval quality limitations of pure RAG systems.
- Hybrid routing avoids high costs of pure long-context processing when retrieval suffices.
- Predictive entropy serves as the metric for switching between RAG and full context modes.
- Addresses the 'middle information loss' problem inherent in standard long-context LLMs.
- Reduces RAG errors caused by partially relevant or contradictory retrieved chunks.
- Enables cost-performance optimization by escalating queries only when necessary.
