A new arXiv paper compares fine-tuning versus Retrieval Augmented Generation (RAG) for training Language Models as world models in text-based environments. The systematic evaluation across five diverse settings, including embodied and web navigation tasks, shows that fine-tuned models generally outperform RAG-based approaches. Specifically, fine-tuned world models enabled agents to achieve higher rewards in 15 out of 20 tested scenarios.
- Fine-tuning is currently superior to RAG for LM-based world modeling in most tested environments.
- Agents using fine-tuned world models achieved higher rewards in 15/20 scenarios compared to RAG.
- RAG remains underexplored for world modeling despite its success in other LM applications.
- Consider fine-tuning over retrieval augmentation when building text-agent simulation systems.