stat.MLcs.LG

Retrieval-Based In-Context Learning: A Domain Adaptation Framework

📅 👥 5 authors arXiv:2610.05717v2

Authors

Abstract

In-context retrieval (ICR) is a retrieval-based form of in-context learning (ICL) in which demonstrations are retrieved from a source database based on similarity to the query, rather than sampled independently. In this work, we formulate ICR as a type of domain adaptation problem, where the source distribution $P$ of the database may differ from the target distribution $Q$ of the test query-label pair. We investigate the performance of ICR under a flexible class of distributional shifts that substantially extends prior work \citep{li2024fine,guo2025retrieval}, and establish theoretical guarantees that quantify the benefits and pitfalls of this learning paradigm. Our theory is verified by experiments on synthetic and language tasks.

Paper Information

arXiv ID:
2610.05717v2
Published:

Related Categories

stat.MLcs.LG