CDS4RAG: RAG向け循環二重逐次型ハイパーパラメータ最適化 CDS4RAG: Cyclic Dual-Sequential Hyperparameter Optimization for RAG

Retrieval-Augmented Generation (RAG) has become a default pattern for grounding large language models in external knowledge, but its end-to-end quality is notoriously sensitive to a long list of hyperparameters: chunk size and overlap, embedding model choice, top-k, reranker settings, prompt templates, and generation parameters such as temperature. The paper introduces CDS4RAG, a cyclic dual-sequential hyperparameter optimization framework aimed at tuning these knobs more systematically.

The core idea is to split the RAG pipeline into two sub-spaces, broadly corresponding to the retriever side and the generator side, and to optimize them in alternation rather than jointly. One sub-space is held fixed while the other is tuned sequentially, and the procedure cycles back and forth until convergence. Joint optimization over the full Cartesian space tends to be sample-inefficient, and naive grid or random search becomes expensive as the number of components grows. By decomposing the problem, each inner optimization sees a smaller, better-shaped search space, and the outer cycle is intended to capture cross-component interactions that a single pass would miss.

This work sits in an increasingly crowded space. Evaluation frameworks like Ragas, TruLens, and ARES have made multi-objective RAG scoring (faithfulness, answer relevancy, context precision, and so on) more tractable, which in turn enables principled hyperparameter search. Bayesian optimization libraries such as Optuna are commonly wired into RAG pipelines, and open-source projects like AutoRAG and RAGBuilder explicitly target automated configuration discovery. Against that backdrop, CDS4RAG's contribution appears to be less about a new search algorithm and more about the structural decomposition: cycling between retriever and generator sub-problems rather than treating them as one monolithic black box.

There are practical reasons this decomposition may be attractive. Retrieval-side experiments are often dominated by embedding and indexing costs, while generator-side experiments are dominated by LLM inference costs; running them in alternation can make budget allocation more predictable and potentially more parallelizable. It also maps naturally to how teams actually iterate in production, where retrieval quality is typically stabilized first before prompt and decoding choices are fine-tuned.

Some caveats are worth keeping in mind. Alternating optimization schemes have well-known failure modes when sub-spaces are strongly coupled, and convergence guarantees for cyclic procedures over noisy, expensive black-box objectives are generally weak. The effectiveness of CDS4RAG will likely depend on how the sub-spaces are partitioned and how the inner search is implemented, and readers should consult the paper itself for benchmark details and baseline comparisons before drawing strong conclusions about generalizability.