Comparative Analysis of Retrieval-Augmented Generation Performance in a Clinical Nutrition Question-Answering System

DOI

10.2991/978-94-6239-828-3_11How to use a DOI?

Keywords

Retrieval-Augmented Generation; Question and Answer System; Clinical Nutrition; BERTScore; RAGAS; Large Language Model; Indonesian Language

Abstract

The rapidly expanding body of biomedical information poses a challenge for healthcare professionals in accessing accurate answers, particularly given the limited availability of Indonesian-language literature. The use of Large Language Models (LLMs) for biomedical question answering systems also faces the issue of hallucination, which poses serious risks in clinical settings. To address this, the present study designs, implements, and evaluates a Retrieval-Augmented Generation (RAG) pipeline for a question-answering system focused on clinical nutrition, using an Indonesian-language dataset. The pipeline consists of five main components: a query embedder, a document retriever, a prompt builder, a generator based on Llama 3.2 3B and GPT-OSS, and an answer builder. Evaluation was conducted using BERTScore and RAGAS across four experiments comparing the two models with and without RAG. The results show that RAG consistently improves answer quality. Llama 3.2 3B with RAG achieved the highest BERT F1 score (0.5703), with a 5.8% improvement, while GPT-OSS with RAG reached 0.5658 with a 10.3% improvement. The RAGAS evaluation revealed a trade-off between faithfulness and answer relevance: GPT-OSS excelled in faithful- ness (0.6881) but scored low on answer relevance (0.1595), while Llama 3.2 3B demonstrated a better balance. These findings confirm that small- scale open-source models with RAG can serve as an efficient alternative for Indonesian-language clinical question-answering systems.

Copyright

© 2026 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Muhammad Yasin Hakim
AU  - Tirana Noor Fatyanosa
AU  - Mahendra Data
AU  - Ummu Ditya Erliana
PY  - 2026
DA  - 2026/11/04
TI  - Comparative Analysis of Retrieval-Augmented Generation Performance in a Clinical Nutrition Question-Answering System
BT  - Proceedings of the 2026 11th International Conference on Sustainable Information Engineering and Technology (SIET 2026)
PB  - Atlantis Press
SP  - 118
EP  - 129
SN  - 2589-4900
UR  - https://doi.org/10.2991/978-94-6239-828-3_11
DO  - 10.2991/978-94-6239-828-3_11
ID  - Hakim2026
ER  -