PENGEMBANGAN CHATBOT INFORMASI ADMINISTRASI AKADEMIK FPMIPA UPI BERBASIS RETRIEVAL AUGMENTED GENERATION (RAG)

Mohammad Labib Husain, - and Yudi Wibisono, - and Ani Anisyah, - (2025) PENGEMBANGAN CHATBOT INFORMASI ADMINISTRASI AKADEMIK FPMIPA UPI BERBASIS RETRIEVAL AUGMENTED GENERATION (RAG). S1 thesis, Universitas Pendidikan Indonesia.

Abstract

Penyediaan informasi administrasi akademik di FPMIPA UPI terkendala oleh sebaran sumber informasi dan sinkronisasi yang kurang sehingga dapat berdampak pada inefisiensi pencarian informasi. Penelitian ini mengembangkan chatbot berbasis RAG sebagai pelengkap kanal resmi yang telah ada. Knowledge base disusun dari dokumen resmi dan hasil web scraping lalu diproses melalui segmentasi hierarkis serta pemetaan relasi antar chunk. Sistem menerapkan pipeline retrieval berlapis sebelum jawaban dibentuk oleh Gemini 2.5 Pro. Prototipe juga dilengkapi dengan memori percakapan. Evaluasi menunjukkan kinerja cukup baik. Pada 60 pertanyaan faktual tercapai Faithfulness 0,990 dan Answer Relevancy 0,967. Pada 20 pertanyaan penalaran sistem bebas halusinasi dengan Faithfulness 1,000 dan relevansi 0,960. Pada 20 pertanyaan di luar konteks sistem menolak secara konsisten dengan semua metrik 1,000 dan noise 0. Evaluasi UAT mengonfirmasi 83,3% jawaban benar. Secara keseluruhan sistem yang dibangun mampu memberikan jawaban yang relevan dan kontekstual untuk pertanyaan berbasis informasi akademik dengan performa yang relatif stabil pada berbagai jenis pertanyaan serta kualitas jawaban sangat ditentukan oleh ketepatan pada proses chunking dan retrieval. The provision of academic administrative information at FPMIPA UPI is hindered by dispersed information sources and weak synchronization which can lead to inefficient information retrieval. This study develops a RAG based chatbot to complement existing official channels. The knowledge base is compiled from official documents and web scraped results then processed through hierarchical segmentation as well as inter chunk relation mapping. The system applies a layered retrieval pipeline before answers are formed by Gemini 2.5 Pro. The prototype is also equipped with conversational memory. The evaluation shows fairly good performance. On 60 factual questions Faithfulness reached 0,990 and Answer Relevancy 0,967. On 20 reasoning questions the system was free of hallucinations with Faithfulness 1,000 and relevancy 0,960. On 20 out of context questions the system consistently refused with all metrics 1,000 and noise 0. UAT evaluation confirmed 83,3% correct answers. Overall the system is able to provide relevant and contextual answers for academic information questions with relatively stable performance across various question types and answer quality is strongly determined by accuracy in the chunking and retrieval processes.

	Text S_KOM_2101989_Title.pdf Download (542kB)
	Text S_KOM_2101989_Chapter1.pdf Download (369kB)
	Text S_KOM_2101989_Chapter2.pdf Restricted to Staf Perpustakaan Download (1MB) | Request a copy
	Text S_KOM_2101989_Chapter3.pdf Download (222kB)
	Text S_KOM_2101989_Chapter5.pdf Download (154kB)
	Text S_KOM_2101989_Appendix.pdf Restricted to Staf Perpustakaan Download (213kB) | Request a copy
	Text S_KOM_2101989_Chapter4.pdf Restricted to Staf Perpustakaan Download (2MB) | Request a copy

Item Type:	Thesis (S1)
Additional Information:	https://scholar.google.com/citations?hl=en&user=nb-gTJkAAAAJ ID SINTA Dosen Pembimbing: Yudi Wibisono: 260167 Ani Anisyah: 6786982
Uncontrolled Keywords:	Chatbot, Informasi Administrasi Akademik, Large Language Model, RAGAS, Retrieval Augmented Generation. Chatbot, Large Language Model, Academic Administration Services, Natural Language Processing, RAGAS, Retrieval Augmented Generation
Subjects:	L Education > L Education (General) Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions:	Fakultas Pendidikan Matematika dan Ilmu Pengetahuan Alam > Program Studi Ilmu Komputer
Depositing User:	Mohammad Labib Husain
Date Deposited:	11 Sep 2025 08:02
Last Modified:	11 Sep 2025 08:02
URI:	http://repository.upi.edu/id/eprint/138355

Actions (login required)

View Item