AI-Generated Text Probability Detection System for Indonesian Scientific Papers Using Perplexity-Burstiness Linguistic Analysis

DOI

10.2991/978-94-6239-805-4_2How to use a DOI?

Keywords

AI Detection; Artificial Intelligence; Burstiness; Perplexity; Scientific Articles

Abstract

The widespread adoption of artificial intelligence (AI) in scientific writing has introduced significant challenges in maintaining academic integrity, particularly in higher education environments. This research develops an AI-generated text probability detection system for Indonesian scientific papers using Perplexity and Burstiness linguistic analysis. Perplexity evaluates the uncertainty of a language model in predicting subsequent tokens, while Burstiness analyzes word distribution patterns to distinguish the linguistic characteristics of human-written and AI-generated texts. Unlike existing AI detection approaches that primarily focus on English general-purpose texts, this study emphasizes Indonesian scientific writing by integrating linguistic probability analysis with sentence-level visualization to provide interpretable detection results. The proposed system was developed using Python 3.11, Flask framework, and the Cahya/GPT2-small-indonesian-522M model as the language processing engine. Functional testing using the Black-box method demonstrated a 100% validation rate across all implemented features. AI probability evaluation on the selected test samples achieved 100% classification accuracy; however, further validation using larger and more diverse datasets is required to evaluate the generalization capability of the system. Performance testing showed response times ranging from 1,200–7,800 ms under different load scenarios, with error rates between 0–15%. Usability evaluation using the System Usability Scale (SUS) involving 30 respondents produced an average score of 80.9, categorized as Excellent (Grade A). The proposed system provides an interpretable approach for assessing AI-generated text probability in Indonesian scientific documents and has potential applications in maintaining academic integrity, including supporting the validation of scientific reports in sustainable tourism research and other academic domains.

Copyright

© 2026 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Gde Brahupadhya Subiksa
AU  - Made Sudarma
AU  - I Nyoman Eddy Indrayana
AU  - I Putu Astya Prayudha
AU  - I Ketut Gede Sudiartha
AU  - Yessi Aprilia Waluyo
PY  - 2026
DA  - 2026/10/08
TI  - AI-Generated Text Probability Detection System for Indonesian Scientific Papers Using Perplexity-Burstiness Linguistic Analysis
BT  - Proceedings of the International Conference on Sustainable Green Tourism Applied Science - Engineering Applied Science 2026 (ICOSTAS-EAS 2026)
PB  - Atlantis Press
SP  - 4
EP  - 14
SN  - 2352-5401
UR  - https://doi.org/10.2991/978-94-6239-805-4_2
DO  - 10.2991/978-94-6239-805-4_2
ID  - Subiksa2026
ER  -