Constructing Custom AI Simulations for Counselor Education Using Feedback-Guided Principles

DOI

10.2991/978-94-6463-950-6_6How to use a DOI?

Keywords

Large language models; AI roleplay; Simulated patients; Counselor training; Human-AI collaboration; Principle-adherence prompting; Expertguided simulation; Social simulation

Abstract

Large language models (LLMs) offer new opportunities for simulating realistic patient interactions, yet constructing believable AI roleplay partners for counselor practice presents significant challenges due to privacy limitations, domain specificity, and the ambiguity of human behavior. We introduce Roleplay-doh, an interactive human-LLM collaboration pipeline that enables domain experts to iteratively design custom simulated patients by eliciting qualitative feedback, which is then converted into explicit behavioral principles. We further propose a principle-adherence prompting pipeline that decomposes multifaceted expert principles into context-sensitive yes/no evaluation criteria and employs self-refinement to improve response fidelity. In a within-subjects user study with 25 experienced counselors, AI patients created using expert-driven principles were consistently rated higher in authenticity, resemblance to real cases, and suitability as training partners, compared to scenario-only baselines. Third-party evaluations confirmed improvements in several key dimensions, demonstrating the validity and scalability of the approach. Our results highlight the utility of natural language rule elicitation and principle-adherence mechanisms in producing nuanced LLM simulations suitable for professional training, and suggest broader applicability for expert-guided social simulation across diverse domains.

Copyright

© 2025 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Mainak Mitra
PY  - 2025
DA  - 2025/12/29
TI  - Constructing Custom AI Simulations for Counselor Education Using Feedback-Guided Principles
BT  - Proceedings of the International Conference on Smart Systems and Social Management (ICSSSM 2025)
PB  - Atlantis Press
SP  - 71
EP  - 80
SN  - 1951-6851
UR  - https://doi.org/10.2991/978-94-6463-950-6_6
DO  - 10.2991/978-94-6463-950-6_6
ID  - Mitra2025
ER  -