Low-Cost Gesture Guided Swarm Control Using MediaPipe and Decentralized Behavior

Ishan Zadbuke; Sahil Panchavishe; Vivek Khandelwal; Yashwardhan Abhale; Anshul Jain

doi:10.2991/978-94-6239-707-1_19

<Previous Article In Volume

Next Article In Volume>

Low-Cost Gesture Guided Swarm Control Using MediaPipe and Decentralized Behavior

Authors

Ishan Zadbuke¹, Sahil Panchavishe¹, Vivek Khandelwal¹, Yashwardhan Abhale¹, Anshul Jain¹^{, *}

¹COEP Technological University, Pune, Maharashtra, India

^*Corresponding author. Email: anshul_ajmera2007@yahoo.co.in

Corresponding Author

Anshul Jain

Available Online 18 June 2026.

DOI: 10.2991/978-94-6239-707-1_19 How to use a DOI?
Keywords: Human Swarm Interaction; Swarm Robotics; Computer Vision; MediaPipe; Decentralized Control
Abstract: Human–Swarm Interaction (HSI) supports straightforward control of groups of robots, but many current systems depend on costly sensing equipment like motion capture rigs, depth cameras, or wearable devices, which limits their use in educational and resource constrained environments. In this work, we propose a low-cost, vision based HSI system that uses a standard monocular webcam and the MediaPipe library to help gesture driven control of a ground robot swarm. The system extracts 21 hand landmarks and converts them into a normalized 42 dimensional feature representation for real time gesture recognition using simple geometric rules. High level commands are broadcast to the swarm using a centralized star network topology with User Datagram Protocol (UDP) broadcasting, while execution remains decentralized at the robot level. Robustness to packet loss and network jitter is achieved through a latching based control mechanism. Experimental results show an average gesture recognition accuracy of 95.0% with a vision processing latency of approximately 32 ms per frame, and an estimated end to end system latency of about 87 ms, confirming suitability for real time interaction.
Copyright: © 2026 The Author(s)
Open Access: Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

<Previous Article In Volume

Next Article In Volume>

Volume Title: Proceedings of the International Conference on Recent Advances in Intelligent and Sustainable Technologies (RAIST 2026)
Series: Atlantis Highlights in Intelligent Systems
Publication Date: 18 June 2026
ISBN: 978-94-6239-707-1
ISSN: 2589-4919
DOI: 10.2991/978-94-6239-707-1_19 How to use a DOI?
Open Access: Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Cite this article

ris enw bib

TY  - CONF
AU  - Ishan Zadbuke
AU  - Sahil Panchavishe
AU  - Vivek Khandelwal
AU  - Yashwardhan Abhale
AU  - Anshul Jain
PY  - 2026
DA  - 2026/06/18
TI  - Low-Cost Gesture Guided Swarm Control Using MediaPipe and Decentralized Behavior
BT  - Proceedings of the International Conference on Recent Advances in Intelligent and Sustainable Technologies (RAIST 2026)
PB  - Atlantis Press
SP  - 222
EP  - 231
SN  - 2589-4919
UR  - https://doi.org/10.2991/978-94-6239-707-1_19
DO  - 10.2991/978-94-6239-707-1_19
ID  - Zadbuke2026
ER  -

download .riscopy to clipboard