AccScience Publishing / AC / Online First / DOI: 10.36922/AC025180032
Submit to AC
Cite this article
1
Download
70
Views
Journal Browser
Volume | Year
Issue
Search
Forthcoming Issue
Current Issue
View All
News and Announcements
View All
ARTICLE

Interactive audio toolkit: Creating sonic experiences and installations with low-cost, low-power microcontrollers

Aman Jagwani1* ,  Victor Lazzarini1
Show Less
1 Department of Music, Maynooth University, Maynooth, Ireland
AC, 025180032 https://doi.org/10.36922/AC025180032
Received: 30 April 2025 | Revised: 28 October 2025 | Accepted: 3 November 2025 | Published online: 20 November 2025
(This article belongs to the Special Issue Ubimus Cross-Cultural Approaches to Interaction and Creativity )
© 2025 by the Author(s). This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution -Noncommercial 4.0 International License (CC-by the license) ( https://creativecommons.org/licenses/by-nc/4.0/ )
Download PDF
XML
Cite
Abstract

Interactive sound installations and experiences are increasingly prevalent in museums, galleries, events, and various spaces worldwide. These installations are often powered by microcontroller board running programs that control interactivity and possibly even audio processing. Commonly used microcontrollers such as the ESP32 and Raspberry Pi Pico are generic, necessitating custom implementations for audio-specific functionalities. Existing audio libraries often have limited signal processing features and may not be designed with interactivity in mind. This paper presents the Interactive Audio Toolkit, which focuses on the seamless integration of sound generation with sensors to create interactive sound experiences with low-power, low-cost microcontrollers. This C++ toolkit is structured around sensor input and audio output classes, providing a simple interface to generate flexible audio responses from sensor interactions. This paper details the toolkit’s structure, components, and workflow, highlighting its ability to foster new forms of sonic and musical interactions for artists and audiences. A case study of an installation utilizing the toolkit is also presented.

Keywords
Sound installation
Interactive audio
Embedded audio
Interaction design
Ubimus toolkits
Funding
This work was funded by the Hume Scholarship from Maynooth University.
Conflict of interest
Victor Lazzarini is the Guest Editor of this special issue but was not in any way involved in the editorial and peer-review process conducted for this paper, directly or indirectly. Separately, other authors declared that they have no known competing financial interests or personal relationships that could have influenced the work reported in this paper.
References
  1. Timoney J, Lazzarini V, Keller D. DIY electronics for ubiquitous music ecosystems. In: Lazzarini V, Keller D, Otero N, Turchet L, editors. Ubiquitous Music Ecologies. Milton Park: Routledge; 2020.

 

  1. Jagwani A. Creative possibilities and customizability of live performance systems with open source programming platforms. In: Yaseen A, Bridges B, Messina M, Keller D, eds. Proceedings of the 2nd International Symposium on Ubiquitous Music (UbiMus 2023). Northern Ireland: Ulster University; 2023:133-144. Available from: https://www.ulster.ac.uk/ conference/ubimus

 

  1. Espressif Systems. ESP32. Available from: https://www. espressif.com/en/products/socs/esp32 [Last accessed on 2024 Apr 24].

 

  1. STM32. STM32 32-bit ARM Cortex MCUs. Available from: https://www.st.com/en/microcontrollers-microprocessors/ stm32-32-bit-arm-cortex-mcus.html [Last accessed on 2024 Apr 24].

 

  1. RaspberryPi. Available from: https://www.raspberrypi.org [Last accessed on 2024 Apr 24].

 

  1. Arduino. Available from: https://www.arduino.cc [Last accessed on 2024 Apr 28].

 

  1. PlatformIO. Available from: https://platformio.org [Last accessed on 2024 Apr 28].

 

  1. Ribeira Netto A, Castheloge L, Oliosi A, Mateus A, Costalonga L, Coura D. Árvore das Memórias: Instalação Multimídia Interativa. In: Proceedings of the Brazilian Symposium of Computer Graphic and Image Processing; 2015.

 

  1. Keller D, Gomes C, Aliel L. The handy metaphor: Bimanual, touchless interaction for the internet of musical things. J New Music Res. 2019;48:1-12. doi: 10.1080/09298215.2019.1654525

 

  1. Schatzmann P. Arduino-Audio-Tools. Available from: https://github.com/pschatzmann/arduino-audio-tools [Last accessed on 2024 Apr 28].

 

  1. Puckette M. Pure Data: Another Integrated Computer Music Environment. In: Proceedings of the Second Intercollege Computer Music Concerts, Tachikawa, Japan; 1997.

 

  1. Lazzarini V, Yi S, Ffitch J, Heintz J, Brandtsegg Ø, McCurdy I. Csound: A Sound and Music Computing System. Cham, Switzerland: Springer International Publishing; 2016. doi: 10.1007/978-3-319-45370-5

 

  1. Lazzarini V, Jagwani A. CSOUND 7: Bare Metal and Co-Processing Hardwares. In: Proceedings of the 19th Linux Audio Conference. Lyon, France; 2025. Available from: https://hal.science/hal-05096036

 

  1. Electro-Smith. Daisy. Available from: https://www.electro-smith.com/daisy [Last accessed on 2023 Jun 08].

 

  1. SparkFun. SparkFun VL53L1X Arduino Library. Available from: https://github.com/sparkfun/sparkfun_vl53l1x_ arduino_library [Last accessed on 2024 Apr 28].

 

  1. SparkFun. Load Cell Amplifier HX711 Breakout Hookup Guide. Available from: https://learn.sparkfun.com/tutorials/ load-cell-amplifier-hx711-breakout-hookup-guide/all [Last accessed on 2024 Apr 28].

 

  1. Arduino. Wire Library Documentation. Available from: https://docs.arduino.cc/learn/communication/wire [Last accessed on 2024 Apr 28].

 

  1. Xilinx. PS UART; 2023. Available from: https://xilinx-wiki. atlassian.net/wiki/spaces/a/pages/18842340/ps+uart [Last accessed on 2024 Feb 10].

 

  1. MaxBotix. Arduino Ultrasonic Sensors. Available from: https://maxbotix.com/pages/arduino-ultrasonic-sensors [Last accessed on 2024 Apr 28].

 

  1. Lazzarini V. Spectral Music Design: A Computational Approach. Oxford: Oxford University Press; 2021.

 

  1. Adafruit. All About Stepper Motors: What is a Stepper Motor? Available from: https://learn.adafruit.com/all-about-stepper-motors/what-is-a-stepper-motor [Last accessed on 2024 Apr 28].

 

  1. Arduino. AccelStepper Library. Available from: https:// www.arduino.cc/reference/en/libraries/accelstepper [Last accessed on 2024 Apr 28].

 

  1. Precision Microdrives. Vibration Motors. Available from: https://www.precisionmicrodrives.com/motors/vibration-motors [Last accessed on 2024 Apr 28].

 

  1. ArduinoJson. ArduinoJson Library. Available from: https:// arduinojson.org [Last accessed on 2024 Apr 28].

 

  1. Waqif A. Chaal. Available from: https://asimwaqif.com/ chaal [Last accessed on 2025 Apr 20].

 

  1. DesignOwl. Chaal by Asim Waqif: Art Installation Using Bamboo. Available from: https://www.youtube.com/ watch?v=kmql_mw3m0i&ab_channel=designowl [Last accessed on 2024 Apr 28].

 

  1. MDN Web Docs. WebSockets API. Available from: https://developer.mozilla.org/en-us/docs/web/api/websockets_api [Last accessed on 2024 Apr 28].

 

  1. React. React Documentation. Available from: https://react. dev [Last accessed on 2024 Feb 16].

 

  1. Espressif. mDNS API Reference for ESP32. Available from: https://docs.espressif.com/projects/esp-idf/en/stable/esp32/api-reference/protocols/mdns. html [Last accessed on 2024 Apr 28].

 

  1. Contributors F. Fastify: Fast and Low Overhead Web Framework for Node. https://fastify.dev [Last accessed on 2024 Feb 16].

 

  1. Turchet L, Essl G, Fischione C. Ubiquitous music and the internet of musical things. In: Lazzarini V, Keller D, Otero N, Turchet L, editors. Ubiquitous Music Ecologies. London: Routledge; 2020.

 

  1. Wiener N. Cybernetics: Or Control and Communication in the Animal and the Machine. United States: MIT Press; 1948.

 

  1. Von Bertalanffy L. General System Theory: Foundations, Development, Applications. New York: George Braziller; 1968.

 

  1. Koszolko MK, Studley T. From site-specific sampling to gamification: An exploration of performative engagement with the environment. Organised Sound. 2023;28(3):338-351. doi: 10.1017/S1355771823000547

 

  1. Bridges B, Lazzarini V, Keller D. Editorial: Ecologically grounded creative practices and ubiquitous music: Interaction and environment. Organised Sound. 2024;28(3):321-327. doi: 10.1017/S1355771823000663

 

  1. TensorFlow. TensorFlow Lite for Microcontrollers. Available from: https://www.tensorflow.org/lite/microcontrollers [Last accessed on 2024 Apr 28].
Share
Back to top
Arts & Communication, Electronic ISSN: 2972-4090 Published by AccScience Publishing
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here
Accept