Opening the Design Space: Two Years of Performance with Intelligent Musical Instruments
Abstract
Machine generation of symbolic music and digital audio are hot topics but there have been relatively few digital musical instruments that integrate generative AI. Present musical AI tools are not artist centred and do not support experimentation or integrating into musical instruments or practices. This work introduces an inexpensive generative AI instrument platform based on a single board computer that connects via MIDI to other musical devices. The platform uses artist-collected datasets with models trained on a regular computer. This paper asks what the design space of intelligent musical instruments might look like when accessible and portable AI systems are available for artistic exploration. I contribute five examples of instruments created and tested through a two-year first-person artistic research process. These show that (re)mapping can replace retraining for discovering AI interaction, that fast input interleaving is a new co-creative strategy, that small-data AI models can be a transportable design resource, and that cheap hardware can lower barriers to inclusion. This work could enable artists to explore new interaction and performance schemes with intelligent musical instruments.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
This paper presents a novel generative AI platform for intelligent musical instruments, emphasizing artist-centered design and small-data approaches. The comprehensive exploration of performance experiences and instrument development contributes valuable insights to the intersection of AI and music, highlighting the potential for innovative co-creative practices.
Methodology
The methodology is grounded in a first-person artistic research approach, which is innovative in the context of generative AI in music. The use of small-data AI models trained on artist-collected datasets is a significant contribution, allowing for a more personalized and artist-centered exploration of generative AI in musical contexts. The paper effectively outlines the design and implementation of a generative AI platform that integrates with existing musical instruments, showcasing a practical application of AI in music performance. The iterative development of five distinct instruments provides a rich qualitative dataset for analysis.
Results & evaluation
The experiments conducted over two years of performance practice are well-documented, providing insights into the evolution of the instruments and their interactions with musicians. The author details the performance experiences and the adaptability of the instruments in various contexts, which adds depth to the evaluation. However, the paper lacks quantitative metrics for assessing the performance of the AI models, which could strengthen the evaluation of their effectiveness.
Reproducibility
The implementation details are provided, including the use of Raspberry Pi and the open-source nature of the software, which enhances reproducibility. The availability of the project on GitHub allows others to replicate the setup and experiment with the platform. However, more detailed instructions on the configuration and training processes would further aid reproducibility.
Limitations
The study is limited by its first-person perspective, which may not capture the full range of experiences from diverse musicians. Additionally, the exploration of model updates over time is not systematically addressed, which could provide further insights into the adaptability and longevity of the AI models in performance settings.
Broader impact
This work has the potential to democratize access to intelligent musical instruments by lowering the cost barrier and encouraging experimentation among artists. The findings could influence future designs of musical AI systems, promoting a shift towards artist-centered approaches in generative AI applications. The implications for HCI and music technology communities are significant, as the research opens new avenues for interaction and collaboration between humans and AI in creative practices. This paper presents a novel generative AI platform for intelligent musical instruments, emphasizing artist-centered design and small-data approaches. The comprehensive exploration of performance experiences and instrument development contributes valuable insights to the intersection of AI and music, highlighting the potential for innovative co-creative practices.
Author of this paper? Send a correction.