2023

  • From Taps to Drums: Audio-to-audio Percussion Style Transfer (ISMIR 2023)
    André C. Santos, Amílcar Cardoso
    Abstract A common, and arguably innate, human response when listening to music is to tap one’s foot to mark the regular pulse of the beat. A more complex form of interactive synchronization occurs when listeners tap out rhythmic patterns using their fingers, hands, or even some form of improvised drumsticks. In this late-breaking demo, we explore this interaction by leveraging the style transfer capabilities of a neural audio synthesis model by training it on a drum dataset and feeding it tapped rhythm recordings at inference time. We also provide a concise and high-level overview of the results, which, in our assessment, not only justify further research but also establish an intriguing baseline for future investigations. Finally, we point out several future research directions.
  • Focusing on artists’ needs: Using a cultural probe for artist-centred creative software development (ICCC 2023)
    Luís Espírito Santo, André C. Santos, Marcio Lima Inácio
    Abstract One of the ultimate goals of Computational Creativity research is to make novel, better, and useful software that can be used for creative purposes. The new wave of learning-endowed generative systems has highlighted the potential of AI for creative tasks, so demand for creative software development is expected to grow significantly, which in turn entails the need for adapted software engineering techniques. We conducted interviews and used a digital cultural probe that posed as a virtual co-creative companion with unlimited capabilities to collect qualitative data on how creative fellows, from different areas and with no knowledge about generative models, would use an ideal piece of creative software. By following an Inductive Thematic Analysis, we bring forward a set of domain-agnostic patterns of how software can help in creative tasks. These themes-12 user needs and 8 contexts of use-can be used to organise functional requirements to sustain an improved usercentred development of creative tools, or might even be used as a classification framework for creativity tools and co-creative systems. Finally, we discuss the benefits and limitations of our methodology that can be repurposed for a more suited and artist-centred initial process of functional requirement gathering.

2022

  • Co-creative Musical Repurposing by Modelling Rhythmic Compatibility (ICCC 2022)
    André C. Santos, Matthew E. P. Davies, Amílcar Cardoso
    Abstract A common, and arguably innate, human response when listening to music is to tap one’s foot to mark the regular pulse of the beat. A more complex form of interactive synchronization occurs when listeners tap out rhythmic patterns using their fingers, hands, or even some form of improvised drumsticks. The proposed goals of this research are two-fold: i) to investigate this complex, but under-explored, phenomenon of rhythmic engagement by building a computational model of rhythmic com- patibility; and ii) to leverage this knowledge to drive novel means for co-creative content repurposing via the layering and integration of user-tapped rhythms in a timbrally-consistent way with the source audio. While this research is predominantly computational, it will be approached in a multi-disciplinary fashion drawing upon music theory, psychology, computational creativ- ity, and machine learning in pursuit of musically mean- ingful next-generation tools for enhanced creativity and content personalization.

2021

  • MuSyFI-Music Synthesis From Images (ICCC 2021)
    André C. Santos, H. Sofia Pinto, Rui Pereira Jorge, Nuno Correia.
    Abstract MuSyFI is a system that tries to model an inspirational computational creative process. It uses images as source of inspiration and begins by implementing a possible translation between visual and musical features. Results of this mapping are fed to a Genetic Algorithm (GA) to try to better model the creative process and produce more interesting results. Three different musical arti- facts are generated: an automatic version, a co-created version, and a genetic version. The automatic version maps features from the image into musical features non- deterministically; the co-created version adds harmony lines manually composed by us to the automatic ver- sion; finally, the genetic version applies a genetic algo- rithm to a mixed population of automatic and co-created artifacts. The three versions were evaluated for six different images by conducting surveys. They evaluated whether people considered our musical artifacts music, if they thought the artifacts had quality, if they considered the artifacts ’novel’, if they liked the artifacts, and lastly if they were able to relate the artifacts with the image in which they were inspired. We gathered a total of 300 answers and overall people answered positively to all questions, which confirms our approach was successful and worth further exploring