I’ve been performing many audio experiments as part of the Meta Creation in NEOS VR project, produced by the BBMC. The goal of the project is to create a Non-Player Character, or NPC, inside of the social VR world NEOS VR, that is enhanced by machine learning software and is able to compose its own original audio independently and in real time.

I’ve built a virtual world for my intelligent NPC to live in and am in the final stages of connecting the in-world character to the external machine learning program that will process audio in real time. In the meantime, I have put the finishing touches on my NPC, which has evolved into an iridescent, singing, humpback whale-being with interactive behaviours. When it is linked to the external program, the whale will sing original compositions and respond to the actions of users in real time.

I’ve been working with musician Mike Kean to create the final compositions for programming into the whale-being, and have also been experimenting with how the machine learning program responds to different audio sources and parameters. The results are interesting, sometimes haunting, sometimes weird.

Here are the results of one experiment:

I created a composition themed on the idea of sorrow and then asked the program to explore the song and create a new composition using only whale song. This was the result:
Mournful Whales

Then, I created a composition themed on extinction and asked an program to explore the song and create a new composition using only whale song. This was the result:
Haunted Whales

Finally, I took the composition Haunted Whales, and asked the program to explore its own composition and create a new song using only music from my Liz Solo Alien album. This was the result:
Haunted Liz

I’ve been playing with the MASOM program to create the voice of the whale.

More details on the exhibition of under world coming soon.

under world and Metacreation in NEOS VR  have been made possible thanks to the Canada Council for the Arts | Conseil des arts du Canada and the Black Bag Media Collective.

under world CREDITS
JE Solo (psychic radio) – Character Creator/Composer/Meta-Composer/Sim Builder
Mike Kean – Additional Audio creation; sound engineering
Dr. Kivanç Tatar (Turing Touch) – Meta-Composition Software Creator
Lucas Rosetti (Lucasro7) – NEOS LogiX Programmer
Tomas Mariancik (Frooxius) – NEOS CTO, Co-Creator
Karel Hulec (Karel) – NEOS CEO, Co-Creator
Special thanks to Koffee, NEOS