Month after month, new products, programs, and experiments show us how vast the potential of generative AI truly is. To that point, Google teamed up with researchers from Japan and found a way to produce music from human brain activity captured using functional magnetic resonance imaging (fMRI) and reconstructed with Google’s MusicLM music generation model.
As Google explains in a research paper titled “Brain2Music: Reconstructing Music from Human Brain Activity” (via ZDNet), 15-second clips from 540 music pieces spanning ten different genres were selected at random. Five participants listened to the clips through a pair of MRI-compatible insert earphones, during which time their brain activity was scanned.
The researchers fed the data into MusicLM to “predict and reconstruct the kinds of music which the human subject was exposed to.” In the end, the generated music did bear similarities to the music the test subjects listened to originally “on a semantic level.”
If you want to hear some of the reconstructed brain activity music, Google set up a webpage for the study featuring the original stimuli and the reconstructions.
The research team also revealed three factors limiting the quality of the AI music:
- the information contained in the fMRI data is very temporally and spatially sparse (the observed regions are 2×2×2mm3 in size, many orders of magnitude larger than human neurons).
- the information contained in the music embeddings from which we reconstruct the music (we used MuLan, in which ten seconds of music are represented by just 128 numbers).
- the limitations of our music generation system. When we studied MusicLM, we saw that it has room to improve both in the way it adheres to the text prompt and in terms of the fidelity of the produced audio.
As with most language models, there is still plenty of work to be done. That said, it’s already clear that AI can effectively use your brain activity to recreate the sounds that you hear. But if you’re worried about a stranger scanning your brain waves and stealing your thoughts, you’re probably safe (for now). The team notes that the volunteers had to spend hours in a large fMRI scanner for this study. So just avoid going on any first dates in large fMRI scanners.
