How Mozart Might Have Played Metallica, According to Artificial Intelligence
This system applies subtle stylistic techniques gleaned from a musician's work to samples of another artist, suggesting how one musician would likely have played another's music.
It's the kind of question that usually arises in those quiet moments of free-association, when listening to your favorite records, for instance, or just before sleep, or after that third cup of coffee: What would Beethoven have sounded like playing a Jimi Hendrix song?
A group of researchers in the UK is hoping to answer these questions by way of a machine learning system that analyzes the style of different musicians, then mashes up the music into virtual collaborations that can span centuries. What would Hank Williams playing an Adele ballad sound like? How would Tchaikovsky tackle the White Stripes?
The machine-learning system works by parsing inputted samples of music and identifying subtle stylistic techniques specific to a particular musician. Those techniques can then be isolated and, through the magic of mathematical algorithms, applied to a second sample of music. The resulting output suggests how one musician would likely have played another's music.
The system began as a smaller project to simply determine stylistic differences between Irish traditional flute players, Islah Ali-MacLachlan, senior lecturer in sound engineering at Birmingham City University in England and lead researcher, told Seeker.
“Each genre of music has its own nuances, so it’s important from a musicology point of view to extract the correct information for further classification,” he said. “With flute this is particularly difficult because a player can change notes without introducing a new attack, unlike piano or guitar. We have concentrated on algorithms that accurately detect note onsets and note timbre.”
Efforts at organizing and classifying the flute music led Ali-MacLachlan to conclude that the system is actually capable of much more. If the technology can handle Irish traditional flute music, he realized, it can handle just about anything.
Ali-MacLachlan and his fellow researchers have since inputted hundreds of pieces of music into the system to create 15,000 individual notes and sounds, each tagged to style of a particular musician or composer.
The musical AI system uses a neural network approach, a subset of artificial intelligence that is modeled on how the human brain processes information. The system essentially teaches itself to recognize specific stylistic techniques, learning as it works, and amassing a potentially near-infinite knowledge base. These advances in machine learning allow technically-inclined musicologists to parse and analyze musical styles in ways that were previously impossible.
Ali-MacLachlan said that the system is currently able to replicate notes with an 86 percent level of accuracy, and imitate nearly 75 percent of all individual note deviations — each directly linked to a musician’s specific style of play.
“So far we have worked with bidirectional long-term, short-term and convolutional neural networks,” Ali-MacLachlan said. “State-of-the-art in note onset detection typically uses machine learning rather than traditional signal analysis.”
As a practical matter, the musical analysis system could allow musicians and composers to noodle around with their own compositions, Ali-MacLachlan said.
“Digital audio workstations like Garage Band or Logic have put the contents of complex recording studios in the hands of anyone with a computer,” he said. “Imagine being able to play a guitar solo at whatever level you can play, then using the style of a famous guitarist to influence the audio. More producers work on their own these days and it would allow them to quickly try different styles of playing in their own tracks.”
Ali-MacLachlan said that, as a kind of side effect, the exacting computer analysis has also given him a new appreciation for virtuoso musicianship.
“I was surprised at how musicians really use the strength of different harmonics in different notes or even through one note,” he said. “Jimi Hendrix is an example of incredible technique — you could use the same guitar and amp settings, but after that it’s all in the fingers.”
WATCH: Is There a Formula Behind Good Music?