If we take the music out of a movie, it loses half the fun.
It is an indispensable element in the cinema, since it is responsible for setting the atmosphere and defining the mood, thus manipulating the emotional reactions of the viewer.
Analyzing this music has been the objective of a research team from the USC Viterbi School of Engineering, who created a technology model based on artificial intelligence which can predict the genre of a movie based only on the soundtrack.
In this way they have learned how filmmakers can reach their audience in a more compelling way. They can reach conclusions that are beyond the logic of many. We all know that a lighthearted romance has rich string passages and lyrical melodies, while a horror movie can feature haunting, piercing frequencies and jarring notes, but more conclusions can be drawn from this work.
With AI they want to find quantitative evidence that elements of a film’s soundtrack could be used to characterize the film’s genre. So far they have been successful with the first tests, and now they want production companies and music supervisors to have a better understanding of how to create and put music in their material to provoke various emotions.
Predicting the genre of the soundtrack
To predict genre, they examined data from 110 popular movies released between 2014 and 2019. They used the Internet Movie Database (IMDb) genre classification: action, comedy, drama, horror, romance, or science fiction. They then applied a deep learning network that extracted auditory information such as timbre, harmony, melody, rhythm, and pitch from the music and score of each movie.
Once done, they managed to accurately classify the genre of each film based solely on these characteristics.
The models did not give details about what types of notes or instruments were associated with each genre, but they were able to establish that tonal and timbral characteristics were the most important in predicting the genre of the film.
Analyzing this audio fingerprint also allows you to see where the musical signals occur in a movie and for how long.
At the moment there is no commercial product that uses this AI, but it is all a matter of time.
The material has been published in journals.plos.org