When we talk about Artificial Intelligence we are talking about an extremely high range of categories where it can be applied. It is difficult to imagine one in which there is no relationship.
Today we will see it with dance, a universal language that is found in almost all cultures, movement patterns that respond to music rhythms and where Artificial Intelligence can help a lot.
Dancing requires practice, professional training on many occasions, so it seems difficult to imagine a machine learning model dedicated to the subject.
The fact is that Google has achieved it with a program called AI Choreographer, a model who is able to imitate and understand dance movements, and can even enhance a person’s ability to do choreography.
Google has unveiled the model and released a set of multimodal 3D dance movement data on a large scale, with over 5 hours of 3D dance movement in 1408 sequences, covering 10 dance genres.
The data set, called AIST ++, also contains data from synchronized images from multiple views, with the aim of being used for various investigations, such as pose estimation in both 2D and 3D, for example.
On the Google blog They comment on the technical details of how they have achieved it:
The model begins by encoding the initial motion and audio inputs using separate motion and audio transformers. The inlays are concatenated and sent to a transmodal transformer, which learns the correspondence between both modalities and generates N future movement sequences. These sequences are then used to train the model in a self-supervised way.
The model is capable of generating frame-by-frame dance movement.
It is impressive how the model can understand the relationship between audio and movement, thus generating 3D motion sequences conditioned to music. Thanks to this technique, it will be possible to practice and create new steps, as well as create programs that can take advantage of this intelligence to meet various needs. The code is available and GitHub and the trained model in this link.