Applications

Essentia has served in a large number of research activities conducted at Music Technology Group since 2006. It has been used for music classification, semantic auto-tagging, music similarity and recommendation, visualization and interaction with music, sound indexing, musical instruments detection, cover detection, beat detection, and acoustic analysis of stimuli for neuroimaging studies.

A list of highlighted academic publications using Essentia can be found here.

Essentia and Gaia have been used extensively in a number of research projects, software and industrial applications. We highlight some of them below.

Both Essentia and Gaia libraries are employed for large-scale indexing and content-based search of sound recordings within Freesound.org, a popular repository of Creative Commons licensed audio samples. In particular, Freesound uses audio based similarity to recommend sounds similar to user queries. The Essentia extractor used for this functionality is in here. Also through the Freesound API the individual analysis descriptors of a sound can be accessed.
AcousticBrainz aims to crowdsource acoustic information for all music in the world and to make it available to the public, providing a massive database of information about music for music technology researchers and open-source hackers. The project relies on Essentia for extracting acoustic characteristics of music, including low-level spectral information, rhythm, keys, scales, and much more, and automatic annotation by genres, moods, and instrumentation.
Dunya is a web-based software application using Essentia that lets users interact with an audio music collection through the use of musical concepts that are derived from a specific musical culture, in this case, Carnatic music.
Good-sounds is a community-based online platform that allows musicians to automatically evaluate their playing exercises in terms of sound quality of instrumental sounds, their “sound goodness”. The evaluation is done by means of audio analysis technology Artistry developed by Korg and the Music Technology Group and is based on Essentia. The system measures how close the sounds performed by training musicians are to the ones that the community has identified as reference sounds.
Cortosia is an interactive tuner iPhone app using Essentia for automated evaluation of playing exercises by training musicians, developed by Korg and linked to the Good-Sounds platform.
Waves is the leading developer of audio plugins for mixing, mastering, post production and live sound.
BMAT started in 2005 when a group of engineers crazy about music decided to spin-off from University Pompeu Fabra and create a company that made music a better and fairer place. Nowadays, BMAT monitors more than 5,000 tvs, radios, clubs and digital services, identifies when and where any song is played and provides more than 90 CMOs worldwide with the metadata that describes who owns the rights over each track. Essentia is the main library used by BMAT for audio analysis.
Crypt of the NecroDancer is an award winning independent game. Players move on the beat of the music to navigate a dungeon and fight enemies. The game uses Essentia's beat detection capabilities to allow users to play the game with songs from their own music library.
Fretello is an app for iOS and Android that helps you learn guitar faster, more effectively, and permanently. It creates individual exercise regimens that are tailored to your specific needs, detects how you play, selects your optimal practicing tempo and adapts dynamically to your progress.
Reactable provides cutting-edge interactive products for musical creation.
MusicMuni Labs offers technologies for music education and infotainment.
Tele Music is a 50 years old music label in France with an extensive catalog of library music for movies, adverts, TV, and radio.
Sonosuite offers tech solutions for digital music distribution, catalogue management and royalties collection.
Tunebat is a key & BPM database prividing a way to browse and analyze millions of songs for DJs, producers, musicians, and music enthusiasts.
LANDR provides services for automated music audio mastering with AI.
Plex is a client-server streaming media player platform.
Utopia provides services for global music consumption monitoring.
Musimap provides AI-powered emotion-sensitive music metadata.
LearnToPlayMusic.com is a music education publishing company providing music lesson materials.
Live music streaming.
HearDis! provides solutions for audio branding.
Artlist is a platform for production music and sound effects.
Arturia makes virtual instruments and effects for music creation.
Scaler makes music theory audio plugins and iOS app for finding chords, melodies, basslines, and progressions.
Music Structure Analysis Framework is a framework for music audio segmentation using Essentia to extract the necessary features.
EssentiaRT~ is a real-time subset of Essentia implemented as an external for Pd and Max/MSP.
ofxAudioAnalyzer is an openFrameworks wrapper for Essentia. It provides audio analysis algorithms modified to process signals in real-time.
GIF Sync reassembles the frames of a GIF to sync its animation to the beat of an audio file.