Essentia

Open-source library and tools for audio and music analysis, description and synthesis

Extensive collection of reusable algorithms

Flexible and easily extendable algorithms for common audio analysis processes and audio and music descriptors.

Cross-platform

Linux, Mac OS X, Windows, iOS, Android, and Web.

Fast prototyping

Python scientific environment, JavaScript bindings, and command-line audio analysis tools.

Industrial applications

Optimized for computational speed, including real-time use cases.

About

Essentia is an open-source C++ library for audio analysis and audio-based music information retrieval. It contains an extensive collection of algorithms, including audio input/output functionality, standard digital signal processing blocks, statistical characterization of data, a large variety of spectral, temporal, tonal, and high-level music descriptors, and tools for inference with deep learning models. Essentia is cross-platform and designed with a focus on optimization in terms of robustness, computational speed, and low memory usage, which makes it efficient for many industrial applications. The library includes Python and JavaScript bindings as well as various command-line tools and third-party extensions, which facilitate its use for fast prototyping and allow setting up research experiments very rapidly.

Similarity

Analyze audio and compute features to find similar sounds or music tracks.

Classification

Classify sounds or music based on computed audio features.

Deep learning inference

Use data-driven TensorFlow models for a wide range applications from music annotation to synthesis.

Mood detection

Find if a song is happy, sad, aggressive or relaxed.

Key detection

Find a key of a music piece.

Onset detection

Detect onsets (and transients) in an audio signal.

Segmentation

Split audio into homogeneous segments that sound alike.

Beat tracking

Estimate beat positions and tempo (BPM) of a song.

Melody extraction

Estimate pitch in monophonic and polyphonic audio.

Audio fingerprinting

Extract fingerprints from any audio source using the Chromaprint algorithm.

Cover song detection

Identify covers and different versions of the same music piece.

Spectral analysis

Analyze spectral shape of an audio signal.

Loudness metering

Use various loudness meters including algorithms compliant with the EBU R128 broadcasting standard.

Audio problems detection

Identify possible audio quality problems in music recordings.

Voice analysis

Voice activity detection and characterization.

Synthesis

Analyze, transform and synthesize sounds using spectral modeling approaches.

Companies and projects using Essentia

Collaborative database of Creative Commons Licensed sounds

Open platform for gathering music information from audio

Interactive iPhone tuner app for learning to play music instruments

Audio plugins for mixing, mastering, and recording

An iOS/Android app that helps you learn guitar

Cutting-edge interactive products for musical creation

Worldwide music identification and monitoring service using fingerprinting technology

Technologies for music education and infotainment

50 years of library music for movies, adverts, TV and radio

Tech solutions for digital music distribution, catalogue management and royalties collection

Key & BPM database and music finder

Award winning independent rhythm game using Essentia's beat detection

Automated music audio mastering with AI

Streaming media player platform

Global music consumption monitoring

AI-powered emotion-sensitive music metadata platform

Music education lessons

Live music streaming

Audio branding solutions

Production music and sound effects

Virtual instruments and effects for music creation

Music theory audio plugins and iOS app for music creation