- Generated by
1.8.11
|
Gaia
|
This page presents a short tutorial to start using Gaia and do some useful stuff.
It is written using the python bindings, but it would be very similar using the C++/Qt or the C++/STL interface, only with some pointers instead of references at some points. The reader paying enough attention should be able to do the required changes without too many difficulties...
This tutorial, to be useful for people, will be based on use-cases and how to solve them. The plan is the following:
all the examples will imply the following:
from gaia2 import *
All the classes you will see that are not from python will then be from the module gaia2 (ie: DataSet, Point, etc... are actually gaia2.DataSet, gaia2.Point, etc...)
There is a verbose flag that you can set which makes transformations and other stuff display information while they do some computing. It is the gaia2.cvar.verbose variable. Use it like this:
# display info: cvar.verbose = True # be silent: cvar.verbose = False
NB: in C++, it is the global variable gaia2::verbose
Merging points in a dataset is actually quite a simple operation, you just need to add the Points one by one in a newly created dataset. Don't forget to set the name for the points! So that gives, for instance:
ds = DataSet()
p1 = Point()
p1.load('01.Elektrik Psylocibe.mp3.sig')
p1.setName('Elektrik Psylocibe')
ds.addPoint(p1)
p2 = Point()
p2.load('02.Funky Monkey.mp3.sig')
p2.setName('Funky Monkey')
ds.addPoint(p2)
p3 = Point()
# you get it by now...
As this is a pretty common operation, there is a function already defined that will do this for you given a file containing all the files to merge:
ds = DataSet.mergeFiles(yaml.load(open('filelist.yaml').read())
The format of 'filelist.yaml' is a simple yaml mapping from the points names to their respective paths, for instance:
"Electrik Psylocibe": "01.Electrik Psylocibe.mp3.sig" "Funky Monkey": "02.Funky Monkey.mp3.sig" ...
NB: double-quotes are only necessary if you have spaces in your ids/filenames
As DataSet.mergeFiles expects a python map, you first have to load the filelist.yaml file yourself, then have yaml parse this as a dictionary, and only then can you give it to the DataSet.mergeFiles function.
You can access a point directly by their ID:
p = ds.point('Elektrik Psylocibe')
To access attributes, you should use either the value() method or the label() method depending on the type of the descriptor you want to fetch or, more pythonically, just use points as if they were dictionaries:
# print some useful stuff
print 'BPM is:', p.value('tempotap_bpm')
print 'Variance of third MFCC coeff is:', p.value('mfcc.var')[2]
print 'Key/mode is: ', p.label('key_key'), p.label('key_mode')
# more pythonic way of doing it:
print 'BPM is:', p['tempotap_bpm']
print 'Key/mode is: ', p['key_key'], p['key_mode']
Loading and saving datasets is just as easy as saying it:
ds.save('mytrancemusic.db')
tranceDB = DataSet()
tranceDB.load('mytrancemusic.db')
# from there it's just as before:
print 'How fast does a monkey dance:', tranceDB.point('Funky Monkey').value('tempotap_bpm')
To know more about the details of transformations, the how/why/etc..., please have a look first at the main or the Gaia 2.0 API Overview
A reference of all transformations can be found at the Algorithms page.
The cleaner transformation has a bit of a special status, because it is required to be applied first on any dataset. Failure to do so will result in undefined behaviour, and also lack of support from the author. So forget it at your own risk!
To apply a transformation, you first need to instantiate the Analyzer that you require, and analyze the dataset. This will return a Transformation instance:
cleaner = AnalyzerFactory.create('cleaner')
cleaner_transfo = cleaner.analyze(tranceDB)
Now, you could apply this transformation to any dataset, but here it just makes more sense to apply it on the dataset you just analyzed:
cleanTranceDB = cleaner_transfo.applyToDataSet(tranceDB)
You can even apply both steps at once after having instantiated your Analyzer:
cleaner = AnalyzerFactory.create('cleaner')
cleanTranceDB = cleaner.analyze(tranceDB).applyToDataSet(tranceDB)
Actually, as most of the time you will want to apply the transformation to the DataSet you just analyzed, there is a shorter way to do this:
cleanTranceDB = transform(tranceDB, 'cleaner')
That's it, you now have a clean dataset ready to be further analyzed!
Next, you probably want to apply all sorts of transformations to make your dataset play nicer with similarity queries. As the way to do this is exactly the same as for the Cleaner transformation, we will just show some transformations here:
# remove MFCC, because we don't want them! (incidentally, mfcc also contains (as of essentia 0.4)
# covariance and inverse covariance matrix, which do NOT like to be normalized)
no_mfccDB = transform(cleanTranceDB, 'remove', { 'descriptorNames': '*mfcc*' })
# Normalize everything
normalizedDB = transform(no_mfccDB, 'normalize')
# PCA-30 on only the mean and variance of Real descriptors
pcaDB = transform(normalizedDB, 'pca', { 'dimension': 30, 'descriptorNames': [ '*.mean', '*.var' ] })
Here you go! So it wasn't that hard, was it?
NB: when creating an analyzer through the AnalyzerFactory, you can specify some parameters by using a specific type which is a gaia2.ParameterMap. In Python (only), you can specify a python map instead of a ParameterMap, and it will be automatically converted to a ParameterMap. This is what happened in the previous example.
As noted in the Gaia 2.0 API Overview, before creating a View that will allow you to do queries on a dataset, you will first need to decide which distance function you want to use. To instantiate it, you will need to go through the DistanceFunctionFactory (also called the MetricFactory), and call the create method with the layout you intend to use the distance function on (from the point or the dataset).
A reference of all distance functions can be found at the Metrics (Distance Functions) page.
For instance, let's get 3 of these:
euclideanMetric = DistanceFunctionFactory.create('euclidean', pcaDB.layout())
manhattanMetric = DistanceFunctionFactory.create('manhattan', pcaDB.layout())
# define a composite distance which is an euclidean distance on only the mean and var descriptors,
# plus 2 times the manhattan distance on the min and max descriptors:
euclidParams = {
'name': 'euclidean',
'params': { 'descriptorNames': [ '*.mean', '*.var' ] },
'weight': 1.0
}
manhattanParams = {
'name': 'manhattan',
'params': { 'descriptorNames': [ '*.min', '*.max' ] },
'weight': 2.0
}
mixedParams = { 'euclidp': euclidParams, 'manp': manhattanParams }
mixedMetric = DistanceFunctionFactory.create('linearcombination', pcaDB.layout(), mixedParams)
Now that we know which distance functions we want to use, we can create an assorted View which is the structure that will allow us to do queries on a dataset:
euclideanView = View(pcaDB, euclideanMetric)
mixedDistanceView = View(cleanTranceDB, mixedMetric) # do queries on the original dataset, not the PCA one
similarSongs1 = euclideanView.nnSearch('Funky Monkey').get(10)
similarSongs2 = mixedDistanceView.nnSearch('Elektrik Psylocibe').get(5)
This is very similar to query by ID, except that you pass a point to the function instead of its ID (because it's not in the dataset yet, for instance):
queryPoint = Point()
queryPoint.load('recentlyAnalyzedByEssentia.sig')
similarSongs3 = euclideanView.nnSearch(queryPoint).get(10)
You can filter queries (either by ID or by example) using a filter term which is very much like an SQL where clause. The reference can be found at the Filtering queries page.
filter = 'WHERE value.tempotap_bpm > 120 AND label.key_key = "C#" AND label.key_mode = "minor"' similarSongs4 = mixedDistanceView.nnSearch(queryPoint, filter).get(5)
1.8.11