MultiPitchMelodia

streaming mode | Pitch category

Inputs

  • signal (real) - the input audio signal

Outputs

  • pitch (vector_vector_real) - the estimated pitch values per frames [Hz]

Parameters

  • binResolution (real ∈ (0, ∞), default = 10) :
    salience function bin resolution [cents]
  • filterIterations (integer ∈ [1, ∞), default = 3) :
    number of interations for the octave errors / pitch outlier filtering process
  • frameSize (integer ∈ (0, ∞), default = 2048) :
    the frame size for computing pitch saliecnce
  • guessUnvoiced (bool ∈ {false, true}, default = false) :
    guess pitch using non-salient contours when no salient ones are present in a frame
  • harmonicWeight (real ∈ (0, 1), default = 0.8) :
    harmonic weighting parameter (weight decay ratio between two consequent harmonics, =1 for no decay)
  • hopSize (integer ∈ (0, ∞), default = 128) :
    the hop size with which the pitch salience function was computed
  • magnitudeCompression (real ∈ (0, 1], default = 1) :
    magnitude compression parameter (=0 for maximum compression, =1 for no compression)
  • magnitudeThreshold (integer ∈ [0, ∞), default = 40) :
    peak magnitude threshold (maximum allowed difference from the highest peak in dBs)
  • maxFrequency (real ∈ [0, ∞), default = 20000) :
    the maximum allowed frequency for salience function peaks (ignore peaks above) [Hz]
  • minDuration (integer ∈ (0, ∞), default = 100) :
    the minimum allowed contour duration [ms]
  • minFrequency (real ∈ [0, ∞), default = 80) :
    the minimum allowed frequency for salience function peaks (ignore peaks below) [Hz]
  • numberHarmonics (integer ∈ [1, ∞), default = 20) :
    number of considered hamonics
  • peakDistributionThreshold (real ∈ [0, 1], default = 0.9) :
    allowed deviation below the peak salience mean over all frames (fraction of the standard deviation)
  • peakFrameThreshold (real ∈ [0, 1], default = 0.9) :
    per-frame salience threshold factor (fraction of the highest peak salience in a frame)
  • pitchContinuity (real ∈ [0, ∞), default = 27.5625) :
    pitch continuity cue (maximum allowed pitch change durig 1 ms time period) [cents]
  • referenceFrequency (real ∈ (0, ∞), default = 55) :
    the reference frequency for Hertz to cent convertion [Hz], corresponding to the 0th cent bin
  • sampleRate (real ∈ (0, ∞), default = 44100) :
    the sampling rate of the audio signal [Hz]
  • timeContinuity (integer ∈ (0, ∞), default = 100) :
    tine continuity cue (the maximum allowed gap duration for a pitch contour) [ms]

Description

This algorithm estimates multiple fundamental frequency contours from an audio signal. It is a multi pitch version of the MELODIA algorithm described in [1]. While the algorithm is originally designed to extract melody in polyphonic music, this implementation is adapted for multiple sources. The approach is based on the creation and characterization of pitch contours, time continuous sequences of pitch candidates grouped using auditory streaming cues. To this end, PitchSalienceFunction, PitchSalienceFunctionPeaks, PitchContours, and PitchContoursMonoMelody algorithms are employed. It is strongly advised to use the default parameter values which are optimized according to [1] (where further details are provided) except for minFrequency, maxFrequency, and voicingTolerance, which will depend on your application.

The output is a vector of estimated melody pitch values and a vector of confidence values.

References:

[1] J. Salamon and E. Gómez, "Melody extraction from polyphonic music signals using pitch contour characteristics," IEEE Transactions on Audio, Speech, and Language Processing, vol. 20, no. 6, pp. 1759–1770, 2012.

[2] http://mtg.upf.edu/technologies/melodia

[3] http://www.justinsalamon.com/melody-extraction

See also

MultiPitchMelodia (standard) PitchContours (standard) PitchContours (streaming) PitchContoursMonoMelody (standard) PitchContoursMonoMelody (streaming) PitchSalience (standard) PitchSalience (streaming) PitchSalienceFunction (standard) PitchSalienceFunction (streaming) PitchSalienceFunctionPeaks (standard) PitchSalienceFunctionPeaks (streaming)

Streaming algorithms

AfterMaxToBeforeMaxEnergyRatio | AllPass | AudioLoader | AudioOnsetsMarker | AudioWriter | AutoCorrelation | BFCC | BPF | BandPass | BandReject | BarkBands | BarkExtractor | BeatTrackerDegara | BeatTrackerMultiFeature | Beatogram | BeatsLoudness | BinaryOperator | BinaryOperatorStream | BpmHistogram | BpmHistogramDescriptors | BpmRubato | CartesianToPolar | CentralMoments | Centroid | ChordsDescriptors | ChordsDetection | ChromaCrossSimilarity | Chromagram | Chromaprinter | ClickDetector | Clipper | ConstantQ | CoverSongSimilarity | Crest | CrossCorrelation | CubicSpline | DCRemoval | DCT | Danceability | Decrease | Derivative | DerivativeSFX | DiscontinuityDetector | Dissonance | DistributionShape | Duration | DynamicComplexity | ERBBands | EasyLoader | EffectiveDuration | Energy | EnergyBand | EnergyBandRatio | Entropy | Envelope | EqloudLoader | EqualLoudness | FFT | FFTC | FadeDetection | FalseStereoDetector | FileOutput | Flatness | FlatnessDB | FlatnessSFX | Flux | FrameCutter | FrameToReal | FrequencyBands | GFCC | GapsDetector | GeometricMean | HFC | HPCP | HarmonicBpm | HarmonicMask | HarmonicModelAnal | HarmonicPeaks | HighPass | HighResolutionFeatures | Histogram | HprModelAnal | HpsModelAnal | HumDetector | IDCT | IFFT | IFFTC | IIR | Inharmonicity | InstantPower | Key | KeyExtractor | LPC | Larm | Leq | LevelExtractor | LogAttackTime | LogSpectrum | LoopBpmConfidence | LoopBpmEstimator | Loudness | LoudnessEBUR128 | LoudnessEBUR128Filter | LoudnessVickers | LowLevelSpectralEqloudExtractor | LowLevelSpectralExtractor | LowPass | MFCC | Magnitude | MaxFilter | MaxMagFreq | MaxToTotal | Mean | Median | MedianFilter | MelBands | MetadataReader | Meter | MinMax | MinToTotal | MonoLoader | MonoMixer | MonoWriter | MovingAverage | MultiPitchMelodia | Multiplexer | NNLSChroma | NSGConstantQ | NSGConstantQStreaming | NSGIConstantQ | NoiseAdder | NoiseBurstDetector | NoveltyCurve | OddToEvenHarmonicEnergyRatio | OnsetDetection | OnsetDetectionGlobal | OnsetRate | Onsets | OverlapAdd | Panning | PeakDetection | PercivalBpmEstimator | PercivalEnhanceHarmonics | PercivalEvaluatePulseTrains | PitchCREPE | PitchContours | PitchContoursMelody | PitchContoursMonoMelody | PitchContoursMultiMelody | PitchFilter | PitchMelodia | PitchSalience | PitchSalienceFunction | PitchSalienceFunctionPeaks | PitchYin | PitchYinFFT | PitchYinProbabilistic | PitchYinProbabilities | PitchYinProbabilitiesHMM | PolarToCartesian | PoolAggregator | PoolToTensor | PowerMean | PowerSpectrum | PredominantPitchMelodia | RMS | RawMoments | RealAccumulator | ReplayGain | Resample | ResampleFFT | RhythmDescriptors | RhythmExtractor | RhythmExtractor2013 | RhythmTransform | RollOff | SBic | SNR | SaturationDetector | Scale | SilenceRate | SineModelAnal | SineModelSynth | SineSubtraction | SingleBeatLoudness | SingleGaussian | Slicer | SpectralCentroidTime | SpectralComplexity | SpectralContrast | SpectralPeaks | SpectralWhitening | Spectrum | SpectrumCQ | SpectrumToCent | Spline | SprModelAnal | SprModelSynth | SpsModelAnal | SpsModelSynth | StartStopCut | StartStopSilence | StereoDemuxer | StereoMuxer | StereoTrimmer | StochasticModelAnal | StochasticModelSynth | StrongDecay | StrongPeak | SuperFluxExtractor | SuperFluxNovelty | SuperFluxPeaks | TCToTotal | TempoCNN | TempoScaleBands | TempoTap | TempoTapDegara | TempoTapMaxAgreement | TempoTapTicks | TensorNormalize | TensorToPool | TensorToVectorReal | TensorTranspose | TensorflowInputFSDSINet | TensorflowInputMusiCNN | TensorflowInputTempoCNN | TensorflowInputVGGish | TensorflowPredict | TensorflowPredict2D | TensorflowPredictCREPE | TensorflowPredictEffnetDiscogs | TensorflowPredictFSDSINet | TensorflowPredictMAEST | TensorflowPredictMusiCNN | TensorflowPredictTempoCNN | TensorflowPredictVGGish | TonalExtractor | TriangularBands | TriangularBarkBands | Trimmer | Tristimulus | TruePeakDetector | TuningFrequency | TuningFrequencyExtractor | UnaryOperator | UnaryOperatorStream | Variance | VectorInput | VectorRealAccumulator | VectorRealToTensor | Vibrato | Viterbi | WarpedAutoCorrelation | Welch | Windowing | ZeroCrossingRate