Author Topic: Visualisation Plug-In, with Matrix Processing  (Read 3564 times)

jean-louis

  • Newbie
  • *
  • Posts: 19
    • View Profile
    • durrieu.ch
Visualisation Plug-In, with Matrix Processing
« on: July 11, 2014, 20:54:35 »
I have been developing an "IMMF0Salience" plug-in that provides an estimate of fundamental frequencies (F0) for audio signals, which can be found at https://github.com/wslihgt/IMMF0salience. It is still work in progress, but can already be used in some cases.

I wanted to mention in this forum that, recently, I found out the solution for an issue that was bugging me for all this time: the algorithm I developed is based on some matrix processing called "Non-negative Matrix Factorization" (NMF), meaning that it operates most meaningfully when I can process several frames in one go. The architecture of the framework for Vamp-Plugins is appropriate for frame by frame processing, and matrix-like processing is possible, but somewhat tricky. It took me a while, but I could at last make that work, and the solution is on this branch: https://github.com/wslihgt/IMMF0salience/tree/matrixProcessing.

I describe in more details how I do this, and also the process that led me to this solution on my "blog": http://durrieu.ch/wordpress/?p=269.

To put it in a nutshell: I did not really understand how the SampleType and the SampleRate attributes for the OutputDescriptor worked, at least not from the documentation. Digging in some other plug-in, I found the correct combination, and that actually helped me understand what the documentation was describing... Don't get me wrong: I do not say the documentation is bad or missing something, but rather that some more examples (or tutorials) would have helped me to figure that out easier.

I hope this little piece of information may help others to develop such plug-ins! Comments are welcome, of course.

Cheers,
Jean-Louis

cannam

  • Administrator
  • Sr. Member
  • *****
  • Posts: 273
    • View Profile
Re: Visualisation Plug-In, with Matrix Processing
« Reply #1 on: July 13, 2014, 14:26:56 »
Hi Jean-Louis -- that's a really good writeup on your blog there, thanks! I've added a link to it on the wiki page at https://code.soundsoftware.ac.uk/projects/vamp-plugin-sdk/wiki/SampleType. You're quite right that the documentation has always been a bit weak on this subject.

Chris