Topics

1

Plugin Development / Visualisation Plug-In, with Matrix Processing

« on: July 11, 2014, 20:54:35 »

I have been developing an "IMMF0Salience" plug-in that provides an estimate of fundamental frequencies (F0) for audio signals, which can be found at https://github.com/wslihgt/IMMF0salience. It is still work in progress, but can already be used in some cases.

I wanted to mention in this forum that, recently, I found out the solution for an issue that was bugging me for all this time: the algorithm I developed is based on some matrix processing called "Non-negative Matrix Factorization" (NMF), meaning that it operates most meaningfully when I can process several frames in one go. The architecture of the framework for Vamp-Plugins is appropriate for frame by frame processing, and matrix-like processing is possible, but somewhat tricky. It took me a while, but I could at last make that work, and the solution is on this branch: https://github.com/wslihgt/IMMF0salience/tree/matrixProcessing.

I describe in more details how I do this, and also the process that led me to this solution on my "blog": http://durrieu.ch/wordpress/?p=269.

To put it in a nutshell: I did not really understand how the SampleType and the SampleRate attributes for the OutputDescriptor worked, at least not from the documentation. Digging in some other plug-in, I found the correct combination, and that actually helped me understand what the documentation was describing... Don't get me wrong: I do not say the documentation is bad or missing something, but rather that some more examples (or tutorials) would have helped me to figure that out easier.

I hope this little piece of information may help others to develop such plug-ins! Comments are welcome, of course.

Cheers,
Jean-Louis

2

Plugin Development / About debugging a plug-in

« on: December 01, 2010, 22:55:28 »

Hi everyone (well, hi chris, actually!)

I have several questions about debugging a plug-in, once again... I feel a bit sorry to post so much lately... I hope that may be of some interest for some other people, still!

First I used the plug-in tester provided on the website: there was a strange error that popped by using it. It said that the parameters were not initialized with the default values. And indeed: in the constructor, I did not use the default values. This however is a bit cumbersome: each time one wants to modify the default values for a parameter, it has to be changed in two places. Is there a way of doing so automatically? What if the parameter is not initialized? how about with another value?

Then, I was wondering whether vamp-plugin-tester was performing some memory checks. That would be a great addition! Otherwise, is it possible to use valgrind for instance to do so? How would you recommend to use it?

Actually, what bothers me, for the latter matter, is that my plug-in needs some arrays, which probably should be freed at some stage... Thing is, I m not so sure about the whole work-flow of the plug-in (maybe the documentation could be a bit more precise about that...). When is the destructor called? the reset? For instance, when a user starts a session, then starts the plug-in as a layer, uses it and closes the layer. When he calls the plug-in as a new layer, is the previous instance called or is it a new instance?

So many questions... I feel sorry about that, since I might have missed the answers in the documentation, but please bear with me!

Best regards,

Jean-Louis

----------
added 2010/12/13:
Just added a second output feature to my plug-in. Launching Sonic Visualiser on it or even the plug-in tester crashes right at the beginning: I figured out that, at the start, SV tries to get some information on the outputDescriptor (from the getOutputDescriptors method), in which I was using attributes that were not initialized yet.

There is something somewhat related to that matter in the plug-in skeleton: handling m_blockSize differently whether it s 0. The comment says:

Code: [Select]

    // Just so as not to return "1". This is the bin count that
    // would result from a block size of 1024, which is a likely
    // default -- but the host should always set the block size
    // before querying the bin count for certain.
Well, I'd say that when starting SV, the block size is not set and the program still calls this method... That's probably not a big issue in the most general cases, in my case, since I am setting the labels for each bin of the output vector, it so happens that, with random values for my attributes (parameters), I end up writing a bit anywhere in memory... Just thought this should be made clear.

Cheers!

jean-louis

3

Plugin Development / Licensing a Vamp plug-in

« on: December 01, 2010, 10:00:55 »

Hi everyone,

I would like to ask: I used the skeleton from the plug-in SDK to make my plug-in. Can I release the source code and provide these source files (notably the modified plugins.cpp)? Is there some restrictions on the license I have to use in this case?

By the way, if I release the binaries (for free), are there restrictions? As I understand, the license on the SDK is quite loose w.r.t. that aspect. Of course, since I use other libraries (atlas, fftw), I ll check on those too... By the way, the fftw license is a GPL, which, from what I understand, makes it compulsory that I also release the source code, am I right?

Regards !

Jean-Louis

4

Host Forum: Sonic Visualiser / "Segmentation fault" on Ubuntu 10.04 ?

« on: November 30, 2010, 09:32:02 »

Hi everyone !

I have been developing a plug-in, lately, for sonic-visualiser (SV). I tested it with SV, under Ubuntu 10.04, using a command line like the following:

Code: [Select]

VAMP_PATH=. sonic-visualiser
When I try to load a file with the option in the menu (or equivalently with the button on the menubar), SV crashes with a "Segmentation fault" error message. I can however load a file by drag-n-dropping it over the window. I can then run my plug-in, which seems, on my few tests, to work without errors. When I try to open the plug-in dialog before the file is entirely loaded (I'm the impatient type of person), it again crashes with a segfault.

The weird thing is that, since it crashes even before I tell him to load the plug-in (and even when the plug-in is nowhere for SV to be found), I tried to launch it the "normal" way, from the ubuntu menu. It then seems to work OK.

If I run it from the terminal, though, I get the same behaviour. Here is the output I get:

Code: [Select]

$ sonic-visualiser
Registering interactive file finder
Loading qt_en_US... Failed
Failed to load Qt translation for locale
Loading sonic-visualiser_en_US... Done
OSCQueue::OSCQueue: Base OSC URL is osc.udp://...
View::View(0x9167a10)
WARNING: MainWindow::setupRecentTransformsMenu: Unknown transform "vamp:f0salience:f0salience:f0salience" in recent transforms list
WARNING: MainWindow::setupRecentTransformsMenu: Unknown transform "vamp:f0salience:f0salience:output" in recent transforms list
WARNING: MainWindow::setupRecentTransformsMenu: Unknown transform "vamp:vampy:vampy-zc2:vampy-counts" in recent transforms list
WARNING: MainWindow::setupRecentTransformsMenu: Unknown transform "vamp:vampy:vampy-zc2:vampy-crossings" in recent transforms list
WARNING: MainWindow::setupRecentTransformsMenu: Unknown transform "vamp:mvamp:marsyas_bextract_zero_crossings:zero_crossing_counts" in recent transforms list
WARNING: MainWindow::setupRecentTransformsMenu: Unknown transform "vamp:qm-vamp-plugins:qm-dwt:wcoeff" in recent transforms list
WARNING: MainWindow::setupRecentTransformsMenu: Unknown transform "vamp:qm-vamp-plugins:qm-constantq:constantq" in recent transforms list
WARNING: MainWindow::setupRecentTransformsMenu: Unknown transform "vamp:qm-vamp-plugins:qm-segmenter:segmentation" in recent transforms list
WARNING: MainWindow::setupRecentTransformsMenu: Unknown transform "vamp:vampy:vampy-sf3:vampy-sd" in recent transforms list
WARNING: MainWindow::setupRecentTransformsMenu: Unknown transform "vamp:vampy:vampy-sf3:vampy-sc" in recent transforms list
WARNING: MainWindow::setupRecentTransformsMenu: Unknown transform "vamp:vampy:vampy-mfcc:warped-fft" in recent transforms list
View::View(0x92ef150)
NOTE: Document::setModel: Layer 0x92f3408 ("Ruler") is already set to model 0 ("(null)")
NOTE: Document::setModel: Layer 0x9307c70 ("Waveform") is already set to model 0 ("(null)")
PluginRDFIndexer::indexConfiguredURLs
PluginRDFIndexer::indexConfiguredURLs: index url is http://www.vamp-plugins.org/rdf/plugins/index.txt
CachedFile::CachedFile: origin is "http://www.vamp-plugins.org/rdf/plugins/index.txt"
CachedFile::check: Valid last retrieval at Tue Nov 30 09:12:35 2010
PluginRDFIndexer::indexConfiguredURLs: url is http://www.vamp-plugins.org/rdf/plugins/vamp-example-plugins
CachedFile::CachedFile: origin is "http://www.vamp-plugins.org/rdf/plugins/vamp-example-plugins"
CachedFile::check: Valid last retrieval at Tue Nov 30 09:12:35 2010
PluginRDFIndexer::indexConfiguredURLs: url is http://www.vamp-plugins.org/rdf/plugins/ofa-vamp-plugin
CachedFile::CachedFile: origin is "http://www.vamp-plugins.org/rdf/plugins/ofa-vamp-plugin"
CachedFile::check: Valid last retrieval at Tue Nov 30 09:12:36 2010
PluginRDFIndexer::indexConfiguredURLs: url is http://www.vamp-plugins.org/rdf/plugins/vamp-libxtract
CachedFile::CachedFile: origin is "http://www.vamp-plugins.org/rdf/plugins/vamp-libxtract"
CachedFile::check: Valid last retrieval at Tue Nov 30 09:12:37 2010
PluginRDFIndexer::indexConfiguredURLs: url is http://www.vamp-plugins.org/rdf/plugins/qm-vamp-plugins
CachedFile::CachedFile: origin is "http://www.vamp-plugins.org/rdf/plugins/qm-vamp-plugins"
CachedFile::check: Valid last retrieval at Tue Nov 30 09:12:37 2010
PluginRDFIndexer::indexConfiguredURLs: url is http://www.vamp-plugins.org/rdf/plugins/mazurka-plugins
CachedFile::CachedFile: origin is "http://www.vamp-plugins.org/rdf/plugins/mazurka-plugins"
CachedFile::check: Valid last retrieval at Tue Nov 30 09:12:38 2010
PluginRDFIndexer::indexConfiguredURLs: url is http://www.vamp-plugins.org/rdf/plugins/vamp-aubio
CachedFile::CachedFile: origin is "http://www.vamp-plugins.org/rdf/plugins/vamp-aubio"
CachedFile::check: Valid last retrieval at Tue Nov 30 09:12:39 2010
PluginRDFIndexer::indexConfiguredURLs: url is http://www.vamp-plugins.org/rdf/plugins/match-vamp-plugin
CachedFile::CachedFile: origin is "http://www.vamp-plugins.org/rdf/plugins/match-vamp-plugin"
CachedFile::check: Valid last retrieval at Tue Nov 30 09:12:39 2010
PluginRDFIndexer::indexConfiguredURLs: url is http://www.vamp-plugins.org/rdf/plugins/vamp-onsetsds
CachedFile::CachedFile: origin is "http://www.vamp-plugins.org/rdf/plugins/vamp-onsetsds"
CachedFile::check: Valid last retrieval at Tue Nov 30 09:12:39 2010
PluginRDFIndexer::indexConfiguredURLs: url is http://www.vamp-plugins.org/rdf/plugins/mvamp
CachedFile::CachedFile: origin is "http://www.vamp-plugins.org/rdf/plugins/mvamp"
CachedFile::check: Valid last retrieval at Tue Nov 30 09:12:40 2010
PluginRDFIndexer::indexConfiguredURLs: url is http://www.vamp-plugins.org/rdf/plugins/nnls-chroma
CachedFile::CachedFile: origin is "http://www.vamp-plugins.org/rdf/plugins/nnls-chroma"
CachedFile::check: Valid last retrieval at Tue Nov 30 09:12:40 2010
Finished setting up OSC interface
Comparing current version "1.7.2" with latest version "1.7.1"
Segmentation fault
The last line only appears when I try to open the "open file" dialog. On this log, the plug-ins were not found, because I removed them from the ~/vamp folder (I guess).

Well, I m not sure what exactly goes wrong there... Could the plug-ins I used (especially the one I develop) have "broken" my current installation of SV? Or is this something related to a build problem within SV?

Best regards,

Jean-Louis

added 1/12/2010: I ran the plugin-tester with the following command line:

Code: [Select]

$ VAMP_PATH=. ~/PATH/TO/vamp-plugin-tester -a
and obtained:

Code: [Select]

... <a lot of debugging "printf" stuff > ...
vamp-plugin-tester: All tests succeeded for this plugin

vamp-plugin-tester: All tests succeeded
Segmentation fault
What should I think of that?...

5

Plugin Development / Visualisation plug-in development

« on: November 05, 2010, 15:40:38 »

Hi everyone!

I develop a plug-in that provides a visualization of the F0 salience for an audio file.

I would like to know a few things, to make it more "user friendly": as I understand, the field 'binNames' of the output descriptor can be set such that it reflects the corresponding bin meaning (the F0 in my case). Would there be a way of calling some internal Sonic Visualiser (SV) things such as the piano display, on the left of the pane, to make this display a bit more meaningful?

By the way, in the visualisation, in SV, would there be a way of setting the min and max of the displayed values? for instance, let's say that for all the frames, all the bins, I have values between 0 and 10^8, and I would like to display a dynamic only between 0 and 100, would that be possible? From what I understand, that is not the way the little "turn button" does, am I wrong? For a better picture of what I m talking about, this would be the Matplotlib equivalent of "plt.clim([0, 100])".

At last, I was wondering how I should distribute it. I tried to compile my plug-in under Linux (Ubuntu 10.04) with the '-static' option, but I got a few errors:

Code: [Select]

g++ -I/usr/lib/ -Wall -fPIC   -c -o f0Salience.o f0Salience.cpp
f0Salience.h: In constructor ‘F0Salience::F0Salience(float)’:
f0Salience.h:94: warning: ‘F0Salience::m_maxIterations’ will be initialized after
f0Salience.h:84: warning:   ‘size_t F0Salience::m_numberFrames’
f0Salience.cpp:3: warning:   when initialized here
g++ -I/usr/lib/ -Wall -fPIC   -c -o plugins.o plugins.cpp
g++ -o f0salience.so f0Salience.o plugins.o -static -Wl,-Bstatic -Wl,-soname=f0salience.so /usr/lib//libvamp-sdk.a --version-script=vamp-plugin.map -llapack -lcblas -latlas -lfftw3 -lm
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 0 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 1 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 2 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 3 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 4 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 5 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 6 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 7 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 8 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 9 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 10 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 11 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 12 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 13 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 14 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 15 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 16 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 17 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 18 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 19 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 20 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 21 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 22 has invalid symbol index 22
/usr/lib/gcc/i486-linux-gnu/4.4.3/../../../../lib/crt1.o: In function `_start':
(.text+0x18): undefined reference to `main'
collect2: ld returned 1 exit status
make: *** [f0salience.so] Error 1

If anyone has an idea of what is going on there... Under MacOsX, it seems that the created dylib is statically linked with the necessary libraries.

Cheers! And thanks again to everyone working on that project! The plug-in interface is really simple and makes it (almost) easy to develop plug-ins! Congrats!

Jean-Louis

6

Host Forum: Sonic Visualiser / How are the spectrogram time stamps calculated?

« on: July 28, 2010, 07:06:41 »

Hi everyone,

I was wondering something, for some time now. In order to annotate some audio data, I display a layer of annotations ("a value at a time") on top of a spectrogram (computed using the built-in layer). The annotations I use have time stamps that correspond to the center of the analysis window. I noticed that sometimes there seems to be an offset between the stream of values (fundamental frequencies) and the displayed spectrogram. I however failed to find out how the time stamps were computed: do they correspond to the beginning of each frame, or to the center of the frames?

Cheers!

Jean-Louis

7

Host Forum: Sonic Visualiser / Reading SVL files with Python/Numpy

« on: July 09, 2010, 10:18:35 »

Hi everyone,

Probably many people might need to use the annotation files in a "batch" way, using any program like Matlab or Python/Numpy to access and process the data.

To do so, I first thought the exported CSV files would do, but I found it difficult to load the generated CSV back into Sonic Visualiser: for instance, exporting a region layer into a CSV provides a format which looks like (one line per region):
<start time (s)>,<value>,<duration (s)>,<label>
I tried several things when importing such a format into SV, but nothing really worked...

Instead, the SVL format is easy to reload, and actually quite "human readable". Unfortunately, I have always found that XML format files were hard to load with Matlab or Python/Numpy... Well, I did a little program in Python/Numpy that does extract some pieces of information. I thought I might not be the only one needing such a tool, so here is the code:

Code: [Select]

# readSVLfiles.py

from xml.dom import minidom
import numpy as np

def extractSvlAnnotRegionFile(filename):
    """
    extractSvlAnnotRegionFile(filename)
   
    Extracts the SVL files (sonic visualiser)
    in this function, we assume annotation files
    for regions (generated by Sonic Visualiser 1.7.2,
    Regions Layer)
   
    Returns the following objects:
        parameters: copy-paste of the "header" of the SVL file,
        frames    : a numpy array containing the time
                    stamps of each frame, at the beginning
                    of the frame,
        durations : a numpy array containing the duration
                    of each frame,
        labels    : a dictionary with keys equal to the frame
                    number, containing the labels,
        values    : a dictionary with keys equal to the frame
                    number, containing the values.
   
    Note that this code does not parse the end of the xml file.
    The 'display' field is therefore discarded here.
   
    Numpy and xml.dom should be imported in order to use this
    function.
       
    Jean-Louis Durrieu, 2010
    firstname DOT lastname AT epfl DOT ch
    """
    ## Load the XML structure:
    dom = minidom.parse(filename)
   
    ## Keep only the data-tagged field:
    ##    note that you could also keep any other
    ##    field here.
    dataXML = dom.getElementsByTagName('data')[0]
   
    ## XML structure for the parameters:
    parametersXML = dataXML.getElementsByTagName('model')[0]
   
    ## XML structure for the dataset field, containing the points:
    datasetXML = dataXML.getElementsByTagName('dataset')[0]
    ## XML structure with all the points from datasetXML:
    pointsXML = datasetXML.getElementsByTagName('point')
   
    ## converting to a somewhat easier to manipulate format:
    ## Dictionary for the parameters:
    parameters = {}
    for key in parametersXML.attributes.keys():
        parameters[key] = parametersXML.getAttribute(key)
   
    ## number of points (or regions):
    nbPoints = len(pointsXML)
   
    ## number of attributes per point, not used here,
    ## but could be useful to check what type of SVL file it is?
    # nbAttributes = len(pointsXML[0].attributes.keys())
   
    ## Initialize the numpy arrays (frame time stamps and durations):
    frames = np.zeros([nbPoints], dtype=np.float)
    durations = np.zeros([nbPoints], dtype=np.float)
    ## Initialize the dictionaries (values and labels)
    values = {}
    labels = {}
   
    ## Iteration over the points:
    for node in range(nbPoints):
        ## converting sample to seconds for the time stamps and the durations:
        frames[node] = np.int(pointsXML[node].getAttribute('frame')) / np.double(parameters['sampleRate'])
        durations[node] = np.int(pointsXML[node].getAttribute('duration')) / np.double(parameters['sampleRate'])
        ## copy-paste for the values and the labels:
        values[node] = pointsXML[node].getAttribute('value')
        labels[node] = pointsXML[node].getAttribute('label')
       
    ## return the result:
    return parameters, frames, durations, labels, values


This is probably not much, but it can save time to some people... It could be a good idea to publish some kind of toolbox that allows to read these annotation files with third party software, in a more "user friendly" format than pure XML to load ?

Of course, any comment on that code is welcome, especially if it has to be corrected! Please let me know before I really start the big batch processing  Note also that the use of Numpy to read the XML file is a little bit superfluous. It might be possible to avoid it, if necessary: just spot the lines with "np." in them, and replace them with whatever suits your need.

Since there already exist some people involved in Python programming, I was wondering if such a tool was actually not already available somewhere... In any case, I would be interested to know whether this is useful to others!

Cheers,

Jean-Louis