Visualisation plug-in development

[jean-louis]

Hi everyone!

I develop a plug-in that provides a visualization of the F0 salience for an audio file.

I would like to know a few things, to make it more "user friendly": as I understand, the field 'binNames' of the output descriptor can be set such that it reflects the corresponding bin meaning (the F0 in my case). Would there be a way of calling some internal Sonic Visualiser (SV) things such as the piano display, on the left of the pane, to make this display a bit more meaningful?

By the way, in the visualisation, in SV, would there be a way of setting the min and max of the displayed values? for instance, let's say that for all the frames, all the bins, I have values between 0 and 10^8, and I would like to display a dynamic only between 0 and 100, would that be possible? From what I understand, that is not the way the little "turn button" does, am I wrong? For a better picture of what I m talking about, this would be the Matplotlib equivalent of "plt.clim([0, 100])".

At last, I was wondering how I should distribute it. I tried to compile my plug-in under Linux (Ubuntu 10.04) with the '-static' option, but I got a few errors:

Code:

g++ -I/usr/lib/ -Wall -fPIC   -c -o f0Salience.o f0Salience.cpp
f0Salience.h: In constructor ‘F0Salience::F0Salience(float)’:
f0Salience.h:94: warning: ‘F0Salience::m_maxIterations’ will be initialized after
f0Salience.h:84: warning:   ‘size_t F0Salience::m_numberFrames’
f0Salience.cpp:3: warning:   when initialized here
g++ -I/usr/lib/ -Wall -fPIC   -c -o plugins.o plugins.cpp
g++ -o f0salience.so f0Salience.o plugins.o -static -Wl,-Bstatic -Wl,-soname=f0salience.so /usr/lib//libvamp-sdk.a --version-script=vamp-plugin.map -llapack -lcblas -latlas -lfftw3 -lm
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 0 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 1 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 2 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 3 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 4 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 5 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 6 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 7 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 8 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 9 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 10 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 11 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 12 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 13 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 14 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 15 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 16 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 17 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 18 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 19 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 20 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 21 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 22 has invalid symbol index 22
/usr/lib/gcc/i486-linux-gnu/4.4.3/../../../../lib/crt1.o: In function `_start':
(.text+0x18): undefined reference to `main'
collect2: ld returned 1 exit status
make: *** [f0salience.so] Error 1

If anyone has an idea of what is going on there... Under MacOsX, it seems that the created dylib is statically linked with the necessary libraries.

Cheers! And thanks again to everyone working on that project! The plug-in interface is really simple and makes it (almost) easy to develop plug-ins! Congrats!

Jean-Louis

[cannam]

Hello!  I'm sorry to take so long to reply, it's been a busy few weeks.

I'm afraid that, at the moment there is no way to do either of the things you request, at least not when SV is the host.  A possibility for the second one (clipping the colour range) might be to have thresholds provided as parameters to the plugin, though I can understand that you might not be keen to clutter up the plugin's own interface and the user might not know in advance what values to use.  A general colour-range editing function would be a very handy addition to SV, in fact...

With regard to compiling for distribution, I think the problem is that you can't build a dynamic object completely statically.  What you can do is enable static linking for the libraries that are not part of the standard runtime.  Something like:

Code:

g++ -o f0salience.so f0Salience.o plugins.o -Wl,-Bstatic -Wl,-soname=f0salience.so /usr/lib//libvamp-sdk.a \
  --version-script=vamp-plugin.map -llapack -lcblas -latlas -lfftw3 -Wl,-Bdynamic -lm

which switches static linkage on at the start of the library list and off again before the end of the library list and the (implicit) standard C and C++ runtimes.  That means your plugin will still link dynamically against libstdc++, but that is probably OK in practice.


Chris

[jean-louis]

A general colour-range editing function would be a very handy addition to SV, in fact...

Oh, yes, that's actually how I meant it. Is there some kind of "wish-list" where we could write that down?

With regard to compiling for distribution, I think the problem is that you can't build a dynamic object completely statically.  What you can do is enable static linking for the libraries that are not part of the standard runtime.  Something like:

Code:

g++ -o f0salience.so f0Salience.o plugins.o -Wl,-Bstatic -Wl,-soname=f0salience.so /usr/lib//libvamp-sdk.a \
  --version-script=vamp-plugin.map -llapack -lcblas -latlas -lfftw3 -Wl,-Bdynamic -lm

which switches static linkage on at the start of the library list and off again before the end of the library list and the (implicit) standard C and C++ runtimes.  That means your plugin will still link dynamically against libstdc++, but that is probably OK in practice.

Considering your answer, I tried several "flavors" for the flags. The following line actually did work:

Code:

g++ -o f0salience.so f0Salience.o plugins.o -shared -Wl,-soname=f0salience.so /usr/lib/libvamp-sdk.a \
    --version-script=vamp-plugin.map -static -Wl,-Bstatic -lcblas -latlas -lfftw3 -lm

which ends up with a significantly bigger .so file (a good sign?) and ldd tells me:

Code:

ldd f0salience.so
statically linked

So I guess I got what I wanted!

Now another trickier problem: do you think it could be possible to compile a DLL for windows, knowing that I use ATLAS and FFTW within my program? I checked a bit Cygwin and MingW, without much success so far... I saw in the plug-in skeleton Makefile some instructions to use mingw, but that did not help much.

It makes me think: would it be possible to make some more tutorials, on that topic (cross-compiling) and others? I have read a lot of the example plug-ins in order to get my own plug-in to work, which is fine, but some more comments on certain fields would have been helpful: I had to check the nnls-chroma plug-in by M. Mauch in order to fill-in the output descriptor's binNames field (BTW, I'd like to thank Matthias for releasing the source code)! But that's probably a lot to ask: I am already more than happy with what we already have thanks to your efforts!

So, thanks again for your reply and your help!

Jean-Louis

[cannam]

Now another trickier problem: do you think it could be possible to compile a DLL for windows, knowing that I use ATLAS and FFTW within my program? I checked a bit Cygwin and MingW, without much success so far... I saw in the plug-in skeleton Makefile some instructions to use mingw, but that did not help much.

Cygwin is an environment for running Unixy programs on Windows -- it's not relevant for building Windows native binaries.

MinGW is probably what you want.  It provides the basic set of GNU compiler tools, with Windows as its native target.  You can either use it on Windows, or install on something like Linux and cross-compile -- look to the mingw32 packages in Ubuntu for example.  The results are the same either way, for a given version of MinGW.

The instructions in the skeleton Makefile are, I believe, the actual commands needed to build a simple plugin using the MinGW32 packages in Ubuntu.  Of course, you'll first need to build the Vamp SDK itself -- you can find a Makefile.mingw in the build/ directory.  The same goes for any other dependencies, generally you'll need to build them afresh for MinGW.


Chris