Handling the output from Magnitude

[Samuel Levy]

Hi all,

I've been playing around with the FFT and Magnitude functions for an upcoming closed-loop ephys project using the VoicePaint example, and I'm hoping someone might be able to help me understand the output of the magnitude function. When using the MatVisualizer for its output, I get a typical-looking power spectrum with peaks where I expect given test sine waves I fed in and x-axis scaling that adapts as I expect to the different sample rates I tested. However, I can't figure out how to get that same array I'm seeing in the visualization either for saving or as an output I could do further transformations on (other than with pre-defined functions like Extrema). Sending the results of extrema to CsvWriter creates a file with all the columns listed in the context menu under OpenCV.Net.Mat, and the data contained in OpenCV.Net.Mat doesn't appear to me to directly translate from what I see in the Magnitude visualization when I save that to a binary file and load it into MATLAB. Hopefully there's something obvious I'm missing?

Thanks,

Sam

[Samuel Levy]

Amendment, the results I'm having trouble saving (out of the OpenCV.Net.Mat object) are from Magnitude, not Extrema

[Gonçalo Lopes]

Hi Sam and welcome to the forums,

Can you provide a simple workflow reproducing the problem? You wouldn't use CsvWriter to save the results from a binary FFT, but rather MatrixWriter to store the values coming off the Magnitude node directly in binary format. If you are doing this right after the fourier transform, bear in mind that half of your data is "garbage" (i.e. they are the residuals of the FFT, required to reconstruct the inverse transform, but not necessarily useful for power spectrum analysis).

Hope this helps.

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/03c6e7e9-834d-42ff-879a-3ba673de8d13%40googlegroups.com.

[Samuel Levy]

Hi Goncalo, 

Thanks for replying. The workflow I'm using is pretty much a direct replication of the Power Spectrum workflow from the VoicePaint example. In this workflow I'm saving the data with matrix writer after the Magnitude node, and that result does seem to be a length related to duration of the program run time * sample rate, but I'm not sure how to take that binary output and make it look like what I see in the visualizer for Magnitude.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai...@googlegroups.com.

[Gonçalo Lopes]

Hi Sam,

This post has an example of how to use the fread function from MATLAB to open a raw binary file: https://groups.google.com/d/msg/bonsai-users/rzXpXU1Z3ZM/k2FjFrWTLicJ

Make sure to use the right dimensions for your matrix and it should work.

Hope this helps.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/2142f136-66b1-4951-8c7b-5c978f629028%40googlegroups.com.

[Samuel Levy]

Hi Goncalo,

Thanks, now we're cooking, I am able to find my expected outputs among the resulting binary data. One more question on this topic, I notice the scaling of these results is not maintained from what's plotted in magnitide. In the attached image, the oscillating signal is the power I expect from one of the test sine waves, and the other 2 lines are random other frequencies, which, as can be seen, while unchanging are showing up on the a similar scaling as my test frequency. Is there some way to recover the relative scaling of these different channels, maybe from one of the other outputs in the OpenCV.Net .Mat object from Magnitude?

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/2142f136-66b1-4951-8c7b-5c978f629028%40googlegroups.com.

[Gonçalo Lopes]

Hi Samuel,

The output from the discrete fourier transform operator is the set of complex numbers describing the signal linearly decomposed in its fourier basis functions, which can be described in polar coordinates as magnitude and phase. Bonsai includes two operators to extract them, respectively Magnitude and Phase. In general for power spectrum analysis the magnitude component should be enough, but depending on how big your window for the DFT is, the relative scaling of frequencies may be affected specifically by aliasing and harmonics.

I am attaching a simple example of the DFT operator applied on the output of two mixed function generators. In this case I am using Submatrix > Magnitude to extract the power spectrum. If you try tweaking the frequencies of the independent generators to produce different mixings you can get a feel for how the scaling of the plot changes.

To better understand what is happening in your case it would be helpful to look at the raw signal. Can you post a small dataset with raw data that we could use to look into this, together with expected values for the frequency responses of the system? Also, can you share the workflow you are using to decompose it to see if that is also a possible source of problems?

Hope this helps.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/6b703b9f-6a12-4b03-967f-4f8aca111ea5%40googlegroups.com.

[Samuel Levy]

Hi Goncalo,

I think I've got a handle on the DFT function itself, and now it's just saving the results of that power spectrum as returned by Magnitude that is the trouble. Attached is the workflow I'm using, modeled on the VoicePaint example, where most of the additions are there to scale the output of magnitude so that each bin is 1hz. I've also attached a test wav file which produces two pure sines, 440hz and 660hz, alternating every 5 seconds. What I see in magnitude from this function is peaks at those x-values for the expected amount of time. If I save that output with matrix writer and open it in Matlab using code modeled from that example response you linked to above, the values don't match the y-axis scaling in the visualizer for magnitude, and there are also non-zero values in channels other than the 440 and 660 bins. The goal here is to be able to recreate the power spectrum as seen with magnitude for post-experiment analysis. 

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/6b703b9f-6a12-4b03-967f-4f8aca111ea5%40googlegroups.com.

[Samuel Levy]

To clarify what I'm seeing a little further, this is a screenshot of the visualizer for Magnitude during the first 5 seconds of sines2.wav (attached above), showing a clear peak only at 440hz, and after that is a screenshot from the data that I loaded from the binary file into matlab. Every 8th bin has non-zero entries, but not exactly the same value. Something similar happens during the 660hz epochs, except the non-zero bins increase to every 4th bin. Hopefully these detail can help lead to possible solutions. 

[Gonçalo Lopes]

Hi Samuel,

Thanks for your examples and data, it was really clear now. I'm convinced this must be an issue with the way the binary file is being interpreted by matlab. I'm attaching a new version of your workflow where the only modification was adding a MatrixWriter at the end to write all the raw data to a file.

I don't have access to matlab, but I wrote the following python equivalent to plot the entire data, both continuously in time, and ordered by trials, and the results look quite sensible given your description (see below). I have two possibilities:

  1) can you check whether you are reading the file into matlab as 64-bit floats, rather than 32-bit floats? I know matlab has a preference for double matrices, so it tends to do that unless you tell it otherwise;

  2) another possibility is the matrix representation is transposed from what matlab likes (i.e. rows are columns, etc).

The data should be a 1D array of 32-bit floats, so in this case I wouldn't expect the memory layout to be ambiguous, but sometimes the plotting functions require specific orientations to represent the data properly.

Hope this helps!

Script:

import numpy as np
import matplotlib.pyplot as plt

# import raw data stored as 32-bit floats
data = np.fromfile('power.bin',dtype=np.float32)

# plot raw data
plt.figure()
plt.plot(data)
plt.xlabel('time (samples)')

# reshape data as 1D of 1024 columns and plot each row buffer w/ offset
plt.figure()
power_rows = data.reshape((-1,1024))
[plt.plot(row + i * 5e7) for i,row in enumerate(power_rows)]
plt.xlabel('samples')

Figures:

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/ee65181f-fa16-4352-bb59-45fc895751ff%40googlegroups.com.

[Samuel Levy]

Hi Goncalo,

That was it, I was reading them in as 32-bit integers, and as soon as I switched to floats I get the data I expect and can replicate the trial-type plot you made. For my future reference, is there a way for my to find out what the data type of the output (either from magnitude or matrix writer) or going to be?

Thanks for your help, and to echo others, this software environment is incredible. Looking forward to getting back to the lab and run new studies.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/ee65181f-fa16-4352-bb59-45fc895751ff%40googlegroups.com.

[Gonçalo Lopes]

Hi Samuel, glad to hear you got it to work!

Yes, you can inspect the data format of the matrix by right-clicking on any node with Mat data type and choosing Show Visualizer > ObjectTextVisualizer. That will pull up a text representation of the data rather than graphical, which will often contain this type of information.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/3439133b-96c7-473a-a0ae-edc5d44d0034%40googlegroups.com.