Cannot open generated hdf5 files in MATLAB

[Philémon Roussel]

Dear all,

First of all thank you for your very nice work!

I have recently installed SpyKING Circus on Windows using Anaconda under Python 3.5 and I have MATLAB R2017b.
Trying to launch the MATLAB GUI both from the Spyking circus launcher and the Anaconda prompt, I get this error concerning data.templates.hdf5:

Error using h5infoc

The filename specified was either not found on the MATLAB path or it contains unsupported characters.

Error in h5info (line 101)

hinfo = h5infoc(filename,location, useUtf8);

I generate the hdf5 files (data.basis.hdf5, data.templates.hdf5, data.result.hdf5...) using:

 spyking-circus C:/ProgSpyKINGCircus/data.dat --method filtering,whitening,clustering,fitting --cpu 4

I am confused because:
- I cannot open in MATLAB any of the generated hdf5 files, neither on my computer with MATLAB R2017b, nor on another computer using MATLAB R2016a;
- I can open the generated hdf5 files using HDF View 3.0;
- I can open other sample hdf5 files taken from the internet in MATLAB R2017b.

I guess I must be misusing the software. Do you have any idea about what may cause this error?

Best regards,

Philémon

[Pierre Yger]

Dear Philemon

Thanks for using the software. You should have a look to the documentation http://spyking-circus.rtfd.org as I think you are not properly using the GUI command. You should simply do

>> circus-gui-matlab mydata.dat

(Or similar with python if you want to use the phy GUI). In theory, it should work!

Best

Pierre

[Philémon Roussel]

Dear Pierre,

Thank you very much for your answer. Unfortunately, it did not solve the issue. I'm going to describe my situation in detail.

Firstly, the installation:

- I created an Python environment based on the .yml file available on the phy github project page (https://github.com/kwikteam/phy)

> conda env create -n phy

- I activated the environment

> activate phy

- I installed phy and phycontrib

> pip install git+https://github.com/kwikteam/phy git+https://github.com/kwikteam/phy-contrib --upgrade

- I installed spyKING circus

> conda install -c conda-forge -c spyking-circus spyking-circus

Secondly, the processing:

- I created the parameter file attached
- I used the following command in Anaconda prompt

> spyking-circus C:/Users/philemon/Documents/Prog/ProgSpyKINGCircus/data.dat --method filtering,whitening,clustering,fitting,converting --cpu 8

- the process ended without errors

Thirdly, the attempts to use both MATLAB and python GUIs

1) MATLAB

- I used the following command in Anaconda prompt

> circus-gui-matlab C:/Users/philemon/Documents/Prog/ProgSpyKINGCircus/data.dat

- I got the following error in MATLAB command window

Error using h5infoc
The filename specified was either not found on the MATLAB path or it contains unsupported characters.

Error in h5info (line 101)
hinfo = h5infoc(filename,location, useUtf8);

Error in SortingGUI>SortingGUI_OpeningFcn (line 134)
    info     = h5info(tmpfile);

Error in gui_mainfcn (line 220)
    feval(gui_State.gui_OpeningFcn, gui_hFigure, [], guidata(gui_hFigure), varargin{:});

Error in SortingGUI (line 47)
    gui_mainfcn(gui_State, varargin{:});

2) python GUI

- I used the following command

> circus-gui-python C:/Users/philemon/Documents/Prog/ProgSpyKINGCircus/data.dat

- I got no error in the prompt but nothing happens



I know the problems concerning the 2 GUIs are probably not related but I took this opportunity to explain both.

Best

Philémon

[Philémon Roussel]

Sorry I forgot to attach my new parameter file in the precedent message.

[Pierre Yger]

Ok, well, everything seems to be correctly installed, I guess the problem is more related to a bad handling of paths in the GUIs. 

Regarding the MATLAB GUI, maybe a first start to debug would be for you to open file (if circus is installed in path CIRCUSPATH), CIRCUSPATH/circus/matlab_GUI/SortingGUI.m, and remove the ";" on L133 such that it is 

tmpfile  = strrep(tmpfile, '.mat', '.hdf5')

This should print on the screen more precisely what is the full path of the file the GUI is trying to open, and maybe it will help us to understand why this is not working. Tell me what is the file name displayed before the crash, and if the file exists

Note that to know CIRCUSPATH, you can simply do

>>python

>>import circus

>>print(circus.__file__)

Regarding the phy GUI, this is weird. Maybe you need to install some more packages for phy (tqdm, vispy, traits) ? To check why phy is not showing anything, you should try to create a params.py file as defined here http://phy-contrib.readthedocs.io/en/latest/template-gui/ (basically, a text file named params.py with the following values:

dat_path - location of raw data file

dir_path  - location of the phy results you obtained after converting (DATAPATH/DATAPATH.GUI)

n_channels_dat - total number of rows in the data file (not just those that have your neural data on them. This is for loading the file)

dtype - data type to read, e.g. 'int16'

offset - number of bytes at the beginning of the file to skip

sample_rate - in Hz

hp_filtered - True

Then you must launch phy with the following command:

>> phy template-gui params.py --debug

This should give you details about why phy is crashing... If this is a datapath issue, it coule be related to the problem for the MATLAB GUI. I'm not sure, also, if phy is working for only 1 channel...

Best

Pierre

--
You received this message because you are subscribed to the Google Groups "SpyKING CIRCUS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spyking-circus-users+unsubscrib...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/spyking-circus-users/621dd765-2c04-400c-9663-144d7a2d7070%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Philémon Roussel]

Thank you for your quick reply!

Just to make sure it's not the source of the problem, I created a new data file data.dat with 2 channels and the associated 2ch.prb (attached) and data.params (attached).

I could not find CIRCUSPATH using the method you suggested, I guess it is because python is handled by Anaconda and thus the python command is not directly accessible. It could have some connections with the problems I encounter (?).

I removed the ";" on line 133 in C:\Users\philemon\Anaconda2\envs\phy\Lib\site-packages\circus\matlab_GUI\sortingGUI.m .

1) Initial steps:

activate phy

spyking-circus C:/Users/philemon/Documents/Prog/SpyKINGCircus/data.dat --method filtering,whitening,clustering,fitting,converting --cpu 8

2) Using:

circus-gui-matlab C:/Users/philemon/Documents/Prog/SpyKINGCircus/data.dat

I obtain in MATLAB Command Window:

tmpfile =

    'C:\Users\philemon\Documents\Prog\SpyKINGCircus\data\data.templates.hdf5'

Error using h5infoc
The filename specified was either not found on the MATLAB path or it contains unsupported characters.

Error in h5info (line 101)
hinfo = h5infoc(filename,location, useUtf8);

Error in SortingGUI>SortingGUI_OpeningFcn (line 134)
    info     = h5info(tmpfile);

Error in gui_mainfcn (line 220)
    feval(gui_State.gui_OpeningFcn, gui_hFigure, [], guidata(gui_hFigure), varargin{:});

Error in SortingGUI (line 47)
    gui_mainfcn(gui_State, varargin{:});

The path seem correct, I attached the data.templates.hdf5 file.

3) Using as before:

circus-gui-python C:/Users/philemon/Documents/Prog/SpyKINGCircus/data.dat

The GUI window opens! It seems to work properly even though I will need more time to figure that out.

It seems therefore that the problem with phy came from the fact that there was only a single channel in my test file.
I still do not understand what is happening with MATLAB though.

Pierre

To unsubscribe from this group and stop receiving emails from it, send an email to spyking-circus-users+unsub...@googlegroups.com.

[Pierre Yger]

phy may have some issues to load/open datasets where templates/PCs would be on only 1 channel. I'll try to make a PR for phy quickly to address that. Regarding what is happening with MATLAB, I don't know either, and I'll have a look

Best

Pierre

Le vendredi 27 octobre 2017 12:35:28 UTC+2, Philémon Roussel a écrit :

[Vincent Prevosto]

I also get the same issue when opening .hdf5 files in Matlab.

The filename specified was either not found on the MATLAB path or it contains unsupported characters.

This is only a problem with files generated with recent versions of Spyking Circus. I can perfectly open files generated until November 7. I supposed I updated the code after that (I don't remember when I previously updated it, so I can't pinpoint the potential problematic commit).

[Pierre Yger]

Thanks for the feedback, i'll try asap to install MATLAB on Windows and test if I can reproduce the error. I pushed yesterday the 0.6.2 version that at least should suppress the problem you had while converting the files for phy (I managed to reproduce and fix the bug, not visible with linux). 

You can upgrade with conda

>> conda upgrade -c spyking-circus spyking-circus

But I'm afraid it will not do anything for matlab issues, so stay tuned...

Best

Pierre

--
You received this message because you are subscribed to the Google Groups "SpyKING CIRCUS" group.

To unsubscribe from this group and stop receiving emails from it, send an email to spyking-circus-u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/spyking-circus-users/6e8881f9-834d-4b20-8259-081d63fd1d69%40googlegroups.com.

[Vincent Prevosto]

Re: the issue opening hdf5 files with Matlab. The problem comes from the latest version of hdf5
Per their website:

New HDF5 1.10 Features
[...]
File Format Changes

This release includes changes in the HDF5 storage format. These changes come into play when one or more of the new features is used or when an application calls for use of the latest storage format (H5Pset_libver_bounds).

Due to the requirements of some of the new features, the format of a 1.10.x HDF5 file is likely to be different from that of a 1.8.x HDF5 file. This means that tools and applications built to read 1.10.x files will be able to read a 1.8.x file, but tools built to read 1.8.x files may not be able to read a 1.10.x file.

If an application built on HDF5 Release 1.10 avoids use of the new features and does not request use of the latest format, applications built on HDF5 Release 1.8.x will be able to read files the first application created. In addition, applications originally written for use with HDF5 Release 1.8.x can be linked against a suitably configured HDF5 Release 1.10.x library, thus taking advantage of performance improvements in 1.10.

So, if you get this issue, it's likely that .hdf5 files were generated with version 1.10. Downgrading the package this way fixes the problem :

conda install hdf5=1.8.18

To unsubscribe from this group and stop receiving emails from it, send an email to spyking-circus-users+unsub...@googlegroups.com.

[Pierre Yger]

Thanks a lot!

Best

Pierre

To unsubscribe from this group and stop receiving emails from it, send an email to spyking-circus-users+unsubscrib...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/spyking-circus-users/6e8881f9-834d-4b20-8259-081d63fd1d69%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "SpyKING CIRCUS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spyking-circus-users+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/spyking-circus-users/f8dabedd-8a9b-4f57-8f51-ddcbab2f4ea3%40googlegroups.com.

[Philémon Roussel]

Downgrading worked for me!

Thanks a lot!

Philémon

To unsubscribe from this group and stop receiving emails from it, send an email to spyking-circus-users+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/spyking-circus-users/6e8881f9-834d-4b20-8259-081d63fd1d69%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.