Neo EpochArray Kwik IO

[John Tukker]

hi,

I'm new to python (using Python3 with Windows Powershell), and have been trying to set up neo recently, hoping to read kwik-format data, generated using phy (http://phy.readthedocs.org/en/latest/kwik-format/). 

I am currently not able to get this to work. Although the docs mention the kwik format being supported, I found this was unfortunately not so up-to-date. 

I found KwikIO.py on github (https://github.com/NeuralEnsemble/python-neo/blob/master/neo/io/kwikio.py), and I manually added it to neo.io (and updated __init__.py accordingly). This generates an error at import, since there is no EpochArray.py in neo.core (again in contrast to the docs). I checked that I have the latest version of Neo (pip freeze tells me neo==0.4.0.dev0), and tried reinstalling but still do not have EpochArray.

Any ideas why EpochArray might be missing? This kind of basic omission makes me worry that:

1. I am doing something wrong in the installation/there was some error with pip, or

2. EpochArray is not used anymore in 0.4.0 and the docs and KwikIO are outdated, or

3. there was simply an oversight that caused EpochArray not to be included in 0.4.0, and it has so far been solved independently by other users without being reported/addressed yet.

I'd be happy with any feedback on this.

I added a new file epocharray.py manually to neo.io (just copy-pasted from github), and also updated the __init__.py there. This allowed me to at least import neo (anyway it looks like kwikio doesn't even need it yet).

When I then import KwikIO, and try to read a kwik file, I get the following:

>>> reader = KwikIO(filename='ch220715_M28_Edgeprobe_MECaim_600sdata_nopulses.kwik')

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "C:\Users\tukkerjj\miniconda3\lib\site-packages\neo\io\kwikio.py", line 109, in __init__

    self._kwd = h5py.File(self._path + os.sep + rawfile, 'r')

  File "C:\Users\tukkerjj\miniconda3\lib\site-packages\h5py\_hl\files.py", line 260, in __init__

    fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)

  File "C:\Users\tukkerjj\miniconda3\lib\site-packages\h5py\_hl\files.py", line 89, in make_fid

    fid = h5f.open(name, flags, fapl=fapl)

  File "h5py\_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (C:\aroot\work\h5py\_objects

.c:2471)

  File "h5py\_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (C:\aroot\work\h5py\_objects

.c:2428)

  File "h5py\h5f.pyx", line 76, in h5py.h5f.open (C:\aroot\work\h5py\h5f.c:1805)

OSError: Unable to open file (Unable to open file: name = '\ch220715_m28_edgeprobe_mecaim_600sdata_n

opulses_100.raw.kwd', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0

)

The problem appears to be that kwikio.py is not written with phy in mind, and is looking for kwd files which are no longer included in the 'kwik-format' (the raw data is kept in dat files). Since really all kwikio.py can do so far seems to be to read in the raw data (read_analogsignal), my conclusion is that it is effectively not usable for phy data in its present form. 

So apart from just wanting to share that info, I have one burning question: 

is anyone presently working on KwikIO.py or some alternative for importing kwik files (of the phy-variety)?

cheers,

John

[Andrew Davison]

Hi John,

As you have discovered, the development version of Neo (which will become 0.4.0, but that hasn't been officially released yet) is rather unstable, with multiple branches of development.

Unfortunately, the stable version (0.3.3) has very outdated kwik support.

There will be a big push on Neo 0.4 in the next four weeks, so the situation ought to improve soon.

Please could you create a new issue in the Neo issue tracker for the two KwikIO errors you found, so we can keep track of this?

Many thanks,

Andrew