comedirecord + hdf5
51 views
Skip to first unread message
Bernd Porr
May 20, 2012, 5:35:35 PM5/20/12
to comed...@googlegroups.com
Hi all,
I've added hdf5 support to comedirecord. Seems to be working quite
nicely. The hdfviewer likes the data.
git://github.com/berndporr/comedirecord.git
project page:
http://www.linux-usb-daq.co.uk/software2/comedi-record/
To the HDF5 gurus here: I can only open the HDF files in the viewer once
I have quit comedirecord (even that I've closed all the handles).
/Bernd
--
www: http://www.linux-usb-daq.co.uk/
http://www.berndporr.me.uk/
http://www.imdb.com/name/nm3293421/
Mobile: +44 (0)7840 340069
Work: +44 (0)141 330 5237
University of Glasgow
School of Engineering
72 Oakfield Avenue (Rankine Building for deliveries)
Glasgow, G12 8LT
I've added hdf5 support to comedirecord. Seems to be working quite
nicely. The hdfviewer likes the data.
git://github.com/berndporr/comedirecord.git
project page:
http://www.linux-usb-daq.co.uk/software2/comedi-record/
To the HDF5 gurus here: I can only open the HDF files in the viewer once
I have quit comedirecord (even that I've closed all the handles).
/Bernd
--
www: http://www.linux-usb-daq.co.uk/
http://www.berndporr.me.uk/
http://www.imdb.com/name/nm3293421/
Mobile: +44 (0)7840 340069
Work: +44 (0)141 330 5237
University of Glasgow
School of Engineering
72 Oakfield Avenue (Rankine Building for deliveries)
Glasgow, G12 8LT
Ross Williamson
May 21, 2012, 4:42:52 PM5/21/12
to comed...@googlegroups.com
Are you using raw HDF5 of netcdf4? I've got a few applications running using netcdf4 (with HDF5 backend) in python and I can access the data whilst being written from KST for example. I'll try and dig up some examples if it's helpful.
--
Ross Williamson
Associate Research Scientist
Columbia Astrophysics Laboratory
212-851-9379 (office)
212-854-4653 (Lab)
312-504-3051 (Cell)
Ross
--
You received this message because you are subscribed to the Google Groups "Comedi: Linux Control and Measurement Device Interface" group.
To post to this group, send email to comed...@googlegroups.com.
To unsubscribe from this group, send email to comedi_list+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/comedi_list?hl=en.
Ross Williamson
Associate Research Scientist
Columbia Astrophysics Laboratory
212-851-9379 (office)
212-854-4653 (Lab)
312-504-3051 (Cell)
Bernd Porr
May 21, 2012, 7:38:45 PM5/21/12
to comed...@googlegroups.com
I'm using HDF5. I've just tried out KST but it won't like the hdf5 file.
That's generated by comedirecord:
http://www.linux-usb-daq.co.uk/tmp/aaa.hdf
As posted earlier with h5ls I can pipe that into gnuplot without any probs:
plot "< h5ls -d -S ggg.hdf | tail -n +3" using 2 with lines
I guess I need to do a similar thing for KST? First converting it back
to ASCII?
/Bernd
Ross Williamson wrote:
> Are you using raw HDF5 of netcdf4? I've got a few applications running
> using netcdf4 (with HDF5 backend) in python and I can access the data
> whilst being written from KST for example. I'll try and dig up some
> examples if it's helpful.
>
> Ross
>
> On Sun, May 20, 2012 at 5:35 PM, Bernd Porr <bern...@f2s.com
> <mailto:bern...@f2s.com>> wrote:
>
> Hi all,
>
> I've added hdf5 support to comedirecord. Seems to be working quite
> nicely. The hdfviewer likes the data.
>
> git://github.com/berndporr/__comedirecord.git
> <http://github.com/berndporr/comedirecord.git>
>
> project page:
> http://www.linux-usb-daq.co.__uk/software2/comedi-record/
> <http://www.linux-usb-daq.co.uk/>
> http://www.berndporr.me.uk/
> http://www.imdb.com/name/__nm3293421/
> <http://www.imdb.com/name/nm3293421/>
> Mobile: +44 (0)7840 340069 <tel:%2B44%20%280%297840%20340069>
> Work: +44 (0)141 330 5237 <tel:%2B44%20%280%29141%20330%205237>
> <mailto:comedi_list%2Bunsu...@googlegroups.com>.
> <http://groups.google.com/group/comedi_list?hl=en>.
That's generated by comedirecord:
http://www.linux-usb-daq.co.uk/tmp/aaa.hdf
As posted earlier with h5ls I can pipe that into gnuplot without any probs:
plot "< h5ls -d -S ggg.hdf | tail -n +3" using 2 with lines
I guess I need to do a similar thing for KST? First converting it back
to ASCII?
/Bernd
Ross Williamson wrote:
> Are you using raw HDF5 of netcdf4? I've got a few applications running
> using netcdf4 (with HDF5 backend) in python and I can access the data
> whilst being written from KST for example. I'll try and dig up some
> examples if it's helpful.
>
> Ross
>
> On Sun, May 20, 2012 at 5:35 PM, Bernd Porr <bern...@f2s.com
> <mailto:bern...@f2s.com>> wrote:
>
> Hi all,
>
> I've added hdf5 support to comedirecord. Seems to be working quite
> nicely. The hdfviewer likes the data.
>
> <http://github.com/berndporr/comedirecord.git>
>
> project page:
> http://www.linux-usb-daq.co.__uk/software2/comedi-record/
> <http://www.linux-usb-daq.co.uk/software2/comedi-record/>
>
> To the HDF5 gurus here: I can only open the HDF files in the viewer
> once I have quit comedirecord (even that I've closed all the handles).
>
> /Bernd
> --
> www: http://www.linux-usb-daq.co.__uk/
>
> To the HDF5 gurus here: I can only open the HDF files in the viewer
> once I have quit comedirecord (even that I've closed all the handles).
>
> /Bernd
> --
> <http://www.linux-usb-daq.co.uk/>
> http://www.berndporr.me.uk/
> http://www.imdb.com/name/__nm3293421/
> <http://www.imdb.com/name/nm3293421/>
> Mobile: +44 (0)7840 340069 <tel:%2B44%20%280%297840%20340069>
> Work: +44 (0)141 330 5237 <tel:%2B44%20%280%29141%20330%205237>
> University of Glasgow
> School of Engineering
> 72 Oakfield Avenue (Rankine Building for deliveries)
> Glasgow, G12 8LT
>
> --
> You received this message because you are subscribed to the Google
> Groups "Comedi: Linux Control and Measurement Device Interface" group.
> To post to this group, send email to comed...@googlegroups.com
> <mailto:comed...@googlegroups.com>.
> School of Engineering
> 72 Oakfield Avenue (Rankine Building for deliveries)
> Glasgow, G12 8LT
>
> --
> You received this message because you are subscribed to the Google
> Groups "Comedi: Linux Control and Measurement Device Interface" group.
> To post to this group, send email to comed...@googlegroups.com
> To unsubscribe from this group, send email to
> comedi_list+unsubscribe@__googlegroups.com
> <mailto:comedi_list%2Bunsu...@googlegroups.com>.
> For more options, visit this group at
> http://groups.google.com/__group/comedi_list?hl=en
> <http://groups.google.com/group/comedi_list?hl=en>.
>
>
>
>
> --
> Ross Williamson
> Associate Research Scientist
> Columbia Astrophysics Laboratory
> 212-851-9379 (office)
> 212-854-4653 (Lab)
> 312-504-3051 (Cell)
>
>
>
>
> --
> Ross Williamson
> Associate Research Scientist
> Columbia Astrophysics Laboratory
> 212-851-9379 (office)
> 212-854-4653 (Lab)
> 312-504-3051 (Cell)
>
> --
> You received this message because you are subscribed to the Google
> Groups "Comedi: Linux Control and Measurement Device Interface" group.
> To post to this group, send email to comed...@googlegroups.com.
> To unsubscribe from this group, send email to
> comedi_list...@googlegroups.com.
> You received this message because you are subscribed to the Google
> Groups "Comedi: Linux Control and Measurement Device Interface" group.
> To post to this group, send email to comed...@googlegroups.com.
> To unsubscribe from this group, send email to
Ross Williamson
May 22, 2012, 10:18:24 AM5/22/12
to comed...@googlegroups.com
Sorry I should have mentioned that the standard netcdf plugin for KST is very old and doesn't work with netcdf4/hdf5 files.
You could also convert to ASCII if needs be on the fly. I know with netcdf4 you have to call "sync" fairly often as the buffer is large and the data will only get written once the file is closed (or buffer fills up)
If you want to use KST I'll post the new netCDF datasource files for kst
<mailto:comedi_list@googlegroups.com>.
comedi_list+unsubscribe@__googlegroups.com
To unsubscribe from this group, send email to
<mailto:comedi_list%2Bunsu...@googlegroups.com>.http://groups.google.com/__group/comedi_list?hl=en
For more options, visit this group at
<http://groups.google.com/group/comedi_list?hl=en>.
--
Ross Williamson
Associate Research Scientist
Columbia Astrophysics Laboratory
212-851-9379 (office)
212-854-4653 (Lab)
312-504-3051 (Cell)
--
You received this message because you are subscribed to the Google Groups "Comedi: Linux Control and Measurement Device Interface" group.
To post to this group, send email to comed...@googlegroups.com.
To unsubscribe from this group, send email to comedi_list+unsubscribe@googlegroups.com.
--
www: http://www.linux-usb-daq.co.uk/
http://www.berndporr.me.uk/
http://www.imdb.com/name/nm3293421/
Mobile: +44 (0)7840 340069
Work: +44 (0)141 330 5237
University of Glasgow
School of Engineering
72 Oakfield Avenue (Rankine Building for deliveries)
Glasgow, G12 8LT
--
You received this message because you are subscribed to the Google Groups "Comedi: Linux Control and Measurement Device Interface" group.
To post to this group, send email to comed...@googlegroups.com.
To unsubscribe from this group, send email to comedi_list+unsubscribe@googlegroups.com.
Bernd Porr
May 23, 2012, 2:41:17 AM5/23/12
to comed...@googlegroups.com
Hi all,
Pretty sobering when looking at the file sizes. Mainly, I was hoping to
create smaller file sizes for very long EEG recordings. Here's the
comparison of 8 channels / 10 secs / floating point (HDF5 compressed,
HDF5 uncompressed, ASCII) stored with comedirecord:
-rw-r--r-- 1 bp1 bp1 340357 2012-05-23 07:29 zzz_bz.hdf
-rw-r--r-- 1 bp1 bp1 400242 2012-05-23 07:25 zzz.hdf
-rw-r--r-- 1 bp1 bp1 859282 2012-05-23 07:25 zzz.txt
after a gzip:
-rw-r--r-- 1 bp1 bp1 265336 2012-05-23 07:25 zzz.txt.gz
Most plotting programs read ASCII only anyway. So, it's better to record
in ASCII and do a gzip after the recording. That's more efficient than
storing in HDF5 with compression.
/Bernd
Ross Williamson wrote:
> Sorry I should have mentioned that the standard netcdf plugin for KST is
> very old and doesn't work with netcdf4/hdf5 files.
>
> I've re-written the plugin for KST to read netCDF4 files - It's alpha
> and I haven't set it upon the general KST gurus yet as I need to clean
> it up. Also I don't think it will read raw HDF5 files but instead
> requires the higher level netcdf4 interface. It will read data in
> "real-time" and also take care of the hierarchical side of things.
>
> You could also convert to ASCII if needs be on the fly. I know with
> netcdf4 you have to call "sync" fairly often as the buffer is large and
> the data will only get written once the file is closed (or buffer fills up)
>
> If you want to use KST I'll post the new netCDF datasource files for kst
>
> On Mon, May 21, 2012 at 7:38 PM, Bernd Porr <bern...@f2s.com
> <mailto:bern...@f2s.com>> wrote:
>
> I'm using HDF5. I've just tried out KST but it won't like the hdf5 file.
>
> That's generated by comedirecord:
> http://www.linux-usb-daq.co.__uk/tmp/aaa.hdf
> <http://github.com/berndporr/__comedirecord.git>
> <http://github.com/berndporr/__comedirecord.git
> <http://github.com/berndporr/comedirecord.git>>
>
> project page:
> http://www.linux-usb-daq.co.____uk/software2/comedi-record/
> <http://www.imdb.com/name/__nm3293421/>
> <tel:%2B44%20%280%29141%20330%__205237>
> <mailto:comedi_list@__googlegroups.com
> <mailto:comed...@googlegroups.com>>.
> <http://googlegroups.com>
> <mailto:comedi_list%__2Buns...@googlegroups.com
> <mailto:comedi_list%252Buns...@googlegroups.com>__>.
> <http://groups.google.com/__group/comedi_list?hl=en>
> 212-854-4653 <tel:212-854-4653> (Lab)
> 312-504-3051 <tel:312-504-3051> (Cell)
Pretty sobering when looking at the file sizes. Mainly, I was hoping to
create smaller file sizes for very long EEG recordings. Here's the
comparison of 8 channels / 10 secs / floating point (HDF5 compressed,
HDF5 uncompressed, ASCII) stored with comedirecord:
-rw-r--r-- 1 bp1 bp1 340357 2012-05-23 07:29 zzz_bz.hdf
-rw-r--r-- 1 bp1 bp1 400242 2012-05-23 07:25 zzz.hdf
-rw-r--r-- 1 bp1 bp1 859282 2012-05-23 07:25 zzz.txt
after a gzip:
-rw-r--r-- 1 bp1 bp1 265336 2012-05-23 07:25 zzz.txt.gz
Most plotting programs read ASCII only anyway. So, it's better to record
in ASCII and do a gzip after the recording. That's more efficient than
storing in HDF5 with compression.
/Bernd
Ross Williamson wrote:
> Sorry I should have mentioned that the standard netcdf plugin for KST is
> very old and doesn't work with netcdf4/hdf5 files.
>
> I've re-written the plugin for KST to read netCDF4 files - It's alpha
> and I haven't set it upon the general KST gurus yet as I need to clean
> it up. Also I don't think it will read raw HDF5 files but instead
> requires the higher level netcdf4 interface. It will read data in
> "real-time" and also take care of the hierarchical side of things.
>
> You could also convert to ASCII if needs be on the fly. I know with
> netcdf4 you have to call "sync" fairly often as the buffer is large and
> the data will only get written once the file is closed (or buffer fills up)
>
> If you want to use KST I'll post the new netCDF datasource files for kst
>
> On Mon, May 21, 2012 at 7:38 PM, Bernd Porr <bern...@f2s.com
> <mailto:bern...@f2s.com>> wrote:
>
> I'm using HDF5. I've just tried out KST but it won't like the hdf5 file.
>
> That's generated by comedirecord:
> <http://www.linux-usb-daq.co.uk/tmp/aaa.hdf>
>
> As posted earlier with h5ls I can pipe that into gnuplot without any
> probs:
> plot "< h5ls -d -S ggg.hdf | tail -n +3" using 2 with lines
>
> I guess I need to do a similar thing for KST? First converting it
> back to ASCII?
>
> /Bernd
>
> Ross Williamson wrote:
>
> Are you using raw HDF5 of netcdf4? I've got a few applications
> running using netcdf4 (with HDF5 backend) in python and I can
> access the data whilst being written from KST for example. I'll
> try and dig up some examples if it's helpful.
>
> Ross
>
> On Sun, May 20, 2012 at 5:35 PM, Bernd Porr <bern...@f2s.com
> <mailto:bern...@f2s.com> <mailto:bern...@f2s.com
>
> As posted earlier with h5ls I can pipe that into gnuplot without any
> probs:
> plot "< h5ls -d -S ggg.hdf | tail -n +3" using 2 with lines
>
> I guess I need to do a similar thing for KST? First converting it
> back to ASCII?
>
> /Bernd
>
> Ross Williamson wrote:
>
> Are you using raw HDF5 of netcdf4? I've got a few applications
> running using netcdf4 (with HDF5 backend) in python and I can
> access the data whilst being written from KST for example. I'll
> try and dig up some examples if it's helpful.
>
> Ross
>
> On Sun, May 20, 2012 at 5:35 PM, Bernd Porr <bern...@f2s.com
> <mailto:bern...@f2s.com>>> wrote:
>
> Hi all,
>
> I've added hdf5 support to comedirecord. Seems to be working
> quite
> nicely. The hdfviewer likes the data.
>
> git://github.com/berndporr/____comedirecord.git
>
> Hi all,
>
> I've added hdf5 support to comedirecord. Seems to be working
> quite
> nicely. The hdfviewer likes the data.
>
> <http://github.com/berndporr/__comedirecord.git>
> <http://github.com/berndporr/__comedirecord.git
> <http://github.com/berndporr/comedirecord.git>>
>
> project page:
> http://www.linux-usb-daq.co.____uk/software2/comedi-record/
>
> <http://www.linux-usb-daq.co.__uk/software2/comedi-record/
> <http://www.linux-usb-daq.co.uk/software2/comedi-record/>>
>
> To the HDF5 gurus here: I can only open the HDF files in the
> viewer
> once I have quit comedirecord (even that I've closed all the
> handles).
>
> /Bernd
> -- www: http://www.linux-usb-daq.co.____uk/
> <http://www.linux-usb-daq.co.__uk/software2/comedi-record/
> <http://www.linux-usb-daq.co.uk/software2/comedi-record/>>
>
> To the HDF5 gurus here: I can only open the HDF files in the
> viewer
> once I have quit comedirecord (even that I've closed all the
> handles).
>
> /Bernd
> <http://www.linux-usb-daq.co.__uk/
> <http://www.linux-usb-daq.co.uk/>>
> http://www.berndporr.me.uk/
> http://www.imdb.com/name/____nm3293421/
> <http://www.linux-usb-daq.co.uk/>>
> http://www.berndporr.me.uk/
> <http://www.imdb.com/name/__nm3293421/>
> <http://www.imdb.com/name/__nm3293421/
> <http://www.imdb.com/name/nm3293421/>>
> Mobile: +44 (0)7840 340069 <tel:%2B44%20%280%297840%20340069>
> <tel:%2B44%20%280%297840%__20340069>
> <http://www.imdb.com/name/nm3293421/>>
> Mobile: +44 (0)7840 340069 <tel:%2B44%20%280%297840%20340069>
> <tel:%2B44%20%280%29141%20330%__205237>
>
> University of Glasgow
> School of Engineering
> 72 Oakfield Avenue (Rankine Building for deliveries)
> Glasgow, G12 8LT
>
> -- You received this message because you are subscribed
> to the Google
> Groups "Comedi: Linux Control and Measurement Device
> Interface" group.
> To post to this group, send email to
> comed...@googlegroups.com <mailto:comed...@googlegroups.com>
> University of Glasgow
> School of Engineering
> 72 Oakfield Avenue (Rankine Building for deliveries)
> Glasgow, G12 8LT
>
> -- You received this message because you are subscribed
> to the Google
> Groups "Comedi: Linux Control and Measurement Device
> Interface" group.
> To post to this group, send email to
> <mailto:comedi_list@__googlegroups.com
> <mailto:comed...@googlegroups.com>>.
>
> To unsubscribe from this group, send email to
> comedi_list+unsubscribe@__goog__legroups.com
> To unsubscribe from this group, send email to
> <http://googlegroups.com>
> <mailto:comedi_list%__2Buns...@googlegroups.com
> <mailto:comedi_list%252Buns...@googlegroups.com>__>.
>
> For more options, visit this group at
> http://groups.google.com/____group/comedi_list?hl=en
> For more options, visit this group at
> <http://groups.google.com/__group/comedi_list?hl=en>
> <http://groups.google.com/__group/comedi_list?hl=en
> <http://groups.google.com/group/comedi_list?hl=en>>.
>
>
>
>
>
> --
> Ross Williamson
> Associate Research Scientist
> Columbia Astrophysics Laboratory
> 212-851-9379 <tel:212-851-9379> (office)
> <http://groups.google.com/group/comedi_list?hl=en>>.
>
>
>
>
>
> --
> Ross Williamson
> Associate Research Scientist
> Columbia Astrophysics Laboratory
> 212-854-4653 <tel:212-854-4653> (Lab)
> 312-504-3051 <tel:312-504-3051> (Cell)
>
> --
> You received this message because you are subscribed to the
> Google Groups "Comedi: Linux Control and Measurement Device
> Interface" group.
> To post to this group, send email to
> comed...@googlegroups.com <mailto:comed...@googlegroups.com>.
> --
> You received this message because you are subscribed to the
> Google Groups "Comedi: Linux Control and Measurement Device
> Interface" group.
> To post to this group, send email to
> To unsubscribe from this group, send email to
> comedi_list+unsubscribe@__googlegroups.com
> <mailto:comedi_list%2Bunsu...@googlegroups.com>.
> For more options, visit this group at
> http://groups.google.com/__group/comedi_list?hl=en
> <http://groups.google.com/group/comedi_list?hl=en>.
>
>
> --
> comedi_list+unsubscribe@__googlegroups.com
> <mailto:comedi_list%2Bunsu...@googlegroups.com>.
> For more options, visit this group at
> http://groups.google.com/__group/comedi_list?hl=en
> <http://groups.google.com/group/comedi_list?hl=en>.
>
>
> --
> www: http://www.linux-usb-daq.co.__uk/
> <http://www.linux-usb-daq.co.uk/>
> http://www.berndporr.me.uk/
> http://www.imdb.com/name/__nm3293421/
> <http://www.imdb.com/name/nm3293421/>
> Mobile: +44 (0)7840 340069 <tel:%2B44%20%280%297840%20340069>
> Work: +44 (0)141 330 5237 <tel:%2B44%20%280%29141%20330%205237>
> University of Glasgow
> School of Engineering
> 72 Oakfield Avenue (Rankine Building for deliveries)
> Glasgow, G12 8LT
>
> --
> You received this message because you are subscribed to the Google
> Groups "Comedi: Linux Control and Measurement Device Interface" group.
> To post to this group, send email to comed...@googlegroups.com
> <mailto:comed...@googlegroups.com>.
> <http://www.linux-usb-daq.co.uk/>
> http://www.berndporr.me.uk/
> http://www.imdb.com/name/__nm3293421/
> <http://www.imdb.com/name/nm3293421/>
> Mobile: +44 (0)7840 340069 <tel:%2B44%20%280%297840%20340069>
> Work: +44 (0)141 330 5237 <tel:%2B44%20%280%29141%20330%205237>
> University of Glasgow
> School of Engineering
> 72 Oakfield Avenue (Rankine Building for deliveries)
> Glasgow, G12 8LT
>
> --
> You received this message because you are subscribed to the Google
> Groups "Comedi: Linux Control and Measurement Device Interface" group.
> To post to this group, send email to comed...@googlegroups.com
> To unsubscribe from this group, send email to
> comedi_list+unsubscribe@__googlegroups.com
> <mailto:comedi_list%2Bunsu...@googlegroups.com>.
> For more options, visit this group at
> http://groups.google.com/__group/comedi_list?hl=en
> <http://groups.google.com/group/comedi_list?hl=en>.
>
>
>
>
> --
> Ross Williamson
> Associate Research Scientist
> Columbia Astrophysics Laboratory
> 212-851-9379 (office)
> 212-854-4653 (Lab)
> 312-504-3051 (Cell)
>
> --
> You received this message because you are subscribed to the Google
> Groups "Comedi: Linux Control and Measurement Device Interface" group.
> To post to this group, send email to comed...@googlegroups.com.
> To unsubscribe from this group, send email to
> comedi_list...@googlegroups.com.
> comedi_list+unsubscribe@__googlegroups.com
> <mailto:comedi_list%2Bunsu...@googlegroups.com>.
> For more options, visit this group at
> http://groups.google.com/__group/comedi_list?hl=en
> <http://groups.google.com/group/comedi_list?hl=en>.
>
>
>
>
> --
> Ross Williamson
> Associate Research Scientist
> Columbia Astrophysics Laboratory
> 212-851-9379 (office)
> 212-854-4653 (Lab)
> 312-504-3051 (Cell)
>
> --
> You received this message because you are subscribed to the Google
> Groups "Comedi: Linux Control and Measurement Device Interface" group.
> To post to this group, send email to comed...@googlegroups.com.
> To unsubscribe from this group, send email to
W. Trevor King
May 23, 2012, 7:31:12 AM5/23/12
to comed...@googlegroups.com
On Wed, May 23, 2012 at 07:41:17AM +0100, Bernd Porr wrote:
> Pretty sobering when looking at the file sizes. Mainly, I was hoping to
> create smaller file sizes for very long EEG recordings. Here's the
> comparison of 8 channels / 10 secs / floating point (HDF5 compressed,
> HDF5 uncompressed, ASCII) stored with comedirecord:
You can cut file sizes by 4x if you save the raw uint16s instead of
> Pretty sobering when looking at the file sizes. Mainly, I was hoping to
> create smaller file sizes for very long EEG recordings. Here's the
> comparison of 8 channels / 10 secs / floating point (HDF5 compressed,
> HDF5 uncompressed, ASCII) stored with comedirecord:
floats. Of course then you'll need to save the conversion constants
too, so you can convert to physical values later. I wrote h5config
[1] so I could do things like this easily for my experiments. I use
it in pypiezo [2], storing channel configuration in
pypiezo.config.ChannelConfig instances. Then I can load the data and
configuration later and convert to physical using
pypiezo.base.convert_bits_to_volts.
On the other hand, disk space is cheap ;).
--
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
W. Trevor King
May 23, 2012, 7:33:31 AM5/23/12
to comed...@googlegroups.com
On Wed, May 23, 2012 at 07:31:12AM -0400, W. Trevor King wrote:
> You can cut file sizes by 4x if you save the raw uint16s instead of
> floats. Of course then you'll need to save the conversion constants
> too, so you can convert to physical values later. I wrote h5config
> [1] so I could do things like this easily for my experiments. I use
> it in pypiezo [2], storing channel configuration in
> pypiezo.config.ChannelConfig instances. Then I can load the data and
> configuration later and convert to physical using
> pypiezo.base.convert_bits_to_volts.
Oops, forgot links:
> You can cut file sizes by 4x if you save the raw uint16s instead of
> floats. Of course then you'll need to save the conversion constants
> too, so you can convert to physical values later. I wrote h5config
> [1] so I could do things like this easily for my experiments. I use
> it in pypiezo [2], storing channel configuration in
> pypiezo.config.ChannelConfig instances. Then I can load the data and
> configuration later and convert to physical using
> pypiezo.base.convert_bits_to_volts.
[1]: http://pypi.python.org/pypi/h5config/
[2]: http://pypi.python.org/pypi/pypiezo/
Bernd Porr
May 23, 2012, 8:13:06 AM5/23/12
to comed...@googlegroups.com
> On the other hand, disk space is cheap ;).
Indeed. ;)
/Bernd
W. Trevor King wrote:
> On Wed, May 23, 2012 at 07:41:17AM +0100, Bernd Porr wrote:
>> Pretty sobering when looking at the file sizes. Mainly, I was hoping to
>> create smaller file sizes for very long EEG recordings. Here's the
>> comparison of 8 channels / 10 secs / floating point (HDF5 compressed,
>> HDF5 uncompressed, ASCII) stored with comedirecord:
>
> You can cut file sizes by 4x if you save the raw uint16s instead of
> floats. Of course then you'll need to save the conversion constants
> too, so you can convert to physical values later. I wrote h5config
> [1] so I could do things like this easily for my experiments. I use
> it in pypiezo [2], storing channel configuration in
> pypiezo.config.ChannelConfig instances. Then I can load the data and
> configuration later and convert to physical using
> pypiezo.base.convert_bits_to_volts.
>
> On the other hand, disk space is cheap ;).
>
--
Reply all
Reply to author
Forward
0 new messages