h5 data set

[Ashwani Rai]

So I followed 

https://github.com/BVLC/caffe/blob/master/src/caffe/test/test_data/generate_sample_data.py example to turn my data into the h5 format.

But now I am not sure how to proceed with the caffe training?

Is there any documentation to use python and train caffe using my data?

[Shaunak De]

I am attaching some files I have prepared to hopefully guide you.

And this is a stub I wrote to generate it:

# -*- coding: utf-8 -*-
"""
Created on Wed Apr 15 09:01:55 2015

@author: shaunak
"""
import h5py
import numpy as np

dataset = np.genfromtxt('test_data.csv',delimiter="\t")
label = np.genfromtxt('test_lab.csv',delimiter="\t")
label = label - 1 #Zero order indexing
f = h5py.File("wine_test.h5","w")
f.create_dataset("data", data= dataset)
f.create_dataset("label", data= label)
f.close()

f = h5py.File("wine_train.h5","w")
dataset = np.genfromtxt('train_data.csv',delimiter="\t")
label = np.genfromtxt('train_lab.csv',delimiter="\t")
label = label - 1 #Zero order indexing
f.create_dataset("data", data= dataset)
f.create_dataset("label", data= label)
f.close()

[Ashwani Rai]

Hi Shaunak, 

Thanks so much for your reply.

My apologies, I should have worded my question better. I have generated the h5 files already. But I do not know how to use them to run caffe training. Should I be modifying the prototxt files?

[Keith Monaghan]

Thank you for this.  I tried this on my own data set (1024 columns), but the data section was just one column with NaN for every entry.  Any idea what could be causing this?

[Ashwani Rai]

data section was just one column with NaN for every entry

Which file are you referring to? 

[Shaunak De]

I am attaching the solver and the model I used.

On Mon, Apr 20, 2015 at 11:02 PM, Ashwani Rai <a.k...@live.com> wrote:

data section was just one column with NaN for every entry

Which file are you referring to? 

--
You received this message because you are subscribed to a topic in the Google Groups "Caffe Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/caffe-users/hTeaOGy_Gys/unsubscribe.
To unsubscribe from this group and all its topics, send an email to caffe-users...@googlegroups.com.
To post to this group, send email to caffe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/caffe-users/a03dc7c3-8fcf-49d9-a878-6d030164d2e6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Shaunak De

http://shaunak.de/

[fabwa]

hi there,

thanks for posting the prototxt files. Do you mind also posting the code where you are actually training a net with these configs + classify unknown data? Somehow i dont find any useful example where this is carried out. I only see examples where they train and output accuracy but not classify unseen data...

Thanks

[Ashwani Rai]

Thanks Shaunak.

Apologies for the basic question, but is there a difference between using one h5 file with multiple rows VS multiple h5 files with one row each?

Regards,

Ash 

[Shaunak De]

Apologies for the delayed response. But if I am not mistaken the random shuffling algorithm does not work across files.