Output of Caffe Feature Extraction example

[Roy]

Hi All

I am new to caffe. I successfully installed and ran the caffe feature extraction demo. The application ouputs two .mdb files data.mdb and lock.mdb.

The tutorial does not say beyond how to use those two files or how to access the data inside them. They are not microsoft access files. Anybody would like to explain how to access the data in those files. Is there any code available that helps explain.

Appreciate your help and thank you!
regards
Roy

[Nikiforos Pittaras]

You can modify the .cpp file to store the features in the format you wish, that's what I have done.

[J. Yegerlehner]

Those are lmdb database files. That is one of two formats that layers of type Data read.
http://caffe.berkeleyvision.org/tutorial/data.html
http://caffe.berkeleyvision.org/tutorial/layers.html#data-layers

[Alessandro Ortis]

can you explain better please? for example I wish to print the 4096-d features of fc8 in the prompt or write them in a txt file.

[npit]

in tools/extract_features.cpp

The loop at line 157, loops over the values of the feature of a single image.

feature_blob_data[d] is the d-th value of the feature vector

The loop at line 143 loops over the feature layers.

The loops at lines  141 and 149  loop over the # of minibatches and each image in the batch, respectively.

So you can store the feature vector of each image using the indices of the above loops :)

[anurika]

Hi!!
I also want to do the same thing. DId you find the solution to this?

[Kenya]

Hi npit,

What format did you save your file as?
Also, could you kindly post your extract_features.cpp file to this thread?

Thanks in advance,

[npit]

 save the file in whatever format I want, since I store the data in a pointer.

Here's the relevant code you asked for. Note that this works only for a batchSize that divides the number of images exactly.

// initialize the feature containers' pointers and the dimension container

                data = new float*[num_layers];

featDimension = new int[num_layers];

numLayers = num_layers;

// variable to control one-time allocation within the loop

bool * allocated = new bool[num_layers];

// init to false

for (int layer_index = 0; layer_index < num_layers; ++layer_index) allocated[layer_index] = false;

                // EXTRACT

// for each batch of images

for (int batch_index = 0; batch_index < num_mini_batches; ++batch_index) 

{

// evaluate the net

feature_extraction_net->Forward(input_vec);

// for each feature layer

for (int layer_index = 0; layer_index < num_layers; ++layer_index) 

{

// get that layer blob and its dimension

const boost::shared_ptr<Blob<Dtype> > feature_blob = feature_extraction_net

->blob_by_name(blob_names[layer_index]);

int batch_size = feature_blob->num();

int dim_features = feature_blob->count() / batch_size;

// init the feat container for that layer if needed, set dimension

if (!allocated[layer_index])

{

data[layer_index] = new float[dim_features * num_mini_batches * batch_size];

featDimension[layer_index] = dim_features;

allocated[layer_index] = true;

}

const Dtype* feature_blob_data;

// for each image in the batch

for (int image_index_inBatch = 0; image_index_inBatch < batch_size; ++image_index_inBatch) {

// get the image's feature vector

feature_blob_data = feature_blob->cpu_data() +

feature_blob->offset(image_index_inBatch);

// copy the values to the data container

for (int d = 0; d < dim_features; ++d) {

data[layer_index][(batch_index * batch_size + image_index_inBatch) * dim_features + d] = feature_blob_data[d];

}

++image_indices[layer_index];

if (image_indices[layer_index] % 1000 == 0) {

//txns.at(layer_index)->Commit();

//txns.at(layer_index).reset(feature_dbs.at(layer_index)->NewTransaction());

LOG(ERROR) << "Extracted features of " << image_indices[layer_index] <<

" query images for feature blob " << blob_names[layer_index];

}

}  // for (int image_index_inBatch = 0; image_index_inBatch < batch_size; ++image_index_inBatch)

}  // for (int layer_index = 0; layer_index < num_layers; ++layer_index)

}  // for (int batch_index = 0; batch_index < num_mini_batches; ++batch_index)

// write the last batch

for (int layer_index = 0; layer_index < num_layers; ++layer_index) {

LOG(ERROR) << "Extracted features of " << image_indices[layer_index] <<

" query images for feature blob " << blob_names[layer_index];

}

[Uni A]

Hi Kenya, 

Did you figure out how to handle this issue? 

Thanks

[Komal Sinha]

Hi !

Please send the full modified code for extract_feature.cpp. as making the mentioned changes throws error .

Thanks! 

[Hongping Cai]

I also got stuck in the same problem. 

[npit]

What's the error?

[Ali Matin]

errors like undefined variables! i spend a long time fixing errors but I'm wasting time, i cant understand whole code so i cant fix all errors. would you send the whole .cpp file wich works? thank you

On Wednesday, September 23, 2015 at 10:07:19 AM UTC+3:30, npit wrote:

What's the error?

[Xing Di]

I also stuck in this problem