Big dataset, dropout and stochastic gradient descent

[Ted]

Hi all,

I have 3 questions regarding the library.

1. I am having problem using a big data sets. I am running eblearn-1.2_x64-r2631-win64 but I am getting this error: fseek to position -2147468236 failed, file is -1890488716 big, in ebl::midx<float>::mget at c:\eblearn\core\libidx\include\idx.hpp:1122.

2. I am experimenting with dropout but so far was not able to configure it in a way I like to. Maybe it is just my bad understanding of the dropout concept or my very limited understanding of the library codes. What really bothers me is the test_time parameter. This is required parameter which in training (during the bprop) only works if set to 0. So the only option is to set it to 0 for the training. But I believe that during the test phase after the training iteration is done and statistics over the train/validation sets are computed it should be set to test_time = 1 (do not drop only divide the outputs by (1-prob)). Am I missing something?

3. As far as I understand you are using stochastic gradient descent. Is mini-batch learning supported in your library?

Thank you for any answer.

[the_minion]

Sorry for the late reply, I've been using Torch-7 lately, so been slow at replying on eblearn forums.

1. w.r.t. 1, i've changed all seek pointers to long, and still idk why win64 cant read a file more then 2GB, thats just freaking weird.

2. Your understanding is perfectly correct about dropout.

3. We dont support mini-batch training. What we do support is the cool 2nd-order learning rate adjustment, as all the modules in eblearn have a bbprop function that calculates the 2nd derivative. This is hard to find in other libraries (unless its numerically calculated 2nd order).

[default...@gmail.com]

Thank you for the answer, but I still don't know how to use dropout. If the test_time parameter should have different settings during the fprop and bprop how it can be used during the training? I can only set it to 0 or 1 in config file, but during each phase of training it shuld be set up differently.

Also you mentioned Torch library multiple times. Can you tell me if the CUDA support in training and in classification/detection is much better in Torch then in EBLearn?