Not Generating Experiment List

64 views
Skip to first unread message

Thilina Madhusanka Perera

unread,
Jul 11, 2015, 8:31:51 AM7/11/15
to express...@googlegroups.com
Hi,

Im new to this one and im trying to do the tutorials that are given here,

network training part is going smoothly but in the evaluation it dose not Generating Experiment List as it says in the tutorial.

cant see any error on the GUI. it shows a message that the Experiment List has Generated but it is not in the experiment list on the GUI

this is the terminal output of the Generating Experiment List process



[u'prob']
['Exp View', 'Creating Experiment', u'cifar10--prob_1436617683.95', 0, u'Creating Experiment with name cifar10--prob', 0]
['Exp View', 'Creating Experiment', u'cifar10--prob_1436617683.95', 0, u'Creating Experiment with name cifar10--prob', 0]
/home/thilina/expresso/net/temp/temp_CIFAR10--TRAINED_train_cif_cifar10--prob_50.sh
['Exp View', 'Creating Experiment', u'cifar10--prob_1436617683.95', 0, u'Creating Experiment with name cifar10--prob', 0, False]
['Train View', 'Training Data', u'cifar10--trained_1436616855.05', 1, u'Training of Data has started with name cifar10--trained', 100, None, False, False]
^^^^^^^^^^^^^^^^^^^^^
['Train View', 'Training Data', u'cifar10--trained_1436616855.05', 1, u'Training of Data has started with name cifar10--trained', 100, None, False, False] 9
^^^^^^^^^^^^^^^^^^^^^
['Exp View', 'Creating Experiment', u'cifar10--prob_1436617104.52', 1, u'Creating Experiment with name cifar10--prob', 100, False, False]
['Exp View', 'Creating Experiment', u'cifar10--prob_1436617156.93', 1, u'Creating Experiment with name cifar10--prob', 100, False, False]
['Exp View', 'Creating Experiment', u'cifar10--prob_1436617683.95', 0, u'Creating Experiment with name cifar10--prob', 0, False]
['Train View', 'Training Data', u'cifar10--trained_1436616855.05', 1, u'Training of Data has started with name cifar10--trained', 100, None, False, False]
^^^^^^^^^^^^^^^^^^^^^
['Train View', 'Training Data', u'cifar10--trained_1436616855.05', 1, u'Training of Data has started with name cifar10--trained', 100, None, False, False] 9
^^^^^^^^^^^^^^^^^^^^^
['Exp View', 'Creating Experiment', u'cifar10--prob_1436617104.52', 1, u'Creating Experiment with name cifar10--prob', 100, False, False]
['Exp View', 'Creating Experiment', u'cifar10--prob_1436617156.93', 1, u'Creating Experiment with name cifar10--prob', 100, False, False]
['/home/thilina/expresso/src/exp/extractFeatures.py', 'cifar10--prob', 'CIFAR10--TRAINED', '50', 'prob', 'train_cif']
START HERE
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0711 17:58:04.999444  2761 net.cpp:42] Initializing net from parameters:
name: "CIFAR10_quick_test"
input: "data"
input_dim: 50
input_dim: 3
input_dim: 32
input_dim: 32
state {
  phase: TEST
}
layer {
  name: "conv1"
  type: "Convolution"
  bottom: "data"
  top: "conv1"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  convolution_param {
    num_output: 32
    pad: 2
    kernel_size: 5
    stride: 1
  }
}
layer {
  name: "pool1"
  type: "Pooling"
  bottom: "conv1"
  top: "pool1"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 2
  }
}
layer {
  name: "relu1"
  type: "ReLU"
  bottom: "pool1"
  top: "pool1"
}
layer {
  name: "conv2"
  type: "Convolution"
  bottom: "pool1"
  top: "conv2"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  convolution_param {
    num_output: 32
    pad: 2
    kernel_size: 5
    stride: 1
  }
}
layer {
  name: "relu2"
  type: "ReLU"
  bottom: "conv2"
  top: "conv2"
}
layer {
  name: "pool2"
  type: "Pooling"
  bottom: "conv2"
  top: "pool2"
  pooling_param {
    pool: AVE
    kernel_size: 3
    stride: 2
  }
}
layer {
  name: "conv3"
  type: "Convolution"
  bottom: "pool2"
  top: "conv3"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  convolution_param {
    num_output: 64
    pad: 2
    kernel_size: 5
    stride: 1
  }
}
layer {
  name: "relu3"
  type: "ReLU"
  bottom: "conv3"
  top: "conv3"
}
layer {
  name: "pool3"
  type: "Pooling"
  bottom: "conv3"
  top: "pool3"
  pooling_param {
    pool: AVE
    kernel_size: 3
    stride: 2
  }
}
layer {
  name: "ip1"
  type: "InnerProduct"
  bottom: "pool3"
  top: "ip1"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  inner_product_param {
    num_output: 64
  }
}
layer {
  name: "ip2"
  type: "InnerProduct"
  bottom: "ip1"
  top: "ip2"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  inner_product_param {
    num_output: 10
  }
}
layer {
  name: "prob"
  type: "Softmax"
  bottom: "ip2"
  top: "prob"
}
I0711 17:58:05.000341  2761 net.cpp:370] Input 0 -> data
I0711 17:58:05.000394  2761 layer_factory.hpp:74] Creating layer conv1
I0711 17:58:05.000427  2761 net.cpp:90] Creating Layer conv1
I0711 17:58:05.000447  2761 net.cpp:410] conv1 <- data
I0711 17:58:05.000474  2761 net.cpp:368] conv1 -> conv1
I0711 17:58:05.000504  2761 net.cpp:120] Setting up conv1
I0711 17:58:05.000566  2761 net.cpp:127] Top shape: 50 32 32 32 (1638400)
I0711 17:58:05.000594  2761 layer_factory.hpp:74] Creating layer pool1
I0711 17:58:05.000622  2761 net.cpp:90] Creating Layer pool1
I0711 17:58:05.000643  2761 net.cpp:410] pool1 <- conv1
I0711 17:58:05.000664  2761 net.cpp:368] pool1 -> pool1
I0711 17:58:05.000689  2761 net.cpp:120] Setting up pool1
I0711 17:58:05.000723  2761 net.cpp:127] Top shape: 50 32 16 16 (409600)
I0711 17:58:05.000747  2761 layer_factory.hpp:74] Creating layer relu1
I0711 17:58:05.000771  2761 net.cpp:90] Creating Layer relu1
I0711 17:58:05.000794  2761 net.cpp:410] relu1 <- pool1
I0711 17:58:05.000818  2761 net.cpp:357] relu1 -> pool1 (in-place)
I0711 17:58:05.000839  2761 net.cpp:120] Setting up relu1
I0711 17:58:05.000865  2761 net.cpp:127] Top shape: 50 32 16 16 (409600)
I0711 17:58:05.000885  2761 layer_factory.hpp:74] Creating layer conv2
I0711 17:58:05.000906  2761 net.cpp:90] Creating Layer conv2
I0711 17:58:05.000926  2761 net.cpp:410] conv2 <- pool1
I0711 17:58:05.000949  2761 net.cpp:368] conv2 -> conv2
I0711 17:58:05.000972  2761 net.cpp:120] Setting up conv2
I0711 17:58:05.001060  2761 net.cpp:127] Top shape: 50 32 16 16 (409600)
I0711 17:58:05.001088  2761 layer_factory.hpp:74] Creating layer relu2
I0711 17:58:05.001112  2761 net.cpp:90] Creating Layer relu2
I0711 17:58:05.001132  2761 net.cpp:410] relu2 <- conv2
I0711 17:58:05.001153  2761 net.cpp:357] relu2 -> conv2 (in-place)
I0711 17:58:05.001174  2761 net.cpp:120] Setting up relu2
I0711 17:58:05.001196  2761 net.cpp:127] Top shape: 50 32 16 16 (409600)
I0711 17:58:05.001215  2761 layer_factory.hpp:74] Creating layer pool2
I0711 17:58:05.001233  2761 net.cpp:90] Creating Layer pool2
I0711 17:58:05.001245  2761 net.cpp:410] pool2 <- conv2
I0711 17:58:05.001258  2761 net.cpp:368] pool2 -> pool2
I0711 17:58:05.001272  2761 net.cpp:120] Setting up pool2
I0711 17:58:05.001287  2761 net.cpp:127] Top shape: 50 32 8 8 (102400)
I0711 17:58:05.001299  2761 layer_factory.hpp:74] Creating layer conv3
I0711 17:58:05.001312  2761 net.cpp:90] Creating Layer conv3
I0711 17:58:05.001323  2761 net.cpp:410] conv3 <- pool2
I0711 17:58:05.001335  2761 net.cpp:368] conv3 -> conv3
I0711 17:58:05.001349  2761 net.cpp:120] Setting up conv3
I0711 17:58:05.001473  2761 net.cpp:127] Top shape: 50 64 8 8 (204800)
I0711 17:58:05.001495  2761 layer_factory.hpp:74] Creating layer relu3
I0711 17:58:05.001508  2761 net.cpp:90] Creating Layer relu3
I0711 17:58:05.001520  2761 net.cpp:410] relu3 <- conv3
I0711 17:58:05.001533  2761 net.cpp:357] relu3 -> conv3 (in-place)
I0711 17:58:05.001545  2761 net.cpp:120] Setting up relu3
I0711 17:58:05.001557  2761 net.cpp:127] Top shape: 50 64 8 8 (204800)
I0711 17:58:05.001569  2761 layer_factory.hpp:74] Creating layer pool3
I0711 17:58:05.001582  2761 net.cpp:90] Creating Layer pool3
I0711 17:58:05.001593  2761 net.cpp:410] pool3 <- conv3
I0711 17:58:05.001605  2761 net.cpp:368] pool3 -> pool3
I0711 17:58:05.001618  2761 net.cpp:120] Setting up pool3
I0711 17:58:05.001632  2761 net.cpp:127] Top shape: 50 64 4 4 (51200)
I0711 17:58:05.001643  2761 layer_factory.hpp:74] Creating layer ip1
I0711 17:58:05.001658  2761 net.cpp:90] Creating Layer ip1
I0711 17:58:05.001670  2761 net.cpp:410] ip1 <- pool3
I0711 17:58:05.001682  2761 net.cpp:368] ip1 -> ip1
I0711 17:58:05.001696  2761 net.cpp:120] Setting up ip1
I0711 17:58:05.001830  2761 net.cpp:127] Top shape: 50 64 (3200)
I0711 17:58:05.001847  2761 layer_factory.hpp:74] Creating layer ip2
I0711 17:58:05.001860  2761 net.cpp:90] Creating Layer ip2
I0711 17:58:05.001871  2761 net.cpp:410] ip2 <- ip1
I0711 17:58:05.001884  2761 net.cpp:368] ip2 -> ip2
I0711 17:58:05.001899  2761 net.cpp:120] Setting up ip2
I0711 17:58:05.001917  2761 net.cpp:127] Top shape: 50 10 (500)
I0711 17:58:05.001932  2761 layer_factory.hpp:74] Creating layer prob
I0711 17:58:05.001945  2761 net.cpp:90] Creating Layer prob
I0711 17:58:05.001956  2761 net.cpp:410] prob <- ip2
I0711 17:58:05.001968  2761 net.cpp:368] prob -> prob
I0711 17:58:05.001981  2761 net.cpp:120] Setting up prob
I0711 17:58:05.001999  2761 net.cpp:127] Top shape: 50 10 (500)
I0711 17:58:05.002012  2761 net.cpp:194] prob does not need backward computation.
I0711 17:58:05.002022  2761 net.cpp:194] ip2 does not need backward computation.
I0711 17:58:05.002033  2761 net.cpp:194] ip1 does not need backward computation.
I0711 17:58:05.002043  2761 net.cpp:194] pool3 does not need backward computation.
I0711 17:58:05.002054  2761 net.cpp:194] relu3 does not need backward computation.
I0711 17:58:05.002065  2761 net.cpp:194] conv3 does not need backward computation.
I0711 17:58:05.002076  2761 net.cpp:194] pool2 does not need backward computation.
I0711 17:58:05.002087  2761 net.cpp:194] relu2 does not need backward computation.
I0711 17:58:05.002097  2761 net.cpp:194] conv2 does not need backward computation.
I0711 17:58:05.002109  2761 net.cpp:194] relu1 does not need backward computation.
I0711 17:58:05.002120  2761 net.cpp:194] pool1 does not need backward computation.
I0711 17:58:05.002130  2761 net.cpp:194] conv1 does not need backward computation.
I0711 17:58:05.002140  2761 net.cpp:235] This network produces output prob
I0711 17:58:05.002156  2761 net.cpp:482] Collecting Learning Rate and Weight Decay.
I0711 17:58:05.002171  2761 net.cpp:247] Network initialization done.
I0711 17:58:05.002182  2761 net.cpp:248] Memory required for data: 15376800
F0711 17:58:05.002238  2761 upgrade_proto.cpp:935] Check failed: ReadProtoFromBinaryFile(param_file, param) Failed to parse NetParameter file: /home/thilina/expresso/net/data/cifar10--trained/cifar10--trained_model.caffemodel
*** Check failure stack trace: ***
Aborted (core dumped)
['Exp View', 'Creating Experiment', u'cifar10--prob_1436617683.95', 2, u'Creating Experiment with name cifar10--prob', 3, False]
['Exp View', 'Creating Experiment', u'cifar10--prob_1436617683.95', 2, u'Creating Experiment with name cifar10--prob', 3, False]
['Train View', 'Training Data', u'cifar10--trained_1436616855.05', 1, u'Training of Data has started with name cifar10--trained', 100, None, False, False]
^^^^^^^^^^^^^^^^^^^^^
['Train View', 'Training Data', u'cifar10--trained_1436616855.05', 1, u'Training of Data has started with name cifar10--trained', 100, None, False, False] 9
^^^^^^^^^^^^^^^^^^^^^
['Exp View', 'Creating Experiment', u'cifar10--prob_1436617104.52', 1, u'Creating Experiment with name cifar10--prob', 100, False, False]
['Exp View', 'Creating Experiment', u'cifar10--prob_1436617156.93', 1, u'Creating Experiment with name cifar10--prob', 100, False, False]
reached here
/home/thilina/expresso/data
train_cif.hdf5
train_cif.hdf5
test_cif.hdf5
test_cif.hdf5
refreshing ExpView
/home/thilina/expresso/net/data/cifar10--trained/cifar10--trained_deploy.prototxt
FIRST
Filling the comboBox for Models
0
0
['Train View', 'Training Data', u'cifar10--trained_1436616855.05', 1, u'Training of Data has started with name cifar10--trained', 100, None, False, False]
^^^^^^^^^^^^^^^^^^^^^
['Train View', 'Training Data', u'cifar10--trained_1436616855.05', 1, u'Training of Data has started with name cifar10--trained', 100, None, False, False] 9
^^^^^^^^^^^^^^^^^^^^^
['Exp View', 'Creating Experiment', u'cifar10--prob_1436617104.52', 1, u'Creating Experiment with name cifar10--prob', 100, False, False]
['Exp View', 'Creating Experiment', u'cifar10--prob_1436617156.93', 1, u'Creating Experiment with name cifar10--prob', 100, False, False]
['Exp View', 'Creating Experiment', u'cifar10--prob_1436617683.95', 1, u'Creating Experiment with name cifar10--prob', 100, False, False]
['Train View', 'Training Data', u'cifar10--trained_1436616855.05', 1, u'Training of Data has started with name cifar10--trained', 100, None, False, False]
^^^^^^^^^^^^^^^^^^^^^
['Train View', 'Training Data', u'cifar10--trained_1436616855.05', 1, u'Training of Data has started with name cifar10--trained', 100, None, False, False] 9
^^^^^^^^^^^^^^^^^^^^^
['Exp View', 'Creating Experiment', u'cifar10--prob_1436617104.52', 1, u'Creating Experiment with name cifar10--prob', 100, False, False]
['Exp View', 'Creating Experiment', u'cifar10--prob_1436617156.93', 1, u'Creating Experiment with name cifar10--prob', 100, False, False]
['Exp View', 'Creating Experiment', u'cifar10--prob_1436617683.95', 1, u'Creating Experiment with name cifar10--prob', 100, False, False]

Jaley Dholakiya

unread,
Jul 12, 2015, 1:33:02 AM7/12/15
to express...@googlegroups.com
This is temporary Fix, for problem might have come because of variety of reasons . .. 

got to location $EXPRESSO_ROOT/src/net/train/cifar10--trained/
externally upload the model in net and save it again as separate net. . . this might help 

Павел Буров

unread,
Sep 2, 2015, 3:12:41 PM9/2/15
to expresso-users
Hi, i'm facing the same problem. except message in main window says that creating experiment in progres, but nothing happened actually (0%). Didn't get workaround posted here. Location $EXPRESSO_ROOT/src/net/ don't have folder /train. What do you mean "externally upload the model in net and save it again as separate net"?
Sorry, if my questions is silly, but i'm stuck at tutorial 3 step 17
Message has been deleted

Jaley Dholakiya

unread,
Sep 8, 2015, 9:29:36 AM9/8/15
to expresso-users
It means create a new net configuration,  and load the model from location $EXPRESSO_ROOT/src/net/train/[YourNetName]/.  I will look into it further

jkthi...@gmail.com

unread,
Oct 15, 2015, 10:13:22 AM10/15/15
to expresso-users
I have exactly the same situation. No folder 'train'- ubuntu 14.04

Jaley Dholakiya

unread,
Oct 15, 2015, 11:04:18 AM10/15/15
to expresso-users
 I will fix this bug soon , you can temporarily follow previous instructions for running . . . it is preemption issue, related to spawning multiple processes.

homer

unread,
Nov 10, 2015, 8:21:17 AM11/10/15
to expresso-users
Hi!
I am not able to follow your instructions, sorry for that. 

What I did is:

mkdir  src/net/train/cifar10
cp tutorials/tutorial_3/* src/net/train/cifar10


In the "Net View": 
Create new net, 
Load existing net configuration -> src/net/train/cifar10/cifar10_quick_train_test.prototxt
train view -> train model
net view -> loading src/net/train/cifar10/cifar10_quick.prototxt
exp view -> generate
experiment list is empty


It should be easy but honestly I do not understand the bug. What am I doing wrong? Someone was able to reproduce the workaround? 

Any help will be appreciated.

Reply all
Reply to author
Forward
0 new messages