NNC version 1.3 does not work with GPU (NVIDIA Quadro K6000)

46 views
Skip to first unread message

vvinewyork

unread,
Dec 6, 2018, 10:03:28 AM12/6/18
to Neural Network Console Users
After release of version 1.2 I reported the error which happened when GPU configuration was used.
The same happens with 1.3 version (the error is slightly different). Meanwhile, version 1.1 works very well.

Below is the error log:

2018-12-06 09:50:56,766 Training process is started.
python "D:\Neural_Network_Console_1.3\libs\nnabla\python\src\nnabla\utils\cli\cli.py" train
 -c "D:\Neural_Network_Console_1.3\samples\sample_project\tutorial\basics\01_logistic_regression.files\20181206_095056\net.nntxt"
 -o "D:\Neural_Network_Console_1.3\samples\sample_project\tutorial\basics\01_logistic_regression.files\20181206_095056"
2018-12-06 09:50:58,457 [nnabla]: Creating cache data for "D:\Neural_Network_Console_1.3\samples\sample_dataset\mnist\small_mnist_4or9_training.csv"
2018-12-06 09:50:59,422 [nnabla]: Creating cache data for "D:\Neural_Network_Console_1.3\samples\sample_dataset\mnist\small_mnist_4or9_test.csv"
2018-12-06 09:50:59,750 [nnabla]: An error occurred while setup of function Sigmoid (nn.SigmoidCudaCudnn) in network Main
2018-12-06 09:50:59,750 [nnabla]: Input variables:
2018-12-06 09:50:59,750 [nnabla]:   Affine (shape: (64, 1), design_shape: (64, 1))
2018-12-06 09:50:59,750 [nnabla]: Output variables:
2018-12-06 09:50:59,750 [nnabla]:   Sigmoid (shape: (64, 1), design_shape: (64, 1))
2018-12-06 09:50:59,750 [nnabla]: Network traceback:
2018-12-06 09:50:59,750 [nnabla]:   Affine
2018-12-06 09:50:59,751 [nnabla]: ->Sigmoid
NNabla command line interface (Version 1.0.8, Build 181005012621)
Traceback (most recent call last):
  File "D:\Neural_Network_Console_1.3\libs\nnabla\python\src\nnabla\utils\cli\cli.py", line 117, in cli_main
    return_value = args.func(args)
  File "D:\Neural_Network_Console_1.3\libs\nnabla\python\src\nnabla\utils\cli\train.py", line 411, in train_command
    info = load.load([args.config], exclude_parameter=True)
  File "D:\Neural_Network_Console_1.3\libs\nnabla\python\src\nnabla\utils\load.py", line 846, in load
    info.networks = _networks(proto, default_context, batch_size)
  File "D:\Neural_Network_Console_1.3\libs\nnabla\python\src\nnabla\utils\load.py", line 636, in _networks
    np, default_context, batch_size, all_variables, rng)
  File "D:\Neural_Network_Console_1.3\libs\nnabla\python\src\nnabla\utils\load.py", line 328, in _network
    network.setup(optimize=True)
  File "D:\Neural_Network_Console_1.3\libs\nnabla\python\src\nnabla\utils\network.py", line 229, in setup
    self.setup_function(func)
  File "D:\Neural_Network_Console_1.3\libs\nnabla\python\src\nnabla\utils\network.py", line 31, in setup_function
    func.variable_inputs, func.variable_outputs)
  File "nnabla\function.pyx", line 192, in nnabla.function.Function.setup (C:\ci\builds\d0a7aff5\0\nnabla\nnabla-builder\nnabla\python\src\nnabla\function.cpp:5367)
RuntimeError: target_specific error in nbla::CudnnHandleManager::handle
C:\ci\builds\17b3a23e\3\nnabla\nnabla-builder\nnabla-ext-cuda\src\nbla\cuda\cudnn\cudnn.cpp:482
Failed `status == CUDNN_STATUS_SUCCESS`: NOT_INITIALIZED
Reply all
Reply to author
Forward
0 new messages