While running the following command, the system RAM gets full and the run is killed.
[RAM: 16 GB, 1 GPU Card: 8GB]
What parameter do I need to tune to overcome this error? I tried changing --max-batch-size, --num-channels, --cuda-worker-thread, -frames-per-chunk. But none of this helped. I tried to lower the numbers to the bare minimum, still the 16GB system RAM is getting overwhelmed within seconds.
__________________________________
batched-wav-nnet3-cuda2 --cuda-use-tensor-cores=false --iterations=1 --acoustic-scale=1.0 --feature-type=mfcc --mfcc-config=exp/model/conf/mfcc.conf --add-pitch=false --ivector-extraction-config=exp/model/conf/ivector_extractor.conf --frame-subsampling-factor=3 --frames-per-chunk=384 --cuda-worker-threads=2 --gpu-feature-extract=false --use-online-features=true --max-batch-size=2 --num-channels=4 --beam=15 --lattice-beam=6.0 --word-symbol-table=exp/graph_combined/words.txt exp/model/final.mdl exp/graph_combined/HCLG.fst scp:test/conversational/wav.scp ark:exp/decode/lat.ark
...
...
LOG (batched-wav-nnet3-cuda2[5.5.907~1-cbed4]:RemoveOrphanNodes():nnet-nnet.cc:948) Removed 0 orphan nodes.
LOG (batched-wav-nnet3-cuda2[5.5.907~1-cbed4]:RemoveOrphanComponents():nnet-nnet.cc:847) Removing 0 orphan components.
LOG (batched-wav-nnet3-cuda2[5.5.907~1-cbed4]:ComputeDerivedVars():ivector-extractor.cc:183) Computing derived variables for iVector extractor
LOG (batched-wav-nnet3-cuda2[5.5.907~1-cbed4]:ComputeDerivedVars():ivector-extractor.cc:204) Done.
LOG (batched-wav-nnet3-cuda2[5.5.907~1-cbed4]:main():batched-wav-nnet3-cuda2.cc:172) Inferencing...
gpu_decoding.sh: line 20: 90584 Killed batched-wav-nnet3-cuda2 --cuda-use-tensor-cores=false --iterations=1 --acoustic-scale=1.0 --feature-type=mfcc --mfcc-config=exp/model/conf/mfcc.conf --add-pitch=false --ivector-extraction-config=exp/model/conf/ivector_extractor.conf
_________________________________________
Thanks,
CM