"Segmentation fault" errors when running test_supervised_learning.py

[Elena Perry]

Hi,

I have QIIME 1.9.1 running on a Mac where I also have R version 3.2.2. I have been trying to run supervised_learning.py on some microbiome data, but I keep getting a "segmentation fault" error. I was able to recreate the same error by running test_supervised_learning.py. The command and full error output from test_supervised_learning.py are below, after the output from print_qiime_config.py. Any ideas what is causing this problem? Is something going wrong within the python script in QIIME or is it a problem with R? My randomForest package version in R is 4.6.12. Also, my current R library pathway was manually stored in the environment variable R_LIBRARY_PATH.

Output from print_qiime_config.py -t:

System information
==================
         Platform:    darwin
   Python version:    2.7.10 |Anaconda 2.2.0 (x86_64)| (default, May 28 2015, 17:04:42)  [GCC 4.2.1 (Apple Inc. build 5577)]
Python executable:    /macqiime/anaconda/bin/python

QIIME default reference information
===================================
For details on what files are used as QIIME's default references, see here:
 https://github.com/biocore/qiime-default-reference/releases/tag/0.1.2

Dependency versions
===================
          QIIME library version:    1.9.1
           QIIME script version:    1.9.1
qiime-default-reference version:    0.1.2
                  NumPy version:    1.9.2
                  SciPy version:    0.15.1
                 pandas version:    0.16.1
             matplotlib version:    1.4.3
            biom-format version:    2.1.4
                   h5py version:    2.4.0 (HDF5 version: 1.8.14)
                   qcli version:    0.1.1
                   pyqi version:    0.3.2
             scikit-bio version:    0.2.3
                 PyNAST version:    1.2.2
                Emperor version:    0.9.51
                burrito version:    0.9.1
       burrito-fillings version:    0.1.1
              sortmerna version:    SortMeRNA version 2.0, 29/11/2014
              sumaclust version:    SUMACLUST Version 1.0.00
                  swarm version:    Swarm 1.2.19 [Jun  2 2015 14:40:16]
                          gdata:    Installed.

QIIME config values
===================
For definitions of these settings and to learn how to configure QIIME, see here:
 http://qiime.org/install/qiime_config.html
 http://qiime.org/tutorials/parallel_qiime.html

                     blastmat_dir:    None
      pick_otus_reference_seqs_fp:    /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta
                         sc_queue:    all.q
      topiaryexplorer_project_dir:    None
     pynast_template_alignment_fp:    /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set_aligned/85_otus.pynast.fasta
                  cluster_jobs_fp:    start_parallel_jobs.py
pynast_template_alignment_blastdb:    None
assign_taxonomy_reference_seqs_fp:    /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta
                     torque_queue:    friendlyq
                    jobs_to_start:    1
                       slurm_time:    None
            denoiser_min_per_core:    50
assign_taxonomy_id_to_taxonomy_fp:    /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/taxonomy/97_otu_taxonomy.txt
                         temp_dir:    /tmp/
                     slurm_memory:    None
                      slurm_queue:    None
                      blastall_fp:    blastall
                 seconds_to_sleep:    60

QIIME base install test results
===============================
.........
----------------------------------------------------------------------
Ran 9 tests in 0.028s

OK


Full error output from test_supervised_learning.py:

MacQIIME Elenas-MacBook-Pro:macqiime $ cd ./QIIME/tests
MacQIIME Elenas-MacBook-Pro:tests $ python test_supervised_learning.py
..sh: line 1: 40423 Segmentation fault: 11  R --slave --no-restore --args --source_dir /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R -e oob -c Individual -m "/tmp/R_test_map_SWB9il.txt" -o "/tmp/tmpR3JF2A" -n 100 -i "/tmp/tmpR3JF2A/R_test_otu_table_mjZJGA.txt" < /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R/randomforests.r > "/tmp/tmp3kvPMu65ZE6g1hG6OLV5.txt" 2> "/tmp/tmpkFS5Y1gng3NcCf4U5yy4.txt"
Esh: line 1: 40427 Segmentation fault: 11  R --slave --no-restore --args --source_dir /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R -e oob -c Individual -m "/tmp/R_test_map_CaIqLU.txt" -o "/tmp/tmpg75SAf" -n 100 -i "/tmp/tmpg75SAf/R_test_otu_table_Izl4ab.txt" < /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R/randomforests.r > "/tmp/tmp9KfyuH3S77BbHc0xyeQS.txt" 2> "/tmp/tmpeG0RYi5wyrP5OKC0R1Xe.txt"
Esh: line 1: 40430 Segmentation fault: 11  R --slave --no-restore --args --source_dir /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R -e oob -c Individual -m "/tmp/R_test_map_KX4RK6.txt" -o "/tmp/tmpiYbM72" -n 100 -i "/tmp/tmpiYbM72/R_test_otu_table_tTzZqL.txt" < /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R/randomforests.r > "/tmp/tmphcxolrvrsGowGewx4YUx.txt" 2> "/tmp/tmpI2zoW6w0RUlgmDoYfGVm.txt"
Esh: line 1: 40433 Segmentation fault: 11  R --slave --no-restore --args --source_dir /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R -e oob -c Individual -m "/tmp/R_test_map_ZkxH2f.txt" -o "/tmp/tmpdMEuI9" -n 100 -i "/tmp/tmpdMEuI9/R_test_otu_table_lJh0eo.txt" < /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R/randomforests.r > "/tmp/tmpQVLRGPvy2SyzQe3Z465Q.txt" 2> "/tmp/tmpKiQMmfWPeBqQPcWmV0qv.txt"
E
======================================================================
ERROR: test_cv_probabilities_format (__main__.RSupervisedLearnerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_supervised_learning.py", line 99, in setUp
    output_dir=output_dir)
  File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/supervised_learning.py", line 64, in run_supervised_learning
    app_result = rsl(predictor_fp)
  File "/macqiime/anaconda/lib/python2.7/site-packages/cogent/app/util.py", line 251, in __call__
    open(errfile).read())
ApplicationError: Unacceptable application exit status: 139
Command:
cd "/macqiime/QIIME/tests/"; R --slave --no-restore --args --source_dir /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R -e oob -c Individual -m "/tmp/R_test_map_SWB9il.txt" -o "/tmp/tmpR3JF2A" -n 100 -i "/tmp/tmpR3JF2A/R_test_otu_table_mjZJGA.txt" < /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R/randomforests.r > "/tmp/tmp3kvPMu65ZE6g1hG6OLV5.txt" 2> "/tmp/tmpkFS5Y1gng3NcCf4U5yy4.txt"
StdOut:

StdErr:

 *** caught segfault ***
address 0x18, cause 'memory not mapped'

Traceback:
 1: .C("classRF", x = x, xdim = as.integer(c(p, n)), y = as.integer(y),     nclass = as.integer(nclass), ncat = as.integer(ncat), maxcat = as.integer(maxcat),     sampsize = as.integer(sampsize), strata = if (Stratify) as.integer(strata) else integer(1),     Options = as.integer(c(addclass, importance, localImp, proximity,         oob.prox, do.trace, keep.forest, replace, Stratify, keep.inbag)),     ntree = as.integer(ntree), mtry = as.integer(mtry), ipi = as.integer(ipi),     classwt = as.double(cwt), cutoff = as.double(threshold),     nodesize = as.integer(nodesize), outcl = integer(nsample),     counttr = integer(nclass * nsample), prox = prox, impout = impout,     impSD = impSD, impmat = impmat, nrnodes = as.integer(nrnodes),     ndbigtree = integer(ntree), nodestatus = integer(nt * nrnodes),     bestvar = integer(nt * nrnodes), treemap = integer(nt * 2 *         nrnodes), nodepred = integer(nt * nrnodes), xbestsplit = double(nt *         nrnodes), errtr = double((nclass + 1) * ntree), testdat = as.integer(testdat),     xts = as.double(xtest), clts = as.integer(ytest), nts = as.integer(ntest),     countts = double(nclass * ntest), outclts = as.integer(numeric(ntest)),     labelts = as.integer(labelts), proxts = proxts, errts = error.test,     inbag = if (keep.inbag) matrix(integer(n * ntree), n) else integer(n),     DUP = FALSE, PACKAGE = "randomForest")
 2: randomForest.default(x, y, keep.inbag = TRUE, importance = TRUE,     do.trace = verbose, ...)
 3: randomForest(x, y, keep.inbag = TRUE, importance = TRUE, do.trace = verbose,     ...)
 4: rf.out.of.bag(x[ix, , drop = FALSE], y[ix], verbose = opts$verbose,     ntree = opts$ntree)
An irrecoverable exception occurred. R is aborting now ...



======================================================================
ERROR: test_features_format (__main__.RSupervisedLearnerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_supervised_learning.py", line 99, in setUp
    output_dir=output_dir)
  File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/supervised_learning.py", line 64, in run_supervised_learning
    app_result = rsl(predictor_fp)
  File "/macqiime/anaconda/lib/python2.7/site-packages/cogent/app/util.py", line 251, in __call__
    open(errfile).read())
ApplicationError: Unacceptable application exit status: 139
Command:
cd "/macqiime/QIIME/tests/"; R --slave --no-restore --args --source_dir /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R -e oob -c Individual -m "/tmp/R_test_map_CaIqLU.txt" -o "/tmp/tmpg75SAf" -n 100 -i "/tmp/tmpg75SAf/R_test_otu_table_Izl4ab.txt" < /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R/randomforests.r > "/tmp/tmp9KfyuH3S77BbHc0xyeQS.txt" 2> "/tmp/tmpeG0RYi5wyrP5OKC0R1Xe.txt"
StdOut:

StdErr:

 *** caught segfault ***
address 0x18, cause 'memory not mapped'

Traceback:
 1: .C("classRF", x = x, xdim = as.integer(c(p, n)), y = as.integer(y),     nclass = as.integer(nclass), ncat = as.integer(ncat), maxcat = as.integer(maxcat),     sampsize = as.integer(sampsize), strata = if (Stratify) as.integer(strata) else integer(1),     Options = as.integer(c(addclass, importance, localImp, proximity,         oob.prox, do.trace, keep.forest, replace, Stratify, keep.inbag)),     ntree = as.integer(ntree), mtry = as.integer(mtry), ipi = as.integer(ipi),     classwt = as.double(cwt), cutoff = as.double(threshold),     nodesize = as.integer(nodesize), outcl = integer(nsample),     counttr = integer(nclass * nsample), prox = prox, impout = impout,     impSD = impSD, impmat = impmat, nrnodes = as.integer(nrnodes),     ndbigtree = integer(ntree), nodestatus = integer(nt * nrnodes),     bestvar = integer(nt * nrnodes), treemap = integer(nt * 2 *         nrnodes), nodepred = integer(nt * nrnodes), xbestsplit = double(nt *         nrnodes), errtr = double((nclass + 1) * ntree), testdat = as.integer(testdat),     xts = as.double(xtest), clts = as.integer(ytest), nts = as.integer(ntest),     countts = double(nclass * ntest), outclts = as.integer(numeric(ntest)),     labelts = as.integer(labelts), proxts = proxts, errts = error.test,     inbag = if (keep.inbag) matrix(integer(n * ntree), n) else integer(n),     DUP = FALSE, PACKAGE = "randomForest")
 2: randomForest.default(x, y, keep.inbag = TRUE, importance = TRUE,     do.trace = verbose, ...)
 3: randomForest(x, y, keep.inbag = TRUE, importance = TRUE, do.trace = verbose,     ...)
 4: rf.out.of.bag(x[ix, , drop = FALSE], y[ix], verbose = opts$verbose,     ntree = opts$ntree)
An irrecoverable exception occurred. R is aborting now ...



======================================================================
ERROR: test_mislabeling_format (__main__.RSupervisedLearnerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_supervised_learning.py", line 99, in setUp
    output_dir=output_dir)
  File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/supervised_learning.py", line 64, in run_supervised_learning
    app_result = rsl(predictor_fp)
  File "/macqiime/anaconda/lib/python2.7/site-packages/cogent/app/util.py", line 251, in __call__
    open(errfile).read())
ApplicationError: Unacceptable application exit status: 139
Command:
cd "/macqiime/QIIME/tests/"; R --slave --no-restore --args --source_dir /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R -e oob -c Individual -m "/tmp/R_test_map_KX4RK6.txt" -o "/tmp/tmpiYbM72" -n 100 -i "/tmp/tmpiYbM72/R_test_otu_table_tTzZqL.txt" < /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R/randomforests.r > "/tmp/tmphcxolrvrsGowGewx4YUx.txt" 2> "/tmp/tmpI2zoW6w0RUlgmDoYfGVm.txt"
StdOut:

StdErr:

 *** caught segfault ***
address 0x18, cause 'memory not mapped'

Traceback:
 1: .C("classRF", x = x, xdim = as.integer(c(p, n)), y = as.integer(y),     nclass = as.integer(nclass), ncat = as.integer(ncat), maxcat = as.integer(maxcat),     sampsize = as.integer(sampsize), strata = if (Stratify) as.integer(strata) else integer(1),     Options = as.integer(c(addclass, importance, localImp, proximity,         oob.prox, do.trace, keep.forest, replace, Stratify, keep.inbag)),     ntree = as.integer(ntree), mtry = as.integer(mtry), ipi = as.integer(ipi),     classwt = as.double(cwt), cutoff = as.double(threshold),     nodesize = as.integer(nodesize), outcl = integer(nsample),     counttr = integer(nclass * nsample), prox = prox, impout = impout,     impSD = impSD, impmat = impmat, nrnodes = as.integer(nrnodes),     ndbigtree = integer(ntree), nodestatus = integer(nt * nrnodes),     bestvar = integer(nt * nrnodes), treemap = integer(nt * 2 *         nrnodes), nodepred = integer(nt * nrnodes), xbestsplit = double(nt *         nrnodes), errtr = double((nclass + 1) * ntree), testdat = as.integer(testdat),     xts = as.double(xtest), clts = as.integer(ytest), nts = as.integer(ntest),     countts = double(nclass * ntest), outclts = as.integer(numeric(ntest)),     labelts = as.integer(labelts), proxts = proxts, errts = error.test,     inbag = if (keep.inbag) matrix(integer(n * ntree), n) else integer(n),     DUP = FALSE, PACKAGE = "randomForest")
 2: randomForest.default(x, y, keep.inbag = TRUE, importance = TRUE,     do.trace = verbose, ...)
 3: randomForest(x, y, keep.inbag = TRUE, importance = TRUE, do.trace = verbose,     ...)
 4: rf.out.of.bag(x[ix, , drop = FALSE], y[ix], verbose = opts$verbose,     ntree = opts$ntree)
An irrecoverable exception occurred. R is aborting now ...



======================================================================
ERROR: test_summary_file (__main__.RSupervisedLearnerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_supervised_learning.py", line 99, in setUp
    output_dir=output_dir)
  File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/supervised_learning.py", line 64, in run_supervised_learning
    app_result = rsl(predictor_fp)
  File "/macqiime/anaconda/lib/python2.7/site-packages/cogent/app/util.py", line 251, in __call__
    open(errfile).read())
ApplicationError: Unacceptable application exit status: 139
Command:
cd "/macqiime/QIIME/tests/"; R --slave --no-restore --args --source_dir /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R -e oob -c Individual -m "/tmp/R_test_map_ZkxH2f.txt" -o "/tmp/tmpdMEuI9" -n 100 -i "/tmp/tmpdMEuI9/R_test_otu_table_lJh0eo.txt" < /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R/randomforests.r > "/tmp/tmpQVLRGPvy2SyzQe3Z465Q.txt" 2> "/tmp/tmpKiQMmfWPeBqQPcWmV0qv.txt"
StdOut:

StdErr:

 *** caught segfault ***
address 0x18, cause 'memory not mapped'

Traceback:
 1: .C("classRF", x = x, xdim = as.integer(c(p, n)), y = as.integer(y),     nclass = as.integer(nclass), ncat = as.integer(ncat), maxcat = as.integer(maxcat),     sampsize = as.integer(sampsize), strata = if (Stratify) as.integer(strata) else integer(1),     Options = as.integer(c(addclass, importance, localImp, proximity,         oob.prox, do.trace, keep.forest, replace, Stratify, keep.inbag)),     ntree = as.integer(ntree), mtry = as.integer(mtry), ipi = as.integer(ipi),     classwt = as.double(cwt), cutoff = as.double(threshold),     nodesize = as.integer(nodesize), outcl = integer(nsample),     counttr = integer(nclass * nsample), prox = prox, impout = impout,     impSD = impSD, impmat = impmat, nrnodes = as.integer(nrnodes),     ndbigtree = integer(ntree), nodestatus = integer(nt * nrnodes),     bestvar = integer(nt * nrnodes), treemap = integer(nt * 2 *         nrnodes), nodepred = integer(nt * nrnodes), xbestsplit = double(nt *         nrnodes), errtr = double((nclass + 1) * ntree), testdat = as.integer(testdat),     xts = as.double(xtest), clts = as.integer(ytest), nts = as.integer(ntest),     countts = double(nclass * ntest), outclts = as.integer(numeric(ntest)),     labelts = as.integer(labelts), proxts = proxts, errts = error.test,     inbag = if (keep.inbag) matrix(integer(n * ntree), n) else integer(n),     DUP = FALSE, PACKAGE = "randomForest")
 2: randomForest.default(x, y, keep.inbag = TRUE, importance = TRUE,     do.trace = verbose, ...)
 3: randomForest(x, y, keep.inbag = TRUE, importance = TRUE, do.trace = verbose,     ...)
 4: rf.out.of.bag(x[ix, , drop = FALSE], y[ix], verbose = opts$verbose,     ntree = opts$ntree)
An irrecoverable exception occurred. R is aborting now ...



----------------------------------------------------------------------
Ran 6 tests in 2.514s

FAILED (errors=4)

[Yoshiki Vázquez Baeza]

Hello Elena,

Can you try reinstalling the dependencies for supervised_learning.py and running the tests again? This looks more like a problem in the R side of things, but do let us know what you find after reinstalling.

Thanks!

Yoshiki.