Error running pick_closed_reference_otus.py

[Sergey Kolchenko]

H!

I have qiime installed on a sge cluster (every node has an access to /shared folder) (I used https://gist.github.com/alanorth/100dd95f809223ae0100 as an example). So far everything in my code was running nice but when I got to:

pick_closed_reference_otus.py -a -O 13 -i seqs_rarified/all_seqs.fna -r "/shared/qiime_reference/qiime-default-reference-master/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta" -t "/shared/qiime_reference/qiime-default-reference-master/qiime_default_reference/gg_13_8_otus/taxonomy/97_otu_taxonomy.txt" -p "cnf/pick_closed_reference_otus.properties" -o otu_rarified

I got an error:

Traceback (most recent call last):
  File "/shared/apps/qiime/1.8.0/bin/pick_closed_reference_otus.py", line 144, in <module>
    main()
  File "/shared/apps/qiime/1.8.0/bin/pick_closed_reference_otus.py", line 140, in main
    status_update_callback=status_update_callback)
  File "/shared/apps/qiime/1.8.0/lib/qiime/workflow/upstream.py", line 442, in run_pick_closed_reference_otus
    close_logger_on_success=close_logger_on_success)
  File "/shared/apps/qiime/1.8.0/lib/qiime/workflow/util.py", line 116, in call_commands_serially
    raise WorkflowError, msg
qiime.workflow.util.WorkflowError:

*** ERROR RAISED DURING STEP: Pick OTUs
Command run was:
 python /shared/apps/qiime/1.8.0/lib/scripts/parallel_pick_otus_uclust_ref.py -i seqs_rarified/all_seqs.fna -o otu_rarified/uclust_ref_picked_otus -r /shared/qiime_reference/qiime-default-reference-master/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta -T --jobs_to_start 13 --enable_rev_strand_match
Command returned exit status: 2
Stdout:

Stderr
python: can't open file '/shared/apps/qiime/1.8.0/lib/scripts/parallel_pick_otus_uclust_ref.py': [Errno 2] No such file or directory

I checked /shared/apps/qiime/1.8.0/lib/ but there is no /scripts folder!

my qiime config:

System information
==================
         Platform:    linux2
   Python version:    2.7.3 (default, Nov 26 2015, 16:12:51)  [GCC 4.8.3 20140911 (Red Hat 4.8.3-9)]
Python executable:    /shared/apps/qiime/1.8.0/bin/python

Dependency versions
===================
                     PyCogent version:    1.5.3
                        NumPy version:    1.7.1
                   matplotlib version:    1.3.1
                  biom-format version:    1.3.1
                         qcli version:    0.1.0
                QIIME library version:    1.8.0
                 QIIME script version:    1.8.0
        PyNAST version (if installed):    1.2.2
                      Emperor version:    0.9.3
RDP Classifier version (if installed):    Not installed.
          Java version (if installed):    Not installed.

QIIME config values
===================
                     blastmat_dir:    None
                         sc_queue:    all.q
      topiaryexplorer_project_dir:    None
     pynast_template_alignment_fp:    None
                  cluster_jobs_fp:    None
pynast_template_alignment_blastdb:    None
assign_taxonomy_reference_seqs_fp:    None
                     torque_queue:    friendlyq
   template_alignment_lanemask_fp:    None
                    jobs_to_start:    1
                cloud_environment:    False
                qiime_scripts_dir:    None
            denoiser_min_per_core:    50
                      working_dir:    None
                    python_exe_fp:    python
                         temp_dir:    /tmp/
                      blastall_fp:    blastall
                 seconds_to_sleep:    2
assign_taxonomy_id_to_taxonomy_fp:    None


How can I fix it?

[Jenya Kopylov]

Hi Sergey,

Have you checked that your QIIME scripts path (in $PATH) and libraries path ($PYTHONPATH) have been setup correctly?

For example, in the github link you gave, installing QIIME itself required the step:

$ python setup.py install --install-scripts=/export/apps/qiime/1.8.0/bin --install-purelib=/export/apps/qiime/1.8.0/lib

If that's what you ran, then your path for --install-scripts should be in $PATH and the path for --install-purelib in $PYTHONPATH.

Do you also get an error with

$ print_qiime_config.py

? 

Thanks,

Jenya

[Sergey Kolchenko]

No, I don't get any errors if I run  print_qiime_config.py or beta_diversity.py or any qiime python file.

I have done the installation again, just to make sure I haven't missed

$ python setup.py install --install-scripts=/export/apps/qiime/1.8.0/bin --install-purelib=/export/apps/qiime/1.8.0/lib

and I still don't have /lib/scripts folder.
Oh, I get that. I dont have /shared/apps/qiime/1.8.0/lib/scripts in my $PATH but qiime is looking for them

UPD: Problem solved, I copied /bin folder to /lib/scripts and added /lib/scripts to $PATH