max() arg is an empty sequence error error for make_rare_faction_plot.py

[Bob-loblaw]

Hi all, 

I'm getting the following error when running make_rarefaction_plots.py. I was wondering if you could help.

bob@ubuntu:~/qiime$ make_rarefaction_plots.py -i ucrarecollated -m uc_mapfile.txt -o ucrareplot

Traceback (most recent call last):

  File "/usr/local/bin/make_rarefaction_plots.py", line 184, in <module>

    main()

  File "/usr/local/bin/make_rarefaction_plots.py", line 174, in main

    suppress_webpage,std_type,output_type)

  File "/usr/local/lib/python2.7/site-packages/qiime/make_rarefaction_plots.py", line 598, in make_averages

    new_ymax=(max([max(v) for v in rares_data['series'].values()])+\

ValueError: max() arg is an empty sequence

I've seen some other topics on here where this was a problem for people but those errors aren't present here to the best of my knowledge. Any would be greatly appreciated. Thanks.

[Will Van Treuren]

Hi Bob,
Can you post the contents of your input file, ucrarecollated?

Thanks,

Will

--
 
 
 

[Bob-loblaw]

Hi Will,

I actually figured it out, there was discrepancies between the naming of the samples in the mapping file and the otu table.

Thanks though,

Bob

[Daniel McDonald]

Bob, you just made my morning when I saw your name...
-Daniel

> --
>
>
>

[jenks]

I think I've had the same issue, but I'm unable to work out how I would fix the problem.
Error message was as follows:

Traceback (most recent call last):

  File "/home/jrj106/qiime_software/qiime-1.7.0-release/bin/alpha_rarefaction.py", line 151, in <module>
    main()
  File "/home/jrj106/qiime_software/qiime-1.7.0-release/bin/alpha_rarefaction.py", line 148, in main
    status_update_callback=status_update_callback)
  File "/home/jrj106/qiime_software/qiime-1.7.0-release/lib/qiime/workflow/downstream.py", line 445, in run_alpha_rarefaction
    close_logger_on_success=close_logger_on_success)
  File "/home/jrj106/qiime_software/qiime-1.7.0-release/lib/qiime/workflow/util.py", line 117, in call_commands_serially
    raise WorkflowError, msg
qiime.workflow.util.WorkflowError:

*** ERROR RAISED DURING STEP: Rarefaction plot: All metrics
Command run was:
 /home/jrj106/qiime_software/python-2.7.3-release/bin/python /home/jrj106/qiime_software/qiime-1.7.0-release/bin/make_rarefaction_plots.py -i Documents/OTUpicking/wf_arare//alpha_div_collated/ -m Mapping_test_modified.txt -o Documents/OTUpicking/wf_arare//alpha_rarefaction_plots/
Command returned exit status: 1
Stdout:

Stderr

Traceback (most recent call last):

  File "/home/jrj106/qiime_software/qiime-1.7.0-release/bin/make_rarefaction_plots.py", line 205, in <module>
    main()
  File "/home/jrj106/qiime_software/qiime-1.7.0-release/bin/make_rarefaction_plots.py", line 195, in main
    suppress_webpage,std_type,output_type)
  File "/home/jrj106/qiime_software/qiime-1.7.0-release/lib/qiime/make_rarefaction_plots.py", line 598, in make_averages

    new_ymax=(max([max(v) for v in rares_data['series'].values()])+\
ValueError: max() arg is an empty sequence

Any help you can provide would be very much appreciated.

Thanks,

Joe

[Tony Walters]

Hello Joe,

If you use convert_biom.py on your OTU table, (the one originally used as input for the alpha_rarefaction.py call) and compare those to the ones in your mapping file, do they match up? You might copy and paste the IDs into Excel, and sort the column so you can match them up.

Example convert_biom.py command: convert_biom.py -i table.biom -o table.from_biom.txt -b

--
 

---
You received this message because you are subscribed to the Google Groups "Qiime Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qiime-forum...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[jenks]

Hi Tony,

I'm slightly puzzled, is the mapping file you are referring to the same as the mapping file which is mentioned throughout the tutorial, (I've attached the mapping file I've been using).

If so, then no, nothing in the table matches ID's in the biom file are OTU ID's such as denovo0, denovo 1 etc.

Am I totally misunderstanding this?

Thanks,

Joe

[Tony Walters]

Hello Joe, 

Can you post your OTU table (or at least the top line that has the SampleIDs)?

[jenks]

The top lines are as follows:

Thanks,

Joe

[Tony Walters]

Hello Joe,

I strongly suspect that the problem is due to the IDs not matching between the OTU table and the mapping file. If Pool7-Pool12 correspond to Bean01-Bean06, you should create a mapping file with the Bean0X IDs replaced with the appropriate PoolX IDs.

[jenks]

This seems to have worked. However, will i need to rerun all previous steps using the same renamed map file?

Thanks,
Joe

[Tony Walters]

You should rerun any steps that used the OTU table and your mapping file (taxa summary plots, alpha diversity, beta diversity).

[Alexandra Alexiev]

Hi everyone, 

I've gotten the same error as above but when I compare the OTU table to the mapping file, they are exactly the same. I don't see any differences in the sample IDs at all, at least in terms of syntax. 

Here is some relevant information (the script I'm running, error, and print_qiime_config.py output):

core_diversity_analyses.py -i outputs/filtered2_otu_table_mc2_w_tax.biom -o outputs/diversity_analyses -m AQmap_minus2.txt -c Description -e 1238 -t outputs/AQUEXPphylo.tre --recover_from_failure

Traceback (most recent call last):

  File "/macqiime/QIIME/bin/core_diversity_analyses.py", line 183, in <module>

    main()    

  File "/macqiime/QIIME/bin/core_diversity_analyses.py", line 180, in main

    status_update_callback=status_update_callback)

  File "/macqiime/lib/python2.7/site-packages/qiime/workflow/core_diversity_analyses.py", line 294, in run_core_diversity_analyses

    retain_intermediate_files=False)

  File "/macqiime/lib/python2.7/site-packages/qiime/workflow/downstream.py", line 351, in run_alpha_rarefaction

    close_logger_on_success=close_logger_on_success)

  File "/macqiime/lib/python2.7/site-packages/qiime/workflow/util.py", line 116, in call_commands_serially

    raise WorkflowError, msg

qiime.workflow.util.WorkflowError: 

*** ERROR RAISED DURING STEP: Rarefaction plot: All metrics

Command run was:

 /macqiime/bin/python /macqiime/QIIME/bin//make_rarefaction_plots.py -i outputs/diversity_analyses/arare_max1238//alpha_div_collated/ -m AQmap_minus2.txt -o outputs/diversity_analyses/arare_max1238//alpha_rarefaction_plots/ 

Command returned exit status: 1

Stdout:

Stderr

Traceback (most recent call last):

  File "/macqiime/QIIME/bin//make_rarefaction_plots.py", line 216, in <module>

    main()

  File "/macqiime/QIIME/bin//make_rarefaction_plots.py", line 206, in main

    generate_average_tables = generate_average_tables)

  File "/macqiime/lib/python2.7/site-packages/qiime/make_rarefaction_plots.py", line 604, in make_averages

    new_ymax=(max([max(v) for v in rares_data['series'].values()])+\

ValueError: max() arg is an empty sequence

MacQIIME 1.8.0-20140103

System information

==================

         Platform: darwin

   Python version: 2.7.3 (default, Dec 19 2012, 09:12:08)  [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]

Python executable: /macqiime/bin/python

Dependency versions

===================

                     PyCogent version: 1.5.3

                        NumPy version: 1.7.1

                   matplotlib version: 1.1.0

                  biom-format version: 1.3.1

                         qcli version: 0.1.0

                QIIME library version: 1.8.0

                 QIIME script version: 1.8.0

        PyNAST version (if installed): 1.2.2

                      Emperor version: 0.9.3

RDP Classifier version (if installed): rdp_classifier-2.2.jar

          Java version (if installed): 1.6.0_51

QIIME config values

===================

                     blastmat_dir: None

                         sc_queue: all.q

      topiaryexplorer_project_dir: None

     pynast_template_alignment_fp: /macqiime/greengenes/core_set_aligned.fasta.imputed

                  cluster_jobs_fp: /macqiime/QIIME/bin/start_parallel_jobs.py

pynast_template_alignment_blastdb: None

assign_taxonomy_reference_seqs_fp: /macqiime/greengenes/gg_13_8_otus/rep_set/97_otus.fasta

                     torque_queue: friendlyq

   template_alignment_lanemask_fp: /macqiime/greengenes/lanemask_in_1s_and_0s

                    jobs_to_start: 1

                cloud_environment: False

                qiime_scripts_dir: /macqiime/QIIME/bin/

            denoiser_min_per_core: 50

                      working_dir: /tmp/

                    python_exe_fp: /macqiime/bin/python

                         temp_dir: /tmp/

                      blastall_fp: blastall

                 seconds_to_sleep: 60

assign_taxonomy_id_to_taxonomy_fp: /macqiime/greengenes/gg_13_8_otus/taxonomy/97_otu_taxonomy.txt

...FFFE......................F...F.

======================================================================

ERROR: test_blastall_fp (__main__.QIIMEDependencyFull)

blastall_fp is set to a valid path

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/macqiime/QIIME/bin/print_qiime_config.py", line 453, in test_blastall_fp

    raise ApplicationNotFoundError("blastall_fp set to %s, but is not in your PATH. Either use an absolute path to or put it in your PATH." % blastall)

ApplicationNotFoundError: blastall_fp set to blastall, but is not in your PATH. Either use an absolute path to or put it in your PATH.

======================================================================

FAIL: test_R_supported_version (__main__.QIIMEDependencyFull)

R is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/macqiime/QIIME/bin/print_qiime_config.py", line 722, in test_R_supported_version

    "which components of QIIME you plan to use.")

AssertionError: R not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_ampliconnoise_install (__main__.QIIMEDependencyFull)

AmpliconNoise install looks sane.

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/macqiime/QIIME/bin/print_qiime_config.py", line 392, in test_ampliconnoise_install

    "$PYRO_LOOKUP_FILE variable is not set. See %s for help." % url)

AssertionError: $PYRO_LOOKUP_FILE variable is not set. See http://qiime.org/install/install.html#ampliconnoise-install-notes for help.

======================================================================

FAIL: test_blast_supported_version (__main__.QIIMEDependencyFull)

blast is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/macqiime/QIIME/bin/print_qiime_config.py", line 463, in test_blast_supported_version

    "which components of QIIME you plan to use.")

AssertionError: blast not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_sourcetracker_installed (__main__.QIIMEDependencyFull)

sourcetracker is installed

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/macqiime/QIIME/bin/print_qiime_config.py", line 422, in test_sourcetracker_installed

    ("SOURCETRACKER_PATH is not set. This is "

AssertionError: SOURCETRACKER_PATH is not set. This is only important if you plan to use SourceTracker.

======================================================================

FAIL: test_usearch_supported_version (__main__.QIIMEDependencyFull)

usearch is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/macqiime/QIIME/bin/print_qiime_config.py", line 701, in test_usearch_supported_version

    "which components of QIIME you plan to use.")

AssertionError: usearch not found. This may or may not be a problem depending on which components of QIIME you plan to use.

----------------------------------------------------------------------

Ran 35 tests in 1.673s

FAILED (failures=5, errors=1)

[Daniel McDonald]

Hi Alexandra,

Would you be able to send on your mapping file and the collated alpha results please?

Thanks,

Daniel

--

---
You received this message because you are subscribed to the Google Groups "Qiime Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qiime-forum...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Alexandra Alexiev]

Daniel-

Here they are. 

[Daniel McDonald]

Thanks, Alexandra. Could you send "AQmap_minus2.txt" please? I want to make sure I'm looking at the exact inputs being used.

Best,

Daniel

[Alexandra Alexiev]

Here it is.

Thanks,

Alex

[Alexandra Alexiev]

Someone else in my lab actually re-ran the workflow we had on their computer and it worked, so I'm wondering if there was simply some syntax error earlier in my workflow.

[Daniel McDonald]

Hi Alexandra,

Tony had a chance to peak at the files and pointed out that the IDs don't overlap (thanks Tony!). For instance, Day.13.H2O.Pond.2 is in the chao1.txt file you sent but it is not listed as a SampleID in the AQmap_minus2.txt mapping file. This is likely the source of the issue as the script is expecting the mapping file to correspond to the alpha diversity results. Perhaps the other lab member used a slightly different mapping file?

Best,

Daniel

[ibis]

Hello,

I ve been getting the same error reported here. I ve checked my mapping file and converted OTU table and I dont see any differences in the ID names in the two files. I vetried many times, with more than one indexes, just in case, but cant seem to figure out what's wrong.
I run it using the following:

multiple_rarefactions.py -i otu_table.biom -m 1000 -x 8000 -s 500 -o rarefied_otu_tables/
alpha_diversity.py -i rarefied_otu_tables/ -m shannon -o adiv_shannon/;
collate_alpha.py -i adiv_shannon/ -o collated_adiv_shannon/;
make_rarefaction_plots.py -i collated_adiv_shannon/ -m mapping_file.txt -o rare_plots/;

and got this error:

Traceback (most recent call last):

  File "/app/genomics/qiime/1.8.0/gnu-4.8.0/qiime-1.8.0-release/bin/make_rarefaction_plots.py", line 216, in <module>
    main()
  File "/app/genomics/qiime/1.8.0/gnu-4.8.0/qiime-1.8.0-release/bin/make_rarefaction_plots.py", line 206, in main
    generate_average_tables = generate_average_tables)
  File "/app/genomics/qiime/1.8.0/gnu-4.8.0/qiime-1.8.0-release/lib/qiime/make_rarefaction_plots.py", line 604, in make_aver                     ages

    new_ymax=(max([max(v) for v in rares_data['series'].values()])+\
ValueError: max() arg is an empty sequence

Any ideas why it is still not working please?
I m attaching the mapping file and first lines of the converted OTU table if it helps.
Many thanks,

ibis

[Tony Walters]

Hello,

I think the error is happening because the SampleIDs in the OTU table do not match those in the mapping file. E.g.:

58-15-wcoim in your OTU table, 58.15.wcoim in your mapping file.

I would change the IDs in your table to match the mapping file (replace the dashes with periods for the SampleIDs) and convert that file to biom, and see if that fixes the issue for you.

[ibis]

Thanks Tony,

this seems to have worked very well. I thought that using (.) would correspond well with the ( _ ) in the OTU table, but obviously it didnt..

ibis