This may be because you passed an incorrect value for phred_offset

[Nilusha Malmuthuge]

Hi there

I recently updated my macqiime to qiime 2. However, I am still getting the incorrect phred offset error when I do the split libraries on join paired reads. The reason for this is I have a weired extra + in the fastqjoin.join.fastq files.

@M03992:221:000000000-BGWB9:1:2107:21167:10778 1:N:0:GTATCGTCGT

+

AGAGTTTGATCCTGGCTCAGGATGAACGCTGGCGGCGTGCCTAACACATGCAAGTCGAGCGACCCCTTCGGGGTGAGCGGCGGACGGGTGAGTAACGCGTGGGTAACCTACCCATCAGAGGGGGATAACACTTGGAAACAGGTGCTAATACCGCATAAGACCACGGGGGGACATCTTCCTGAGGTAAAAGGTGCAAACCGGTTTGGGGTGGGCTCGCGTCCGATTAGATAGTTGGTGAGGTAACGGCCCACCAAGTCGACGATCGGTAGCCGACCTGAGAGGGTGATCGGCCACATTGGAACTGAGAGACGGTCCAAACTCCTACGGGAGGCAGCAGTGGGGGATATTGCACAATGGGCGCAAGCCTGATGCAGCGACGCCGCGTGAAGGAAGACGTATTTCGGTATGTAAACTTCTATCAGCGGGGAAGAAGATGACGGTACCCGACTAAGAAGCCCCGGCTAATTACGTGCCAGCCGCCGCGG

+

9@<ACGC<9<EFGFE<FF<6,CCFFCEGGEFGGE7@FGGCGGGGGGGGGGC@EGAFG,FGGG+6FGGGFGGGGGGGGGGGG7+FEGGGGGFGD<<FFGGFGGGCGGGGFGGFE,DCCCFGGGGGGGGGCEFCFGFEGG8BFFGGGGGG?:BGGC7FEGGFGFEGEDCDGGCEFGC?FGG7CFGFFF?9ECCFFCEG7C;EG+:;CCEGEF?FBFFBBEEGGCEE9<E59EFGCDF:GGFFGFFFGGFD<7?C<FBCGFFGEGFGGGGGGGGGFF?CFBGGGGGGGGGGGGFCGF3(FGGGGGFGFGGGGGGFGECFCFGFECGFGGGGCGGGGGGGGGGGGGGGGGGFGGGGGGGEGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGDGGGGGGGGFFFFGGGGGFEFGGGGGGGGGGGGEFGGGFGFB+EFF@GGGGGGEFGCF>GFGGGGFGFEFFEEFGFEFGFACCBA

@M03992:221:000000000-BGWB9:1:2107:25672:10854 1:N:0:GTATCGTCGT

+

AGAGTTTGATCCTGGCTCAGGATGAACGCTGGCGGCGTGCCTAACACATGCAAGTCGAACGGGACTATTCAGATGAACTTTTCGGAGCGATTCTTTTTAGTTTAGTGGCGGACGGGTGAGTAACGCGTGGATAACCTGCCTCTCACAGGGGGATAGCAGCTGGAAACGGCTGGTAATACCGCATACGCTCTTGAGACCGCATGGTCTTAAGAGGAAAGATGTATCGGTGAGAGATGGATCCGCGTCTGATTAGCCAGTTGGCGGGGTAACGGCCCACCAAAGCGACGATCAGTAGCCGGCCTGAGAGGGCGGACGGCCACATTGGGACTGAGACACGGCCCAGACTCCTACGGGAGGCAGCAGTGGGGAATATTGCACAATGGGGGAAACCCTGATGCAGCAACGCCGCGTGAGTGAGGAAGTATTTCGGTATGTAAAGCTCTATCAGCAGGGAAGATAATGACGGTACCTGACTAAGAAGCCCCGGCTAACTACGTGCCAGCCGCCGCGG

+

CCC<ABDF,EEACEF,@FAE8FEFFFFFG:8@C77+@FGEFEGGGCFGGGGC,EF9FCDECCF7FF?AFFFA<ACGGDFFGGAF:CCE7:=CFEFFCGCGGFGGGGGGGDGDCFCEEE7DFFFGDFGEDGFGGGFGGGE7EFGGFFCFGGCEGGGGGGFFGGGCC9F*CC*:FF?C9CCCG><C7FCEEEGCGGG59:C@C5?EEG+2<C?7EAAAE5AA'4<:7>EGGGFFE<FD>9<<EG9B?7>AB9CGC?FDF5<9CCFFFFFFFE@@>33GFG?3F@D=F<FGFGGFDDGD>GGGGGGGGGGGGGGGGEGGGGGFCFCCFFE9FGFGGEGGGGGGGGGD6GGDGGGGGGGGGFEFGGGGGGGGGGGGGGGGGFGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGCGGGGGGGGGGGGGGGGGGGGGFGGGGGGGGF?FGGEFFFDGGFFGF9GFEE9GGGGGFGCCCGGGGGGFGGGFFFB@GF@>FFACCCC

@M03992:221:000000000-BGWB9:1:2107:13552:10867 1:N:0:GTATCGTCGT

+

AGAGTTTGATCCTGGCTCAGGATGAACGCTGGCGGCGTGCCTAACACATGCAAGTCGAGCGAACCCTTCGGGGTGAGCGGCGGACGGGTGAGTAACGCGTGGGTAACCTGCCCTGTACACCCGGATAACATACCGAAAGGTATGCTAATACGAGATAAAATGTAATGATCGCATGGTCGATACATCAAAGCTCCGGCGGTACAGGATGGACCCGCGTCGGATTAGCTAGTTGGTGAGGTAATGGCTCACCAAGGCGACGATCAGTAGCCGACCTGAGAGGGTGATCGGCCACATTGGAACTGAGACACGGTCCAAACTCCTACGGGAGGCAGCAGTGGGGAATATTGCACAATGGGCGAAAGCCTGATGCAGCAACGCCGCGTGAGCGATGAAGGCCTTCGGGCCGTAAAGCTCTGTCCTCAAGGAAGATAATGACGGTACTTGAGGAGGAAGCCCCGGCTAACTACGTGCCAGCAGCCGCGG

+

@C,CACB;,CA?E@CFCFAFGGGF8FFGGC;+BCEFCF+@FFGGGFGGEDF9CEFGEBFGG@BCGGGGG?CGGGDGGGG7FEGEEE:FFFGDF9=9FEEGFGGGGGG9FBF<FFFGGFFG,@FCGG@FFG?F9FGEFDCAFCBCFGG@FGFGGGGGGGFGCFGGFGGGGGG7FF7C?EGDCG8@2CFG:E6:;@EEGGGGDEGE@G7<;?F8;8CC3=%C@37>AF:7><<DF?GFFF?=<9CFC*:GE?:DFDDFFGFF=GGGD<C6GGGFCDGEGGGGGEE;GGGGFFGFGFGGF=FC9=,GFGGGGGGGGGGED:GGGGDGGGGGGFFGFCGCFFGGGGGGGGGGFCGGGGGGEFGGGGGGFGGFGGGGGGEGGFCGGGGFDF<GGGGGGGGDGCGGGGGEGGGGGGFGGGGGGFADGGGGGGFEFCFE:GEGGFD@F<FFFFGGGFF@FC7CCGGFAEGGGGDGGFECGGFEGFCCCCC

I had the same issue when I was using macqiime with qiime version 1.9.1, which couldnot solve. So I used a linux system computer to run join_paired_ends.py script and then run all other commands on macqiime. I am not sure why am I keep getting this error even after installing qiime 2 on my mac. 

Could someone help me to figure out if something wrong with my qiime version. 

PS: I can run full qiime workflow with forward reads only. So, I beleive some how my fast_join is not working properly.

print_qiime_config.py -tf

System information

==================

         Platform: darwin

   Python version: 2.7.14 |Anaconda, Inc.| (default, Dec  7 2017, 11:07:58)  [GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)]

Python executable: /Users/nsm542/miniconda2/envs/qiime1/bin/python

QIIME default reference information

===================================

For details on what files are used as QIIME's default references, see here:

 https://github.com/biocore/qiime-default-reference/releases/tag/0.1.3

Dependency versions

===================

                QIIME library version: 1.9.1

                 QIIME script version: 1.9.1

      qiime-default-reference version: 0.1.3

                        NumPy version: 1.10.2

                        SciPy version: 1.0.0

                       pandas version: 0.22.0

                   matplotlib version: 1.4.3

                  biom-format version: 2.1.6

                         h5py version: 2.7.1 (HDF5 version: 1.10.1)

                         qcli version: 0.1.1

                         pyqi version: 0.3.2

                   scikit-bio version: 0.2.3

                       PyNAST version: 1.2.2

                      Emperor version: 0.9.51

                      burrito version: 0.9.1

             burrito-fillings version: 0.1.1

                    sortmerna version: SortMeRNA version 2.0, 29/11/2014

                    sumaclust version: Not installed.

                        swarm version: Swarm 1.2.19 [Mar  5 2016 17:12:19]

                                gdata: Installed.

RDP Classifier version (if installed): Not installed.

          Java version (if installed): 1.8.0_131

QIIME config values

===================

For definitions of these settings and to learn how to configure QIIME, see here:

 http://qiime.org/install/qiime_config.html

 http://qiime.org/tutorials/parallel_qiime.html

                     blastmat_dir: None

      pick_otus_reference_seqs_fp: /Users/nsm542/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta

                         sc_queue: all.q

      topiaryexplorer_project_dir: None

     pynast_template_alignment_fp: /Users/nsm542/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set_aligned/85_otus.pynast.fasta

                  cluster_jobs_fp: start_parallel_jobs.py

pynast_template_alignment_blastdb: None

assign_taxonomy_reference_seqs_fp: /Users/nsm542/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta

                     torque_queue: friendlyq

                    jobs_to_start: 1

                       slurm_time: None

            denoiser_min_per_core: 50

assign_taxonomy_id_to_taxonomy_fp: /Users/nsm542/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/taxonomy/97_otu_taxonomy.txt

                         temp_dir: /var/folders/v3/t8n721jj5ts8485c2snhz0hncg92mw/T/

                     slurm_memory: None

                      slurm_queue: None

                      blastall_fp: blastall

                 seconds_to_sleep: 1

QIIME full install test results

===============================

.F.FFE.FFFF.F....FF..FFF..F

======================================================================

ERROR: test_blastall_fp (__main__.QIIMEDependencyFull)

blastall_fp is set to a valid path

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/Users/nsm542/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/print_qiime_config.py", line 446, in test_blastall_fp

    blastall)

ApplicationNotFoundError: blastall_fp set to blastall, but is not in your PATH. Either use an absolute path to or put it in your PATH.

======================================================================

FAIL: test_INFERNAL_supported_version (__main__.QIIMEDependencyFull)

INFERNAL is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/Users/nsm542/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/print_qiime_config.py", line 498, in test_INFERNAL_supported_version

    "which components of QIIME you plan to use.")

AssertionError: Infernal not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_ampliconnoise_install (__main__.QIIMEDependencyFull)

AmpliconNoise install looks sane.

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/Users/nsm542/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/print_qiime_config.py", line 382, in test_ampliconnoise_install

    "$PYRO_LOOKUP_FILE variable is not set. See %s for help." % url)

AssertionError: $PYRO_LOOKUP_FILE variable is not set. See http://qiime.org/install/install.html#ampliconnoise-install-notes for help.

======================================================================

FAIL: test_blast_supported_version (__main__.QIIMEDependencyFull)

blast is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/Users/nsm542/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/print_qiime_config.py", line 456, in test_blast_supported_version

    "which components of QIIME you plan to use.")

AssertionError: blast not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_cdbtools_supported_version (__main__.QIIMEDependencyFull)

cdbtools is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/Users/nsm542/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/print_qiime_config.py", line 477, in test_cdbtools_supported_version

    "which components of QIIME you plan to use.")

AssertionError: cdbtools not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_cdhit_supported_version (__main__.QIIMEDependencyFull)

cd-hit is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/Users/nsm542/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/print_qiime_config.py", line 621, in test_cdhit_supported_version

    "which components of QIIME you plan to use.")

AssertionError: cd-hit not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_chimeraSlayer_install (__main__.QIIMEDependencyFull)

no obvious problems with ChimeraSlayer install

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/Users/nsm542/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/print_qiime_config.py", line 429, in test_chimeraSlayer_install

    self.assertTrue(chim_slay, "ChimeraSlayer was not found in your $PATH")

AssertionError: ChimeraSlayer was not found in your $PATH

======================================================================

FAIL: test_clearcut_supported_version (__main__.QIIMEDependencyFull)

clearcut is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/Users/nsm542/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/print_qiime_config.py", line 601, in test_clearcut_supported_version

    "which components of QIIME you plan to use.")

AssertionError: clearcut not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_denoiser_supported_version (__main__.QIIMEDependencyFull)

denoiser aligner is ready to use

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/Users/nsm542/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/print_qiime_config.py", line 570, in test_denoiser_supported_version

    "Denoiser flowgram aligner not found or not "

AssertionError: Denoiser flowgram aligner not found or not executable. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_mothur_supported_version (__main__.QIIMEDependencyFull)

mothur is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/Users/nsm542/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/print_qiime_config.py", line 540, in test_mothur_supported_version

    "which components of QIIME you plan to use.")

AssertionError: mothur not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_muscle_supported_version (__main__.QIIMEDependencyFull)

muscle is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/Users/nsm542/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/print_qiime_config.py", line 519, in test_muscle_supported_version

    "which components of QIIME you plan to use.")

AssertionError: muscle not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_raxmlHPC_supported_version (__main__.QIIMEDependencyFull)

raxmlHPC is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/Users/nsm542/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/print_qiime_config.py", line 580, in test_raxmlHPC_supported_version

    "which components of QIIME you plan to use.")

AssertionError: raxmlHPC not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_rtax_supported_version (__main__.QIIMEDependencyFull)

rtax is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/Users/nsm542/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/print_qiime_config.py", line 629, in test_rtax_supported_version

    "which components of QIIME you plan to use.")

AssertionError: rtax not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_sourcetracker_installed (__main__.QIIMEDependencyFull)

sourcetracker is installed

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/Users/nsm542/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/print_qiime_config.py", line 412, in test_sourcetracker_installed

    ("SOURCETRACKER_PATH is not set. This is "

AssertionError: SOURCETRACKER_PATH is not set. This is only important if you plan to use SourceTracker.

======================================================================

FAIL: test_usearch_supported_version (__main__.QIIMEDependencyFull)

usearch is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/Users/nsm542/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/print_qiime_config.py", line 650, in test_usearch_supported_version

    "which components of QIIME you plan to use.")

AssertionError: usearch not found. This may or may not be a problem depending on which components of QIIME you plan to use.

----------------------------------------------------------------------

Ran 27 tests in 0.146s

FAILED (failures=14, errors=1)

[TonyWalters]

Hello Nilusha,

I'm thinking that there may be a difference in the version of fastq-join between your linux and mac install. Do you see a difference if you use:

fastq-join

or 

fastq-join -V

on the command line?

I think someone saw the same issue here on an earlier version: https://groups.google.com/forum/#!topic/ea-utils/nR5qvhgZKIY

with this potential solution:

http://www.wernerlab.org/software/macqiime/add-fastq-join-to-macqiime-1-8-0

[Nilusha Malmuthuge]

Thank you for the reply Tony

here's what I get with fastq-join

Usage: fastq-join [options] <read1.fq> <read2.fq> [mate.fq] -o <read.0q>

Version: 1.3.1

Joins two paired-end reads on the overlapping ends.

Options:

-o FIL     See 'Output' below

-v C       Verifies that the 2 files probe id's match up to char C

            use ' ' (space) for Illumina reads

-p N       N-percent maximum difference (8)

-m N       N-minimum overlap (6)

-r FIL     Verbose stitch length report

-R         No reverse complement

-x         Allow insert < read length

Output: 

  You can supply 3 -o arguments, for un1, un2, join files, or one 

argument as a file name template.  The suffix 'un1, un2, or join' is 

appended to the file, or they replace a %-character if present.

  If a 'mate' input file is present (barcode read), then the files

'un3' and 'join2' are also created.

and I only get version information with fastq-join -V

Version: 1.3.1

I tried to add fastq-join to MacQiime with the link you send but I am still getting te same error. I wondering if I should uninstall everything and reinstall qiime and all other required programs to see if I can get this error out of my way. I am not really good with computer stuff, but I have IT support at my workplace. So, if you can give me a solution even with all the computer language I should be able to get this done

Thanks again Tony

[TonyWalters]

Is it showing version 1.3.1 for both your Mac and Linux system?

[Nilusha Malmuthuge]

Tony, I only have a mac and tried the command on mac terminal.

I am nor sure if my previous message confused you. I have macqiime and qiime 2 (both) installed in my mac. 

Thanks

[TonyWalters]

In your first post you mentioned that the fastq-join step created output that didn't have the extra + characters in the lines-I'm wanting to see if we can match up the version that is on that system and on your Mac, which will hopefully work once the software is changed.

[Nilusha Malmuthuge]

Oh I see. I used my friend's computer then. I will ask my friend to run this command on linux system and let me know and I will get back to you.

Thanks Tony

[Nilusha Malmuthuge]

Hi Tony

I just wanted to let you know that I solved the problem. I uninstall my previous version of fastq-join (which was originally in macqiime/bin) and reinstall it using conda (now it is in macqiime/anaconda). Now, I do not get that extra "+" and everything is working smoothly.

Thanks for helping to figure out the problem

Cheers!

Nilusha