unable to open fasta file with add_qiime_labels

[Karen]

I'm trying to use add_qiime_labels.py to append file names and combine 4 data sets and keep getting the error below. I've received this message in qiime 1.8 on a linux system and in qiime 1.7 on a high performance computer i have access to. I've attached the mapping file which checks out with the validate script. Any idea what is going on? Thanks Karen

/home/mikem/python/bin/python

Traceback (most recent call last):

  File "/home/mikem/qiime_1_7_0/bin/add_qiime_labels.py", line 106, in <module>

    main()

  File "/home/mikem/qiime_1_7_0/bin/add_qiime_labels.py", line 102, in main

    output_dir, count_start)

  File "/home/mikem/qiime_1_7_0/lib/qiime/add_qiime_labels.py", line 44, in add_qiime_labels

    fasta_files = get_fasta_fps(fasta_dir, fasta_name_to_sample_id.keys())

  File "/home/mikem/qiime_1_7_0/lib/qiime/add_qiime_labels.py", line 121, in get_fasta_fps

    raise IOError,("Unable to open %s" % curr_fp)

IOError: Unable to open fasta/WA4Ie_CPK_11.19.2013

[Antonio González Peña]

The error you are seeing is because the file doesn't exist: IOError:
Unable to open fasta/WA4Ie_CPK_11.19.2013, could you verify this? The
easiest is to do: ls fasta/WA4Ie_CPK_11.19.2013, from the folder where
you are running the command.

> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "Qiime Forum" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to qiime-forum...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.



--
Antonio

[Jarvis, Karen]

Thanks Antonio you are correct I get a message ls: cannot access fasta/WA4Ie_CPK_11.19.2013: No such file or directory. However when I cd fasta and then ls all of the files are present. I have all of the fasta files in a directory named fasta so the script doesn't get confused with all of the other fasta files in my home directory. How do I fix this?

[Antonio González Peña]

My guess is that the file has some extension that you are omitting;
try ls and copy/paste (or use your tab key to autocomplete) to get the
correct filename. BTW as long as you give the correct name the script
will not get confused.

[Jarvis, Karen]

Antonio,
Your guess was correct. I did not realize the .fasta was necessary. Thanks so much. Karen

[Diana Nemergut]

Hey Antonio- I am having the same issue that Karen had and I'm confused as to how she resolved the problem. As far as I understand this script, the -i argument is supposed to be a directory containing fasta files. That's how I ran the script, but I got an error that one of the files (within the directory) could not be found. My error is the same as what Karen displayed above. And, yes, the individual files have .fasta extensions, but I don't know how specifying this in the command would make a difference (nor do I know how I would do this). Can you clarify? THANKS SO MUCH! Diana

[Diana Nemergut]

I figured it out- the mapping file needs to include the .fasta extension in the column as in the example shown here (http://qiime.org/scripts/add_qiime_labels.html)

[Wenshu Yap]

Hi,

I encountered the same problem and followed the same solutions suggested here yet I still receive the same problem. Can anybody help?

add_qiime_labels.py -i joined_reads/ -m mapfile2.txt -c InputFileName -n 1 -o combined_fasta

attached is my mapping file(mapfile2.txt)

Cheers.

Wenshu

[Colin Brislawn]

Hello Wenshu,

When you run that command, what is the error?

Also, you could run validate_mapping_file.py on that mapping file to make sure it is formatted correctly.

Colin

[Wenshu Yap]

Hi Colin,

Thanks for the respond.

I validated the mapping file and there is no error with the mapping file.

Initially I received the same IOError:unable to open fasta... as in this post. I tried to change the file names and now I receive different message which I don't quite understand what does it mean. 

Here is the command I run:

add_qiime_labels.py -i rawfile/joined_reads/fastqjoin/ -m mapfile2.txt -c InputFileName -n 1 -o combined_fasta

The output file is empty and the screen will have a lot of the following ( I couldn't manage to copy all, this is just partial):

GCTCACCGGAGAGATCCGGTTTCCCCGCAAGGGGCGCAGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGTGTCGTGAGATGTTGGGTTAAGTCCCGCAACGAGCGCAACCCTCGCCTTTAGTTGCCATCATTTAGTTGGGCACTCTAAAGGGACCGCCGGCGACAAGCCGGAGGAAGGTGGGGATGACGTCAAGTCCTCATGGCCCTTACACCCTGGGCTACACACGTGCTACAATGGCGGGGACAGTGGGCAGCGACCACGCAAGTGGATGCGAATCCCAAAAAACCGTCCCAGTTCAGATTGTCCTCTGCAACTCGAGGGCATGAAGGCGGAATCGCTAGTAATCGCAGAACAGCAGGCTGCGGTGAATACNTTCCCGGGCCTTNTACACACCNCCCGT', '+', 'CCC#6CDFFGGGGGFE@F@@CGGCGGGGGGGGFGGFCCEFFGFCFAFGFGGFGCFGGDFEGGGGGGGGGGGCFGCFGGCGFEGGFGCDAEFGGGGGGGGG9FGGGEGGGFGG?FF@CFFGGFGGD:FFGDFGGGDG@FFCC:FFGG9CFGGGGGGGGGGCFEGFGDGGGCE88CCGF<,<:>5CGFFGGGC?>>:CCE8EG*8EGG,8;ACEGEGFEG79,,<CDECEGGFCCGCF?CC8;ECF?7/@CFC5?CGEGD50:CGC@FEF<;C0=;EFCFFFDEGEC?8,:FGFFFGFC*GC7@F@6*F<F<3,CC?ECFGD8C8FD9E9FD>:CCC:D@,GCD7F:3,@C+C7GGDEFGGE9CGGGGGGF=FFCFB4++DFFCF>7<8GGEEFCFCFFGGGGGGFC<CFFFFDCDGGGFDDAGGGGGGGGGGGGFGFGCGGGFEFAFDA9EFFGGFGEEGGFFCGGF8FC:#FCFDFEFF@CC:#FECFGFC:#CCCCC', '@MISEQ1:77:000000000-AGEG8:1:2116:14839:20236 1:N:0:CTCTCTACAAGGAGTA', 'AAANTCAAATGAATTGACGGGGGCCCGCACAAGCAGCGGAGCGTGTGGTTTAATTCGAGGCTACACGAAGAACCTTACCTGGGTTTGACATACAGGTAGTAGGGAACCGAAAGGGGACCGACCGCAAGGAGCCTGTACAGGTGCTGCATGGCTGTCGTCAGCTCGTGCCGTGAGGTGTTCGGTTAAGTCCGCAAACGAGCGCAACCCTCGGCGTTAGTTACAAGTGTCTAACGCGACTGCCCGTGAGAAACGGGAGGAAGGTGGGGATGACGTCAAGTCAGCATGGCCTTTATATCCAGGGCTACACACACGCTACAATGGATGGTACAGTCGGTTGCGAAGCCGCGATGTGGAGCCAATCCCCAAAGCCATTCGTAGTTCGGATTGCAGGCTGCAACTCGCCTGCATGAAGTCGGAGTTGCTAGTAACCGCAGGTCAGCTATACTGCGGTGAATACNTTCCCGGGCCTTNTACACACCNCCCGT', '+', 'CCC#8DFCFGFFGFGGGG7@:=FGGGGGGEFFGGGGGGGGGGGGGGGGGFGGGGGGGGGGGGFGGGG@FGGCFGGGGGGFCGGF>FG<EFGGGGGGGGGGGGGGGGGGGCGGGGGGGGG>FGGGGGC@FGGGGGFGGG=FGGGGDGGGFGFFFFGE@FCFGGGGGGGGGGGGGGGGGDDFFCFGGFFGGGGGGGGFGGGGDGGG

Thanks.

--

---
You received this message because you are subscribed to a topic in the Google Groups "Qiime Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qiime-forum/h7hJWgLfM7A/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qiime-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Colin Brislawn]

Thanks for sending me the script you ran.

I noticed that your input that you listed after -i was a folder called fastqjoin. Are there fasta files in that folder or fastq files? (The script add_qiime_labels.py needs fasta files.)

Colin

[Wenshu Yap]

Hi Colin,

Here is the ls of rawfile/joined_reads/fastqjoin/

MacQIIME des-wsyap:fastqjoin $ ls

fastaqual       pcr3rxpb.fastq  pcrrmd.fastq    pcrrxd.fastq    pcrryp04.fastq

pcr0m.fastq     pcrr10m.fastq   pcrrms.fastq    pcrrxp04.fastq  pcrrypc.fastq

pcr112m.fastq   pcrr20m.fastq   pcrrps.fastq    pcrrxpc.fastq   pcrswalex.fastq

pcr20mst.fastq  pcrr2m.fastq    pcrrts.fastq    pcrrxu.fastq    pcrswaley.fastq

pcr2rxpu.fastq  pcrrbe.fastq    pcrrx04.fastq   pcrry04.fastq   pxrrxc.fastq

pcr2ryc.fastq   pcrrbs.fastq    pcrrxa04.fastq  pcrrya04.fastq

pcr2rypb.fastq  pcrrin.fastq    pcrrxb.fastq    pcrryb.fastq

Cheers.

Wenshu 

--

[Colin Brislawn]

Hello Wenshu,

I have found your problem! Those are .fastq files, but the script only accepts .fasta files.

You can convert each fastq file to fasta file like this:

sed -n '1~4s/^@/>/p;2~4p' pcr0m.fastq > pcr0m.fasta

After you have run that on every one of your fastq files, you can rerun add_qiime_labels.py with the mapping file listing the fasta files you just made.

Let me know if that works!

Colin

[Wenshu Yap]

Hi Colin,

Yes, you are right, I am working on fastq file instead of fasta file.

I tried with the command you provided but it doesn't work. Below is the error message that I received.

MacQIIME des-wsyap:pcr0m $ sed -n '1~4s/^@/>/p;2~4p' pcr0m.fastq > pcr0m.fasta

sed: 1: "1~4s/^@/>/p;2~4p": invalid command code ~

MacQIIME des-wsyap:pcr0m $ sed -n  pcr0m.fastq > pcr0m.fasta

sed: 1: "pcr0m.fastq": extra characters at the end of p command

MacQIIME des-wsyap:pcr0m $ sed -n pcr0m.fastq > pcr0m.fasta

sed: 1: "pcr0m.fastq": extra characters at the end of p command

MacQIIME des-wsyap:pcr0m $ sed -n pcr0m.fastq>pcr0m.fasta

sed: 1: "pcr0m.fastq": extra characters at the end of p command

I tried to search through qiime, it seem that convert_fastaqual_fastq.py -c fastq_to_fastaqual -f pcr0m.fastq -o fastaqual/pcr0m is the method to convert fastq file to fasta file but the output is .fna and .qual file.

Sorry for all the trouble, and really appreciate your patient and kind help.

Cheers.

Wenshu

[Colin Brislawn]

Good morning,

Sorry that command does not work. I forget that sed works differently on Linux and MacOSX.

Try this command:

paste - - - - < demo_head.fastq | sed 's/^@/>/g'| cut -f1-2 | tr '\t' '\n' > demo_head.fasta 

It's a bit longer, but will work on MacOSX.

Colin

[Samuel Major]

Hello, 

I am having similar troubles and trying to re-run an analysis. I went through all my files and my mapping file to make sure that the extension was correct and it all agrees (my files are .fa). This is the script I ran 

add_qiime_labels.py -m Fall_15_map2_corrected.txt  -i seqs -o 16S_qlab -c InputFileName

Traceback (most recent call last):

  File "/macqiime/anaconda/bin/add_qiime_labels.py", line 111, in <module>

    main()

  File "/macqiime/anaconda/bin/add_qiime_labels.py", line 107, in main

    output_dir, count_start)

  File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/add_qiime_labels.py", line 44, in add_qiime_labels

    fasta_files = get_fasta_fps(fasta_dir, fasta_name_to_sample_id.keys())

  File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/add_qiime_labels.py", line 123, in get_fasta_fps

    raise IOError("Unable to open %s" % curr_fp)

IOError: Unable to open seqs/

"ls" shows that the files are there and in the .fa format. I used "less" to at least show that there is data in at least one of those files, just to make sure I didn't have any empty files; that was successful. My hard drive says I can read and write on it, it's in MS-DOS(FAT) format, so this should be readable and writeable on PC or Mac. I'm not sure what my problem could be.

Thank you for your help in advance

Sam

[zhiying Guo]

Hello,

I still doubt it may be a permission-matter problem.

Whether the script (or the current user) has the permission? Can you check that?

Best,

Zhiying

[Samuel Major]

Hi Zhiying,

When I look at the folders on my windows machine, there's a black square, indicating that it is "Read-Only". So I went on the mac machines at school and they indicated that the files were "Read and Write". I received the same errors on those machines from the add_qiime_labels.py command as I did with my qiime Virtual Machine

Thank you!

Sam