--sample command - Unable to find sample as specified in the population map

[helen....@otago.ac.nz]

Hi, I've seen this posted before, but not with an answer that has solved my issue. I am running denovo_map.pl and trying to use the --samples option to avoid having to list out each of my 120 samples with -s. When I run the wrapper script using the --samples command (see whole script at end of message), I get the error message:

Unable to find sample 'B104599-AGAACTAGA.fq.gz' in directory './samples_by_barcode_strict_nonbam

But the samples are all right there in that directory, listed as follows:

B104599-AGAACTAGA.fq.gz  B109694-GAATTAC.fq.gz    B121095-CCGACTA.fq.gz    B122332-ACCAACGTA.fq.gz  B122362-GACACAT.fq.gz

B104930-CTTGGACT.fq.gz   B109695-TAAGACAA.fq.gz   B121096-TTCGTAT.fq.gz    B122333-GCCAATA.fq.gz    B122363-AGTGTCGGT.fq.gz

B104957-TGAACACAA.fq.gz  B109696-GATCCGT.fq.gz    B121099-TGTTGTCTA.fq.gz  B122334-AGAATCCA.fq.gz   B122364-AACAGAATA.fq.gz

and so on for 120 samples.

I can't see any difference between how this is written in my popmap file vs in the file name itself.

Previous answers to this topic have been that the user hadn't specified the file extension in their individual names in popmap, and an extra blank line in the popmap file. I've checked both these things and they're not the problem.  

We do have a workaround script to avoid having to put all the individual names into the denovo_map.pl wrapper script with -s, but it would be really nice to just be able to use the wrapper script as intended.

Anyone have any ideas how to fix this?

Thanks!

Helen

The script I'm trying to run is as follows:

denovo_map.pl -S -m 3 -M 2 -n 2 -b 1 -T 20 -o ./stacks_out_samples_by_barcode_strict_denovo -O ./popmap2.txt --samples ./samples_by_barcode_strict_nonbam -X "populations:--vcf --plink"

[Bautisse Postaire]

Hi Helen

I am totally new to stacks and I have the same problem. While looking for answers, it seems that adding the extension  " .txt " to your popmap file may create an error.

I tried the three recommended troubleshooting steps (i.e. no extension, no additional line, specifying the extension) but denovo_map.pl still doesn't validate the correspondance between my popmap and my fasta files (the complete extension I have is XX.fastq.fas). Tell me if you find a solution!

Thanks

Bautisse

[Helen Taylor]

Hi Bautisse,

I think I have found the answer (for me, at lest). I got an email from another user the other day the suggested the following:

"If I understand your message correctly, your population map lists "B104599-AGAACTAGA.fq.gz" as a sample. Assuming your sequences files are named NAME.fq.gz, the population map should only contain the sample names, and no suffixes. Otherwise the program will treat the extension as part of the name and will look for "B104599-AGAACTAGA.fq.gz.fq.gz" (with the extension twice). Does this answer your question?”

So, basically, there was an issue because I had file extension names as part of the sample name within my pop map file. I’ve now changed this and managed to get the command to run. My pop map file does have a .txt extension and that doesn’t seem to be causing a problem.  Hopefully this will help you out.

Good luck!

Helen.

--
Stacks website: http://catchenlab.life.illinois.edu/stacks/
---
You received this message because you are subscribed to a topic in the Google Groups "Stacks" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stacks-users/NyE0Rwu4s-s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stacks-users...@googlegroups.com.
Visit this group at https://groups.google.com/group/stacks-users.
For more options, visit https://groups.google.com/d/optout.

[Bautisse Postaire]

Hi Helen

Thanks for your quick update

I tried your solution and it still doesn't work for me. Could it be the name of my extensions, like .fq is recognized but not .fastq?

Helen.

To unsubscribe from this group and all its topics, send an email to stacks-user...@googlegroups.com.

[Helen Taylor]

Hi Bautisse,

I just looked at the results of my run and, although it has completed, it has only considered the first sample listed in the popmap and ignored the other 118. So still some more trouble shooting to do here clearly. :(

H

To unsubscribe from this group and all its topics, send an email to stacks-users...@googlegroups.com.

[Nicolas Rochette]

Hi Helen,

Have you been able to solve this issue? If no, could you copy paste the
denovo_map.pl/ref_map.pl command you used and the first line of your
population map?

Best,
Nicolas

[helen....@otago.ac.nz]

Hi Nicolas,

No, not as yet. The denovo_map.pl command I used is:

/usr/local/stacks-1.45/bin/denovo_map.pl -S -m 3 -M 2 -n 2 -b 1 -T 20 -o ./stacks_out_samples_by_barcode_strict_denovo -O ./popmap3nobs.txt --samples ./samples_by_barcode_strict_nonbam -X "populations:--vcf --plink" > denovo_map.out 2> denovo_map.se 3> populations_denovo.out 4> populations_denovo.se &

The first few lines of my popmap look like this:

B104599-AGAACTAGA Kai

B104930-CTTGGACT Kai

B104957-TGAACACAA Kai

B109462-TTGTTCTTA Kai

B109466-GGTCGAGA Nuku

B109467-CCAAGTGA Nuku

B109469-CCTACCAGT Nuku

Thanks,

Helen.

[Nicolas Rochette]

After a couple private exchanges, it appeared that this was a text/invisibile characters issue. Newlines in the population map were not as expected.

You can check invisible characters with:

head popmap.tsv | od -c

Nicolas

[helen....@otago.ac.nz]

Yes - Stacks de novo now seems to be running. Thanks to Nicolas for his help. Worth noting that my popmap files looked fine in text editor, but had hidden characters that I couldn't see unless I opened the files in terminal. These came up as ^M in vim or /r using the check that Nicolas suggests below. Unsure why some of my .csv fles read in fine and others have hidden characters. Will continue to look into this and post if I find the answer in case it's an issue for others.

Helen