Error: No samples remaining after main filters.

[Peiyuan Zhu]

I'm clearly keeping a subset of samples but why it says no samples remaining after main filters?

[zhupy@gra797 zhupy]$ head -10 sample_ids_NA_reduced.tsv

1000018

1000020

1000037

1000043

1000051

1000066

1000079

1000092

1000107

1000110

[zhupy@gra797 zhupy]$ wc -l sample_ids_NA_reduced.tsv

463521 sample_ids_NA_reduced.tsv

[zhupy@gra797 zhupy]$ ./plink2 --bgen ukb_imp_chr1_v3_pruned.bgen ref-first --sample ukb_imp_chr1_v3_pruned.sample --keep sample_ids_NA_reduced.tsv --export bgen-1.2 bits=8 --out ukb_imp_chr1_v3_pruned_NA_reduced

PLINK v2.00a3 32-bit (23 Sep 2020)             www.cog-genomics.org/plink/2.0/

(C) 2005-2020 Shaun Purcell, Christopher Chang   GNU General Public License v3

Logging to ukb_imp_chr1_v3_pruned_NA_reduced.log.

Options in effect:

  --bgen ukb_imp_chr1_v3_pruned.bgen ref-first

  --export bgen-1.2 bits=8

  --keep sample_ids_NA_reduced.tsv

  --out ukb_imp_chr1_v3_pruned_NA_reduced

  --sample ukb_imp_chr1_v3_pruned.sample

Start time: Tue Feb  9 19:35:45 2021

128539 MiB RAM detected; reserving 2047 MiB for main workspace.

Using up to 32 threads (change this with --threads).

--bgen: 571622 variants detected, format v1.2.

487409 samples imported from .sample file to

ukb_imp_chr1_v3_pruned_NA_reduced-temporary.psam .

--bgen: ukb_imp_chr1_v3_pruned_NA_reduced-temporary.pgen +

ukb_imp_chr1_v3_pruned_NA_reduced-temporary.pvar written.

487409 samples (264296 females, 222987 males, 126 ambiguous; 487409 founders)

loaded from ukb_imp_chr1_v3_pruned_NA_reduced-temporary.psam.

571622 variants loaded from ukb_imp_chr1_v3_pruned_NA_reduced-temporary.pvar.

Note: No phenotype data present.

--keep: 0 samples remaining.

Error: No samples remaining after main filters.

End time: Tue Feb  9 19:40:16 2021

[Christopher Chang]

1. Is the sample ID scheme in your .bgen file identical to the one in your --keep file?  If you add --keep-autoconv to your command line, what are the first 5 lines of the resulting .psam file?

2. You should almost certainly use a 64-bit build instead of a 32-bit build.  That will support larger datasets and run much faster, and you can use --memory to control memory consumption if that's an issue.

[Peiyuan Zhu]

Here're the first 5 lines of the resulting .psam file. 

[zhupy@cedar1 zhupy]$ head -5 ukb_imp_chr1_v3_pruned_NA_reduced.psam

#FID IID SEX

1389117 1389117 1

2912943 2912943 2

3094358 3094358 2

5502557 5502557 2

[Christopher Chang]

Since you have two-part .psam IDs, you need two-part --keep IDs.

[Lihle Moyakhe]

Hi Everyone, 

Just wanted to check if this issue was resolved? I am having the same issue on my end. I have used 

--keep-autoconv

I also tried adding 

--double-id

but still, keep getting the same error message. 

please help, 

[Christopher Chang]

Can't help you without at least a full .log file, and the top two lines of your --keep file.

[Peiyuan Zhu]

Hi Chris, I lost my snp ids before when I merge. Is this issue resolved in the newest update? 

[Christopher Chang]

Please explain EXACTLY what you mean by this, with enough information for someone else to reproduce what you are seeing; or, in the much more likely case that you discover that you are mischaracterizing what happened, post a comment to that effect.

[Peiyuan Zhu]

I think I used --set-all-var-ids @:#[b19]\$r,\$a before because when I didn't use it plink2 recommended me to do that otherwise it cannot be merged. I remember the reason was because I had multi-allelic variants. But I can try this again see how it works. 

[Tony Xu]

I'm having the same issue. Can someone help me to figure out what went wrong?

Here are related information:

>head -5 chunk.psam 

#IID SEX

NWD100343 NA

NWD100368 NA

NWD100720 NA

NWD100793 NA

I made a keep sample file with only 1 and first entry from psam file

>cat m.sample 

IID SEX

NWD100343 0

>plink2 --pfile chunk --keep m.sample --make-bed --out test

PLINK v2.00a2LM 64-bit Intel (22 Oct 2019)     www.cog-genomics.org/plink/2.0/

(C) 2005-2019 Shaun Purcell, Christopher Chang   GNU General Public License v3

Logging to test.log.

Options in effect:

  --keep m.sample

  --make-bed

  --out test

  --pfile chunk

Start time: Thu Dec 30 19:55:22 2021

515455 MiB RAM detected; reserving 257727 MiB for main workspace.

Using up to 56 threads (change this with --threads).

5382 samples (0 females, 0 males, 5382 ambiguous; 5382 founders) loaded from

chunk.psam.

6589 variants loaded from chunk.pvar.

Note: No phenotype data present.

--keep: 0 samples remaining.

Error: No samples remaining after main filters.

End time: Thu Dec 30 19:55:22 2021

THANK YOU!

[Christopher Chang]

The sex column is not part of the ID.

[Tony Xu]

That works. Thank you!