Permutation using --perm but did not get empirical P-value in output

[XY Jia]

Hi Sam,

I'm using PRSice-2 with the permutation function by --perm for a binary phenotype.

Here is my command:

Rscript ./PRSice.R --dir . --prsice ./PRSice_linux --base My_base.txt --target My_target --cov My_COV.txt --or --binary-target T --clump-r2 0.2 --clump-p 1 --perm 10000 --thread 4 --upper 1 --print-snp --fastscore --all-score --bar-levels 5e-08,5e-06,1e-05,0.0005,0.05,0.1,0.5,1 --quantile 100 --quant-break 5,10,20,40,60,80,90,95,100 --quant-ref 60 --prevalence My_prevalence --out My_PRS_result

However, I still got the message "You can use the --perm option (see manual) to calculate an empirical P-value" in my .log file and did not get the empirical P-value in my output:

Pheno Set Threshold R2 R2.adj P Coefficient Standard.Error Num_SNP

Could you please let me know what is the possible reason for this problem?

Thank you.

Best regards,

Xiaoyuan

[Sam Choi]

Do you have the full log?

Sam

[Sam Choi]

2.1.9 is really old. Please try with 2.3.5. You should in theory find the output in the summary file with the empirical p

On Thursday, 25 November 2021 at 10:50:08 UTC-5 wqche...@gmail.com wrote:

Hi,

I wonder any chance you got it solved. I got the same issue. 

Briefly, I got the same warning about inflation/overfitting even after I used --perm. I compared the PRSice output before and after using --perm. Neither xx.prsice nor xx.all.score files changed after adding --perm. I noticed a warning about --logit-perm: "One problem with using --logit-perm is that some of the permuted phenotype might be suffer from perfect separation. This leads to the GLM logistic model not being able to be converge (thus terminating PRSice)". Given the PRSice is not terminated, I assume this is not the major driver. The command and log are listed below.

- The PRSice I'm using is PRSice 2.1.9 (25 Feburary 2019) .

- The command is : 

/software/bin/PRSice_linux \

    --A1 A1 \

    --A2 A2 \

    --all-score  \

    --bar-levels 5e-08,1e-06,1e-05,0.0001,0.001,0.01,0.05,0.1,0.2,0.3,0.4,0.5,1 \

    --base GWAS_Base_QCed.gz \

    --beta  \

    --binary-target T \

    --bp BP \

    --chr CHR \

    --clump-kb 250 \

    --clump-p 1.000000 \

    --clump-r2 0.200000 \

    --cov-col sex,genetics_batch,ageatinc,PC1,PC2,PC3,PC4,PC5,PC6,PC7,PC8,PC9,PC10,PC11,PC12,PC13,PC14,PC15,PC16,PC17,PC18,PC19,PC20 \

    --cov-factor sex,genetics_batch \

    --cov-file Target_cohort.cov \

    --fastscore  \

    --logit-perm  \

    --model add \

    --out Target_cohort_PRS \

    --perm 10000 \

    --pheno-col psych \

    --pheno-file Target_cohort.pheno \

    --pvalue PVAL \

    --seed 2459248683 \

    --snp SNP \

    --stat Z \

    --target Target_cohort \

    --thread 1

- Relevant information from PRSice log file:

• Warning: Using --logit-perm can be ridiculously slow 
• There are 1 region(s) with p-value less than 1e-5. Please note that these results are inflated due to the overfitting inherent in finding the best-fit PRS (but it's still best to find the best-fit PRS!). You can use the --perm option (see manual) to calculate an empirical P-value.