numpy.linalg.LinAlgError: Last 2 dimensions of the array must be square

[Priya Choudhary]

Hello there!

First of all, I wanted to thank this community for always being very helpful and responding to the issues I have been running into lately. 

Lately, I have been receiving this issue when using the Elastic Net models to run S-PrediXcan. Here is the error message: 

INFO - Processing GWAS command line parameters

INFO - Building beta for /share/carvajal-archive/GWAS_DATA/Colorectal_Cancer/Colorectal_TWAS_4_2021/PrediXcan/SPrediXcan/New_switched_Liftover_data.txt and /share/carvajal-archive/GWAS_DATA/Colorectal_Cancer/Colorectal_TWAS_4_2021/PrediXcan/MetaXcan/data/Elastic_Net_Models/elastic_net_models/en_Colon_Transverse.db

INFO - Reading input gwas with special handling: /share/carvajal-archive/GWAS_DATA/Colorectal_Cancer/Colorectal_TWAS_4_2021/PrediXcan/SPrediXcan/New_switched_Liftover_data.txt

INFO - Processing input gwas

Level 9 - Calculating zscore from pvalue

Level 9 - Acquiring sign from beta

INFO - Aligning GWAS to models

INFO - Trimming output

INFO - Successfully parsed input gwas in 3.0178139600902796 seconds

INFO - Started metaxcan process

INFO - Loading model from: /share/carvajal-archive/GWAS_DATA/Colorectal_Cancer/Colorectal_TWAS_4_2021/PrediXcan/MetaXcan/data/Elastic_Net_Models/elastic_net_models/en_Colon_Transverse.db

INFO - Loading covariance data from: /share/carvajal-archive/GWAS_DATA/Colorectal_Cancer/Colorectal_TWAS_4_2021/PrediXcan/MetaXcan/data/Elastic_Net_Models/elastic_net_models/en_Colon_Transverse.txt.gz

INFO - Processing loaded gwas

INFO - Started metaxcan association

Level 9 - Processing gene 0:ENSG00000000419.12

Traceback (most recent call last):

  File "/share/carvajal-archive/GWAS_DATA/Colorectal_Cancer/Colorectal_TWAS_4_2021/PrediXcan/MetaXcan/MetaXcan-master/software/SPrediXcan.py", line 65, in <module>

    run(args)

  File "/share/carvajal-archive/GWAS_DATA/Colorectal_Cancer/Colorectal_TWAS_4_2021/PrediXcan/MetaXcan/MetaXcan-master/software/SPrediXcan.py", line 31, in run

    M04_zscores.run(args, g)

  File "/share/carvajal-archive/GWAS_DATA/Colorectal_Cancer/Colorectal_TWAS_4_2021/PrediXcan/MetaXcan/MetaXcan-master/software/M04_zscores.py", line 70, in run

    results = run_metaxcan(args, context)

  File "/share/carvajal-archive/GWAS_DATA/Colorectal_Cancer/Colorectal_TWAS_4_2021/PrediXcan/MetaXcan/MetaXcan-master/software/M04_zscores.py", line 34, in run_metaxcan

    r, snps = AssociationCalculation.association(gene, context, return_snps=True)

  File "/share/carvajal-archive/GWAS_DATA/Colorectal_Cancer/Colorectal_TWAS_4_2021/PrediXcan/MetaXcan/MetaXcan-master/software/metax/metaxcan/AssociationCalculation.py", line 60, in association

    d_ = numpy.linalg.eig(cov)[0]

  File "<__array_function__ internals>", line 6, in eig

  File "/home/pchoudhary/.conda/envs/imlabtools/lib/python3.7/site-packages/numpy/linalg/linalg.py", line 1294, in eig

    _assert_stacked_square(a)

  File "/home/pchoudhary/.conda/envs/imlabtools/lib/python3.7/site-packages/numpy/linalg/linalg.py", line 213, in _assert_stacked_square

    raise LinAlgError('Last 2 dimensions of the array must be square')

numpy.linalg.LinAlgError: Last 2 dimensions of the array must be square

Any help would be much helpful!

Warm regards, 

Priya

[Alvaro Barbeira]

Hi Priya,

Are you getting this error with all elastic net models or just a few?

The error you observe suggests the numpy library failed to compute eigenvalues of the covariance matrix. I don't remember running into this condition before. 

I attach a modified version of AssociationCalculation.py that will start a python debugger session when this error is met - you can replace this in your local download of the MetaXcan repository (at software/metax/metaxcan/AssociationCalculation.py).to troubleshoot. 

There is the remote possibility that something changed in numpy library, and it became more strict about something we used liberally.

Best,

Alvaro

--
You received this message because you are subscribed to the Google Groups "PrediXcan/MetaXcan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to predixcanmetax...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/predixcanmetaxcan/bb165eb6-5e21-4d1a-a781-39e596978594n%40googlegroups.com.

[Guanchao Wang]

Hi Alvaro and Priya,

I am encountering the exact same error. I just want to follow up on this thread to see if you were able to fix the issue. I tried the new version of AssociationCalculation.py attached, but I don't quite understand the additional errors shown. 

Any information would be helpful! 

Best Regards,

Guanchao

[Festus]

Hi all,

The error numpy.linalg.LinAlgError: Last 2 dimensions of the array must be square   arises when there is a mismatch between the ids used in the covariance matrix and the model_id used to match to the summary stats file. If the covariance matrix has rsids make sure to use model_key rsid and not varID.

Festus