mqmscan singular matrix

[David]

I'm working with a fairly large dataset of 220 RI lines and 2018
markers with 56 traits. These RI lines were evaluated in 4 reps and so
I've formatted my dataset to actually include 880 individuals (220 x
4). Rather than add each rep as an additional phenotype for the line,
I want to include rep as a experimental covariate. For some traits
phenotypes on all 880 individuals are available but for other traits
only a subset of the lines were phenotyped. I've been able to perform
the single qtl scan for all traits but when I perform mqmscan I get
the following error:

FATALError in mqmscan(ibms, cofactors = mqmibm1, pheno.col = 50) :
Singular matrix

It doesn't matter which pheno.col I chose, I still get this error. I
alsos performed fill.geno prior to running the cofactor and mqmscan
functions. I understand that this singular matrix erro probably means
that their is severe segregation distortion in my dataset but I'm
wondering if their is a parameter I can pass to force the mqmscan. Any
help would be much appreciated. Thanks!

Here is my execution script:

library(qtl)
ibms<- read.cross("csv", file="C:/IBMRILx4reps.csv",
estimate.map=FALSE)
summary.cross(ibms)

ibms<-convert2riself(ibms)
ibms<-jittermap(ibms)
ibms<- drop.nullmarkers(ibms)
ibms<- est.rf(ibms)
ibms<- calc.genoprob(ibms, step=1, error.prob=0.01)

ibms<-fill.geno(ibms)

mqmibm1<-mqmautocofactors(ibms, 10,plot=TRUE, verbose=TRUE)
mqm_auto<-mqmscanall(ibms,cofactors=mqmibm1

[Angovy Kelty]

Hi David,
I am performind qtl analysis using mqmscan function. I got exactly the same fatal error of singular matrix. Could you please tell me how you fixed that issue.
Best
Angovy

[Danny Arends]

Hey Angovy,

The singular matrix can have multiple causes:

(1) Most often: Map problems, this is caused by either 

  1 Markers at the same position (with different genotypes) or 

  2 A map on which there are markers which behave as unlinked (so the chromosome has map expansion, and the total length is too long)

This can be checked by using:

plot.map(pull.map(cross),est.map(cross))

If a or multiple chromosome(s) from the map expands significantly then this is a good indication that this is happening

To debug this problem and find the offending chromosome, just scan the chromosomes 1 by 1, and record the chromosomes that cannot be scanned.

(I use the pheno2geno package (not yet on CRAN) developed by a student of mine to drop chromosomes by name (function removeChromosomes from 

https://github.com/KonradZych/phenotypes2genotypes )

in Arabidopsis (which has 5 chromosomes)

new_cross <- removeChromosomes(cross,c('2','3','4','5'))

res_c1 <- mqmscan(new_cross)

new_cross <- removeChromosomes(cross,c('1','3','4','5'))

res_c2 <- mqmscan(new_cross)

Then once you identified the chromosomes that cannot be scanned have a closer look at the structure of the markers there

e.g.: if chromosome 4 was the one that couldn't be scanned:

plot.rf(cross_newmap,1)

(2) Sometimes: Scanning using model="dominance" while there are markers that don't have individuals that are AA or BB 

(or in some weird edge cases if an entire chromosome shows no heterogeneity ) the easiest solution is to use the "additive" model

the dominance model is just a subset of the additive model (and thus increases LOD scores in the case of a real dominance effect)

(3) Rarely: Too many cofactors are selected in the initial model. or there are too many cofactors at a single position (this 

is related to the window.size parameter, multiple cofactors inside a single 'window' can cause a singular matrix during matrix single value decomposition)

Hope this helps, otherwise you can always contact me directly

Kind regards,

Danny Arends

[pjia...@gmail.com]

Hi Danny,

I ran into the same problem. I followed you suggestions--the problem remains without using dominance model and fitting cofactors--So I think it has to do with the map and markers. I have a total of ~2400 SNP markers in a F2 cross. I subset cross for each chromosome, and it worked. But when the total number of markers goes beyond ~1344 in a set of chromosomes, it will give me singular matrix error, regardless the combination of chromosomes included. I also compared the map with est.map, and it did not seem to have obvious problem. Could you give me some suggestion on this? Much appreciated!

Best regards,

Peng

[heinrichd...@googlemail.com]

On Friday, August 16, 2013 9:34:42 PM UTC+2, Peng Jiang wrote:

> Hi Danny,
>
>
>
> I ran into the same problem. I followed you suggestions--the problem remains without using dominance model and fitting cofactors--So I think it has to do with the map and markers. I have a total of ~2400 SNP markers in a F2 cross. I subset cross for each chromosome, and it worked. But when the total number of markers goes beyond ~1344 in a set of chromosomes, it will give me singular matrix error, regardless the combination of chromosomes included. I also compared the map with est.map, and it did not seem to have obvious problem. Could you give me some suggestion on this? Much appreciated!
>
>
>
> Best regards,
>
>
>
> Peng

I'm facing the problem mentioned aboive. Does anyone have a solution yet?
I have a DH/BC population with >3k markers. Subsetting single LGs and combinations of LG will work until a total marker number of 1344 , but above this marker no I get the "Singular matrix" error as well. Like Peng said, the recalculation of the map shows no change in single LG.

Thanks in advance and regards,

Haiko

[Danny Arends]

Dear all,

I am working on finding the source of this problems with the MQM algorithm,
What I need for this is examples of cross objects that exhibit this behavior, so if you could send me your genotype data with a subset of your phenotype data (1 phenotype is enough)
I can hopefully quickly get to the bottom of this annoying error

Kind regards,
Danny Arends

[Alicia Gardner]

Danny,

Has a solution for MQM been found. I'm experiencing the same sort of problem where all of the individual linkage groups can be scanned, but in combination the singular matrix error pops up.

Regards,
Alicia

[Gill]

I'm have the Singular matrix error as well. Has there been any update on this topic?

[Karl Broman]

No, I don't think we have a solution yet.

karl

> --
> You received this message because you are subscribed to the Google Groups "R/qtl discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to rqtl-disc+...@googlegroups.com.
> To post to this group, send email to rqtl...@googlegroups.com.
> Visit this group at http://groups.google.com/group/rqtl-disc.
> For more options, visit https://groups.google.com/d/optout.

[Gill]

Thank you very much. Then is there a way to carry on the mqm analysis when encountering this error message?

[Gill]

I just found have 8/14 LGs expanded. I'm pretty sure in my case 1 will exist. If I get rid of 1, should I re-construct the map to solve 2?

1 Markers at the same position (with different genotypes) or 

 2 A map on which there are markers which behave as unlinked (so the chromosome has map expansion, and the total length is too long)

[Lara]

Hi all,

Did anyone ever find a solution to this issue? I am working with an F2 population of 199 individuals (100% genotyped) and 1761 markers. My computer does not have enough memory to use mqmaugment so I am using fill.geno. However, I am getting the singular matrix error issue when using mqmscan. 

I have tried troubleshooting according to the advice above. I have already removed duplicate markers using findDupMarkers() and removed unlinked markers. I double checked this with plot.map(pull.map(chilling.jit),est.map(chilling.jit)), but my map doesn't expand much (the most expansion I see is one chromosome expands from 107 cM to ~155 cM). I tried using the additive model instead of the dominance model and I tried removing the cofactors, but the error still occurs. 

Has there been any developments? Or does anyone have advice on how to use mqmaugment instead? I am working on R/Studio in Windows 11 on a PC (16 GB). Would really appreciate the insight.

Code:

chilling.imp <- fill.geno(chilling.LR10, method="imp")

chilling.jit <- jittermap(chilling.imp)

cofactors <- mqmautocofactors(chilling.jit, plot=TRUE)

mqm.N1 <- mqmscan(chilling.jit, pheno.col=6, model="additive", cofactors=cofactors)

[Lara]

Thanks, I updated the package and re-ran checkAlleles() with markerlrt(). It returned that there are no apparent problems. However, I still receive the same warnings for mqmaugment() and mqmscan().

mqmaugment()

> memory.limit(size=15000000000000) #523983
[1] 1.5e+13

> maug_min1 <- mqmaugment(chilling.LR10, minprob=1.0) #not enough memory
INFO: VALGRIND MEMORY DEBUG BARRIERE TRIGGERED
There were 50 or more warnings (use warnings() to see the first 50)

> warnings()

Warning messages:
1: In mqmaugment(chilling.LR10, minprob = 1) : Not enough memory for new mqm marker vector of dimension 1

...

mqmscan()

> chilling.imp <- fill.geno(chilling.LR10, method="imp")

> chilling.jit <- jittermap(chilling.imp)

> cofactors <- mqmautocofactors(chilling.jit, plot=TRUE)

> mqm.N1 <- mqmscan(chilling.jit, pheno.col=6, model="additive", cofactors=cofactors)

Error in mqmscan(chilling.jit, pheno.col = 6, model = "additive", cofactors = cofactors) : Singular matrix