[R] error in La.svd Lapack routine 'dgesdd'
steven mosher
what resources are there to track down errors like this
[[alternative HTML version deleted]]
Douglas Bates
that the error code indicates that the SVD algorithm failed to
converge. Evaluation of the singular values and vectors is done via
an iterative optimization and on some occasions will fail to converge.
Frequently this is related to the scaling of the matrix. If some
rows or columns are a very large magnitude relative to others the
convergence of the optimization can be impeded.
Providing a reproducible example of such an error condition will help
in diagnosing what is happening.
If you wonder why the error message is so enigmatic, it is because the
underlying code is Fortran and does not provide much flexibility for
informative error trapping.
> R-h...@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>
______________________________________________
R-h...@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
You received this message because you are subscribed to the Google Groups "R-help-archive" group.
To post to this group, send email to r-help-...@googlegroups.com.
To unsubscribe from this group, send email to r-help-archiv...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/r-help-archive?hl=en.
Philipp Grueber
I have an unbalanced panel with (on average) approx. 100 individuals over
1370 time intervals (with individual time series of different lengths,
varying between 60 and 1370 time intervals). I use the following model:
res1<-plm(x~c+d+e,data=pdata_frame, effect="twoways", model="within",
na.action=na.omit))
I repeatedly get the following error (which has been discussed in the past):
Error in La.svd(x, nu, nv) : error code 1 from Lapack routine ‘dgesdd’
I found it hard to create a reproducible example. As noted by Douglas Bates,
the error might be related to the scaling of the matrix.
For variables x,c,d,and e in object pdata_frame, I find that all sd() are
reasonably similar both among the cross-sections as well as among the
variables. However, I find that extracting the demeaned data from plm(),
variables demXt$d and demXt$e (i.e. the demeaned variables) have sd()s that
are very small compared to those of dem_yt and demXt$c (approx. by factor
1e-15). I extract the demeaned data as follows:
dem_yt<-pmodel.response(res)
demXt<-model.matrix(res)
How is this possible? What is it that plm() does with my data so that the
standard deviations change?
I suspect effect="twoways" to play a central role because plm() works fine
for effect="individual". I thought about the idea that maybe, time-effects
simply do not apply here. However: In order to test my regression for
time-effects (which I detect for subsamples (by time) and for equation x~e
at high levels of significance), I need both the model with and and the
model without time effects (as otherwise, I can't compare the two models in
an F-test), right? Any alternative tests?
Another thought was that the impact of d and e changes over time (as in the
subsamples I do see such a change).
Any help is appreciated!
Best wishes,
Philipp Grueber
-----
____________________________________
EBS Universitaet fuer Wirtschaft und Recht
FARE Department
Wiesbaden/ Germany
http://www.ebs.edu/index.php?id=finacc&L=0
--
View this message in context: http://r.789695.n4.nabble.com/error-in-La-svd-Lapack-routine-dgesdd-tp2125052p4603097.html
Sent from the R help mailing list archive at Nabble.com.
Philipp Grueber
Since I wish to understand what plm does to my data, I tried to manually
calculate the demeaned values and use OLS. See below how far I got with the
Grunfeld data; formula's are based on Greene's Econometric Analysis.
Obviously, I am missing at least one important step as my results differ
from both the plm and the LSDV estimations. Transferring the model to my
original dataset, I i) also get different results for the lm vs. the plm
models, however with the "manually" demeaned dataset, I do not end up with
incredibly small standard deviations in the demeaned values. That is, the
step that I am missing when reproducing the plm results is highly probable
to cause the error in my analysis.
###############
library(plm)
library(lmtest)
data(Grunfeld)
head(Grunfeld)
pdata<-pdata.frame(Grunfeld,index=c("firm","year"))
head(pdata)
fm<-inv~value+capital
mod<-plm(fm, pdata, effect="twoways", model="within")
y<-Grunfeld$inv
x1<-Grunfeld$value
x2<-Grunfeld$capital
y_i<-c()
x1_i<-c()
x2_i<-c()
for (i in unique(Grunfeld$firm)){
y_i<-c(y_i,rep(mean(Grunfeld$inv[Grunfeld$firm==i],na.rm=TRUE),nrow(Grunfeld)))
x1_i<-c(x1_i,rep(mean(Grunfeld$value[Grunfeld$firm==i],na.rm=TRUE),nrow(Grunfeld)))
x2_i<-c(x2_i,rep(mean(Grunfeld$capital[Grunfeld$firm==i],na.rm=TRUE),nrow(Grunfeld)))
}
y_t<-rep(NA,nrow(Grunfeld))
x1_t<-rep(NA,nrow(Grunfeld))
x2_t<-rep(NA,nrow(Grunfeld))
for (t in unique(Grunfeld$year)){
y_t[Grunfeld$year==t]<-rep(mean(Grunfeld$inv[Grunfeld$year==t],na.rm=TRUE),sum(Grunfeld$year==t))
x1_t[Grunfeld$year==t]<-rep(mean(Grunfeld$value[Grunfeld$year==t],na.rm=TRUE),sum(Grunfeld$year==t))
x2_t[Grunfeld$year==t]<-rep(mean(Grunfeld$capital[Grunfeld$year==t],na.rm=TRUE),sum(Grunfeld$year==t))
}
y_it<-rep(mean(Grunfeld$inv,na.rm=TRUE),nrow(Grunfeld))
x1_it<-rep(mean(Grunfeld$value,na.rm=TRUE),nrow(Grunfeld))
x2_it<-rep(mean(Grunfeld$capital,na.rm=TRUE),nrow(Grunfeld))
y_star<-y-y_i-y_t+y_it
x1_star<-x1-x1_i-x1_t+x1_it
x2_star<-x2-x2_i-x2_t+x2_it
mod_lm<-lm(y_star~x1_star+x2_star-1,na.action=na.omit)
mod_dumm<-lm(Grunfeld$inv~Grunfeld$value+Grunfeld$capital+as.factor(Grunfeld$year)+as.factor(Grunfeld$firm)-1,na.action=na.omit)
coeftest(mod)
coeftest(mod_lm)
coeftest(mod_dumm)
##############
Again, any help is highly appreciated.
Best wishes,
Philipp
-----
____________________________________
EBS Universitaet fuer Wirtschaft und Recht
FARE Department
Wiesbaden/ Germany
http://www.ebs.edu/index.php?id=finacc&L=0
--
Millo Giovanni
Dear Philipp,
this is just a tentative answer because debugging is really not possible
without a reproducible example (or, at a very bare minimum, the output
from traceback()).
Anyway, thank you for reporting this interesting numerical issue; I'll
try to replicate some similar behaviour on a similarly dimensioned
artificial dataset when I have some time (which might not be soon). As
for now, please see below my remarks with '##', I hope they are useful
anyway. Bottom line: time fixed effects might be out of place here.
Best wishes,
Giovanni
Giovanni Millo, PhD
Research Dept.,
Assicurazioni Generali SpA
Via Machiavelli 4,
34132 Trieste (Italy)
tel. +39 040 671184
fax +39 040 671160
----------------- original message ------------
Message: 8
Date: Wed, 2 May 2012 05:45:47 -0700 (PDT)
From: Philipp Grueber <philipp...@ebs.edu>
To: r-h...@r-project.org
Subject: Re: [R] error in La.svd Lapack routine 'dgesdd'
Message-ID: <1335962747113...@n4.nabble.com>
Content-Type: text/plain; charset=UTF-8
Dear R Users,
I have an unbalanced panel with (on average) approx. 100 individuals
over
1370 time intervals (with individual time series of different lengths,
varying between 60 and 1370 time intervals). I use the following model:
res1<-plm(x~c+d+e,data=pdata_frame, effect="twoways",
model="within",
na.action=na.omit))
## I have difficulty in understanding why you would want to introduce
ca. 1470 incidental parameters... I'd rather go with a more parsimonious
specification: a trend, AR(n) or else...
I repeatedly get the following error (which has been discussed in the
past):
Error in La.svd(x, nu, nv) : error code 1 from Lapack routine
?dgesdd?
I found it hard to create a reproducible example. As noted by Douglas
Bates,
the error might be related to the scaling of the matrix.
## Too difficult for me to tell without output, references etc.,
although of course I trust D.B.'s opinion.
For variables x,c,d,and e in object pdata_frame, I find that all sd()
are
reasonably similar both among the cross-sections as well as among the
variables. However, I find that extracting the demeaned data from plm(),
variables demXt$d and demXt$e (i.e. the demeaned variables) have sd()s
that
are very small compared to those of dem_yt and demXt$c (approx. by
factor
1e-15). I extract the demeaned data as follows:
dem_yt<-pmodel.response(res)
demXt<-model.matrix(res)
How is this possible? What is it that plm() does with my data so that
the
standard deviations change?
## it demeans them... (although the scale of the reduction is
impressive, yet you're estimating out 1500 constants!)
I suspect effect="twoways" to play a central role because plm() works
fine
for effect="individual".
## sure, also because "individual" 'just' introduces 100 more
parameters.
I thought about the idea that maybe, time-effects
simply do not apply here.
## You know your model. Yet time effects on T=1300 seems hazardous to
me.
However: In order to test my regression for
time-effects (which I detect for subsamples (by time) and for equation
x~e
at high levels of significance), I need both the model with and and the
model without time effects (as otherwise, I can't compare the two models
in
an F-test), right? Any alternative tests?
## please see ?plmtest
Another thought was that the impact of d and e changes over time (as in
the
subsamples I do see such a change).
Any help is appreciated!
## HTH, G.
Best wishes,
Philipp Grueber
-----
____________________________________
EBS Universitaet fuer Wirtschaft und Recht
FARE Department
Wiesbaden/ Germany
http://www.ebs.edu/index.php?id=finacc&L=0
--
View this message in context:
http://r.789695.n4.nabble.com/error-in-La-svd-Lapack-routine-dgesdd-tp21
25052p4603097.html
Sent from the R help mailing list archive at Nabble.com.
Ai sensi del D.Lgs. 196/2003 si precisa che le informazi...{{dropped:12}}