[R] sparseM and kronecker product_R latest version
alessia matano
I just installed the new version of R, 2.10.1, and I am currently
using the package sparseM. (I also use a 64 bit windows version)
I got a problem that I never had: when I try to multiply with a
kronecker product (%x%) two sparse matrixes I get the following
message:
Error in dim(x) <- length(x) : invalid first argument
I never had this problem with previous versions of R.
May you help me?
thanks
alessia
______________________________________________
R-h...@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Peter Ehlers
on the help page?
?'%x%-methods'
Works for me on Windows Vista (32-bit OS) and
R version 2.10.1 Patched (2010-01-05 r50896).
-Peter Ehlers
--
Peter Ehlers
University of Calgary
403.202.3921
David Winsemius
On Jan 11, 2010, at 7:55 AM, Peter Ehlers wrote:
> Do you have the same problem with the example
> on the help page?
>
> ?'%x%-methods'
>
> Works for me on Windows Vista (32-bit OS) and
> R version 2.10.1 Patched (2010-01-05 r50896).
>
> -Peter Ehlers
>
> alessia matano wrote:
>> Dear all,
>> I just installed the new version of R, 2.10.1, and I am currently
>> using the package sparseM. (I also use a 64 bit windows version)
SparseM
>> I got a problem that I never had: when I try to multiply with a
>> kronecker product (%x%) two sparse matrixes I get the following
>> message:
>> Error in dim(x) <- length(x) : invalid first argument
>> I never had this problem with previous versions of R.
I get the same error as you do when trying the example on the cited
help page using SparseM 0.83 in a 64 bit Mac version of 2.10.1
> A.csr %x% matrix(1:4,2,2)
Error in dim(x) <- length(x) : invalid first argument
> sessionInfo()
R version 2.10.1 RC (2009-12-09 r50695)
x86_64-apple-darwin9.8.0
locale:
[1] en_US.UTF-8/en_US.UTF-8/C/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] grid splines stats graphics grDevices utils
datasets methods base
other attached packages:
[1] SparseM_0.83 Matrix_0.999375-32 Epi_1.1.10
plotrix_2.7-2 ROCR_1.0-4
[6] gplots_2.7.4 caTools_1.10 bitops_1.0-4.1
gdata_2.6.1 gtools_2.6.1
[11] lattice_0.17-26 Design_2.3-0 Hmisc_3.7-0
survival_2.35-7
loaded via a namespace (and not attached):
[1] cluster_1.12.1 tools_2.10.1
>> May you help me?
>> thanks
>> alessia
>> ______________________________________________
> --
> Peter Ehlers
> University of Calgary
> 403.202.3921
>
> ______________________________________________
David Winsemius, MD
Heritage Laboratories
West Hartford, CT
alessia matano
I just installed the new version of R, 2.10.1, and I am currently
using the package sparseM. (I also use a 64 bit windows version)
I got a problem that I never had: when I try to multiply with a
kronecker product (%x%) two sparse matrixes I get the following
message:
Error in dim(x) <- length(x) : invalid first argument
I never had this problem with previous versions of R.
May you help me?
thanks
alessia
______________________________________________
alessia matano
fortunately when I closed R and opeedn it again, it did not give again
that error. However, now there is something more strange happening,
related again to sparse matrix, and I am afraid it could concern
memory problems (I put 4000 of memory limit).
I am defining a matrix where I have nrow=113289, ncol=36698 such as:
D<- as.matrix.csr(0, nrow, ncol)
Error in if (length(x) == nrow * ncol) x <- matrix(x, nrow, ncol) else { :
missing value where TRUE/FALSE needed
In addition: Warning message:
In nrow * ncol : NAs produced by integer overflow
With reduced number of rows and columns it worked, so I am wondering
wether it is a problem of memory. My pc has 6gb of nmemory, and it is
64-bit windows system, but R reads only 4 has capacity.
Thanks for your help
alessia
2010/1/11 David Winsemius <dwins...@comcast.net>:
Martin Maechler
>>>>> on Mon, 11 Jan 2010 15:38:39 +0100 writes:
am> Many thanks for your suggestions,
am> fortunately when I closed R and opeedn it again, it did not give again
am> that error. However, now there is something more strange happening,
am> related again to sparse matrix, and I am afraid it could concern
am> memory problems (I put 4000 of memory limit).
am> I am defining a matrix where I have nrow=113289, ncol=36698 such as:
am> D<- as.matrix.csr(0, nrow, ncol)
am> Error in if (length(x) == nrow * ncol) x <- matrix(x, nrow, ncol) else { :
am> missing value where TRUE/FALSE needed
am> In addition: Warning message:
am> In nrow * ncol : NAs produced by integer overflow
am> With reduced number of rows and columns it worked, so I am wondering
am> wether it is a problem of memory.
It's not; rather a "trap" that the programmers of SparseM fell
into (I did too, in the past): Your nrow and ncol are so
large that nrow * ncol is larger than R's maximal integer,
( == .Machine$integer.max which is 2^31 - 1 on all current versions of R)
and so overflows to NA and such leads to the
"missing value" warning you see.
am> My pc has 6gb of nmemory, and it is
am> 64-bit windows system, but R reads only 4 has capacity.
am> Thanks for your help
am> alessia
As a side note:
As co-author of R package 'Matrix' in to which we have put a
large amount of work and which is nowadays a recommended package
(i.e., also part of every R distribution),
I do wonder why you don't use 'Matrix' for sparse matrix
computing in R.
Best regards,
Martin Maechler, ETH Zurich
am> 2010/1/11 David Winsemius <dwins...@comcast.net>:
am> ______________________________________________
am> R-h...@r-project.org mailing list
am> https://stat.ethz.ch/mailman/listinfo/r-help
am> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
am> and provide commented, minimal, self-contained, reproducible code.
alessia matano
However it is strange that when I put the numbers rather than ncol(R)
(a matrix with ncol=36698) it worked. Look below
> dim(res2)
[1] 170471 25822
> D<- as.matrix.csr(0,nrow(tmpb),25822)
> D<- as.matrix.csr(0,nrow(tmpb),ncol(res2))
Error in if (length(x) == nrow * ncol) x <- matrix(x, nrow, ncol) else { :
missing value where TRUE/FALSE needed
In addition: Warning message:
In nrow * ncol : NAs produced by integer overflow
But probably it is true what you said, anyway.
So, do you suggest me to use directly the simple matrix command? or a
kind of sparse matrix within your package?!? Many thanks again.
Moreover unfortunately I then run the procedure of quantile fixed
effects elaborated by Koenker on a sample of 276000 observations, and
even if I am using a 64 bit machine with 6 gigabyte of Ram (and we
will raise it to 16 giga) he gives me the error :
Error: cannot allocate vector of size 155.4 Mb
and R cannot read more than 4 giga of RAM (even if I have 6).
Any suggestion to overcome this problem? Otherwise I have to randomly
subsample my data, which is a pity when you have such an amount of
information.
2010/1/11 Martin Maechler <maec...@stat.math.ethz.ch>:
David Winsemius
caused obscure problems. Prof Ripley suggested trying example() and
that did show that the example on the help page "worked". I just ran
example(`%x%`), as well as example(matrix.csr) and encountered no
errors. I also tried re-entering the first example on the help page
and am still getting the error. It looks as though the help page that
comes up with help(`%x%`) is similar to the second example suite. But
it's not precisely the same.
For me it's not a problem, since I'm not using SparseM at the moment.
I was just trying to help out the OP. I still should probably a)
restart an run without Matrix loaded, and b) upgrade to a more recent
R, but I have a number of models and datasets that I am working on and
those efforts take precedence.
> n1 <- 10
> n2 <- 10
> p <- 6
> y <- rnorm(n1)
> a <- rnorm(n1*p)
> a[abs(a)<0.5] <- 0
> A <- matrix(a,n1,p)
> A.csr <- as.matrix.csr(A)
> b <- rnorm(n2*p)
> b[abs(b)<1.0] <- 0
> B <- matrix(b,n2,p)
> B.csr <- as.matrix.csr(B)
>
> # matrix transposition and multiplication
> A.csr%*%t(B.csr)
elided output.... no error
>
> # kronecker product
> A.csr %x% matrix(1:4,2,2)
Error in dim(x) <- length(x) : invalid first argument
Also tried kronecker():
> kronecker(A.csr, matrix(1:36, 10,10))
Error in dim(x) <- length(x) : invalid first argument
--
David.
On Jan 11, 2010, at 10:02 AM, Roger Koenker wrote:
> I can't reproduce this on my mac:
>
>> sessionInfo()
> R version 2.10.1 (2009-12-14)
> x86_64-apple-darwin9.8.0
>
> locale:
> [1] C
>
> attached base packages:
> [1] graphics grDevices datasets utils stats methods base
>
> other attached packages:
> [1] fortunes_1.3-7 quantreg_4.44 SparseM_0.83
>
> loaded via a namespace (and not attached):
> [1] tools_2.10.1
>
> url: www.econ.uiuc.edu/~roger Roger Koenker
> email rkoe...@uiuc.edu Department of Economics
> vox: 217-333-4558 University of Illinois
> fax: 217-244-6678 Urbana, IL 61801
Roger Koenker
> sessionInfo()
R version 2.10.1 (2009-12-14)
x86_64-apple-darwin9.8.0
locale:
[1] C
attached base packages:
[1] graphics grDevices datasets utils stats methods base
other attached packages:
[1] fortunes_1.3-7 quantreg_4.44 SparseM_0.83
loaded via a namespace (and not attached):
[1] tools_2.10.1
url: www.econ.uiuc.edu/~roger Roger Koenker
email rkoe...@uiuc.edu Department of Economics
vox: 217-333-4558 University of Illinois
fax: 217-244-6678 Urbana, IL 61801
On Jan 11, 2010, at 8:00 AM, David Winsemius wrote:
Martin Maechler
am> Many thanks for it.
am> However it is strange that when I put the numbers rather than ncol(R)
am> (a matrix with ncol=36698) it worked. Look below
>> dim(res2)
am> [1] 170471 25822
>> D<- as.matrix.csr(0,nrow(tmpb),25822)
>> D<- as.matrix.csr(0,nrow(tmpb),ncol(res2))
am> Error in if (length(x) == nrow * ncol) x <- matrix(x, nrow, ncol) else { :
am> missing value where TRUE/FALSE needed
am> In addition: Warning message:
am> In nrow * ncol : NAs produced by integer overflow
am> But probably it is true what you said, anyway.
yes, it is true.
The clue is that typeof(25822) is "double" and not "integer".
am> So, do you suggest me to use directly the simple matrix command? or a
am> kind of sparse matrix within your package?!?
the latter, e.g.,
library(Matrix)
D <- Matrix(0, nrow = 113289, ncol=36698)
## or
D. <- sparseMatrix(x=double(0), i=integer(0), j=integer(0),
dims = c(113289,36698))
identical(D, D.) ##--> TRUE
## and, e.g.,
> Dk <- kronecker(D, Diagonal(x=5:2))
> identical(Dk, D %x% Diagonal(x = 5:2))
[1] TRUE
> dim(D)
[1] 113289 36698
> dim(Dk)
[1] 453156 146792
>
Regards,
Martin Maechler, ETH Zurich
alessia matano
and thanks for the example with matrix.
best
alessia
2010/1/12 Martin Maechler <maec...@stat.math.ethz.ch>: