very large matrice with ejml

[salvador]

How large can a matrix be? I am trying to create a matrix with dimensions 10.000 X 10.000 and i get error. Is there a solution for this?

Thanks!

[Peter A]

Probably out of memory problems.  If it is there really isn't anything you can do besides buy more ram.  Is there some other exception being thrown?

- Peter

On Mon, Apr 22, 2013 at 7:16 AM, salvador <sot....@gmail.com> wrote:

How large can a matrix be? I am trying to create a matrix with dimensions 10.000 X 10.000 and i get error. Is there a solution for this?

Thanks!

--
You received this message because you are subscribed to the Google Groups "efficient-java-matrix-library-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to efficient-java-matrix-li...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
"Now, now my good man, this is no time for making enemies."    — Voltaire (1694-1778), on his deathbed in response to a priest asking that he renounce Satan.

[salvador]

Yes it is memory error, i wonder if there is a solution like sparse matrices.

[Peter A]

BoofCV doesn't support sparse matrices.  If you find a good java library for sparse matrices let me know and I'll add a reference to it in the FAQ.

- Peter

On Tue, Apr 23, 2013 at 11:10 AM, salvador <sot....@gmail.com> wrote:

Yes it is memory error, i wonder if there is a solution like sparse matrices.

--

You received this message because you are subscribed to the Google Groups "efficient-java-matrix-library-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to efficient-java-matrix-li...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Lu Jiang]

Hi Peter,

I checked several constructor in ejml, and it seems it either create a consecutive double array. For really large dense matrices, it leads to out-of-memory error, cuz usually one may not have large piece chunk of consecutive memory. 

Stroring 2d matrices by a 2d double array seems more reasonable to me but I cannot find a constructor.

How would I handle this problem? 

On Thursday, April 25, 2013 9:16:05 PM UTC-4, Peter wrote:

BoofCV doesn't support sparse matrices.  If you find a good java library for sparse matrices let me know and I'll add a reference to it in the FAQ.

- Peter

On Tue, Apr 23, 2013 at 11:10 AM, salvador <sot....@gmail.com> wrote:

Yes it is memory error, i wonder if there is a solution like sparse matrices.

--
You received this message because you are subscribed to the Google Groups "efficient-java-matrix-library-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to efficient-java-matrix-library-discuss+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Peter A]

Internally the memory is always continuous in BoofCV, even if the constructor takes a 2D array as input.  That was decided early on after doing several benchmarks and it makes resizing matrices easier. You could avoid effectively declaring the matrix twice by first declaring the matrix with new DenseMatrix64F(nRows,nCols) then writing the data directly to the matrix.  I would be surprised if the inability to find a large continuous chunk of data is the primary cause of out of memory exceptions.  Often an operation will declare extra data for book keeping purposes.  For example. many operations will convert the row-major DenseMatrix64F into a block matrix.  When possible it will do the conversion in place to avoid excessive memory allocation. If you inspect the source code you could figure out how to declare a block matrix directly and use that. But the documentation isn't flushed out and there are a few operations which don't support block matrices yet.

- Peter

To unsubscribe from this group and stop receiving emails from it, send an email to efficient-java-matrix-li...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.