Hi, can the current SVD routine in the mlpack handle large sparse
matrices? When I used the SVD routine in the mlpack, I got "out of
memory" error. The dimension of the matrix is about 15000x5000, but
it's very sparse. Is there an easy way that I don't need to rely on
other SVD tools?
The SVD routine defined in fastlib/la/la.h is a dense-matrix based method,
and will
require 8 bytes for each of the 15000 * 5000 entries. You would have to rely
on
an external package that is not included in the current release of MLPACK on
mloss.org.
On Thu, Dec 4, 2008 at 10:23 PM, byungki <yor...@gmail.com> wrote:
> Hi, can the current SVD routine in the mlpack handle large sparse
> matrices? When I used the SVD routine in the mlpack, I got "out of
> memory" error. The dimension of the matrix is about 15000x5000, but
> it's very sparse. Is there an easy way that I don't need to rely on
> other SVD tools?
-- Dongryeol Lee
Ph.D. Student, College of Computing
Georgia Institute of Technology
The internal version of MLPACK uses trillinos, although it hasn't been
maintained. If you really need it we could do something for it although we
are busy until NIPS
On Thu, Dec 4, 2008 at 10:25 PM, Dongryeol Lee <dongr...@cc.gatech.edu>wrote:
> The SVD routine defined in fastlib/la/la.h is a dense-matrix based method,
> and will
> require 8 bytes for each of the 15000 * 5000 entries. You would have to
> rely on
> an external package that is not included in the current release of MLPACK
> on
> mloss.org.
> On Thu, Dec 4, 2008 at 10:23 PM, byungki <yor...@gmail.com> wrote:
>> Hi, can the current SVD routine in the mlpack handle large sparse
>> matrices? When I used the SVD routine in the mlpack, I got "out of
>> memory" error. The dimension of the matrix is about 15000x5000, but
>> it's very sparse. Is there an easy way that I don't need to rely on
>> other SVD tools?
> --
> Dongryeol Lee
> Ph.D. Student, College of Computing
> Georgia Institute of Technology
Well, if so, I'll just use other tools such as svdpack and try to
integrate it to fastlib as much as I can. It's just a suggestion, but
it would be nicer if mlpack could handle larger matrices because then
dimensionality reduction techniques, for example, become more useful.
On Dec 4, 11:32 pm, "Nikolaos Vasiloglou II" <nva...@ieee.org> wrote:
> The internal version of MLPACK uses trillinos, although it hasn't been
> maintained. If you really need it we could do something for it although we
> are busy until NIPS
> On Thu, Dec 4, 2008 at 10:25 PM, Dongryeol Lee <dongr...@cc.gatech.edu>wrote:
> > The SVD routine defined in fastlib/la/la.h is a dense-matrix based method,
> > and will
> > require 8 bytes for each of the 15000 * 5000 entries. You would have to
> > rely on
> > an external package that is not included in the current release of MLPACK
> > on
> > mloss.org.
> > On Thu, Dec 4, 2008 at 10:23 PM, byungki <yor...@gmail.com> wrote:
> >> Hi, can the current SVD routine in the mlpack handle large sparse
> >> matrices? When I used the SVD routine in the mlpack, I got "out of
> >> memory" error. The dimension of the matrix is about 15000x5000, but
> >> it's very sparse. Is there an easy way that I don't need to rely on
> >> other SVD tools?
> > --
> > Dongryeol Lee
> > Ph.D. Student, College of Computing
> > Georgia Institute of Technology
