sparse-matrix construction

[je...@thinktopic.com]

Hi peeps,

  I'm creating a sparse matrix to represent a database of products where each product might have around 30 weighted tags out of thousands.  I have the data in the form of {product-id [[tag-index tag-weight] ...]} so it seems like a perfect scenario for lazy construction of a sparse matrix.  Is this supported currently, and if so how shall I go about it?

At the moment I have to call sparse on a dense matrix, but this won't work too well as the dataset expands.

Thanks,

Jeff

[Mike Anderson]

Hi Jeff,

We don't have a specific sparse matrix constructor in the core.matrix API yet, ideally we would want something like:

(sparse-vector indexes values)

This is probably a sensible API addition, feel free to create an issue.

Currently you have two fairly decent options:

1. Use interop to call the underlying constructor in the implementation. In the :vectorz case you probably want:

(mikera.vectorz.impl.SparseIndexedVector/wrap (int length) (int-array indexes) (double-array values))

This will offer the best possible performance, in this case O(number of non-zero elements)

2. Create a mutable indexed vector and use mset! to set the individual non-zero elements, with something like:

(let [v (new-sparse-array :vectorz [length])]

  (doseq [[index value] data]

    (mset! v index value))

  v)

This should be fine as long as you don't have too many non-zero elements, since mset! is typically going to be worse than O(1) for sparse vectors. 

In the :vectorz case, this will be O((number of non-zero elements)^2))