ND4J - a distributed scientific computing lib for Java

[Chris V. Nicholson]

Hey folks - Just wanted to flag ND4J.org. It's an attempt to build tools for n-dimensional arrays on a distributed system. It also happens to be the linear algebra engine behind deeplearning4j.org. Eager to get your feedback if this is useful. 

[Curtis Rueden]

Hi Chris,

> Just wanted to flag ND4J.org

Looks like a cool and well structured project!

Are you interested in integrating with the ImgLib2 library [1]? Right now ImgLib2 uses JAMA in a few places, but as ND4J is more flexible, perhaps we could use that instead. Upon cursory inspection it would be a very natural and powerful fit. The GPU processing capabilities are particularly compelling (Brian: I'm CCing you since you are particularly interested in this topic!).

To summarize the SciJava effort, it actually covers a couple of different goals:

- It represents a pledge to reuse code to create a common "software stack" as much as feasible.

- It describes that stack from the perspective of a few interoperable applications: ImageJ, KNIME and OMERO in particular—although CellProfiler (a Python application) is a big player as well. The scijava.org web site lists the involved interoperable applications and libraries.

- It is a shorthand to refer to projects that build on a Maven-based architecture; see http://imagej.net/Architecture for details there.

As you can probably tell, there are life sciences and image processing biases to the project since it emerged from those areas, but the cornerstone SciJava projects are by no means intended to be limited that way. As a rule of thumb: the scijava GH org covers non-image-specific things, whereas imagej+imglib+scifio cover non-domain-specific image processing, and fiji+trakem2+bigdataviewer+openmicroscopy cover life sciences more specifically. The KNIME project is also a very general data analytics tool, unspecific to images or life sciences.

It would be fantastic to add ND4J to the mix if you see areas of overlap or interoperability. Like I said, the most obvious win I see would be for ImgLib2 to leverage ND4J as a potential "container" for image data, and/or to create an INDArray implementation backed by an ImgLib2 data object. This would allow developers to freely bounce between paradigms.

The immediate question is: what concrete benefit will it bring to the table, justifying someone to explore the integration? As a starting point, the GPU processing capabilities spring to mind...

Regards,

Curtis

[1] https://github.com/imglib/imglib2

On Thu, Dec 4, 2014 at 12:17 AM, Chris V. Nicholson <chrisvn...@gmail.com> wrote:

Hey folks - Just wanted to flag ND4J.org. It's an attempt to build tools for n-dimensional arrays on a distributed system. It also happens to be the linear algebra engine behind deeplearning4j.org. Eager to get your feedback if this is useful. 

--
You received this message because you are subscribed to the Google Groups "scijava" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scijava+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.