Apache NiFi support for JanusGraph

[Mike Thomsen]

I'm on the Apache NiFi PMC, and thought I'd give an unofficial announcement that in NiFi 1.10, JanusGraph will be supported in our new "graph bundle." Anyone who's interested in trying it out can download a snapshot of our source for 1.10-SNAPSHOT and should be able to build it using this command from the root folder:

mvn install -DskipTests=true -Pinclude-graph

The zip and tar.gz distributions will be built in $ROOT/nifi-assembly/target.

JanusGraph is our test database for testing Gremlin compatibility and Neo4J obviously is our test for Cypher.

When 1.10 is released, the graph support will not be part of the binaries we release due to space constraints (our convenience binaries are about 1.5GB already).

If anyone tries it out and wants to give feedback, just jump on the nifi-dev mailing list.

(We're currently using JanusGraph 0.3.2 for testing)

Thanks,

Mike

[Florian Hockmann]

Hi Mike, thanks for notifying us about the added support! That's really good to know! Is there some documentation where the connection to JanusGraph is described?

[Mike Thomsen]

Florian,

At this point, not really, except for the documentation that we generate for users as part of the component documentation set. With 1.10, that should appear here: http://nifi.apache.org/docs.html

The general flow, though, is pretty simple. The only graph processor right now is ExecuteGraphQuery, and it requires a configured client service (we provide three). For Janus, you'd just add the Gremlin client service and configure it with the endpoint for the target JanusGraph server.

For now, these components are technically still a work in progress, so there's room to suggest breaking changes to make the Gremlin support better and things like that.

[PhilBy]

I'd be very interested in understanding more about the use of Nifi's ExecuteGraphQuery processor or any other to load data into JanusGraph from Nifi. Currently, I'm using the ExecuteStream processor in Nifi to basically load data into JanusGraph via a groovy script. However, a Nifi processor to do this would save us a lot of time. Just wondering if you have any documentation about this, particularly how to connect to JanusGraph via Nifi.