[DISCUSS] Official JanusGraph docker image repo

[Ted Wilmes]

Hello,

I wanted to restart a discussion on producing an official JanusGraph Docker image. There is lots of community interest and progress has been made but we're not quite there yet. There is an existing PR [1] that a coworker of mine, Chris Pounds, had been working on. After some brainstorming with Misha Brukman, we were wondering if a change in approach may make sense. Namely, that it might be a good idea to host the artifacts required to maintain Janus docker images separate from the main JanusGraph repo. With that in mind, Chris has recently created this repo: https://github.com/experoinc/janusgraph-images. It's sitting under out GitHub at the moment but is Apache 2 licensed, and if this plan makes sense to folks, we'd just transition it over to the Janus org and put it through the usual PR and approval process. With all that said, I think there are a number of benefits to maintaining the Docker plumbing separate from JanusGraph itself. I would imagine a matrix of different versions and image options at some point, potentially supporting a variety of different linux distros including alpine. Also, it would be nice to be able to version the Docker artifacts separately from JanusGraph so that we can rev them at different speeds. So, first step is I'd like to get some community feedback on this approach and see what folks thought about setting up a separate repo under the Janus org to host the Docker plumbing?

Thanks,

Ted

[1] https://github.com/JanusGraph/janusgraph/pull/700

[Chris Hupman]

Hello,

I'm regularly using containers for Cassandra and Elasticsearch for my JanusGraph work and I had a couple of thoughts on this. 

• In pull 700 only janusgraph is being run, in the experoinc repo es, cassandra, and janusgraph are all running in the same container. I personally think JanusGraph only running in-memory by default is the right approach. While an all-in-one is great for development it wouldn't really make sense for a real deployment. 
  
• Right now both containers are using JanusGraphFactory(JGF) instead of ConfiguredGraphFactory(CGF). Since this container is for running JanusGraph server I think CGF would make more sense than JGF. 
• To be more modular and avoid passing as many -e flags it should support multiple gremlin server files through ENVs. 
• For JGF deployments we should add in a script to bind g = graph.traversal() to improve user experience as referenced in the docs
• For CGF deployments it would be nice if the container tried to create a template using the configuration parameters it already had access to. 
• I think it makes sense to house the docker components and tests in a separate repo. With the approach of downloading the release and moving it to /opt/janusgraph it would be easy to run a script afterwards to update the yaml and property files to take ENVs. 

I think this would be great for the overall health of the project and I'm willing to help out with code, docs, and/or testing.

[Florian Hockmann]

Hi,

first of all, big thanks for bringing this up again! I'm really looking forward to an official JanusGraph image.

I can think of one advantage of including the Docker images in the main JanusGraph repo: When you want to test a client library like the ones we will probably create for the different languages (I'm currently working on one for .NET) then you need an instance of the server and this should be a version of the server that will be released with the same version number. A JanusGraph docker image would make testing of such a client library much easier but when we put the Dockerfile in another repo then we would only have already published images available in the main repo. So you couldn't test such a client library with a Docker image of the server as it is at that moment in the repo, you would rather have to use an already released version.

A big advantage of a separate repo would be that we could make use of DockerHub's automated builds which can then also include the README.md and the Dockerfile.

One thing I'm wondering about:

potentially supporting a variety of different linux distros including alpine

What's the advantage of having images for other distros than alpine? Having alpine as the base image means that the image will be really small but do users also want a debian based image or something like that?

I haven't looked into the new repo Chris created in detail as I think that we should first only discuss about whether or not a separate repo should be used and I assume that there will be the usual review process afterwards.

[Ted Wilmes]

Thanks for the responses Chris and Florian. I'm in agreement that we should first discuss the separate repo issue separate from the implementation details. That was my intent, but by including those two links I conflated the two. I would expect either approach we take to go through the usual PR process, or perhaps even another round of discussion. My Alpine base image example isn't a great one, may main point is that for whatever reason, there may be a community desire to support a variety of different configurations for the images (not Janus configs themselves to be clear).

Your point about client library development is a good one. My inclination is to treat the JanusGraph developer Docker environment use case differently than the JanusGraph end-user use cases (user dev & integration testing, production deployment). There would undoubtedly be some overlap, but what if we included a minimal Docker build as part of the repo for client library testing and then left the separate repo to serve only the purpose of producing official, published Docker images?

Thanks,

Ted

[Florian Hockmann]

what if we included a minimal Docker build as part of the repo for client library testing and then left the separate repo to serve only the purpose of producing official, published Docker images?

That makes perfectly sense to me. Seems to be the best solution overall to me.

[Chris Hupman]

Since we seem to be making repos right now I wanted to stir this up a bit. I think right now would be a good time to make a janusgraph-docker repo for an official Docker image. 

I mainly wanted to ask if a single repo would be adequate. I personally like the approach of a single image defaulting to in-memory so the image can be deployed standalone and then configuring through -e flags at deploy. In one of the PRs it was suggested to default to using es and Cassandra. I am assuming, quite possibly incorrectly,  that would mean throwing es, Cassandra, and JanusGraph in a single container for ease of deploy. If that was the case an additional repo would probably be required. 

Hoping to get some +1s for a consensus on adding a janusgraph-docker repo.

Chris

[Jason Plurad]

+1 for a single janusgraph-docker repo.

Wouldn't you be able to use the directory structure to manage the different flavors of a Docker image that we want to publish to Docker Hub? I wouldn't want to create multiple JanusGraph repos for Docker technology.

[Chris Hupman]

Stirring one more time. Ted and Florian are you guys in favor of the creation of a repo for an official JanusGraph docker image?

[Ted Wilmes]

Hey Chris,

Appreciate the extra stir and yes, I'm in agreement. +1 for a separate repo.

Thanks,

Ted

--
You received this message because you are subscribed to a topic in the Google Groups "JanusGraph developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/janusgraph-dev/V70uVxh9who/unsubscribe.
To unsubscribe from this group and all its topics, send an email to janusgraph-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/d2da37b0-44c1-4c3b-b51b-58e279bdeefa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Florian Hockmann]

Yes, +1 for the separate repo.

[Jason Plurad]

https://github.com/JanusGraph/janusgraph-docker is created

[Chris Hupman]

One last question, mainly for Expero folks. Is anyone already intending to make the initial commit? I have a conference this week, but can work on this next week if not.

[Ted Wilmes]

I think I can get to that later this week. I could push Chris' work into a new branch on the new repo and then put a PR in so we could come to some consensus on if that's a good starting point?

To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-dev/1da5c589-aef2-437c-b89e-9064108f0e7c%40googlegroups.com.

[Chris Hupman]

I think that makes sense as a starting point. I'm assuming there will be a fair amount of testing, discussions,  and iterations before it gets pushed up as an official image. 

[Ted Wilmes]

Ok, I moved Mr. Pounds code over into my fork of the new repo and put a "scratch" PR in: https://github.com/JanusGraph/janusgraph-docker/pull/1

Let's move our discussion/feedback on approach over to that PR unless you all think it would be beneficial to continue here on the list.

--Ted

[andy.r....@gmail.com]

Any idea when this is going to get a release?