D3 visualization of gremlin graph

[Benjamin Ricaud]

Hi all,

I needed a visualization for my graph database, so I made one with D3.js. I want to share it with everyone who is interested.

You can visualize a graph on a Gremlin server using REST or websockets, directly in your web browser.

It is on my github:

https://github.com/bricaud/graphexp

I call it Graphexp or Graph explorer.

A live demo is here if you want to try it without installing (tutorial on the github page): https://bricaud.github.io/graphexp/graphexp.html

Many things can be improved. Comments, suggestions or pull requests are welcome.

Benjamin

[Stephen Mallette]

Interesting - can you briefly describe the differences between your work and http://gremlinbin.com/ ?

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/c2f74844-2a2f-4d4d-bed8-e690f20f9a8d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Derek Williams]

I took a quick look at Benjamin's graphexp.

I really like:

* Fading the vertices and edges

* Nice side bar when a vertex or edge is selected (showing details)

* Use of D3 (as opposed to canvas), though I may be biased

* Simple search (great for users less familiar with gremlin)

* Good readme with a Docker to get up and running quickly

Not seeing (yet) 

* Any way to enter a gremlin query (though this would be simple to add)

* Any editing of edge or vertex values, though this should be relatively easy to add

* Console of queries being run

Unsure of:

* The function of the smaller circles adjacent to the vertices

* How to reverse the drilling down to a vertex or edge.

Will definitely start looking at the code.

Nice work.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H4390tGRM2B5wThzyR_hw9uiXnA%3Dw0L5RnG5fhc%3DESXhdAQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Derek Williams
Cell: 970.214.8928

[Benjamin Ricaud]

Yes sure. 

Graphexp is just a visualization where you can display nodes either 

* by searching a particular keyword or value in the node properties (enter it in the textbox)

* or by clicking on a node already displayed, which displays its neighbors. You can explore the graph by clicking from neighbor to neighbor.

You can not submit a Gremlin query as in Gremlinbin. 

It is aimed at people not knowing Gremlin syntax or people who just want to have a simple tool for graph exploration. (You can not learn Gremlin syntax by using Graphexp, which is one of the goal of Gremlinbin)

You can put the code on your computer, just modify the configuration file to connect to your gremlin server, open a web browser and load graphexp.html in your local folder to explore your graph.
You could see it as a plugin-in to see a Gremlin graph.

You can display all the properties of a node or an edge by clicking on it (on the sidebars or near the nodes and edges). Although I still have to code the display of Vertexproperties.

Le lundi 10 juillet 2017 17:27:32 UTC+2, Stephen Mallette a écrit :

Interesting - can you briefly describe the differences between your work and http://gremlinbin.com/ ?

On Mon, Jul 10, 2017 at 11:16 AM, Benjamin Ricaud <benjami...@gmail.com> wrote:

Hi all,

I needed a visualization for my graph database, so I made one with D3.js. I want to share it with everyone who is interested.

You can visualize a graph on a Gremlin server using REST or websockets, directly in your web browser.

It is on my github:

https://github.com/bricaud/graphexp

I call it Graphexp or Graph explorer.

A live demo is here if you want to try it without installing (tutorial on the github page): https://bricaud.github.io/graphexp/graphexp.html

Many things can be improved. Comments, suggestions or pull requests are welcome.

Benjamin

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

[Benjamin Ricaud]

Thanks Derek,

The small circle adjacent to a node is there to pin the node. If you click on it, the node will stay without fading. I will add a better documentation in the following days.

A button for coming back in the exploration would be a nice thing to have. I will think about adding it.

I will put your suggestion of editing the property values in my todo list as well.

Thanks for your feedback.

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/c2f74844-2a2f-4d4d-bed8-e690f20f9a8d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H4390tGRM2B5wThzyR_hw9uiXnA%3Dw0L5RnG5fhc%3DESXhdAQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[yashic...@srijan.net]

Hi Benjamin, 

I am running gremlin server on neptune and already loaded a graph on it. What changes need to be made in graphexp files to see the neptune graph ? 

[Benjamin Ricaud]

yes sure, you have to modify the line in the file graphConf.js:

const SINGLE_COMMANDS_AND_NO_VARS = false;

set it to true:

const SINGLE_COMMANDS_AND_NO_VARS = false;

It should work.

--
You received this message because you are subscribed to a topic in the Google Groups "Gremlin-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gremlin-users/uXkauKjxeA0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gremlin-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/0e25f13a-a4fa-431f-9541-fd90e767b47b%40googlegroups.com.

[Benjamin Ricaud]

sorry:

const SINGLE_COMMANDS_AND_NO_VARS = true;

[yashic...@srijan.net]

I made that  change but cannot see any node/ edges in the graph. Am i missing something ? 
 Requirement :  

My neptune instance 172.31.59.70:8182 is already loaded with a custom graph which i want to see in your graphex. I have added my host "172.31.59.70"   in host of graphConf.js file. How should i run the gremlin server ? 

sorry:

To unsubscribe from this group and all its topics, send an email to gremlin-user...@googlegroups.com.

[josh....@pearson.com]

Neptune is running a gremlin server if you loaded it with the gremlin bulk format (CSV) as described here: https://docs.aws.amazon.com/neptune/latest/userguide/load-api-reference-load.html. If you loaded RDF formatted data then the gremlin server is not running and you can't use any of the tinkerpop stack. 

For Neptune, in your configuration, you will also need to set:

const COMMUNICATION_METHOD = "GraphSON3"

Did you try hitting search with no inputs? Or hitting the "Get Graph Info" button? No data is retrieved without some kind of user action.

If none of that works set the communication to REST via the following property:

const COMMUNICATION_PROTOCOL = "REST";

open the browser developer tools and look at the outgoing network requests and put the responses in this thread so we can debug.

[yashic...@srijan.net]

[josh....@pearson.com]

It looks like you are having a networking issue to me. GraphExp makes request from the local client (browser) to the Neptune server directly. If you don't have connectivity from your local machine to the Neptune server, then you have two options. 

1. Change the VPC, or subnet group, or security group of your Neptune Instance to allow connections over port 8182 from your local machine. If you are lucky, it'll be as easy as changing or adding a security group rule. You could try adding a rule to allow you specific IP in. If that doesn't work you might be better off with option 2 depending on how comfortable you are with AWS Networking and if you even have permissions (I'm assuming you are using some kind of corporate account like I am).

2. Setup a proxy and change your GraphExp configuration HOST and PORT setting to point to the proxy. The proxy would need to allow connections from your IP over ports 80/443 and be able to make requests to your Neptune instance over port 8182. 

Also, since you are hosting the GraphExp project, you might run into CORS issues. I worked around that myself by writing a simple java server to listen to the requests from the client then make requests to the Neptune server and return the response. There's probably even easier ways to do that, but thats how I did it. In Chrome, using the file:// URL to open GraphExp, I did not run into any CORS issues with Neptune so you could always clone the project locally and go that route.

[nalgonda santosh]

Hi could u help me in connecting with neptune database connectivity.I had edited my host in graphconf.js .Still its not able to run.

[nalgonda santosh]

Could you please help me in connecting to Neptune through graphic.neptune endpoint is giving connection time out error.

[josh....@pearson.com]

Graphexp is either using websockets or REST calls to the Neptune gremlin server. There isn't anything special it is doing to communicate with Neptune.

Did you load data into your Neptune instance? Did you use the CSV format? I believe currently, a Neptune cluster can only support one format: Gremlin or SPARQL. If you loaded data in one of the RDF formats, you can't use gremlin. 

Can you use a browser or other tool like curl to do an http get to the following: http://$neptune_endpoint:$port/loader? 

Most likely there is something in your AWS networking keeping you from connecting. Have you checked the security groups attached to your neptune instance? Is port 8182 open to the IP of wherever you have GraphExp? If you were to setup a simple EC2 instance in the same subnets as your neptune cluster, could you ssh to it? 

[nalgonda santosh]

I am trying to connect through this URL by placing in host I'm graph conf.js.this is the URL Gremlin HTTP requests all use a single endpoint: http://your-neptune-endpoint:8182/gremlin.could you send your graph conf.js with host for your Neptune endpoint changed.should I use above endpoint or other to connect through graphex.

[Walton, Joshua]

This config works for me (changed the host name). I have to be on my company's VPN to connect to it as the neptune instance is in our private VPC and subnets. 

// configuration for the graph database access

const HOST = "mycluster.us-east-1-beta.rds.amazonaws.com"

const PORT = "8182"

// The communication protocol with the server can be "REST" or "websocket"

const COMMUNICATION_PROTOCOL = "REST";

//const COMMUNICATION_PROTOCOL = "websocket";

// TODO: configuration for the secure server

// The communication method can be GraphSON 1.0 (used by Gremlin 3.2)

// or GraphSON 3.0 (used by Gremlin 3.3)

//const COMMUNICATION_METHOD = "GraphSON1"

const COMMUNICATION_METHOD = "GraphSON3"

On Sat, Mar 10, 2018 at 8:27 PM, nalgonda santosh <nalgond...@gmail.com> wrote:

I am trying to connect through this URL by placing in host I'm graph conf.js.this is the URL Gremlin HTTP requests all use a single endpoint: https://urldefense.proofpoint.com/v2/url?u=http-3A__your-2Dneptune-2Dendpoint-3A8182_gremlin.could&d=DwIFaQ&c=0YLnzTkWOdJlub_y7qAx8Q&r=QdPeqj1OK0YSTCPW5_QAEQNyl3aW1kgWhepgeTd-TgU&m=iU1K3zI1OHzToDrIdrto-0-U-FSXfRrNevXVPNLseM0&s=tjLL7vs-XbPkDdeKMQMhC8Wk1LPXWPuEA9yMxVItPcU&e= you send your graph conf.js with host for your Neptune endpoint changed.should I use above endpoint or other to connect through graphex.

--
You received this message because you are subscribed to a topic in the Google Groups "Gremlin-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gremlin-users/uXkauKjxeA0/unsubscribe.

To unsubscribe from this group and all its topics, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/44f6e5c6-fa9d-4353-b9f1-c58821507bad%40googlegroups.com.

[nalgonda santosh]

You tried hitting api from rest client?

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAC_r-j0LC7QAjVJObnqRm6xu2225KrMAzdYpei-zz0VZkUmAjw%40mail.gmail.com.

[nalgonda santosh]

Could you send the Ajax URL you are sending for post request? Please help me

[kARTHIK R]

I was able to get an Amazon Neptune instance to work with GraphExp. Here are some of the callouts:

1. As mentioned in this email thread, the main challenge is to make sure that your DB instance is queryable from your Network (GraphExp makes AJAX calls to the DB directly). For making this happen, I created an Application Load Balancer on a public subnet, and added a listener that connects ALB's port to the Neptune Instance's IP and port. (Tested that I can telnet into ALB + port pair). I also had to attach a VPC security group to the DB so that it accepts inbound connections to the DB port from this public subnet.
   
2. I defaulted to REST over Websockets, since I'm using an ALB (HTTP). I'd imagine that it would work with WS as well, but you'll probably need an NLB (Network Load Balancer, to load balance at a TCP level).
3. Use TP version 3.3.* as that's what Neptune currently supports.
4. Set the hostname in GraphExp as the ALB endpoint and not the DB Instance/Cluster endpoint.

If I have some spare time, I'd like to add some features to GraphExp, especially SSL support, option to normalize the visualization (the graph gets pretty clunky once you expand a few nodes) etc..

Cheers,
Karthik

On Saturday, 10 March 2018 20:33:28 UTC-8, nalgonda santosh wrote:

Could you send the Ajax URL you are sending for post request? Please help me

On Mar 11, 2018 9:57 AM, "nalgonda santosh" <nalgond...@gmail.com> wrote:

You tried hitting api from rest client?

On Mar 11, 2018 9:56 AM, "Walton, Joshua" <josh....@pearson.com> wrote:

This config works for me (changed the host name). I have to be on my company's VPN to connect to it as the neptune instance is in our private VPC and subnets. 

// configuration for the graph database access

const HOST = "mycluster.us-east-1-beta.rds.amazonaws.com"

const PORT = "8182"

// The communication protocol with the server can be "REST" or "websocket"

const COMMUNICATION_PROTOCOL = "REST";

//const COMMUNICATION_PROTOCOL = "websocket";

// TODO: configuration for the secure server

// The communication method can be GraphSON 1.0 (used by Gremlin 3.2)

// or GraphSON 3.0 (used by Gremlin 3.3)

//const COMMUNICATION_METHOD = "GraphSON1"

const COMMUNICATION_METHOD = "GraphSON3"

On Sat, Mar 10, 2018 at 8:27 PM, nalgonda santosh <nalgond...@gmail.com> wrote:

I am trying to connect through this URL by placing in host I'm graph conf.js.this is the URL Gremlin HTTP requests all use a single endpoint: https://urldefense.proofpoint.com/v2/url?u=http-3A__your-2Dneptune-2Dendpoint-3A8182_gremlin.could&d=DwIFaQ&c=0YLnzTkWOdJlub_y7qAx8Q&r=QdPeqj1OK0YSTCPW5_QAEQNyl3aW1kgWhepgeTd-TgU&m=iU1K3zI1OHzToDrIdrto-0-U-FSXfRrNevXVPNLseM0&s=tjLL7vs-XbPkDdeKMQMhC8Wk1LPXWPuEA9yMxVItPcU&e= you send your graph conf.js with host for your Neptune endpoint changed.should I use above endpoint or other to connect through graphex.

--
You received this message because you are subscribed to a topic in the Google Groups "Gremlin-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gremlin-users/uXkauKjxeA0/unsubscribe.

To unsubscribe from this group and all its topics, send an email to gremlin-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/44f6e5c6-fa9d-4353-b9f1-c58821507bad%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Gremlin-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gremlin-users/uXkauKjxeA0/unsubscribe.

To unsubscribe from this group and all its topics, send an email to gremlin-user...@googlegroups.com.

[Girirajan Soundar Rajan]

Karthik,

Did you try any other visualization tool with amazon neptune. we are currently doing POC on amazon neptune and we are looking for  visualization tool.

Thanks and Regards

Girirajan S.

[kARTHIK R]

Hi Girirajan,

I have not tried any in person, but I've come across these online:
https://cambridge-intelligence.com/visualizing-the-amazon-neptune-database-with-keylines/
https://www.tomsawyer.com/aws-amazon-neptune-partner/

Karthik

[Girirajan Soundar Rajan]

Thanks Karthik. I checked that but those are propertied version.

--

You received this message because you are subscribed to the Google Groups "Gremlin-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/a51ae070-97a4-4a89-b1bb-f40badd28793%40googlegroups.com.

[Dinesh nautiyal]

Hi Karthik I am trying to host the graphexp in S3 and try to connect to neptune. but getting this error

Can't access database using REST at http://te.ciwpjbsljoew.us-east-1.neptune.amazonaws.com:8182

Could you please help me to fix this?

[Dinesh nautiyal]

Hi Walton,

I am also trying to connect graphexp with neptune. hosting graphexp in s3 and trying to connect from s3 to neptune. but connection is not establishing. Could you please help me in

On Sunday, 11 March 2018 09:56:09 UTC+5:30, Walton, Joshua wrote:

This config works for me (changed the host name). I have to be on my company's VPN to connect to it as the neptune instance is in our private VPC and subnets. 

// configuration for the graph database access

const HOST = "mycluster.us-east-1-beta.rds.amazonaws.com"

const PORT = "8182"

// The communication protocol with the server can be "REST" or "websocket"

const COMMUNICATION_PROTOCOL = "REST";

//const COMMUNICATION_PROTOCOL = "websocket";

// TODO: configuration for the secure server

// The communication method can be GraphSON 1.0 (used by Gremlin 3.2)

// or GraphSON 3.0 (used by Gremlin 3.3)

//const COMMUNICATION_METHOD = "GraphSON1"

const COMMUNICATION_METHOD = "GraphSON3"

On Sat, Mar 10, 2018 at 8:27 PM, nalgonda santosh <nalgond...@gmail.com> wrote:

I am trying to connect through this URL by placing in host I'm graph conf.js.this is the URL Gremlin HTTP requests all use a single endpoint: https://urldefense.proofpoint.com/v2/url?u=http-3A__your-2Dneptune-2Dendpoint-3A8182_gremlin.could&d=DwIFaQ&c=0YLnzTkWOdJlub_y7qAx8Q&r=QdPeqj1OK0YSTCPW5_QAEQNyl3aW1kgWhepgeTd-TgU&m=iU1K3zI1OHzToDrIdrto-0-U-FSXfRrNevXVPNLseM0&s=tjLL7vs-XbPkDdeKMQMhC8Wk1LPXWPuEA9yMxVItPcU&e= you send your graph conf.js with host for your Neptune endpoint changed.should I use above endpoint or other to connect through graphex.

--
You received this message because you are subscribed to a topic in the Google Groups "Gremlin-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gremlin-users/uXkauKjxeA0/unsubscribe.

To unsubscribe from this group and all its topics, send an email to gremlin-user...@googlegroups.com.