[TinkerPop3] Gremlin and Visualization with Gephi

[Stephen Mallette]

We are often asked how to go about graph visualization in TinkerPop.  We typically refer folks to Gephi or Cytoscape as the standard desktop data visualization tools.  The process of using those tools involves: getting your graph instance, saving it to GraphML (or the like) then importing it to those tools

TinkerPop3 now does two things to help make that process easier:

1. A while back we introduced the "subgraph" step which allows you to pop-off a Graph instance from a Traversal, which help greatly simplify the typical graph visualization process with Gremlin, where you are trying to get a much smaller piece of your large graph to focus the visualization effort.

2. Today we introduce a new :remote command in the Console.  Recall that :remote is used to configure a different context where Gremlin will be evaluated (e.g. Gremlin Server).  For visualization, that remote is called "gephi" and it configures the :submit command to take any Graph instance and push it through to the Gephi Streaming API.  No more having to import/export files!

The process for using this is pretty simple.  First, you have to get Gephi "ready":

1. Download Gephi 

2. Install the Streaming plugin

3. Create a new workspace (for this example, use "Workspace 0")

4. Click on the Streaming window, expand Master, right-click Master Server and select "Start". This will start the streaming server that will listen for events from the Console.

5. Click on the Layout window, select Force Directed from the dropdown and click "Run".  This step is sorta optional, but fun because it will apply a force-directed layout while the graph is populating from the Console. 

Next, start a Gremlin Console session and execute these command to generate a graph using the gremlin-algorithms module to produce some dummy data to visualize:

gremlin> g = new TinkerGraph()

==>tinkergraph[vertices:0 edges:0]

gremlin> (0..<500).each{g.addVertex()}

==>0

...

==>497

==>498

==>499

gremlin> dist = new PowerLawDistribution(2.1)

==>PowerLawDistribution{gamma=2.1, multiplier=0.0}

gremlin> gen = DistributionGenerator.create(g).label('knows').outDistribution(dist).inDistribution(dist).expectedNumEdges(5000).build()

==>com.tinkerpop.gremlin.algorithm.generator.DistributionGenerator@319bc845

gremlin> gen.generate()

==>4806

That generates a 500 vertex graph with 4806 edges.  Next configure the :remote and :submit (aka :>) the graph to Gephi:

gremlin> :remote connect gephi

==>connection to Gephi - http://localhost:8080/workspace0

gremlin> :> g

==>tinkergraph[vertices:500 edges:4806]

The graph will start to draw itself in Gephi as soon as you :submit.  Instead of taking my word for it, I made a video where you see the process

http://www.tinkerpop.com/downloads/gremlin-gephi.mp4

There's a lot of directions this feature could go.  I just wanted to cover the basic use case I outline above and eliminate all the import/export, file copy, etc.  I think there's many interesting features that could be applied here.  I also find it interesting how flexible the :remote/:submit commands are.  Pretty nice that the same infrastructure can support Gremlin Server, Hadoop/Giraph and now Gephi.  Can't wait to see how it might be used next!

Enjoy!

Stephen

[Stephen Mallette]

For those who didn't feel like downloading the video file for this visualization demo  - i finally got the quality right for youtube: 

https://www.youtube.com/watch?v=JPCsO1O1BMs

Stephen

[ziggy]

pretty cool.  That's going to be useful :) (assuming I'm able to move to Tinkerpop3)

The example worked for me on OSX-Mavericks and was even worth the Java 8 install pain.

[Stephen Mallette]

Out of curiousity...what would be your barrier to "move to TinkerPop3"?

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Louis Clark]

nice of you to ask.  Input is giving me trouble at the moment.  I can't get a graph into gremlin from neo4j and the graphml files I have from gremlin 2.5.0 are giving me a "Edge label can not be null" error.  I just posted a question about the neo4j input under an existing topic.  thanks

--
You received this message because you are subscribed to a topic in the Google Groups "Gremlin-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gremlin-users/pT87rPvty0g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gremlin-user...@googlegroups.com.

[Randall Barnhart]

I got some time to spend "hacking" on this, with the intent of dynamically visualizing traversals as it progresses through vertices.

By modifying GephiRemoteAcceptor to accept Traversal object types submitted to the remote console, and by taking advantage of the new Memory object within a traversal to store("x") vertices at particular steps, I was able to iterate through these stores and highlight/color the nodes that were in the traversal at each store step:

https://gist.github.com/randompi/d3d6c5c2989f97d63d1b

I put in a 2 second delay between step visualization to aid in seeing the traversals... but this could certainly be parameterized.  You'll also notice that I "decay" the colors of the nodes that have been visited in past steps, so that at the end you can see the traversal path through the graph following the gradient of blue to light green.

Here's a short screencast demonstrating the visualization: http://youtu.be/KXGEony69JM

Let me know if you guys would be interested in a pull request around this and I'll try and spruce it up, making it more parameterized.

I would also like to play around with making special step: g.autoviz().v(1)...

This would have to create a new kind of TraversalStrategy that would inject "store" steps into appropriate places in the pipeline... thoughts?

Enjoy!

-Randall

@randompi

[Stephen Mallette]

Mr. Randompi, this is pretty cool.  A polished up version of this, would make a good pull request.  I definitely had it in mind as a feature from the time I first put the Gephi Plugin together, plus you and I have talked about this separately on a few occasions going back to your original work around this type of visualization with earlier versions of TinkerPop.  

Your example was cool because it let us clearly see what was happening in the visualization, though I can't help wondering what a traversal would look like over larger graph touching more things. :)

I like the idea of a custom strategy to auto-inject the store() steps.  You might not need to do an autoviz step though.  Might be cleaner to eval the groovy string to a Traversal then register a strategy on it before using it....i think that will work.  

We should probably think through the flow of usage a bit too if you want to do a pull request.  So, as I see it in your video you first have to do  :> g and then you basically just update colors in graph that's in gephi.  Maybe that's just how you have it for now....would be cool if you could go right from Traversal to an empty gephi workspace (or maybe it does that now).  Could you clarify a bit please?

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/dff41fb3-b589-4d11-9570-832de7f8dc2f%40googlegroups.com.

[Randall Barnhart]

Thanks Stephen... I did run the visualization on your 500 node power law distribution network, and it looks pretty cool, but the larger the dataset, the harder it is to comprehend whats going on.  I'll post a followup video with a killer soundtrack later... :)

As far as the workflow, you do have to submit a graph to get the "nodes" added into Gephi.  Once they are there, the traversal code is using the change node API ("cn" >> https://wiki.gephi.org/index.php/Graph_Streaming#Supported_formats), to add the color attribute.  Interestingly it looks like when you call this on the Gephi Graph Server, it resets the position on a center line in the view (hence why the Fuchterman Reingold layout continuously running looks like its rotating the nodes).  We could also explicitly set the nodes position, but its hard to guess how zoomed in the gephi view is...  

I didn't see through the Traversal API how you could access the underlying graph to essentially add all the nodes/edges up front in one fell swoop.  If there is some other way, then that would be cool.  Although, I do think in a real-world scenario, an analyst will want to look at the graph first (:> g), possibly run a layout or tweak layout options, change sizes of nodes possibly from pagerank or degree attributes, possibly color by types, and then run a query to be able to understand what parts of the graph the query is touching.  Also note, that you can clear all the colors on the nodes and run a second query (same or different) and it will recolor the graph again.

I wanted to run this on some of our "course" data, but I couldn't get the .graphml or graphson to be read in correctly in tinkerpop3 (from our blueprints 2.3.0 code that generated it).

One other oddity is that the gremlin console reports an error on accessing the Memory, and I chose not to display the stack trace.  And the Traversal object does seem to be appropriately populated... Any thoughts on this (The memory does not have a value for provided variable: %&%cap)?

gremlin> d2 = g.v(2).store("1").in('knows').store("2").out('knows').filter{it.get().value('age') > 30}.store("3").outE('created').filter{it.get().value('weight') > 0.5}.inV.store("4")

The memory does not have a value for provided variable: %&%cap

Display stack trace? [yN] y

java.lang.IllegalArgumentException: The memory does not have a value for provided variable: %&%cap

at com.tinkerpop.gremlin.process.Traversal$Memory$Exceptions.variableValueDoesNotExist(Traversal.java:90)

at com.tinkerpop.gremlin.process.util.DefaultMemory.get(DefaultMemory.java:29)

at com.tinkerpop.gremlin.process.graph.step.sideEffect.SideEffectCapStep.processNextStart(SideEffectCapStep.java:37)

at com.tinkerpop.gremlin.process.util.AbstractStep.hasNext(AbstractStep.java:75)

at com.tinkerpop.gremlin.process.util.DefaultTraversal.hasNext(DefaultTraversal.java:53)

at java_util_Iterator$hasNext.call(Unknown Source)

at com.tinkerpop.gremlin.console.Console$_closure3.doCall(Console.groovy:127)

I'll work on sprucing up the existing "gist" to prep for a pull request.

Thanks,

Randall

[Marko Rodriguez]

Hey,

That is a bug………………. given that you named the variable store("4") it should not be looking for Graph.Key.hidden("cap"), but "4".

Will fix.

Marko.

[Marko Rodriguez]

Hi,

Fixed: https://gist.github.com/okram/a003eebe90f940e2f58f

Pushed.

https://github.com/tinkerpop/tinkerpop3/blob/e7ca4d82d6bfb53604467e04b6fcebf0e9223ed4/gremlin-core/src/main/java/com/tinkerpop/gremlin/process/graph/strategy/SideEffectCapStrategy.java

Thanks Mr. Cherry,

Marko.

http://markorodriguez.com

[Stephen Mallette]

Thanks Stephen... I did run the visualization on your 500 node power law distribution network, and it looks pretty cool, but the larger the dataset, the harder it is to comprehend whats going on.  I'll post a followup video with a killer soundtrack later... :)

looking forward to that. awesome.  i'm wondering how a running this on a larger graph would help people debug gremlin - or if it would help at all?

I didn't see through the Traversal API how you could access the underlying graph to essentially add all the nodes/edges up front in one fell swoop.  If there is some other way, then that would be cool.  Although, I do think in a real-world scenario, an analyst will want to look at the graph first (:> g), possibly run a layout or tweak layout options, change sizes of nodes possibly from pagerank or degree attributes, possibly color by types, and then run a query to be able to understand what parts of the graph the query is touching.  Also note, that you can clear all the colors on the nodes and run a second query (same or different) and it will recolor the graph again.

The analyst looking at the graph first and tweaking gephi options makes sense to me.  My vision has always been that visualization would be performed over a subgraph (either via graph wrapper subgraph or the subgraph step). That said, it would be nice if we could just go from traversal right to gephi without visualizing "g" first.  The "g" is actually stored as a hidden variable on TinkerGraph (though not on Neo4jGraph):

gremlin> t = g.V;null

==>null

gremlin> t.memory().get(Graph.Key.hidden("g"))

==>tinkergraph[vertices:6 edges:6]

Marko, is that "wrong".  Should all traversals have a reference to the Graph instance on them?  If so, we should write a test to enforce that.

 

I wanted to run this on some of our "course" data, but I couldn't get the .graphml or graphson to be read in correctly in tinkerpop3 (from our blueprints 2.3.0 code that generated it).

Yes, GraphSON is not compatible between 2.x and 3.x.  There were inconsistencies in GraphML in 2.x that I corrected  for 3.x, but I'm sorta surprised that doesn't transfer over.  That's not so good...i'll look into that.

I'll work on sprucing up the existing "gist" to prep for a pull request.

ok...look forward to your next iteration.

Stephen

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/5fc6d1c4-de5d-4502-a2fd-a4c8cd3c28af%40googlegroups.com.

[Marko Rodriguez]

Hi,

gremlin> t = g.V;null

==>null

gremlin> t.memory().get(Graph.Key.hidden("g"))

==>tinkergraph[vertices:6 edges:6]

Marko, is that "wrong".  Should all traversals have a reference to the Graph instance on them?  If so, we should write a test to enforce that.

For OLTP systems, yes. Once you go OLAP, its usually smart to remove the reference else you get into Serialization nonsense. See:

https://github.com/tinkerpop/tinkerpop3/blob/master/tinkergraph-gremlin/src/main/java/com/tinkerpop/gremlin/tinkergraph/structure/TinkerHelper.java#L146

Test written now (MemoryTest): 

https://github.com/tinkerpop/tinkerpop3/blob/master/gremlin-test/src/main/java/com/tinkerpop/gremlin/process/graph/step/util/MemoryTest.java

Updated Neo4j-Gremlin and Giraph-Gremlin accordingly as well.

Finally, for Groovy users, I overloaded the "-" operator so you can do this easily:

-"g" -> Graph.Key.hide("g")

-Graph.Key.hide("g") -> "g"

Some sugar for that syn-tax.

Thanks,

Marko.

http://markorodriguez.com

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H438ftmwEfWuyRx-Ju72NoMRdyQHQ0XwJcZGqnQ5C1%2BZ4mg%40mail.gmail.com.

[daniel...@gmail.com]

That is awesome!  Good work man!

Dan

Sent from my iPhone

[Randall Barnhart]

I pulled and built the latest and now I'm running into this issue when I try to launch the console:

localhost:bin barnhart$ ./gremlin.sh 

         \,,,/

         (o o)

-----oOOo-(3)-oOOo-----

Exception in thread "main" java.io.FileNotFoundException: /Users/barnhart/dev/Gambit/Projects/tinkerpop3/gremlin-console/bin/ext/plugins.txt (No such file or directory)

at java.io.FileOutputStream.open(Native Method)

at java.io.FileOutputStream.<init>(FileOutputStream.java:206)

at java.io.FileOutputStream.<init>(FileOutputStream.java:156)

at java.io.FileWriter.<init>(FileWriter.java:90)

at org.codehaus.groovy.runtime.ResourceGroovyMethods.newWriter(ResourceGroovyMethods.java:1699)

at org.codehaus.groovy.runtime.ResourceGroovyMethods.withWriter(ResourceGroovyMethods.java:1790)

at org.codehaus.groovy.runtime.dgm$882.invoke(Unknown Source)

at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:271)

at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:53)

at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)

at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)

at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)

at com.tinkerpop.gremlin.console.Mediator.writePluginState(Mediator.groovy:62)

???

-Randall

On Wednesday, June 25, 2014 2:23:45 PM UTC-6, Stephen Mallette wrote:

[Stephen Mallette]

sorry randall.  developers have a bit of a burden at the moment.  you can add an "ext" directory in the root of your tinkerpop git source with a plugins.txt file there.  or you can build, then change directory down to the distribution: cd gremlin-console/target/*stand*/ and run the bin/gremlin.sh from there.  we have some pathing problems i'm not sure how to best solve for all environments.  

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/fa739e17-d2c6-443a-9fd5-ed4b6152aa92%40googlegroups.com.

[Marko Rodriguez]

Here is my plugins.txt in ext.

com.tinkerpop.gremlin.console.plugin.DriverGremlinPlugin

com.tinkerpop.gremlin.console.plugin.SugarGremlinPlugin

com.tinkerpop.gremlin.console.plugin.UtilitiesGremlinPlugin

Marko.

http://markorodriguez.com

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H439U884osU4uzXUNDBme%3DuyMnuK4THX0aMTGUdp3_s8UMg%40mail.gmail.com.

[Randall Barnhart]

I was able to get it launched when I created the ext/ dir and touched plugins.txt... But now that I'm connected to a gephi remote, and try to submit a power law distributed graph I'm getting a new error on this line in GephiRemoteAcceptor.groovy: g.V().sideEffect { addVertexToGephi(it.instance()) }.iterate()

gremlin> gen.generate()

==>2806

gremlin> :rem connect gephi

==>connection to Gephi - http://localhost:8080/workspace0 with stepDelay:1000, startRGBColor:[0.0, 1.0, 0.5], and colorToFade:g

gremlin> :> g

==>tinkergraph[vertices:500 edges:2806]

Connection to http://localhost:8080 refused

Display stack trace? [yN] N

gremlin> :> g

==>tinkergraph[vertices:500 edges:2806]

No signature of method: com.tinkerpop.gremlin.process.SimpleTraverser.instance() is applicable for argument types: () values: []

Possible solutions: inspect(), inflate(com.tinkerpop.gremlin.structure.Vertex), isCase(java.lang.Object)

Display stack trace? [yN] Ny

gremlin> :> g

==>tinkergraph[vertices:500 edges:2806]

No signature of method: com.tinkerpop.gremlin.process.SimpleTraverser.instance() is applicable for argument types: () values: []

Possible solutions: inspect(), inflate(com.tinkerpop.gremlin.structure.Vertex), isCase(java.lang.Object)

Display stack trace? [yN] y

groovy.lang.MissingMethodException: No signature of method: com.tinkerpop.gremlin.process.SimpleTraverser.instance() is applicable for argument types: () values: []

Possible solutions: inspect(), inflate(com.tinkerpop.gremlin.structure.Vertex), isCase(java.lang.Object)

at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:55)

at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:46)

at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)

at com.tinkerpop.gremlin.console.plugin.GephiRemoteAcceptor$_submit_closure1.doCall(GephiRemoteAcceptor.groovy:153)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:483)

at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)

at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:324)

at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:278)

at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1016)

at groovy.lang.Closure.call(Closure.java:423)

at org.codehaus.groovy.runtime.ConvertedClosure.invokeCustom(ConvertedClosure.java:51)

at org.codehaus.groovy.runtime.ConversionHandler.invoke(ConversionHandler.java:81)

at com.sun.proxy.$Proxy7.accept(Unknown Source)

at com.tinkerpop.gremlin.process.graph.step.sideEffect.SideEffectStep.lambda$new$cdc45521$1(SideEffectStep.java:17)

at com.tinkerpop.gremlin.process.graph.step.sideEffect.SideEffectStep$$Lambda$24/1235635586.test(Unknown Source)

at com.tinkerpop.gremlin.process.graph.step.filter.FilterStep.processNextStart(FilterStep.java:29)

at com.tinkerpop.gremlin.process.util.AbstractStep.next(AbstractStep.java:64)

at com.tinkerpop.gremlin.process.util.AbstractStep.next(AbstractStep.java:14)

at com.tinkerpop.gremlin.process.util.DefaultTraversal.next(DefaultTraversal.java:62)

at com.tinkerpop.gremlin.process.Traversal.iterate(Traversal.java:172)

at com.tinkerpop.gremlin.process.Traversal$iterate.call(Unknown Source)

at com.tinkerpop.gremlin.console.plugin.GephiRemoteAcceptor.submit(GephiRemoteAcceptor.groovy:153)

[Marko Rodriguez]

Hi,

You need to unwrap the vertex from SimpleTraverser. You probably mean:

addVertexToGephi(it.get())

Marko.

http://markorodriguez.com

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/ff0f3ad5-4ff1-4f9b-a258-008e7a9f4c0e%40googlegroups.com.

[Stephen Mallette]

Just a quick update on the TinkerPop3 Gephi integration

We accepted a nice pull request from RandomPi on the Gephi Plugin for the console.  He provided some neat configurations to the plugin which yields some extra control and some good documentation.  You can read more about the latest on that effort here:

http://www.tinkerpop.com/docs/3.0.0-SNAPSHOT/#gephi-plugin

This plugin is developing into a really nice feature for Gremlin.  Thanks, Randall, for the contribution!

Enjoy!

Stephen

On Monday, August 11, 2014 7:32:01 PM UTC-4, Marko A. Rodriguez wrote:

Hi,

You need to unwrap the vertex from SimpleTraverser. You probably mean:

addVertexToGephi(it.get())

Marko.

http://markorodriguez.com

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/fa739e17-d2c6-443a-9fd5-ed4b6152aa92%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H439U884osU4uzXUNDBme%3DuyMnuK4THX0aMTGUdp3_s8UMg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.

[Randall Barnhart]

Thanks guys!  Glad this made it in...

I've posted a longer YouTube video here demoing the capability on a larger generated scale free graph with the latest tinkerpop3 build (excuse the slowness due to my outdated hardware):

http://youtu.be/wtiUjna9IMo

And here's the gremlin session to go with it:

localhost:gremlin-console barnhart$ bin/gremlin.sh 

         \,,,/

         (o o)

-----oOOo-(3)-oOOo-----

plugin activated: gephi

gremlin> :plugin use gephi

==>gephi activated

gremlin> g = new TinkerGraph()

==>tinkergraph[vertices:0 edges:0]

gremlin> (0..<500).each{g.addVertex()}

==>0

==>1

==>2

...

==>498

==>499

gremlin> dist = new PowerLawDistribution(2.1)

==>PowerLawDistribution{gamma=2.1, multiplier=0.0}

gremlin> gen = DistributionGenerator.build(g).label('knows').outDistribution(dist).inDistribution(dist).expectedNumEdges(5000).create()

==>com.tinkerpop.gremlin.algorithm.generator.DistributionGenerator@5cbe877d

gremlin> gen.generate()

==>3507

gremlin> :remote connect gephi

==>connection to Gephi - http://localhost:8080/workspace0 with stepDelay:1000, startRGBColor:[0.0, 1.0, 0.5], colorToFade:g, colorFadeRate:0.7

gremlin> :> g

==>tinkergraph[vertices:500 edges:3507]

gremlin> t = g.V[119].store('1').both.dedup.store('2').both.dedup.store('3').both.dedup.store('4')

==>[v[494], v[191], v[156], v[35], v[495], v[12], v[204], v[218], v[332], v[27], v[488], v[69], v[485], v[320], v[241], v[290], v[23], v[32], v[243], v[133], v[280], v[41], v[327], v[150], v[104], v[219], v[324], v[205], v[102], v[68], v[244], v[403], v[275], v[363], v[499], v[270], v[162], v[216], v[284], v[160], v[496], v[285], v[175], v[77], v[476], v[153], v[375], v[402], v[419], v[236], v[154], v[88], v[163], v[114], v[136], v[184], v[233], v[424], v[331], v[277], v[262], v[34], v[428], v[407], v[157], v[374], v[101], v[115], v[297], v[257], v[131], v[46], v[326], v[268], v[132], v[220], v[142], v[201], v[59], v[21], v[180], v[139], v[213], v[343], v[366], v[84], v[64], v[185], v[279], v[49], v[147], v[78], v[4], v[245], v[208], v[321], v[287], v[42], v[140], v[441], v[2], v[36], v[399], v[260], v[80], v[91], v[417], v[388], v[348], v[0], v[266], v[111], v[166], v[299], v[445], v[33], v[6], v[487], v[352], v[412], v[86], v[210], v[148], v[57], v[82], v[446], v[259], v[89], v[118], v[427], v[448], v[415], v[273], v[122], v[413], v[52], v[1], v[271], v[100], v[464], v[30], v[230], v[389], v[458], v[395], v[93], v[8], v[342], v[10], v[345], v[11], v[432], v[433], v[436], v[438], v[440], v[362], v[455], v[364], v[463], v[103], v[107], v[203], v[109], v[128], v[481], v[135], v[54], v[385], v[392], v[278], v[491], v[164], v[167], v[76], v[179], v[397], v[329], v[304], v[200], v[90], v[308], v[267], v[311], v[96], v[479], v[346], v[226], v[349], v[449], v[276], v[24], v[25], v[65], v[242], v[105], v[174], v[250], v[176], v[367], v[79], v[112], v[302], v[207], v[198], v[482], v[92], v[416], v[5], v[238], v[224], v[452], v[361], v[447], v[306], v[222], v[110], v[189], v[350], v[223], v[356], v[43], v[146], v[472], v[378], v[212], v[227], v[29], v[121], v[196], v[490], v[28], v[215], v[188], v[123], v[26], v[3], v[126], v[73], v[310], v[291], v[7], v[380], v[143], v[296], v[152], v[468], v[199], v[129], v[87], v[489], v[456], v[414], v[169], v[16], v[55], v[31], v[62], v[193], v[53], v[434], v[355], v[371], v[338], v[391], v[221], v[18], v[444], v[450], v[239], v[75], v[247], v[282], v[466], v[396], v[398], v[373], v[313], v[194], v[390], v[379], v[339], v[359], v[303], v[319], v[19], v[255], v[83], v[9], v[231], v[214], v[474], v[443], v[442], v[261], v[37], v[159], v[470], v[462], v[67], v[281], v[483], v[234], v[151], v[351], v[272], v[404], v[74], v[381], v[309], v[334], v[421], v[13], v[130], v[202], v[497], v[165], v[14], v[98], v[318], v[423], v[439], v[106], v[467], v[48], v[117], v[124], v[228], v[171], v[173], v[493], v[254], v[405], v[265], v[305], v[314], v[368], v[325], v[187], v[209], v[72], v[66], v[192], v[354], v[85], v[498], v[58], v[344], v[393], v[315], v[256], v[181], v[17], v[22], v[293], v[317], v[435], v[113], v[477], v[384], v[225], v[286], v[401], v[197], v[336], v[141], v[377], v[298], v[406], v[232], v[237], v[47], v[301], v[95], v[300], v[454], v[365], v[451], v[158], v[70], v[372]]

gremlin> :> t

Visualizing vertices at step: 1... visited: 1

Visualizing vertices at step: 2... visited: 1

Visualizing vertices at step: 3... visited: 271

Visualizing vertices at step: 4... visited: 381

gremlin> :remote config colorToFade g

==>connection to Gephi - http://localhost:8080/workspace0 with stepDelay:1000, startRGBColor:[0.0, 1.0, 0.5], colorToFade:g, colorFadeRate:0.7

gremlin> :remote config startRGBColor [1.0, 0.5, 0.7]

==>expects [host <hostname>|port <port number>|workspace <name>|stepDelay <milliseconds>|startRGBColor <RGB array of floats>|colorToFade: <char r|g|b>]|colorFadeRate: <float>

gremlin> :remote config startRGBColor [1.0,0.5,0.7]

==>connection to Gephi - http://localhost:8080/workspace0 with stepDelay:1000, startRGBColor:[1.0, 0.5, 0.7], colorToFade:g, colorFadeRate:0.7

gremlin> :> t

Visualizing vertices at step: 1... visited: 1

Visualizing vertices at step: 2... visited: 1

Visualizing vertices at step: 3... visited: 271

Visualizing vertices at step: 4... visited: 381

gremlin> 

Also, if any of you are struggling to get Gephi 0.8.2 working on a Mac OS X with JDK 1.7 or higher, you may have to point to an older version by adding the line:

jdkhome="/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/"

... in the file: /Applicaitons/Gephi.app/Contents/Resources/gephi/etc/gephi.conf

For reference on this issue see: https://github.com/gephi/gephi/issues/748

Happy Traversualizing!

-Randall

http://markorodriguez.com

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

[Stephen Mallette]

Nice video...Randall - thanks for sharing that.  It was nice to see how it worked on something "big".  

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/492c892a-13e1-4ebe-83ae-cb632833d5a3%40googlegroups.com.

[Marko Rodriguez]

Wow man. That traversal that highlights vertices/edges as it goes is psychedelic!

Marko.

http://markorodriguez.com

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H43_gFN4Pmhq81LVcM%3DjNUq6pvzH%3DPo4d0rCH1t%3D0DW1Qyw%40mail.gmail.com.

[rana maher]

Hi Stephen,

I am trying to create a dummy graph and I used the code you mentioned above but it does not work with me.

I have titan1.0.0 and when I tried to run this line  "dist = new PowerLawDistribution(2.1)" in the gremlin console but it gives me this error:

 "groovysh_evaluate: 3: unable to resolve class PowerLawDistribution"

Do I need to import certain library ??

Cheers,

Rana

[Stephen Mallette]

I suggest you follows the instructions in the reference documentation:

http://tinkerpop.apache.org/docs/3.0.2-incubating/#gephi-plugin

Graph generators were moved from gremlin-core a long long time ago - before GA.  They live in gremlin-test now if you really want to use them.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/743e9f19-0b57-4b3b-9343-894ba3d37493%40googlegroups.com.

[sheshadr...@getfreebird.com]

Hi,

I've tried exactly the code that is in latest tinkerpop3 page and there is no graph showing up in Gephi 0.91 (yes the streaming plugin was installed and master server was started on port 8080.  Only exception to the instructions is that Gephi seemed to only open workspace1 and never workspace0.  So I changed the workspace for gephi in Gremlin via the config property for gephi plugin in gremlin.  Still nothing... 

• version of Gremlins used (with JDK1.8)
• 3.1.0-incubating (as part of titan1.1-snapshot), 
• gremlin 3.1.1-incubating (directly from Tinkerpop3 site)

In both cases, I tried several times, and looked for any errors in the Gephi logs -- none!

Then I noticed that Streaming plugin of Gephi (https://marketplace.gephi.org/plugin/graph-streaming/) is reportedly meant for Gephi 0.82.  So I did the following:

1. Changed JDK to 1.7
2. Uninstalled Gephi 0.91
3. Installed Gephi 0.82 (NOTE: there is only a 0.82-BETA available not sure if this is the issue)
4. tried to install the plugin -- Gephi couldn't find the catalog at http://gephi.org/updates/official/0.8.2/catalog.xml... 
5. downloaded the plugin as a zip file and tried to install the downloaded zip -- nothing; no errors in the log file and no plugin loaded.

Could you please share:

• version of gephi used -- if it was 0.82 and it wasn't the BETA that's avail on gephi.org under "Older versions", then where can I get a working version
• version of gremlin used

Lastly, this is a Gremlin plugin - right -- so if I'm using Titan 1.x (1.0.0 or 1.1.0-SNAPSHOT) with a version of gremlin can I display a graph using Gephi ?? 

Thanks,

S-

[Stephen Mallette]

It's 0.8.2-beta - I don't think they ever had a full release of that.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/5f081848-0977-470b-8e06-6cc1357ee144%40googlegroups.com.

[Sheshadri Mantha]

Thanks Stephen...

Then I believe I'm stuck ! As the catalog for plugins isn't accessible anymore and using the zip file doesn't seem to work.

Do you perhaps know a way to tell Gephi 0.8.2 to over ride the catalog location ??  I'll ask in gephi channel also.  

--
You received this message because you are subscribed to a topic in the Google Groups "Gremlin-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gremlin-users/pT87rPvty0g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H43843aQUfQAV08U3fn5Ay2k5v64b736J_62ZZaDn98oSAA%40mail.gmail.com.

[Stephen Mallette]

I'm not sure what you're asking unfortunately - i guess you will have to ask on the Gephi mailing list as you mentioned.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAO-_dxSoFtcau4q%3DCkthMkoRkWV8C97oCE6QGPKdg1An0MtHrQ%40mail.gmail.com.

[sheshadr...@getfreebird.com]

Ok ... have posted to stackoverflow [gephi] here are my questions:

1. You are still using 0.8.2 - when you go to Tools -> Plugins, can you load any new plugins ?  Or get Updates ?  This doesn't work for me and complains about the catalog of plugins being not available at: https://gephi.org/updates/official/0.8.2/catalog.xml -- are you seeing the same error?
1. Browsing to https://gephi.org/updates/official/0.8.2/catalog.xml in browser displays XML but there's no Streaming Plugin available there -- what is the location of the catalog for your installation ?  is it different than the default with 0.8.2 ?? If not, did you change this location?  IF so, how ?  :)
2. [secondly] have you tried with gephi 0.9.1 and latest gremlin ?  Are you seeing the tinkergraph in Gephi ?
3. Lastly, I've just downloaded the gremlin code and just finished compiling it... I'll try to run gremlin console in debugger w/ Gephi 0.9.1 and report back what I find... 

BIT LATER

NM -- I got to debugging gremlin-gephi plugin in IDE and I noticed that while gremlin-gephi *WAS* sending data, it wasn't being displayed by gephi - with play with the visualization.

Long & Short -- your gephi gremlin plugin appears to work... even in 0.9.1

Now to read up on how to do visualizations in Gephi.

[Stephen Mallette]

I'm still not 100% certain as to why it won't work with 0.9.1. I have not tested it myself. I only have reports from users who have struggled with it. I will likely look into the problem for TinkerPop 3.1.3. Not sure when I will get to it though. If someone else can jump ahead and figure it out, that would be awesome.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/6652ef1f-6ab3-4fd4-83aa-662d5c74d658%40googlegroups.com.

[Sheshadri Mantha]

Perhaps these points would help... 

1. the context window in gephi does indicate the correct number of edges and nodes (6 and 6 in the modern tinker graph case).
2. If I futz around with the color of the node labels, edge labels then I see something... by "default" I see nothing
3. There appears to be a layout issue as well as everything seems to be laid linearly (i'm imagining) one on top of another.  See my snip

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H43_Bw_hVA3zR1YQfSTrbeXvfv7GdxyFLBAPJjfT%3DS_ehmA%40mail.gmail.com.

[Stephen Mallette]

thanks for all that. i'm going to guess that's largely a gephi issue. what is your OS?

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAO-_dxR7vyVezbLKHcROqS0DWergYNh1%2B6dAhAdq%2BF5MYdvqew%40mail.gmail.com.

[Sheshadri Mantha]

Windows 10...

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H43-UgBx5_bQ25ytvNpf4Cs__RvLX6G0sd8uzxrJZ%2BmYpYA%40mail.gmail.com.

[Stephen Mallette]

I've known windows to sometimes be shaky with display/refresh/layout on Gephi in earlier versions. I'm feeling more strongly that this is a bug on their side, especially since the data is all showing up in gephi. 

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAO-_dxTWN03RpFFDHOx11JKRghUwVN5Dg_vjKK9r2DruhDXz3Q%40mail.gmail.com.

[Sheshadri Mantha]

I tend to agree with your assessment.

I'll try on Ubuntu ec2...

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H438-co8B3Ms_b7BLpkiEhSYV1CCw9eh3x%2BTtoz6qkWSnmQ%40mail.gmail.com.

[Dylan Bethune-Waddell]

Hmm - I ran into this issue prior to a presentation a month or two ago and had to jump through the same kind of hoops you're talking about. Here is how I got it to work:

1) use Gephi 0.8.2 beta

2) use TinkerPop 3.1.0-SNAPSHOT (maybe another version works I used this)

3) download the gephi streaming plugin for 0.8.2 from https://marketplace.gephi.org/plugin/graph-streaming/

4) unzip the archive, and for each nbm file unpack it into a jar using UnpackNBM.jar

    - described here https://github.com/gephi/gephi/wiki/How-to-use-plug-ins-with-the-Toolkit

5) move all jars named org-gephi*.jar it to the GEPHI_HOME/modules/gephi directory

I was in a rush and don't quite remember what I did to get gephi to recognize the jars - will post back here when I figure that out but this might get you started. Hope this helps.

Best,

Dylan 

[Stephen Mallette]

Dylan, it would be awesome if you could do that. In the worst case, we could even append your instructions to the reference docs for the Gephi Plugin - assuming this remains unfixed for a while.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/f671166b-61ae-4963-ac44-c262e24837f0%40googlegroups.com.

[Dylan Bethune-Waddell]

Hi Stephen,

Right on - okay, I'll look into it when I am back at my computer tonight in a few hours and hopefully I can even try a PR for the docs... would be nice to learn how that works.

Thanks,

Dylan

[Sheshadri Mantha]

Dylan - you Rock!

Stephen - thanks for your patience.

I've verified that unzipping and loading the nbm's to 0.8.2 works.  Here is a step-by-step on how to add them to 0.8.2: http://stackoverflow.com/questions/36654801/need-to-use-0-8-2-with-streaming-plugin-but-gephi-cant-find-catalog/36658521#36658521

Still not sure what the issue with 0.9.1 is - I'll continue playing with layouts and perhaps 0.9.2 Gephi code.

Thanks,

S-

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/b0cdf36c-c54a-47a3-b8bc-3db594520486%40googlegroups.com.

[Sheshadri Mantha]

Hey - 

I just posted an email with instructions... hopefully you are able to draw from it to update the docs.

If you are able to teach me how, I'd be willing to take a shot at doc update.

Thank you *BOTH*

s-

[Mark Baynes]

Hiya

Seems more than a few people have been having lots of problems connecting Gremlin to Gephi using the Streaming Plugin. I was trying to get this work today when I noticed that the plugin (https://gephi.org/plugins/#/plugin/graphstreamingwas) was updated on 30th July to a 0.9.1 Gephi version by Andre Panisson and it works like a dream! Woo hoo! The connection process from Gephi Streaming is slightly different and am not sure I used it correctly but it sure works :) 

I have screengrabs of complete process which I will post separately at a later date (docs will need updating) but two key ones are attached. Hope this is of help to others. 

cheers from London, 

Mark

[Stephen Mallette]

I started going through the process to see what changed. I got the latest version of the streaming plugin at 1.0.3 and gephi 0.9.1 (which i don't think changed for me since the last time i tested) . If you explicitly follow the instructions here:

http://tinkerpop.apache.org/docs/current/reference/#gephi-plugin

I dont' see any difference under these versions. Everything works. I did get hung up for a minute though because I tried to use the Streaming Plugin after immediately starting Gephi and not clicking File > New Project (as specified in the docs) That seems to trigger something that immediately adds the streaming server which you can then start. I fell into the hole of screenshots you provided by trying to add a streaming source which I believe is the client-side streaming and since that source points to nothing - you get an immediate error when you hit the OK button. of course, after that error, the streaming server pops up....not sure if there is some kind of bug there of something.

Anyway, I don't see anything wrong with our current screens or instructions. I"ll add something to the docs as a warning to not miss that File > New Project step.  

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/0a51de45-25dd-49ec-bfd2-d2face2b44bf%40googlegroups.com.

[andys...@gmail.com]

I tried Cephi 0.9.1 and Gremlin 3.3.0, and had File > New Project step .

Following the instruction in TinkerPop3 document to test Gephi plugin , but I failed for several time.

D:\run\apache-tinkerpop-gremlin-console-3.3.0-bin\apache-tinkerpop-gremlin-console-3.3.0>bin\gremlin.bat

         \,,,/

         (o o)

-----oOOo-(3)-oOOo-----

plugin activated: tinkerpop.server

plugin activated: tinkerpop.gephi

plugin activated: tinkerpop.utilities

plugin activated: tinkerpop.tinkergraph

gremlin> graph = TinkerFactory.createModern()

==>tinkergraph[vertices:6 edges:6]

gremlin> :remote connect tinkerpop.gephi

==>Connection to Gephi - http://localhost:8080/workspace1 with stepDelay:1000, startRGBColor:[0.0, 1.0, 0.5], colorToFade:g, colorFadeRate:0.7, startSize:10.0,sizeDecrementRate:0.33

gremlin> :> graph

log4j:WARN No appenders could be found for logger (org.apache.http.client.protocol.RequestAddCookies).

log4j:WARN Please initialize the log4j system properly.

log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Unsuccessful request to Gephi - [404] Not Found - <html>

<head>

<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>

<title>Error 404 Not Found</title>

</head>

<body>

<h2>HTTP ERROR: 404</h2>

<p>Problem accessing /workspace1. Reason:

<pre>    Not Found</pre></p>

<hr /><i><small>Powered by Jetty://</small></i>

</body>

</html>

Type ':help' or ':h' for help.

Display stack trace? [yN] 

[Stephen Mallette]

well - it seems like you are getting responses from Gephi so the server appears to be running. Perhaps your workspace isn't named "workspace1" somehow?

On Mon, Sep 4, 2017 at 9:31 PM, <andys...@gmail.com> wrote:

I've tried streaming plugin at 1.0.3 and gephi 0.9.1, and had File > New Project step, but cephi didn't get data correctly.

在 2017年8月10日星期四 UTC+8上午1:29:12，Stephen Mallette写道：

I started going through the process to see what changed. I got the latest version of the streaming plugin at 1.0.3 and gephi 0.9.1 (which i don't think changed for me since the last time i tested) . If you explicitly follow the instructions here:

http://tinkerpop.apache.org/docs/current/reference/#gephi-plugin

I dont' see any difference under these versions. Everything works. I did get hung up for a minute though because I tried to use the Streaming Plugin after immediately starting Gephi and not clicking File > New Project (as specified in the docs) That seems to trigger something that immediately adds the streaming server which you can then start. I fell into the hole of screenshots you provided by trying to add a streaming source which I believe is the client-side streaming and since that source points to nothing - you get an immediate error when you hit the OK button. of course, after that error, the streaming server pops up....not sure if there is some kind of bug there of something.

Anyway, I don't see anything wrong with our current screens or instructions. I"ll add something to the docs as a warning to not miss that File > New Project step.  

On Tue, Aug 1, 2017 at 9:55 AM, Mark Baynes <he...@baynesmedia.com> wrote:

Hiya

Seems more than a few people have been having lots of problems connecting Gremlin to Gephi using the Streaming Plugin. I was trying to get this work today when I noticed that the plugin (https://gephi.org/plugins/#/plugin/graphstreamingwas) was updated on 30th July to a 0.9.1 Gephi version by Andre Panisson and it works like a dream! Woo hoo! The connection process from Gephi Streaming is slightly different and am not sure I used it correctly but it sure works :) 

I have screengrabs of complete process which I will post separately at a later date (docs will need updating) but two key ones are attached. Hope this is of help to others. 

cheers from London, 

Mark

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/0a51de45-25dd-49ec-bfd2-d2face2b44bf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/3b9f2d92-be08-4738-ada0-bde669162171%40googlegroups.com.