Loading GraphML file containing floats --> exception (TP3 M9)

[Kelvin Lawrence]

I have a GraphML file that has latitude and longitude coordinates in it expressed as decimals of the form:

<data key='lat'>33.6366996765137</data>
<data key='lon'>-84.4281005859375</data>

If I define lat and lon as string types in my GraphML file, the file loads fine.

Am I doing something wrong here (too many digits perhaps) or is this a bug in the GraphML parser? If it is a case of too many didgits I think I would expect the parser to still work but to truncate the values.

FWIW Gephi reads the file fine and just truncates as I would have expected.

If I define them as double or float I get an exception as the graph is loading as follows:

java.lang.NumberFormatException: empty String
        at sun.misc.FloatingDecimal.readJavaFormatString(Unknown Source)
        at sun.misc.FloatingDecimal.parseDouble(Unknown Source)
        at java.lang.Double.parseDouble(Unknown Source)
        at java.lang.Double.valueOf(Unknown Source)
        at org.apache.tinkerpop.gremlin.structure.io.graphml.GraphMLReader.typeCastValue(GraphMLReader.java:311)
        at org.apache.tinkerpop.gremlin.structure.io.graphml.GraphMLReader.readGraph(GraphMLReader.java:147)
        at org.apache.tinkerpop.gremlin.structure.io.graphml.GraphMLIo.readGraph(GraphMLIo.java:86)
        at org.apache.tinkerpop.gremlin.structure.io.Io$readGraph.call(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:110)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:122)
        at groovysh_evaluate.run(groovysh_evaluate:3)
        at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:215)
        at org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:69)
        at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:185)
        at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:119)
        at org.codehaus.groovy.tools.shell.commands.LoadCommand$_load_closure1.doCall(LoadCommand.groovy:84)
        at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:324)
        at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:292)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1016)
        at groovy.lang.Closure.call(Closure.java:423)
        at groovy.lang.Closure.call(Closure.java:439)
        at org.codehaus.groovy.runtime.DefaultGroovyMethods.callClosureForLine(DefaultGroovyMethods.java:5233)

        at org.codehaus.groovy.runtime.IOGroovyMethods.eachLine(IOGroovyMethods.java:483)
        at org.codehaus.groovy.runtime.IOGroovyMethods.eachLine(IOGroovyMethods.java:436)
        at org.codehaus.groovy.runtime.ResourceGroovyMethods.eachLine(ResourceGroovyMethods.java:319)
        at org.codehaus.groovy.runtime.ResourceGroovyMethods.eachLine(ResourceGroovyMethods.java:303)
        at org.codehaus.groovy.runtime.dgm$929.doMethodInvoke(Unknown Source)
        at org.codehaus.groovy.tools.shell.commands.LoadCommand.load(LoadCommand.groovy:83)
        at org.codehaus.groovy.tools.shell.commands.LoadCommand.execute(LoadCommand.groovy:72)
        at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:215)
        at org.codehaus.groovy.tools.shell.Shell.execute(Shell.groovy:101)
        at org.codehaus.groovy.tools.shell.Groovysh.super$2$execute(Groovysh.groovy)
        at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:324)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1207)
        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:130)
        at org.codehaus.groovy.tools.shell.Groovysh.executeCommand(Groovysh.groovy:254)
        at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:153)
        at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:119)
        at org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:94)
        at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy)
        at sun.reflect.GeneratedMethodAccessor37.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:324)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1207)
        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:130)
        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:150)
        at org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:123)
        at org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:58)
        at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:324)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1207)
        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:130)
        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:150)
        at org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:82)
        at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:215)
        at org.apache.tinkerpop.gremlin.console.Console.<init>(Console.groovy:134)
        at java.lang.invoke.MethodHandle.invokeWithArguments(Unknown Source)
        at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:215)
        at org.apache.tinkerpop.gremlin.console.Console.main(Console.groovy:292)

[Stephen Mallette]

well - the exception is indicative of "empty string" in your dataset.  is it possible that you have some bad data (i.e. a lat/long with an empty string value somehow)?

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/7af77191-ab71-45d3-b4e4-560b4ef76ab1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kelvin Lawrence]

Interesting - OK I'll go digging. It's a big file so I could have bad data. If it is that should an empty string really cause the load to fail or should the parser skip over it?

Thanks for the pointer of what to go look for.

Kelvin

...

[Stephen Mallette]

I'm not sure what the right behavior is there if we don't have it now....skip over it? treat it as zero?  what if you legitimately don't expect empty in your data and it's there - maybe you'd want to know?  

--

You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/5552b510-c5a2-497b-82ac-fe411dd23f1d%40googlegroups.com.

[Kelvin Lawrence]

I agree with you. As I was typing my last message, I was thinking what would I really want to happen?

The error did serve a purpose, for, as you correctly asserted I just found a single node deep down in my XML file that had no lat/lon values between the angle brackets. So the exception actually helped me find a bug in my code but it also caused me to waste your time as it was not clear from the exception that the null string was the fault of my XML. Another slight twist is that a null string is not an error if the datatype is 'string' (so no exception) but it is equally likely that my code was broken by not generating a string. So the behavior then becomes a bit inconsistent.

So, my two cents, don't throw an exception, convert the value to zero or null (or whatever makes sense) and issue a WARN if possible with a message that would send me off looking at my XML and not bothering you :-)

Does that make some sense?

Cheers

Kelvin

...

[Stephen Mallette]

> So, my two cents, don't throw an exception, convert the value to zero or null (or whatever makes sense) and issue a WARN if possible with a message that would send me off looking at my XML and not bothering you :-)

Not sure.  My first instinct with the "right behavior" was to improve the exception message to be a little less low-level.  Converting to "null" isn't an option because properties can't have null values.  We could just omit the property key all together i suppose, but that might be confusing in its own right.  Maybe need to think on it some more.

could you please create a ticket for this in apache jira and reference this discussion?  a course of action can be worked out from there....

--

You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/462b61bb-7c15-493e-b212-a930f2795dae%40googlegroups.com.

[Kelvin Lawrence]

Hi Stephen, I just opened ticket 756 with a pointer to this discussion.

Thanks for your help and tolerance as always!

Kelvin

...