Math Functions?

228 views
Skip to first unread message

Marko Rodriguez

unread,
Oct 2, 2017, 11:47:49 AM10/2/17
to gremli...@googlegroups.com
Hello,

There has been some discussions in the DataStax internal DSEGraph mailing list about adding math and string manipulation steps to Gremlin so that lambdas can be avoided in such situations.

The problem is that there are times when numbers are flowing through a traversal and they need to be operated on. We currently support testing numbers (predicates):

is(gt(30))
where(lt(“a”))

But we do not have a way to manipulate numbers (bi-operators).

math(add(30))
math(sub(“a”))

Note that we have the following tickets that discuss this problem:


I don’t have a clean idea for how to do this that is respective of the known step patterns in Gremlin. As such, I’m sending this email around to get ideas on how people would like this to be formulated.

Thoughts?,
Marko.

Stephen Mallette

unread,
Oct 2, 2017, 1:11:39 PM10/2/17
to Gremlin-users
Just for added context the general issue of "math steps" was also brought up on the dev mailing list in relation to security a couple of weeks back:




--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/F5A0E440-708C-494A-BA1B-2CB5D29771F8%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Daniel Kuppitz

unread,
Oct 2, 2017, 1:49:54 PM10/2/17
to gremli...@googlegroups.com
What if we introduce a MathTraversal? As an example, this is a simple Sigmoid formula:  y=1/(1+e^(-x)).

Now a MathTraversal to describe this formula could look like this: c(1).div(c(1).plus(e.pow(input().negate())))
...where c() is for constant values and binding() is a placeholder for the incoming value (we can have binding("label") for cases when you have more than a single variable).

Next we can have a single calc-step that takes a MathTraversal as an argument:

g.V().outE().values("weight").calc(c(1).div(c(1).plus(e.pow(binding().negate()))))               // or:
g.V().outE().calc(c(1).div(c(1).plus(e.pow(binding("a").negate())))).bind("a", values("weight"))

I think this approach would be better as all the math steps would be encapsulated and we wouldn't pollute GraphTraversal.

Cheers,
Daniel


Stephen Mallette

unread,
Oct 2, 2017, 1:54:03 PM10/2/17
to Gremlin-users

Marko Rodriguez

unread,
Oct 2, 2017, 2:19:43 PM10/2/17
to gremli...@googlegroups.com
I like it!

c() and binding() I don’t like so much. I think we should just stick with TinkerPop3 constructs for variables. Thus, calc(), when it sees a String, it assumes its a variable.

Marko.





To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H438go2gZ_JYkuwNMwwvqhwmoG%3D3nKF_3LJw33PvakXd4eg%40mail.gmail.com.

Daniel Kuppitz

unread,
Oct 2, 2017, 2:44:05 PM10/2/17
to gremli...@googlegroups.com
I mainly chose c() and binding() to prevent naming clashes with GraphTraversal methods. I still like c(), but binding() could be v() or variable() with an optional by() modulator. I thought short names like c() and v() will keep longer functions, like the sigmoid above, more readable.

However, I've just made a comparison and it really doesn't make a big difference, thus I don't have a strong feeling regarding the namings.

Cheers,
Daniel


--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.

Ted Wilmes

unread,
Oct 2, 2017, 2:47:38 PM10/2/17
to Gremlin-users
That MathTraversal approach is pretty slick. Another idea, instead of providing the expression as another traversal, maybe it could be submitted as a string and evaluated behind the scene by a math expression parser library like https://github.com/fasseg/exp4j. The parsed expression would give the strategies access to what labels are being referenced so they wouldn't be inscrutable like lambdas are generally.

ex: g.V().outE().values("weight").calc("1/(1+e^-x)")

I could see one downside to this approach being that the exact syntax of the expression is dependent on the underlying implementation so a different Gremlin machine may support a slightly different syntax. The traversal approach would bring the math steps up to to top level constructs per the Gremlin "standard".

--Ted
gremlin-math - crazy :)

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

HadoopMarc

unread,
Oct 2, 2017, 3:50:08 PM10/2/17
to Gremlin-users
Following Ted's line and avoiding a host of div(), plus() etc. steps, the following would also be possible:

def sygmoid(x) { 1 / (1 + Math.E ** - x) }
g.V().outE().values("weight").
calc(sygmoid)

Btw, do we also want math expressions with more than one variable? What to do with naming of explicit variables or the order of implicit variables then?

Cheers,   Marc

Op maandag 2 oktober 2017 20:47:38 UTC+2 schreef Ted Wilmes:

Stephen Mallette

unread,
Oct 2, 2017, 3:54:29 PM10/2/17
to Gremlin-users
You can definitely do the sygmoid function as a lambda even without the calc() step - it's just map(sygmoid) but the goal is to avoid use of lambdas. They are a security risk that would be nice to avoid.

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/1ebf2a8b-5311-4c79-93c7-9cf25e27f5c2%40googlegroups.com.

Jean-Baptiste Musso

unread,
Oct 2, 2017, 4:24:58 PM10/2/17
to gremli...@googlegroups.com
Ah, this is cool. Though, don't we already have a kind of MathTraversal with the following?

gremlin> inject(1, 2, 3).sum()
==>6

When explaining Gremlin to newcomers, I tend to use this example to illustrate that Gremlin can traverse much more than property graphs but any kind of data.

Also, I really like Ted's approach with .calc("1/(1+e^-x)"). First feeling is that this makes GLV easier to write/maintain (though code generation doesn't make it that hard). Readability is also neat!

Jean-Baptiste

Marko Rodriguez

unread,
Oct 2, 2017, 6:53:21 PM10/2/17
to gremli...@googlegroups.com
Hi,

So we seem to like two “cool” models:

1. String representation of an equation. [Ted]
2. Step representation of an equation. [Daniel]

Chapter 1. The String Representation

For the Traversal:

g.V().project(“w”,”h”).
    by(“weight”).
    by(“height”).
  calc(“w / h")

Via exp4j (https://github.com/fasseg/exp4j), we would be able to do:

Expression e = new ExpressionBuilder(“w / h")
  .variables(“w”,”h")
  .build()
  .setVariable(“w”, traverser.path().get(“w”))
  .setVariable(“h”, traverser.path().get(“h”))
return traverser.split(e.evaluate());

exp4j supports +, -, ^, …, abs, sin, log10, log2… sorta like a high school scientific calculator. You can also add you own custom operators.

This would be really EASY to do. This ticket could be knocked out in 1 (solid testing 2) days. 

Chapter 2. The Step Representation

For the Traversal:

g.V().project(“w”,”h”).
    by(“weight”).
    by(“height”).
  calc(binding(“w”).div(binding(“h”))

In this model, we would need lots of BiOperators that don’t take Number, but instead, Object, where Object can be a Number or a BiOperator. In this way, we can nest BiOperators. As it stands, we would have to extend Kuppitz’ line of work here:


This is also pretty easy to do and would take, a bit longer at 2 (solid testing 3) days.

——

Marko.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAC3-3jg%3DTCz1du7U6DZk6-6ipCL1E60%2BUFwdZ6P1JxN5U7KuPQ%40mail.gmail.com.

Robert Dale

unread,
Oct 2, 2017, 7:58:24 PM10/2/17
to gremli...@googlegroups.com
Is that 3 or 5 days for both?


Jean-Baptiste

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAC3-3jg%3DTCz1du7U6DZk6-6ipCL1E60%2BUFwdZ6P1JxN5U7KuPQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.
--
Robert Dale

Daniel Kuppitz

unread,
Oct 2, 2017, 8:11:24 PM10/2/17
to gremli...@googlegroups.com
We can have both. The String representation could be a language variant of gremlin-math ;)


--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.

Marko Rodriguez

unread,
Oct 3, 2017, 9:58:15 AM10/3/17
to gremli...@googlegroups.com
Hello,

We can have both, but do we want both? As always, more code means more code to write, test, document, evolve, confuse users with, etc.

* I believe that Ted’s representation is easiest on the eyes.
* I believe that Daniel’s representation is more in line with Gremlin’s functional notation.

* I believe that Ted’s representation is simpler to back out of in the future. It would be a single step — calc(String).
* I believe that Daniel’s representation is leaning too much towards TinkerPop4 concepts — "binding()" and "constant()".

* I believe that Ted’s representation is more burdensome for other Gremlin machine implementations — e.g. CosmosDB would need a exp4j impl.
* I believe that Daniel’s representation is much easier for other machine implementations, its just steps and operators.

Marko.

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CA%2Bf9seVZEVbeBkGZPm3shC64OBr7uLVCdj7-yNK-GPY5ESWf3g%40mail.gmail.com.

Robert Dale

unread,
Oct 3, 2017, 10:04:12 AM10/3/17
to gremli...@googlegroups.com
Dan & Ted's Excellent Adventure!

Robert Dale

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.

Stephen Mallette

unread,
Oct 3, 2017, 10:06:14 AM10/3/17
to Gremlin-users
Strange things are afoot at the Circle K

Marko Rodriguez

unread,
Oct 3, 2017, 10:52:35 AM10/3/17
to gremli...@googlegroups.com
Hello,

I implemented Ted’s model to see what we are missing.


This allows you to do:

gremlin> g.V().outE().values("weight").as("a").math("a + 10").as("b").math("a / b”)
==>0.038461538461538464
==>0.038461538461538464
==>0.047619047619047616
==>0.09090909090909091
==>0.09090909090909091
==>0.019607843137254905

Here are the problems:

1. By Modulation

We really want to be able to by()-modulate the variables. For instance, we want to be able to do this:

g.V().as(“a”).out(“knows”).as(“b”).math(“a / b”).by(“age”)

So this is possible, however, what is the order in which the by()-modulation is applied? For a single by()-modulation, its easy, but imagine:

g.V().as(“a”).out(“knows”).as(“b”).
  math(“a / b”).
    by(“age”).
    by(outE().count())
Does a => age and b => outE().count() ? … a first and then b cause a shows up first in the equation? If so, then we have to have parse the string equation ?!

Hmmmm….

Marko.

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H439YMEiANJHhEJqLovhc9eZm%3DfTkE_BnJjpQ74R1CDDxLg%40mail.gmail.com.

Ted Wilmes

unread,
Oct 3, 2017, 11:06:52 AM10/3/17
to Gremlin-users
Yeah, that gets confusing. I can't think of a good way around it other than introducing the binding modulator or doing something like 
your earlier example where you had a project preceding the calc step that seeded all of the variables in a map. But if that pattern (project().math())
was required a large percentage of the time, I think it'd be worth encapsulating that functionality into the step itself
with the binding modulators. 

--Ted


--
You received this message because you are subscribed to a topic in the Google Groups "Gremlin-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gremlin-users/EFog3lhh4Xw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/AD79C151-4F02-4FA4-AC65-914DD602EA63%40gmail.com.

Marko Rodriguez

unread,
Oct 3, 2017, 12:25:55 PM10/3/17
to gremli...@googlegroups.com
Hi,

So I have by()-modulation working where the order of the by()-application is with respects to the location of the variable in the traversal.


Thus,

gremlin> g.V().as("a").out("knows").as("b”).
           math("a ^ b”).
             by("age”).
             by(bothE().count())
==>29.0
==>24389.0

Because “a” is the first in the expression it is the first in the by() and thus, it is the “age” of “a”.

Thoughts?,
Marko.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAGrm2%3DAodU6GsHmO7KbG-HY_3ajNg3OF9tHo6KuztU4sSvQTcw%40mail.gmail.com.

Marko Rodriguez

unread,
Oct 3, 2017, 12:31:39 PM10/3/17
to gremli...@googlegroups.com
Hello,

Sometimes you already have the object and don’t want to as() name it. Thus, I made it such that _ means “current” (like “it” in Groovy).

gremlin> g.V().as("a").out("knows”).
           math("a ^ _”).
             by("age”).
             by(bothE().count())
==>29.0
==>24389.0
Message has been deleted
Message has been deleted

Stephen Mallette

unread,
Oct 4, 2017, 6:09:40 AM10/4/17
to Gremlin-users
I keep waffling back and forth on which solution is best between a MathTraversal and embedding this math library. I think I would have gone with MathTraversal pretty quickly had I not thought of the extra work involved for GLVs to implement the additional steps. Now that we're seeing that the math library approach has some tricky bits to it, I'm feeling more inclined to accept the additional effort that MathTraversal brings. Marko, do you envision any "tricky" bits with that approach? 

On Wed, Oct 4, 2017 at 5:55 AM, HadoopMarc <marc.de...@gmail.com> wrote:
Tricky indeed.  How would you deal the opposite order for math("b ^ a")?  Should b in the expression match with the as("b")?

Also, any implicit ordering should not interfere with explicit selects:

gremlin> g.V().as("a").out("knows”).as("b").
           math("x ** y”).
             by(select("b").values("age”)).
             by(select("a").bothE().count())
Cheers,   Marc

Op dinsdag 3 oktober 2017 18:31:39 UTC+2 schreef Marko A. Rodriguez:

Robert Dale


Jean-Baptiste

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "Gremlin-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gremlin-users/EFog3lhh4Xw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gremlin-user...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAGrm2%3DAodU6GsHmO7KbG-HY_3ajNg3OF9tHo6KuztU4sSvQTcw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.

Marko Rodriguez

unread,
Oct 4, 2017, 11:33:26 AM10/4/17
to gremli...@googlegroups.com
Hello,

What does x and y refer to?

In TINKERPOP-1632, you have to use variables that exist in the traverser’s path (and soon, scope).

Marko.
On Oct 4, 2017, at 3:55 AM, HadoopMarc <marc.de...@gmail.com> wrote:

Tricky indeed.  How would you deal the opposite order for math("b ^ a")?  Should b in the expression match with the as("b")?

Also, any implicit ordering should not interfere with explicit selects:

gremlin> g.V().as("a").out("knows”).as("b").
           math("x ** y”).
             by(select("b").values("age”)).
             by(select("a").bothE().count())
Cheers,   Marc

Op dinsdag 3 oktober 2017 18:31:39 UTC+2 schreef Marko A. Rodriguez:
Hello,

Robert Dale


Jean-Baptiste

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/F5A0E440-708C-494A-BA1B-2CB5D29771F8%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CA%2Bf9seXV6YxVEJYefcrJ2rdNcrt7jZ8swZrW%2BTh-hh1-kny4Yw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAC3-3jg%3DTCz1du7U6DZk6-6ipCL1E60%2BUFwdZ6P1JxN5U7KuPQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/808EF861-E2FC-49CF-A7CF-15542813EC0C%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CA%2Bf9seVZEVbeBkGZPm3shC64OBr7uLVCdj7-yNK-GPY5ESWf3g%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/3E789EB8-4251-4099-B069-87213F596C46%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CABed_4ojUrCd8U1jc3uKfdyskAco3EMr7WG69vPv6RvkzfS%2BxQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H439YMEiANJHhEJqLovhc9eZm%3DfTkE_BnJjpQ74R1CDDxLg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to a topic in the Google Groups "Gremlin-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gremlin-users/EFog3lhh4Xw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/AD79C151-4F02-4FA4-AC65-914DD602EA63%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAGrm2%3DAodU6GsHmO7KbG-HY_3ajNg3OF9tHo6KuztU4sSvQTcw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

Marko Rodriguez

unread,
Oct 4, 2017, 1:32:23 PM10/4/17
to gremli...@googlegroups.com
Hi,

Here is the full working implementation.


There is nothing “kooky” — it is completely self consistent with standard techniques used for data access and by()-modulation.

Enjoy!,
Marko.
Reply all
Reply to author
Forward
0 new messages