New Cascalog Feature: Generator as filter / negations

[nathanmarz]

I added a feature that allows you to treat generators like filters. If
you're familiar with logic programming, this feature lets you do
traditional logic negations (among other things). Let's look at a few
examples using the playground:

This first example gets all the people who are 25 and don't follow
anybody.

(?<- (stdout) [?person]
(age ?person 25) (follows ?person _ :> false) (:distinct false))

As you can see, we're treating the follows dataset as a filter.
Cascalog treats the generator like it's a set. This next example gets
all the people who are 25 and follow somebody:

(?<- (stdout) [?person]
(age ?person 25) (follows ?person _ :> true) (:distinct false))

This is different than doing a normal join, as multiple follows
records for ?person won't cause multiple records for ?person to be
emitted as output. It works just like an operation (it is implemented
behind the scenes using a special kind of join). You can also capture
the output of the generator as a filter:

(?<- (stdout) [?person ?follows-someone]
(age ?person 25) (follows ?person _ :> ?follows-someone) (:distinct
false))

You can include constants or variables for the "non join variables"
and it will work just as you expect. The following query finds all
people who are male that don't follow "bob":

(?<- (stdout) [?person]
(gender ?person "m") (follows ?person "bob" :> false) (:distinct
false))

Note that Cascalog will disallow any predicates on the non-join vars
of a generator as a filter. So the following query is invalid:

(?<- (stdout) [?person]
(gender ?person "m") (age ?person ?age :> false) (> ?age 40)
(:distinct false))

If you wanted to know all the people who were male and not in the
"older than 40 set", you can use a subquery:

(let [older (<- [?person] (age ?person ?age) (> ?age 40) (:distinct
false))]
(?<- (stdout) [?person]
(gender ?person "m") (older ?person :> false)
))

This feature also works seamlessly with the new predicate macro stuff
I released in Cascalog 1.6.0. The following two queries are
equivalent, one using c/negate and one using generator as filter
feature directly:

(?<- (stdout) [?person]
(age ?person 25) ((c/negate follows) ?person _) (:distinct false))

(?<- (stdout) [?person]
(age ?person 25) (follows ?person _ :> false) (:distinct false))


I'm very excited about this new functionality. It's available on
GitHub and in 1.7.0-SNAPSHOT on Clojars. Enjoy!

-Nathan