can't join on derived predicates?

[Andy Xue]

user=> (use 'cascalog.api)

nil

user=> (def test-data-1 [["A" 1]["b" 2]])

#'user/test-data-1

user=> (def test-data-2 [["a" "alpha"]])

#'user/test-data-2

user=> (defn to-lower-case [v] (if (string? v) (.toLowerCase v) nil))

#'user/to-lower-case

user=> (def test-q (<- [!a !b !!c] (test-data-1 !a !b) (test-data-2 !a_raw !!c) (to-lower-case !a_raw :> !a)))

user=> IllegalArgumentException Unable to join predicates together  sun.reflect.NativeConstructorAccessorImpl.newInstance0 (NativeConstructorAccessorImpl.java:-2)

vs.

user=> (def test-q (<- [!a !b !!c] (test-data-1 !a !b) (test-data-2 !a !!c)))

#'user/test-q

i can see how you can't join on the output of a reducer, but the to-lower-case function should be applied at mapping before the join occurs?

[Andy Xue]

er, derived fields i mean, not predicates

[Nathan Marz]

You can't outer join on derived predicates... I'd say that's actually a bug in Cascalog. You can work around this by making a subquery that does the to-lower-case, and then outer joining on that subquery.

--
Twitter: @nathanmarz
http://nathanmarz.com