custom joins

[John Libby]

Hello,

I am having an issue creating a custom joiner.  I have a unit test created that uses 2 text delimited files, 3 rows in each.  However for some reason in my joiner class, only one of the rows gets processed.  I couldn't find a good example of a custom join, so if someone could point me to one, that would be greatly appreciated.  Otherwise, here is the implementation of getIterator for my custom joiner:

    @Override
    public Iterator<Tuple> getIterator(JoinerClosure closure) {

        List<Tuple> results = new ArrayList<Tuple>();
        Iterator<Tuple> lhs = closure.getIterator(0);
        Iterator<Tuple> rhs = closure.getIterator(1);
        Fields[] fields = closure.getValueFields();

        while (lhs.hasNext()) {
            Tuple lhsTuple = lhs.next();
            System.out.println("evaluating "+lhsTuple.getString(0));

            while (rhs.hasNext()) {
                Tuple rhsTuple = rhs.next();
                System.out.println("***against "+rhsTuple.getString(0));

                if (expression.isTrue(fields[0], lhsTuple, fields[1], rhsTuple)) {
                    results.add(new Tuple(lhsTuple).append(new Tuple(rhsTuple)));
                }
            }
        }

        return results.iterator();
    }


Am I missing something?  From what I was able to find out, it should be pretty simple to just override this method determining when to pass on tuples that pass my test.

Thanks,
John

[Chris K Wensel]

You might find using the BufferJoin easier..

http://docs.cascading.org/cascading/2.5/javadoc/cascading/operation/Buffer.html

--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascading-use...@googlegroups.com.
To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at http://groups.google.com/group/cascading-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/14049f7e-24ca-4240-91ac-cb24d3a97ffe%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Chris K Wensel

ch...@concurrentinc.com

http://concurrentinc.com

[John Libby]

Sorry, I should have mentioned, I am using cascading 2.1 which doesn't have an accessor method for the joiner closure in the BufferCall object.  If there is a way I can use it please let me know, otherwise can you tell me what I am missing on my implementation of a custom join?

[Andre Kelpe]

Is there a specific reason to stick to cascading 2.1? 2.x is fully
backwards compatible and there should be no problem for you just using
the latest version.

- André

On 4/21/14, John Libby <jwl...@gmail.com> wrote:
>
> Sorry, I should have mentioned, I am using cascading 2.1 which doesn't have
>
> an accessor method for the joiner closure in the BufferCall object. If
> there is a way I can use it please let me know, otherwise can you tell me
> what I am missing on my implementation of a custom join?
>
> On Monday, April 21, 2014 12:07:54 PM UTC-4, Chris K Wensel wrote:
>>
>> You might find using the BufferJoin easier..
>>
>>
>> http://docs.cascading.org/cascading/2.5/javadoc/cascading/operation/Buffer.html
>>

>> On Apr 21, 2014, at 8:30 AM, John Libby <jwl...@gmail.com <javascript:>>

>> email to cascading-use...@googlegroups.com <javascript:>.

>> To post to this group, send email to

>> cascadi...@googlegroups.com<javascript:>
>> .

>> Visit this group at http://groups.google.com/group/cascading-user.
>> To view this discussion on the web visit

>> https://groups.google.com/d/msgid/cascading-user/14049f7e-24ca-4240-91ac-cb24d3a97ffe%40googlegroups.com<https://groups.google.com/d/msgid/cascading-user/14049f7e-24ca-4240-91ac-cb24d3a97ffe%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .

>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> Chris K Wensel

>> ch...@concurrentinc.com <javascript:>
>> http://concurrentinc.com

>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "cascading-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cascading-use...@googlegroups.com.
> To post to this group, send email to cascadi...@googlegroups.com.
> Visit this group at http://groups.google.com/group/cascading-user.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/cascading-user/8a0e35c2-1c89-462a-b93b-f2adef924b71%40googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.
>


--

André Kelpe
an...@concurrentinc.com
http://concurrentinc.com