Wiring of collections in a body of a rule

[Vera Moffitt]

I am encountering a problem where modification of a collection does not propagate into other collections that appear in lhs of rules where the problem collection is in the body only.

So, a simple example (my actual rules are significantly more complicated but the idea is the same):

rel1 <= rel2 {|t| [t.atom] if rel3.exists?}

or something like

rel1 <= rel2 {|t| [t.atom] if rel3[['somekey']].field1 > 0}

I am observing that if rel3 is updated, rel1 is not marked for rescanning. rel3 is in the depends list but wire_to is not called.

What am I missing here? 

Thank you,

Vera

[Neil Conway]

Hi Vera,

Generally speaking, I would try to avoid using rules of that form, and
using either joins or negation operators where possible. Is that an
option for the program you're writing?

Neil

> --
> You received this message because you are subscribed to the Google Groups
> "Bloom Language" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to bloom-lang+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Vera Moffitt]

Hi Neil,

I did implement them as joins before, but the performance was far below what I wanted. If rel2 has 10000 tuples and rel3 has 10 (of which I only care about 1 specific one), that's a 10x slowdown. If I could do push joins on value, then I'd do that. That would require supporting syntax like

rel1 <= (rel2*rel3).combos(rel3.somekey => 'somevalue') {}

But I don't believe this syntax is supported.

I'd welcome any other suggestions.

Thanks!

-Vera

[Peter Alvaro]

Hi Vera,

perhaps I am misunderstanding your intent, but could you work around this by projecting the (statically) relevant tuples into a scratch?  e.g.

relevant_rel3 <= rel3{|r| r if r.somekey == 'somevalue'}

rel1 <= (rel2 * relevant_rel3).combos

peter

[Vera Moffitt]

Hi Peter,

You are understanding me correctly. I did in fact try that approach at one point. Unfortunately, the improvement was only marginally better because addition of new collections, whether tables or scratches, results in restratification, the runtime of which depends on the number of collections. Every time a new collection is added, restratification takes longer (which is to be expected) and not in a fixed amount way, but in a close to exponential way (which I didn't expect but I suppose makes sense). Since I install these rules runtime, I am unwilling to pay this kind of penalty if I don't have to. And by using the collection directly in a body of a rule I get around all these problems. It stratifies correctly, it just doesn't invalidate correctly.

I've found a workaround for now that will suffice, but I really think supporting push joins with values is what is called for. 

Thanks,

-Vera