Drools Fusion

[Maciej]

Is it possible to use Drools Fusion functionality  (Complex Event Processing) in rules?

[Michal Rutka]

Yes it is.

[Maciej]

Thanks Michal!

I'm developing a near-real-time app and CEP, and especially using the sliding window, seems very interesting for my purposes. I went through the docs and they say that sliding window requires STREAM parameter to be set in the following way:

KnowledgeBaseConfiguration config = KnowledgeBaseFactory

    .newKnowledgeBaseConfiguration();

config.setOption(EventProcessingOption.STREAM);

KnowledgeBase kbase = KnowledgeBaseFactory

    .newKnowledgeBase(config);

kbase.addKnowledgePackages(kbuilder.getKnowledgePackages());

StatefulKnowledgeSession ksession = kbase

    .newStatefulKnowledgeSession();

How do I approach setting it, is it possible within DRL file? Could you give some hint?

On Thursday, September 21, 2017 at 5:41:42 PM UTC+2, Maciej wrote:

[Michal Rutka]

I was shouting few years that for a serious Drools work the STREAM mode should be default. Then I've ended up by building my own controller as this parameter can be set only in the source. I'm not sure how it looks now in the current controller release 2.5 and 2.6? Eric, can you shed some light on it?

Anyway, if you are planning a serious work on rules the chances are that you will end up just like me with a custom build. It is not very difficult after all.

Kind regards,
Michał

[Maciej]

I just realised I will have to dig deep. Custom build is fine, I will do whatever it takes to get this sliding window going because it seems to be a really useful feature for real-time smart home tasks. I'd appreciate if someone let me know the key steps to get the STREAM processing going.

Thanks,

Maciej

On Thursday, September 21, 2017 at 5:41:42 PM UTC+2, Maciej wrote:

[Michal Rutka]

Well, it depends. Openremote controller 2.x used the cloud mode. In Openremote 3 the rules engine runs in the stream mode out of the box. In which version are you interested? Version 2.x is semi dead already as almost no new development is there. All effort is pointed right now towards version 3. However, this one is bit harder to use, have larger hardware requirements and less protocols available. 

[Maciej]

I'm using 2.5.0 / Java 6 but migrating now to 2.6.0 / Java 8. My current app runs in "production environment" so stability is key. Thus I would rather wait with OR 3 until it's a late beta. So the version for which I would like to set stream is 2.6.0.

On Thursday, September 21, 2017 at 5:41:42 PM UTC+2, Maciej wrote:

[Michal Rutka]

Yes, going to 2.6 is a wise decision, you will get Java 8. Anyway, in the git controller repo there is no STRAM mode, but it is quite simple change in the RuleEngine.java code. However, the official demo design with the home automation example uses the rule engine in the STREAM mode. You can take a look on it's rule's file and you will see syntax which is valid only in this mode. I don't remember from which source this controller was build but I know that the demo itself is run on AWS and setup from the docker image available on hub.docker.com. Therefore, if you are working with docker (and if not you should ;) then there is an easy way to get what you want.

Kind regards,

Michal

[Maciej]

Thank you again! Can you confirm that the demo you mentioned is the one at https://hub.docker.com/r/openremote/demo-controller/ ? I pulled it and explored its files, it has drools 6.4.0 and runs on Java 8 so it seems that it's a controller 2.6. It has no rules folder and rules files though (I did not find them in /opt/OpenRemote-Controller/webapps/controller).

On Thursday, September 21, 2017 at 5:41:42 PM UTC+2, Maciej wrote:

[Michal Rutka]

Yes it is this one. However, resources are not in the folder they used to be. There are now in /opt/or-resources and rules in the /opt/or-resources/rules. But the design itself is not a part of this docker image. There should be somewhere a zip file with this demo design which you can import into OR designer. Then you can sync this controller. Find it on the wiki page describing this demo.

[Maciej]

I tested this demo controller, tried to run Drools Fusion expressions (namely the sliding window) and I got:

"Cannot start event processor 'Drools Rule Engine' : The requested KieBase "OpenRemoteKBase" has been set to run in CLOUD mode but requires features only available in STREAM mode

java.lang.RuntimeException: The requested KieBase "OpenRemoteKBase" has been set to run in CLOUD mode but requires features only available in STREAM mode"

The rules in the demo indeed have some Fusion syntax, however in a scope that does not require the STREAM mode. STREAM is required for the sliding window, which is not in the demo.

Do I understand correctly that that the only way to get the controller working in STREAM mode is to change the RuleEngine.java and do a custom build? How should I proceed?

thanks

Maciej

  
On Thursday, September 21, 2017 at 5:41:42 PM UTC+2, Maciej wrote:

[Michal Rutka]

Can you show the rule which upset the demo controller?

[Maciej]

Please see below (not sure if there are no errors in it)

declare Event

@role( event )

end

rule "check sliding window"

when

$e : Event(source == "Temp3", value != "N/A")

Number( doubleValue > 0 ) from accumulate(

Event( source == "Temp3", $temp : value ) over window:time( 10m ),

average( $temp ) )

then

System.out.println("Sliding window");

end

[Michal Rutka]

Indeed, I’ve just checked and you are correct. Just why this controller accepts other STREAM constructs and not the sliding window is another mystery? This is a new Drools engine compared to the old 2.5 controller. I haven’t work with it much and it continuously surprises me with new stuff which works differently than previously. Nevertheless, you can either create your own build of 2.6 controller with the engine put in the STREAM mode or you can try to work with version 3.0. There is the STREAM mode default for sure as I’m using sliding windows there.

--
You received this message because you are subscribed to a topic in the Google Groups "OpenRemote" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openremotecommunity/lzFYq-0bDZQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openremotecommu...@googlegroups.com.
Visit this group at https://groups.google.com/group/openremotecommunity.
To view this discussion on the web visit https://groups.google.com/d/msgid/openremotecommunity/b589877f-0a0f-49d3-891a-f7963d86a6db%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Maciej]

Ok, I made the STREAM mode work, but now I have another problem as described further below. 

To get the STREAM mode I modified the RuleEngine.java and added:

import org.kie.api.conf.EventProcessingOption;

....

.setEventProcessingMode(EventProcessingOption.STREAM)

I compiled the zip distributable with Gradle. The modified controller runs with no errors. However the rule I mentioned in previous post is incorrect and does not work, because the Event's value is a string, and 'accumulate ... average' only works with numbers, so the value needs to be converted first.

And here's another problem - the Events do not seem to 'get remembered' and the sliding window at any moment sees only 1 recent event (or 0 if there were no events). But the window should see all events that came within the window duration, so if the window's duration is 1 minute then all events within the last 1 minute should be seen. The drl is as below (imports omitted, the Event comes from temperature sensor):

declare Event

@role( event )

@expires( 5m )

end

declare window AvgTemp3Window

Event( source == "Temp3", value != "N/A" )

over window:time( 1m )

end

function String _TimeStamp(){

  Date now = new Date();

  SimpleDateFormat dateFormatter = new SimpleDateFormat("d H:mm:ss");

  return(dateFormatter.format(now));

}

rule "check fusion"

when

$b1:Event( source == "Temp3" )

then

String v = $b1.getValue().toString();

System.out.println(_TimeStamp() + " -- OK. Temp is: "+v);

end

rule "check sliding window"

when

accumulate( $temp : Event() from window AvgTemp3Window,

$avg : average( Double.parseDouble($temp.getValue().toString()) ),

$cnt : count( $temp  ) )

then

System.out.println(_TimeStamp() + " -- Sliding window. Avg is: " + String.valueOf($avg) + " Count is: " + String.valueOf($cnt));

end

The output to terminal is:

7 12:08:28 -- Sliding window. Avg is: 0.0 Count is: 0

7 12:11:39 -- OK. Temp is: 12.9375

7 12:11:39 -- Sliding window. Avg is: 12.9375 Count is: 1

7 12:12:09 -- OK. Temp is: 12.875

7 12:12:09 -- Sliding window. Avg is: 12.875 Count is: 1

7 12:13:09 -- Sliding window. Avg is: 0.0 Count is: 0

7 12:14:19 -- OK. Temp is: 12.9375

7 12:14:19 -- Sliding window. Avg is: 12.9375 Count is: 1

7 12:14:44 -- OK. Temp is: 12.875

7 12:14:44 -- Sliding window. Avg is: 12.875 Count is: 1

7 12:15:04 -- OK. Temp is: 12.9375

7 12:15:04 -- Sliding window. Avg is: 12.9375 Count is: 1

For a 1-minute sliding window the count should be more than 1 if there are more events within 1 last minute. This may have something to do with how 'Event' facts are handled by engine, they somehow do not persist, they immediately expire even though the expiration is to happen after 5 minutes. 

Does anynone have any idea why?

[Michal Rutka]

This problem is very easy to solve. All you need is to create custom fact events and insert them into working memory. Something like this (not tested):

declare CustomEvent
    @role(event)
    @expires(5m)
    source: String
    value: double
end

rule InsertTempValues

when

   Event($source: source matches "Temp\\d+", $value: value)

then

   insert(new CustomEvent($source, Double.parseDouble($value.toString())));

end

and then use the sliding window over CustomEvent instead. Of course some error checking would be nice for NaNs.

[Michal Rutka]

This is how it is implemented in the RulesEngine.java. When a new value comes from a sensor then the previous fact associated with this sensor is deleted and a brand new fact inserted. Therefore, for having multiply facts associated with one sensor you need the solution I've posted previously. Of course, don't forget about @expires(...) otherwise your memory get exhausted. 

On Saturday, October 7, 2017 at 1:17:23 PM UTC+2, Maciej wrote:

[Michal Rutka]

Last but not least. If you can, please make a docker image with it and put it on hub.docker.com, or if this is too much then point me to github source of you build and I'll create the docker image (with automated build).

On Saturday, October 7, 2017 at 1:17:23 PM UTC+2, Maciej wrote:

[Maciej]

Thanks a lot, very helpful. 
I did the source modification locally, not on github. I will upload it to github and let you know.

Regarding the Docker image - my controller has some other additions (odbc jar, mysql, etc) and I do not know exactly how to make Docker image with that.

[Maciej]

Based on your suggestions and upon some adjustments I have the below piece of code working. It properly counts events and averages their values within the time sliding window.

package org.openremote.controller.model.event

global org.openremote.controller.statuscache.CommandFacade execute;

global org.openremote.controller.statuscache.SwitchFacade switches;

import org.openremote.controller.protocol.*;

import java.text.DecimalFormat;

import java.util.Date;

import java.text.SimpleDateFormat;

import java.io.*;

import java.util.Properties;

import javax.naming.*;

import org.joda.time.DateTime;

import java.util.Timer;

import java.util.TimerTask;

import java.lang.Float;

import java.lang.Number;

import java.util.concurrent.TimeUnit;

declare CustomEvent

    @role(event)

    @expires(2m)

    

    source: String

    value: String

end

declare window AvgTemp3Window

CustomEvent()

over window:time( 1m )

end

function String _TimeStamp(){

  Date now = new Date();

  SimpleDateFormat dateFormatter = new SimpleDateFormat("d H:mm:ss");

  return(dateFormatter.format(now));

}

rule "InsertTempValues"

when

   Event($source: source matches "Temp\\d+", $value : value!= "N/A")

then

   insert(new CustomEvent( String.valueOf($source), String.valueOf($value) ));

end

rule "Read temp"

when

$b1:Event( source == "Temp3" )

then

String v = $b1.getValue().toString();

System.out.println(_TimeStamp() + " -- OK. Temp is: "+v);

end

rule "Check sliding window"

when

accumulate( $temp : CustomEvent() from window AvgTemp3Window,

$avg : average( Double.parseDouble($temp.getValue().toString()) ),

$cnt : count( $temp  ) )

then

System.out.println(_TimeStamp() + " -- Sliding window. Avg is: " + String.valueOf($avg) + " Count is: " + String.valueOf($cnt));

end

The modified OR controller 2.6.0 source code is at https://github.com/maq1000/OR_STREAM_mode

Thanks for help!

Maciej