caching and reactor

[ElKlaaso]

Hi,

I am using reactor, and found out it has no caching system and i find
this pretty important.

Does this mean none of you guys using reactor cache your queries? Or
how do you do this?

I have read that Reactor 2.0 which according to http://trac.reactorframework.org/roadmap,
should be due in 6 months, will have a caching mechanism implemented.
How reliable is this?


greets,
klaas-jan winkel

[Mark Drew]

Since we havent implemented it yet, not sure how reliable it will be ;)

What will you cache? your objects? your queries?

One of the things that I hate about query caching is that you cant
reliably remove a specific query from cache... you have to clear the
whole cache.

Any suggestions are of course welcome

Mark Drew

- Adobe Community Expert
- Reactor ORM Project Manager
- CFEclipse Project Lead Developer
Blog: http://www.markdrew.co.uk/blog/
LinkedIn: http://www.linkedin.com/in/mdrew

[Klaas-Jan Winkel]

Funny :) But i meant how reliable the milestone date is...

I dont really mind if it caches queryies or objects, i just need it quick :)

I reckon it will be a good idea to not use coldfusion caching, a good
caching provider would be the one of coldbox,
i think its implemented quite well.. it might be a quick way to add
caching to reactor. If its not there in a few months
i probably add some caching provider interface to reactor, to which i
can hook the coldbox caching provider.

greets,
klaas

--
Because you're so cute, i baked you a pony

[Mark Drew]

Just so you know, when you emailed I had the following open in another
browser window and was going to be having a look at it for reactor
(and some other projects)

http://jakarta.apache.org/jcs/getting_started/intro.html

Mark Drew

- Adobe Community Expert
- Reactor ORM Project Manager
- CFEclipse Project Lead Developer
Blog: http://www.markdrew.co.uk/blog/
LinkedIn: http://www.linkedin.com/in/mdrew

[Klaas-Jan Winkel]

Yep, that looks even more solid, it can even cache on a remote server...

Are there any exact/concrete plans on implementing a caching provider?

[Mark Drew]

Not yet, fancy drawing them up?

We have a section on the trac site to write RFC's for features such as
this, if you fancy starting one and then the rest of the community can
get involved and we can code it once we all have agreed

http://trac.reactorframework.org/wiki/RFC

Mark Drew

- Adobe Community Expert
- Reactor ORM Project Manager
- CFEclipse Project Lead Developer
Blog: http://www.markdrew.co.uk/blog/
LinkedIn: http://www.linkedin.com/in/mdrew

[Klaas-Jan Winkel]

Hi,

Ill see if i have time somewhere, probably not the coming weeks as i
have to get an initial release ready.

About the JCS, need to load external Jar files for this, but this can
be done with:

http://www.transfer-orm.com/?action=displayPost&ID=312

Anyway, ill leave it at this for now.. but ill make a note about the
RFC so ill get back to it.

greets,
klaas-jan

[Mark Drew]

I use the JavaLoader a lot in our projects and would be using that,
but we dont need to just use this as I have also been using in
production a system called SoftCache, which is memory sensitive
caching... which has worked quite nicely.

Mark Drew

- Adobe Community Expert
- Reactor ORM Project Manager
- CFEclipse Project Lead Developer
Blog: http://www.markdrew.co.uk/blog/
LinkedIn: http://www.linkedin.com/in/mdrew

[Klaas-Jan Winkel]

I think its maybe best to just build on an interface in reactor, maybe
with a default implementation (it might just be dummy non-caching
implementation), and then anyone can write his own implementation for
it, the interface would be really simple i think.

[Klaas-Jan Winkel]

One thing to also watch out for, is that if you implement caching in
reactor, and then add custom queries in the xxxxmssql.cfc (example)
objects, then these will not get cached...

[Mark Drew]

Right, but there should be an api or rather a pointer instance to the
cache available to each of these objects so you can implement it
yourself

something like

if not isQuery this.CACHE.get("key")
this.CACHE.put("key", query, CACHE.timespan);
else return this.CACHE.get("key

Mark Drew

- Adobe Community Expert
- Reactor ORM Project Manager
- CFEclipse Project Lead Developer
Blog: http://www.markdrew.co.uk/blog/
LinkedIn: http://www.linkedin.com/in/mdrew

[Klaas-Jan Winkel]

Hi mark and others,

I was just thinking about it again, and i realized object/query
caching would be exactly a thing best solved with AOP.

So i had a look at this article:

http://cdscott.blogspot.com/2005/10/coldspring-aop-tutorial-part-one.html

Which is a coldspring AOP tutorial. And i thought, maybe the best way
to solve the whole caching problem the easiest way is like this.

For now, i think the gateway classes would be the ones benefitting the
most from caching as they return a lot of data, the record methods
probably rarely need to be cached, or never (correct me if im wrong).

We cant really use coldspring to do our aop. So, we need to do AOP in
the ReactorFactory.createGateway.

We would need a class, actually just one class, that has one method
that would look something like this:

<cffunction name="before" access="public" returntype="any">
<cfargument name="method" type="coldspring.aop.Method" required="true" />
<cfargument name="args" type="struct" required="true" /> <-------
These args are the key in whatever cache solution used.
<cfargument name="target" type="any" required="true" />


</cfloop>

This method has to be executed on every method call in any gateway
object, or maybe better: any method that returns a query.

If i have time i will try to rip things out of coldspring to make this
happen, or maybe even use complete cfc's from coldspring if thats
possible. The good thing about this solution would be that even custom
methods in the xxxmssql.cfc's will be cached.
It might be pretty difficult to get this done, but very unobtrusive,
the whole reactor library will stay the same, except for the one
reactorfactory.createGateway method.

The key in whatever cache solution we use, would be derived from the
set of arguments and values.

So if all argument-values are the same, we can decide to return a
cached query and not execute the normal method.

On top of this, this before method (or whatever name used), can write
up an xml file, for every method (and even possible argument values),
a default cache timeout is added (default would be 0). It can write
this up when gateway methods are called, which would mean you would
have to first execute methods to have a complete xml file. Another
option would be to write one up straightaway by parsing all gateway
cfc's, which would require extra code.

Now we can completely optimize the cache timeouts per method per
gateway and even per argument values. With this xml file you can
basically 'tune' your application, i think its even nicer that the
cfquery cache tags... which are not central, and you forget about
them. Caching is something that should be configured centrally if you
ask me, as its an aspect problem.

Another solution would be to add a custom reactortimeout attribute to
gateway methods, like <function name="getAll" reactortimeout="20" ....
But the disadvantage of this is that you can not do this with cfc's
in the project directory of reactor.

One 'but' is, if certain methods are not determenistic (e.g. do not
make the same query with the same arguments because they use variables
from other places/cfc's), the caching will mess things up. I have to
think about this better, a solution would be to just keep the
timeout's of these methods 0. It should not happen in to much cases
anyway.

The before method now has all information, a key, a timeout. Reactor
now just needs a general interface cfc that specifies an interface for
a cache solution. Whatever caching solution (implementing this
interface) can now be injected with coldspring using the
reactor.config.Config cfc, which would have to be modified for this.

What do you think about this solution? Is this worthy an RFC ?

greets,
Klaas-Jan

[Tom Chiverton]

2008/12/10 Mark Drew <mark...@gmail.com>:

> browser window and was going to be having a look at it for reactor
> (and some other projects)
>
> http://jakarta.apache.org/jcs/getting_started/intro.html

One of the nice things about Reactor is it doesn't need a .jar adding...

--
Tom

[Tom Chiverton]

2008/12/10 Klaas-Jan Winkel <k.wi...@gmail.com>:

> About the JCS, need to load external Jar files for this, but this can
> be done with:
> http://www.transfer-orm.com/?action=displayPost&ID=312

JavaLoader is a problem, because of a bug (at least in CF8), which
means it has to be loaded into the server scope (once), and then
obviously all applications have to use the same JavaLoader.
I would rather not have this dependency.

--
Tom