Cache Discard Algorithm change in SVN

[Mark Mandel]

Another change, this is a bit trickier.

I've changed the way discard() operations on the cache occur, whether by timeout from the cache, or explicitly.

Previously, in a situation where you would have:

A -manytone-> B

If B timed out, or was discarded (essentially the same thing), *both* A and B would be discarded.

So Transfer would follow the compositions, to ensure there were no other TransferObjects in the cache that pointed to the TransferObject that was being discarded.

This means that in cache configuration, lowest timeout would always win, and you didn't have much granular control over chache timeouts.. and because of that, pieces would skip in and out of the cache quite regularly.

This has now changed.

Now if you have:

A -manytoone-> B

and B is timed out / discard, *only* B is discarded, and A simplye 'unloads' the compsition that points to B.

So, unless you go back and request B from A, it won't be loaded back in again.

This means you have FAR more control over how your cache is tuned, and you can do clever things like this:

Say, you have:

User -manytone-> Role -onetomany-> Permission
User -onetomany -> Permission

So users hook up to a Role which has permissions, but you can also connect ad-hoc permissions to the user.

So you write up a decorator to cache the set of permissions inside your User.. but the roles.. and permissions are still hanging around, taking up memory.

So now, you can set the user to be cached for an Hour, but set the Permissions to be cached, with a accessedminutestimeout of 10 minutes, so once they have stopped being used, they get dropped off the User object.

This does mean, if you were using a top level object to control an entire cache hierarchy, that won't work anymore.  I would suggest actually setting a defautlCache timeout (either max, or accessed), and then tweaking it from there.


I've also changed some pieces in the Observer collections, that may cause some slowdown in create/update/delete... but from my tests shouldn't be anything more than 10-50ms per 10000 items - but in the long run will just allow for some faster processing of dumping unused objects.  Let me know if anyone sees any drastic, and I can always put it back the way it was.

Let me know if you have any questions.

Mark
--
E: mark....@gmail.com
W: www.compoundtheory.com

[James Allen]

This all does sound really great – increasing the level of control on something which has the potential to really speed up an application is essential..

 

Although saying that ( J ), as commented before we really do need a good tutorial and theory document on Transfer caching and cache strategy.

At the moment I really wouldn’t know where to start in terms of knowing how to analyse an application to work out where caching should be tweaked or what is available to make this happen in Transfer.

 

Has anyone stepped up to put something like this together on the Wiki?

[mark mazelin]

Mark:

Does this work with an object that only has the onetomany relationship
set? If so, I think this will solve the problem I submitted to this
list a couple weeks ago:
http://groups.google.com/group/transfer-dev/browse_thread/thread/ce508f0dbe43a737/e4985033bc70ea24

Is the svn code stable enough to check out and use, or should I wait a
bit before trying?

On another note, when setting onetomany relationships, should you
always set a matching manytoone on the other object? For instance in
my app, a donation has a onetomany to a payment, but there is no
reciprocal manytoone setting in the payment object. Should there be?

Thanks,
Maz

> E: mark.man...@gmail.com
> W:www.compoundtheory.com

[Mark Mandel]

Just in case this wasn't clear - this is the case for all composite relationships.  I was just using m2o as an example.

On Thu, Jun 26, 2008 at 11:12 PM, mark mazelin <markm...@gmail.com> wrote:

Mark:

Does this work with an object that only has the onetomany relationship
set? If so, I think this will solve the problem I submitted to this
list a couple weeks ago:
http://groups.google.com/group/transfer-dev/browse_thread/thread/ce508f0dbe43a737/e4985033bc70ea24

In a way it does. (There was a user->payment relationship) If you discard the payment object, the user will still stay cached, and will simply unload the payment object. Make sense?
 

Is the svn code stable enough to check out and use, or should I wait a
bit before trying?

Whatever is checked into SVN has passed all my unit tests.  I encourage people to try it out, so as to test it in real world apps.
 

On another note, when setting onetomany relationships, should you
always set a matching manytoone on the other object? For instance in
my app, a donation has a onetomany to a payment, but there is no
reciprocal manytoone setting in the payment object. Should there be?

No, there shouldn't. I recently added several notes to the documentation to this effect.

[mark mazelin]

I was finally able to get back to this for testing. I downloaded the
latest rev from subversion (533). I added the following cache config
to my transfer xml:

<objectCache>
<cache class="donor.payment">
<scope type="request"/>
</cache>
</objectCache>

A donor.donation object has a onetomany with payment:
<onetomany name="payment" lazy="true">
<link to="donor.payment" column="DonationID" />
<collection type="array">
<order property="CreatedDate" order="asc" />
</collection>
</onetomany>

But the cache seems to still be too sticky. I have the same transfer
config loaded in two different coldfusion apps. I want the payment(s)
to only cache for the current request. But if I view the donation (w/
payment) in the admin in between the time the initial donation and
payment records are created and the payment is completed, the payment
information is updated on the end user application, but not in the
admin application. I cannot figure out what I'm doing wrong. Any help
would be appreciated.

Thanks.

On Jun 27, 12:58 am, "Mark Mandel" <mark.man...@gmail.com> wrote:
> Just in case this wasn't clear - this is the case for all composite
> relationships. I was just using m2o as an example.
>

> On Thu, Jun 26, 2008 at 11:12 PM, mark mazelin <markmaze...@gmail.com>

> wrote:
>
>
>
> > Mark:
>
> > Does this work with an object that only has the onetomany relationship
> > set? If so, I think this will solve the problem I submitted to this
> > list a couple weeks ago:
>

> >http://groups.google.com/group/transfer-dev/browse_thread/thread/ce50...

>
> In a way it does. (There was a user->payment relationship) If you discard
> the payment object, the user will still stay cached, and will simply unload
> the payment object. Make sense?
>
>
>
> > Is the svn code stable enough to check out and use, or should I wait a
> > bit before trying?
>
> Whatever is checked into SVN has passed all my unit tests. I encourage
> people to try it out, so as to test it in real world apps.
>
>
>
> > On another note, when setting onetomany relationships, should you
> > always set a matching manytoone on the other object? For instance in
> > my app, a donation has a onetomany to a payment, but there is no
> > reciprocal manytoone setting in the payment object. Should there be?
>
> No, there shouldn't. I recently added several notes to the documentation to
> this effect.
>
> --

> E: mark.man...@gmail.com
> W:www.compoundtheory.com

[Mark Mandel]

Mark,

Having objects that have cache settings that span multiple scopes has
always been a bad idea.

I had a note in the documentation, but it didn't provide much
information, I've now expanded it out:

http://docs.transfer-orm.com/wiki/Managing_the_Cache.cfm

Let me know if that doesn't make any sense.

Mark

--
E: mark....@gmail.com
W: www.compoundtheory.com