Caching problem in onetomany collection
James Allen
I've got a weird caching issue with Transfer which I can't figure out,
which seems to be related to a onetomany array collection.
I'm running CF8 and the latest BER of Transfer 1.0.
I have a onetomany relationship set up from a question to it's answers
- returned as an array collection.
Here's an XML snippet:
<object name="Question" table="tblQuestions"
decorator="model.questions.question">
<id name="ID" type="numeric" />
<!--<property name="lnkIDUser" type="numeric" column="lnkIDUser" /
>
<property name="lnkIDAnswerStyle" type="numeric"
column="lnkIDAnswerStyle" />-->
<property name="PublicID" type="string" column="PublicID" />
<property name="Question" type="string" column="Question" />
<property name="SecondaryInfo" type="string"
column="SecondaryInfo" />
<property name="Anonymous" type="numeric" column="Anonymous" />
<property name="PostDate" type="date" column="PostDate" />
<property name="ExpiryDate" type="date" column="ExpiryDate" />
<property name="PosterIP" type="string" column="PosterIP" />
<property name="LastUpdateDate" type="date"
column="LastUpdateDate" />
<property name="Rating" type="numeric" column="Rating" />
<property name="NoRatings" type="numeric" column="NoRatings" />
<property name="HasBeenExtended" type="numeric"
column="HasBeenExtended" />
<!-- Link between a question and the user who posted it -->
<manytoone name="User">
<link to="users.Users" column="lnkIDUser"/>
</manytoone>
<!-- Link between a question and it's answers -->
<onetomany name="Answers" lazy="true">
<link to="question.Answer" column="lnkIDQuestion" />
<collection type="array">
<order property="PostDate" order="asc" />
</collection>
</onetomany>
</object>
<!-- Answer to a question -->
<object name="Answer" table="tblAnswers"
decorator="model.questions.answers.answer">
<id name="ID" type="numeric" />
<property name="AnswerText" type="string" column="AnswerText" />
<property name="AnswerOption" type="numeric"
column="AnswerOption" />
<property name="Rating" type="numeric" column="Rating" />
<property name="NoRatings" type="numeric" column="NoRatings" />
<property name="OwnerRating" type="numeric" column="OwnerRating" /
>
<property name="PosterIP" type="string" column="PosterIP" />
<property name="PostDate" type="date" column="PostDate" />
<!-- Link between an answer and the user who posted it -->
<manytoone name="User">
<link to="users.Users" column="lnkIDUser"/>
</manytoone>
<!-- Link between an answer and the answer choice if answer choice
type -->
<manytoone name="AnswerChoice">
<link to="question.AnswerChoices" column="lnkIDAnswerChoice"/>
</manytoone>
<onetomany name="ExtraAnswers">
<link to="question.ExtraAnswer" column="lnkIDParentAnswer" />
<collection type="array">
<order property="ID" order="asc" />
</collection>
</onetomany>
</object>
Now on my question page I call question.getAnswersArray() to obtain
the answer beans. I then loop the array and output the answers.
Now when a user rates an answer I take the answer ID and perform a
transfer.get to obtain the answer bean. I then update a property on
this to set the rating. E.G:
<cfset answerBean.setOwnerRating(rating)>
<cfset answerBean.save()>
Now once this is executed the database is updated and if I then do
another transfer.get() with the answer ID I get a fully updated answer
bean back.
However, my problem is that when the question page is refreshed the
answers array is NOT updated. I.E the answer that was updated does not
contain the new rating.
It looks like the answers array is being cached with seperate copies
of the answer beans.
Is this expected or should the above code cause the answer bean in the
answer array (the onetomany in questions) to be updated as well?
If I restart Transfer the answers list is displayed correctly.
Any help on this would be appreciated.
Cheers,
James.
Mark Mandel
What is your cache configuration?
I am a little worried about this:
<onetomany name="ExtraAnswers">
<link to="question.ExtraAnswer" column="lnkIDParentAnswer" />
<collection type="array">
<order property="ID" order="asc" />
</collection>
</onetomany>
If so, that may explain your problem.
Mark
E: mark....@gmail.com
W: www.compoundtheory.com
James Allen
Hi Mark,
Yes I use the same table for additional answers but define them as their own object in Transfer.
Why would this cause the issue?
I’ve noticed that the cache is updated sometimes but not at other times.
Cheers,
James.
Mark Mandel
Generally you should only have 1 object definition per table.
Mark
James Allen
The ID’s are all unique though (identity field) so won’t that avoid the problem or is there something I’m not considering? Also Im seeing the cache problem without necessarily using the extra answer data.
Would you think this kind of configuration would stop the cache from updating properly?
Mark Mandel
a = new("Answer", 1);
b = new("ExtraAnswer", 1);
a.setFoo("thing");
save(a);
Start to see the issue?
Mark
James Allen
Hi Mark,
I’m still not sure as I cant see how this could occur mainly because Answer is always saved first. Extra answer can only be added to Answer when Answer is persisted. Therefore both Answer and ExtraAnswer will always have unique ID’s issued to them by SQL Server. Is this situation won’t the problem you’re talking about be avoided as there are no ID collisions?
Or will Transfer do something when run in this configuration which would explain why the cache isn’t always updated when Answer is modified? In my tests today I didn’t use any ExtraAnswer data but still saw the caching issue.
Thanks for your help on this.
Mark Mandel
I'm just telling you that it's an overall bad idea.
I've never seen the scenario you are talking about ever occur - which leads me to believe you may have done this in several places.
Also, you haven't sent through your cache configuration.
What is the surrounding code you are calling to run your test?
You may need to send through a small test bed to show how this is occurring.
Mark Mandel
You running the 1.0RC release, or you running the BER from SVN?
Mark
James Allen
Hi Mark,
In terms of cache configuration I am just running the defaults (i.e. not specifying anything specifically).
I am running the latest 1.0 from SVN - “1.0RC.a”.
I will try and put together a test bed as currently it’s quite a complex AJAX update mechanism with a couple of façades and service layer / gateway.
Thanks for the advice on ExtraAnswer. I thought this would be ok as the extra answers are only accessed via an individual answer so I can’t see where the situation you describe could occur. Everything is controlled via Transfer really. I.E. I grab a question, get the answers array.. Loop the answer beans and call getExtraAnswersArray to get the extra answers.
If you think this could be causing the cache issue I could remove it from the XML and see if the issue still exists.
Thanks again,
James Allen
Hi Mark,
I just took out the ExtraAnswers functionality from the XML and the codebase and the caching issue is still there.
Cheers,
James
James Allen
Hi Mark,
I just threw together a quick test script that re-creates this problem for me. I created a new question (id=307) and added 3 answers to it. I then ran this script that firstly retrieves the answer list and displays it, then I get one of the answers (answerid=84), update the owner rating and then re-run the code to output the answer list via the parent question bean.
The owner rating is not updated when the second routine runs to display the answer list, though the change has definitely made it into the D/B.
Here is the code:
<cfset transfer = application.beanfactory.getBean("transfer")>
<!--- Get answer list to show current answers --->
<cfset questionBean = transfer.get("question.Question",307)>
<cfset answerList = questionBean.getAnswersArray()>
<cfloop from="1" to="#ArrayLen(answerList)#" index="x">
<cfoutput>
[#answerList[x].getID()#] - AnswerText: #answerList[x].getAnswerText()#<br>
OwnerRating: #answerList[x].getOwnerRating()#<br>
</cfoutput>
</cfloop>
<!--- UPDATE RATING --->
<cfset answerBean = transfer.get("question.Answer",84)>
<cfset answerBean.setOwnerRating(7)>
<cfset answerBean.save()>
<!--- Get answer list after update --->
<cfset questionBean = transfer.get("question.Question",307)>
<cfset answerList = questionBean.getAnswersArray()>
<cfloop from="1" to="#ArrayLen(answerList)#" index="x">
<cfoutput>
[#answerList[x].getID()#] - AnswerText: #answerList[x].getAnswerText()#<br>
OwnerRating: #answerList[x].getOwnerRating()#<br>
</cfoutput>
</cfloop>
Cheers,
James.
From: transf...@googlegroups.com [mailto:transf...@googlegroups.com] On Behalf Of Mark Mandel
Sent: 15 May 2008 05:01
Mark Mandel
I'm fixing a pretty nasty caching bug I introduced trying to fix another different bug at the moment... and it's possible that could cause the issue you are seeing (although, I'm not entirely sure).
It's worth a shot.
Mark
James Allen
Hi Mark,
Thanks for that, although to be honest I think I was running 1.0RC before. I upgraded to the SVN version yesterday to see if it would fix this problem.
It is a weird one..
Casey Corcoran
after looking at your code...
I don't see you saving the questionBean, it seems like that would be
your problem... if you don't save it explicitly the cache wont update.
Unless answerBean.setOwnerRating calls transfer.save on the questionBean
in question which it very well may.
Anyway, I just thought I would share my thoughts. I had the same problem
with a manytomany relationship, I figured it was bi-directional...
meaning if I did something to a child it would always reflect in the
parent which is true on the server-side but NOT in the cache.
Good luck,
Casey
> *From:* transf...@googlegroups.com
> [mailto:transf...@googlegroups.com] *On Behalf Of *Mark Mandel
> *Sent:* 15 May 2008 05:01
> *To:* transf...@googlegroups.com
> *Subject:* [transfer-dev] Re: Caching problem in onetomany collection
> *From:* transf...@googlegroups.com
> <mailto:transf...@googlegroups.com>
> [mailto:transf...@googlegroups.com
> <mailto:transf...@googlegroups.com>] *On Behalf Of *Mark Mandel
> *Sent:* 15 May 2008 00:25
>
>
> *To:* transf...@googlegroups.com <mailto:transf...@googlegroups.com>
> *Subject:* [transfer-dev] Re: Caching problem in onetomany collection
>
>
>
> What if I do this:
>
> a = new("Answer", 1);
> b = new("ExtraAnswer", 1);
>
> a.setFoo("thing");
>
> save(a);
>
> Start to see the issue?
>
> Mark
>
> On Thu, May 15, 2008 at 9:24 AM, James Allen
> <sling...@googlemail.com <mailto:sling...@googlemail.com>> wrote:
>
> The ID's are all unique though (identity field) so won't that avoid
> the problem or is there something I'm not considering? Also Im seeing
> the cache problem without necessarily using the extra answer data.
>
>
>
> Would you think this kind of configuration would stop the cache from
> updating properly?
>
>
>
> *From:* transf...@googlegroups.com
> <mailto:transf...@googlegroups.com>
> [mailto:transf...@googlegroups.com
> <mailto:transf...@googlegroups.com>] *On Behalf Of *Mark Mandel
> *Sent:* 15 May 2008 00:15
>
>
> *To:* transf...@googlegroups.com <mailto:transf...@googlegroups.com>
> *Subject:* [transfer-dev] Re: Caching problem in onetomany collection
>
>
>
> Well, what happens if you update the extra.. but use the same id for
> the usual answer?
>
> Generally you should only have 1 object definition per table.
>
> Mark
>
> On Thu, May 15, 2008 at 9:14 AM, James Allen
> <sling...@googlemail.com <mailto:sling...@googlemail.com>> wrote:
>
> Hi Mark,
>
>
>
> Yes I use the same table for additional answers but define them as
> their own object in Transfer.
>
>
>
> Why would this cause the issue?
>
>
>
> I've noticed that the cache is updated sometimes but not at other times.
>
>
>
> Cheers,
>
> James.
>
>
>
> *From:* transf...@googlegroups.com
> <mailto:transf...@googlegroups.com>
> [mailto:transf...@googlegroups.com
> <mailto:transf...@googlegroups.com>] *On Behalf Of *Mark Mandel
> *Sent:* 14 May 2008 23:36
> *To:* transf...@googlegroups.com <mailto:transf...@googlegroups.com>
> *Subject:* [transfer-dev] Re: Caching problem in onetomany collection
> E: mark....@gmail.com <mailto:mark....@gmail.com>
> W: www.compoundtheory.com <http://www.compoundtheory.com>
>
>
>
>
>
>
>
>
> --
> E: mark....@gmail.com <mailto:mark....@gmail.com>
> W: www.compoundtheory.com <http://www.compoundtheory.com>
>
>
>
>
>
>
>
>
> --
> E: mark....@gmail.com <mailto:mark....@gmail.com>
> W: www.compoundtheory.com <http://www.compoundtheory.com>
>
>
>
>
>
>
>
> --
>
> E: mark....@gmail.com <mailto:mark....@gmail.com>
> W: www.compoundtheory.com <http://www.compoundtheory.com>
>
>
>
>
> --
> E: mark....@gmail.com <mailto:mark....@gmail.com>
> W: www.compoundtheory.com <http://www.compoundtheory.com>
>
>
> >
James Allen
Thanks for that.. I think I'd thought of the question bean but didn't think
I'd need to save it for the cache to update, though it's worth a try. I
think Mark contends that this shouldn't be the expected behavior though so
maybe this shouldn't be required.
I'll try this right now.. :)
Ah damn, no even when saving the question bean after the answer update the
cache isn't updated.
Bizarre..
-----Original Message-----
From: transf...@googlegroups.com [mailto:transf...@googlegroups.com]
Casey Corcoran
This brings another question to mind, I'm too lazy to look it up right
now (must make coffee) but is there a way in Transfer to manually update
the cache of a given TransferObject. Something like:
transfer.reCache( "foo.MyFoo", 123 );
or
foo = transfer.get( "foo.MyFoo", 123 );
transfer.reCache( foo );
Just a thought.
James Allen
There is a discard method to drop an object from the cache. I'm not sure
this will work though as once the answer is saved, if I then do a get() it
comes back correctly - so seems to be an issue with the answers array cache
in the question bean.
I will test though.
I'm just about to roll back to Mark's bane - version 0.6.3 to see what
happens then.
-----Original Message-----
From: transf...@googlegroups.com [mailto:transf...@googlegroups.com]
On Behalf Of Casey Corcoran
Sent: 15 May 2008 15:10
To: transf...@googlegroups.com
Subject: [transfer-dev] Re: Caching problem in onetomany collection
James Allen
Hi Mark,
Some bad news. I rolled back to good old 0.6.3 and the caching issue disappeared – everything is now working as expected.
Is there anything in 1.0 which could be causing this with the way I am doing things? I lazy load the answers array from the questions bean so don’t know if it might be something to do with that?
James Allen
Ah… ignore my last message..
I hadn’t applied the mapping change to switch to 0.6.3… :(
So it *was* working in my last test. I just tried again though and it didn’t work – the caching issue was back.
This seems to be quite intermittent.
James Allen
Right an update, sort of.
When I store the owner rating in the answer bean, after saving it I discard it from the cache manually.
Everything is now working fine (thanks for the suggestion Casey).
I would like to know what is causing the issue though – my code or something in Transfer.
Mark, if you need me to try anything to try and identify the issue let me know.
James Allen
Hi Mark,
Right I think I’ve worked out what the problem is here and it’s not Transfer (it rarely is J ).
This goes back to something you helped with a months ago…
When I create a new question I store the new Transfer bean in session. This maintains the question data until it’s time to save it.
Ok, I’m sure you can see where I’m going with this. So, when I save the bean the session version of the bean is now going to be persisted – very bad!
I assume this must be the cause of the issues. It also explains that if I leave the site for a while then come back the problem goes away.. Down to the session expiring no doubt…
The thing is though, I have now put code in place to discard the question from the cache AND remove the session variable completely. I tried StructDelete and setting the session question to 0 but the cache problem still exists. It goes away if I restart my application and restart Transfer – I assume because it purges any session pointers.
Even if I StructDelete or set the session variable to 0, is it possible that Transfer is still pointing to the data? I can’t see how this can be possible but I know you do some pretty hardcore stuff inside the core. Or is the whole session thing causing Transfer to get messed up internally?
The only workaround I have to this which I was using before is to purge the answers from the cache as they get rated – this solves the issue.
Any advice or info on what is happening would be appreciated. I’d like to know more about the inner workings of the cache and why it is happening in this case even when the bean is discarded from the cache and the session variable removed.
Mark Mandel
I haven't forgotten about you, I'm just working out a bug.
Once I've done that, I will build a unit test to look into what you are describing.
Mark
James Allen
Hi Mark,
Thanks for looking into this for me.. It is a weird one but the Session storage factor must be having a big effect I’d think.
Mark Mandel
<package name="onetomany">
<object name="Basic" table="tbl_onetomany" sequence="tbl_onetomany_seq2">
<id name="idbasic" type="numeric" column="idbasic"/>
<property name="string" type="GUID" column="basic_string"/>
<onetomany name="child">
<link to="onetomany.Child" column="lnkBasicID"/>
<collection type="array">
<order property="name" order="asc"/>
</collection>
</onetomany>
</object>
<object name="Child" table="tbl_onetomanyChild">
<id name="IDChild" type="numeric" column="IDChild"/>
<property name="name" type="string" column="child_name"/>
</object>
</package>
<cffunction name="testChildOneToManySave" hint="" access="public" returntype="void" output="false">
<cfscript>
var simple = getTransfer().new("onetomany.Basic");
var child = 0;
var clone = 0;
var counter = 1;
var reget = 0;
var regetChild = 0;
var array = 0;
getTransfer().save(simple);
//add five children
for(; counter lte 5; counter = counter + 1)
{
child = getTransfer().new("onetomany.Child");
child.setName(RandRange(1, 9999));
child.setParentBasic(simple);
getTransfer().save(child);
}
getTransfer().discardAll();
reget = getTransfer().get("onetomany.Basic", simple.getIDBasic());
AssertEquals("Should be 5", ArrayLen(reget.getChildArray()), 5);
regetChild = getTransfer().get("onetomany.Child", reget.getChild(2).getIDChild());
AssertEquals("should have same id", regetChild.getIdChild(), reget.getChild(2).getIdChild());
AssertSame("(1) should be the same child", regetChild, reget.getChild(2));
regetChild.setName("Yoda");
AssertEquals("(1) should be 'Yoda'", "Yoda", regetChild.getName());
AssertEquals("(2) should be 'Yoda'", "Yoda", reget.getChild(2).getName());
getTransfer().save(regetChild);
//now it has been re-ordered, so lets find it again
array = reget.getChildArray();
child = 0; //so it fails if it can't be found
for(counter = 1; counter lte 5; counter = counter + 1)
{
if(array[counter].getIDChild() eq regetChild.getIDChild())
{
child = array[counter];
break;
}
}
AssertEquals("(3) should be 'Yoda'", "Yoda", child.getName());
AssertSame("(2) should be the same child", regetChild, child);
</cfscript>
</cffunction>
Can you see where I am going wrong.
It looks like you'll have to send me a test bed so I can see what is going wrong.
Mark
James Allen
Hi Mark,
I’m wondering whether the fact that I store the unpersisted question bean in session, then save it, then get it could be affecting things?
That seems to be the factor that is causing the issue, even if I remove the session variable and discard the question from the cache first – which is confusing as I’d assume that once removed from session and discarded from the cache it would work correctly.
Mark Mandel
Like i keep saying, you'll have to build me a test bed to outline the issue.
Mark
On 5/16/08, James Allen <sling...@googlemail.com> wrote:
> Hi Mark,
>
>
>
> I'm wondering whether the fact that I store the unpersisted question bean in
> session, then save it, then get it could be affecting things?
>
>
>
> That seems to be the factor that is causing the issue, even if I remove the
> session variable and discard the question from the cache first - which is
> This goes back to something you helped with a months ago.
>
>
>
> When I create a new question I store the new Transfer bean in session. This
> maintains the question data until it's time to save it.
>
>
>
> Ok, I'm sure you can see where I'm going with this. So, when I save the bean
> the session version of the bean is now going to be persisted - very bad!
>
>
>
> I assume this must be the cause of the issues. It also explains that if I
> leave the site for a while then come back the problem goes away.. Down to
> the session expiring no doubt.
>
>
>
> The thing is though, I have now put code in place to discard the question
> from the cache AND remove the session variable completely. I tried
> StructDelete and setting the session question to 0 but the cache problem
> still exists. It goes away if I restart my application and restart Transfer
> - I assume because it purges any session pointers.
>
>
>
> Even if I StructDelete or set the session variable to 0, is it possible that
> Transfer is still pointing to the data? I can't see how this can be possible
> but I know you do some pretty hardcore stuff inside the core. Or is the
> whole session thing causing Transfer to get messed up internally?
>
>
>
> The only workaround I have to this which I was using before is to purge the
> answers from the cache as they get rated - this solves the issue.
James Allen
Thanks for all your help on this.
I have been sitting here for a while now trying to think of an easy way to
create a test bed but it's not been that easy. I'm trying to put something
together but want to make sure it properly maps out the process the façade,
service, gateway and bean are a part of.
I'm tempted to just stick with discarding the answer bean whenever a rating
is received although I realize this is not efficient. The weird thing with
this is that once Transfer is restarted this problem goes away for all
subsequent rating operations. It is very stange.
Cheers,
James.