Active record performance issue

[Jon Levy]

I'm working on a script that reveals a strange performance issue that I'm wondering if anyone is aware of.  My script retrieves around 1,000 instances of Model1, and calls a method on each of those instances, with each method call generating an instance of Model2.

I call Model1's search method to get the 1,000-active-record list.  Here's the strange part: calling the instance method on the active records retrieve from search gives poor performance (around 1,250 seconds for my script); but if I create new active records using the ids of the search-retrieved records and calling Model1(id) on each of those ids, performance is much better (about 95 seconds).

The script breaks my 1,000 item list into 50-item sublists, and I get the described performance improvement by inserting the following line into my sublist-processing code:

recs = [Model1(i) for i in [i.id for i in recs]]  # recs is the sublist.

Anyone have a clue why this is happening?

[Cédric Krier]

I suspect that you are writing on the database after/during each call to
the instance method and you are accessing some Function field.
If it is the case, here is what could happen. At the first call Tryton
will compute the Function fields for the all 1000 records. Then if you
call write/create (or save) the local cache will be cleared. And so on
the next call to the method, Tryton will compute again the Function
fields for the 999 records and so on.
So it is always better to create/write at the end of the computation if
possible or by batch of cache_size().
And this can also explain why by instantiating record individually the
record, you have better performance because in this case the local cache
is only computed for 1 instance.

--
Cédric Krier - B2CK SPRL
Email/Jabber: cedric...@b2ck.com
Tel: +32 472 54 46 59
Website: http://www.b2ck.com/

[Jon Levy]

On Monday, January 4, 2016 at 3:15:04 PM UTC-5, Cédric Krier wrote:

At the first call Tryton
will compute the Function fields for the all 1000 records. Then if you
call write/create (or save) the local cache will be cleared. And so on
the next call to the method, Tryton will compute again the Function
fields for the 999 records and so on.

 

Thank you, that makes sense.  So I guess I can think of each records retrieved from a search as carrying along with it all of the other records retrieved, such that a field retrieved on one gets retrieved on all--which is a good optimization for most cases, but helpful to know that I can avoid by replacing them with individually instantiated instances.

[Cédric Krier]

It is a bad idea to individually instantiate instances. It is your
design that you should review to work properly with bunch of records.