I am at the early stages (still) of looking into migrating a legacy
code base to one of the CF OO frameworks. (Probably modelglue)
We would like to use transfer in many places. But we are not sure
about how well it scales. We have a large website with tens of
thousands of articles, all of which are stored in the database. I am
wondering what sort of size systems people have built with transfer?
Scalability is very dependent on how you set up your system, what caching options you use, how you model your data, the size of the data that you pull in, how many objects you want to use at once.. the list goes on.
So to answer your question - it depends.
Maybe if you can outline some of the performance issues you are having (I believe that you mentioned it in another thread), we can work out solutions for them.
Regards,
Mark
On Feb 6, 2008 4:32 AM, jez_f <jeremy.fre...@gmail.com> wrote:
> I am at the early stages (still) of looking into migrating a legacy > code base to one of the CF OO frameworks. (Probably modelglue)
> We would like to use transfer in many places. But we are not sure > about how well it scales. We have a large website with tens of > thousands of articles, all of which are stored in the database. I am > wondering what sort of size systems people have built with transfer?
I'll just chip in with this thought as well - the architectural style of the app makes a huge difference. If you keep your app close to the database, use SQL (either directly or through Transfer) to do most of the heavy lifting, use queries within your app for bulk data, and use transfer objects for dealing with single objects or small collections of objects, CF + Transfer scales very well. If you want to write a totally database agnostic app and use Transfer to marshall arbitrary numbers of records between the object model and the database, you'll run into performance problems very quickly.
So in terms of baseline performance tens of thousands of articles isn't going to be a problem if you return single articles as transfer objects, but use queries to list articles.
To answer your question re size: I have an app with about 15,000 users where the users/org units/managers are modelled as transfer objects. On the level of a single request the pages which interact with transfer are not perceptibly slower than pages that just use plain old cfquery. At the server level, a single server copes with about 40 simultaneous users (plus hosts lots of other apps) without breaking a sweat. Some informal load testing indicates that this configuration will top out at about 10 requests/second (which would equate to a couple of thousand users), but of course that's very hardware dependent.
Jaime
On Feb 6, 2008 6:05 AM, Mark Mandel <mark.man...@gmail.com> wrote:
> This is one of those million dollar questions ;o)
> Scalability is very dependent on how you set up your system, what caching > options you use, how you model your data, the size of the data that you pull > in, how many objects you want to use at once.. the list goes on.
> So to answer your question - it depends.
> Maybe if you can outline some of the performance issues you are having (I > believe that you mentioned it in another thread), we can work out solutions > for them.
> Regards,
> Mark
> On Feb 6, 2008 4:32 AM, jez_f <jeremy.fre...@gmail.com> wrote:
> > Hi there
> > I am at the early stages (still) of looking into migrating a legacy > > code base to one of the CF OO frameworks. (Probably modelglue)
> > We would like to use transfer in many places. But we are not sure > > about how well it scales. We have a large website with tens of > > thousands of articles, all of which are stored in the database. I am > > wondering what sort of size systems people have built with transfer?
I started to echo what Jaime said and when I got to a couple of paragraphs, I figured I should just blog about it, since I think this is an important point (and was planning a post on it anyway). For the visually oriented folks in the crowd, there's a diagram with some boxes and lines.
> I'll just chip in with this thought as well - the architectural style of > the app makes a huge difference. If you keep your app close to the > database, use SQL (either directly or through Transfer) to do most of the > heavy lifting, use queries within your app for bulk data, and use transfer > objects for dealing with single objects or small collections of objects, CF > + Transfer scales very well. If you want to write a totally database > agnostic app and use Transfer to marshall arbitrary numbers of records > between the object model and the database, you'll run into performance > problems very quickly.
> So in terms of baseline performance tens of thousands of articles isn't > going to be a problem if you return single articles as transfer objects, but > use queries to list articles.
> To answer your question re size: I have an app with about 15,000 users > where the users/org units/managers are modelled as transfer objects. On the > level of a single request the pages which interact with transfer are not > perceptibly slower than pages that just use plain old cfquery. At the > server level, a single server copes with about 40 simultaneous users (plus > hosts lots of other apps) without breaking a sweat. Some informal load > testing indicates that this configuration will top out at about 10 > requests/second (which would equate to a couple of thousand users), but of > course that's very hardware dependent.
> Jaime
> On Feb 6, 2008 6:05 AM, Mark Mandel <mark.man...@gmail.com> wrote:
> > Jeremy,
> > This is one of those million dollar questions ;o)
> > Scalability is very dependent on how you set up your system, what > > caching options you use, how you model your data, the size of the data that > > you pull in, how many objects you want to use at once.. the list goes on.
> > So to answer your question - it depends.
> > Maybe if you can outline some of the performance issues you are having > > (I believe that you mentioned it in another thread), we can work out > > solutions for them.
> > Regards,
> > Mark
> > On Feb 6, 2008 4:32 AM, jez_f <jeremy.fre...@gmail.com> wrote:
> > > Hi there
> > > I am at the early stages (still) of looking into migrating a legacy > > > code base to one of the CF OO frameworks. (Probably modelglue)
> > > We would like to use transfer in many places. But we are not sure > > > about how well it scales. We have a large website with tens of > > > thousands of articles, all of which are stored in the database. I am > > > wondering what sort of size systems people have built with transfer?
The honour is all mine. The OO designer and the CF programmer in me are still beating each other up over your conclusion, but that's a whole other topic.
I should add that the performance problems when using Transfer to marshall large numbers of objects are not really Transfer's fault. You won't do significantly better in any ColdFusion code. And no, CF8's object creation speedups don't really change this conclusion, not even with the 1.5 JVM (although they are pretty nice). Also no, IMHO this isn't really a good enough reason to jump to Java - it *is* orders of magnitude faster at creating objects, but that just brings it it the same ballpark as CF's query handling.
Jaime
On Feb 6, 2008 10:29 AM, Paul Marcotte <pmarco...@gmail.com> wrote:
> I started to echo what Jaime said and when I got to a couple of > paragraphs, I figured I should just blog about it, since I think this is an > important point (and was planning a post on it anyway). For the visually > oriented folks in the crowd, there's a diagram with some boxes and lines.
> In a nutshell, I've had a revelation in understanding how the view > requirements heavily influence the model design.
> Jaime, I hope you don't mind that I paraphrased you for the post. Your > take on subject is spot on.
> Paul
> On Feb 5, 2008 1:30 PM, Jaime Metcher <jmetc...@gmail.com> wrote:
> > I'll just chip in with this thought as well - the architectural style of > > the app makes a huge difference. If you keep your app close to the > > database, use SQL (either directly or through Transfer) to do most of the > > heavy lifting, use queries within your app for bulk data, and use transfer > > objects for dealing with single objects or small collections of objects, CF > > + Transfer scales very well. If you want to write a totally database > > agnostic app and use Transfer to marshall arbitrary numbers of records > > between the object model and the database, you'll run into performance > > problems very quickly.
> > So in terms of baseline performance tens of thousands of articles isn't > > going to be a problem if you return single articles as transfer objects, but > > use queries to list articles.
> > To answer your question re size: I have an app with about 15,000 users > > where the users/org units/managers are modelled as transfer objects. On the > > level of a single request the pages which interact with transfer are not > > perceptibly slower than pages that just use plain old cfquery. At the > > server level, a single server copes with about 40 simultaneous users (plus > > hosts lots of other apps) without breaking a sweat. Some informal load > > testing indicates that this configuration will top out at about 10 > > requests/second (which would equate to a couple of thousand users), but of > > course that's very hardware dependent.
> > Jaime
> > On Feb 6, 2008 6:05 AM, Mark Mandel <mark.man...@gmail.com> wrote:
> > > Jeremy,
> > > This is one of those million dollar questions ;o)
> > > Scalability is very dependent on how you set up your system, what > > > caching options you use, how you model your data, the size of the data that > > > you pull in, how many objects you want to use at once.. the list goes on.
> > > So to answer your question - it depends.
> > > Maybe if you can outline some of the performance issues you are having > > > (I believe that you mentioned it in another thread), we can work out > > > solutions for them.
> > > Regards,
> > > Mark
> > > On Feb 6, 2008 4:32 AM, jez_f <jeremy.fre...@gmail.com> wrote:
> > > > Hi there
> > > > I am at the early stages (still) of looking into migrating a legacy > > > > code base to one of the CF OO frameworks. (Probably modelglue)
> > > > We would like to use transfer in many places. But we are not sure > > > > about how well it scales. We have a large website with tens of > > > > thousands of articles, all of which are stored in the database. I am > > > > wondering what sort of size systems people have built with transfer?
I gotta mention the idea of an IBO here (or your own version of an object iterator). Wrap the cfquery into a single business object with iterating methods, a loadQuery() method and support for generic and custom getters and setters. That way you can get the encapsulation benefits of an object and the performance of a query. It also means that whether you have one OrderItem or a hundred of them, if you want to have a getItemTotal() custom method that multiplies the item unit price and item quantity, you can use that same code for a single business object or for a collection of them without worrying about performance.
Best Wishes, Peter
On 2/5/08 11:25 PM, "Jaime Metcher" <jmetc...@gmail.com> wrote:
> The honour is all mine. The OO designer and the CF programmer in me are still > beating each other up over your conclusion, but that's a whole other topic.
> I should add that the performance problems when using Transfer to marshall > large numbers of objects are not really Transfer's fault. You won't do > significantly better in any ColdFusion code. And no, CF8's object creation > speedups don't really change this conclusion, not even with the 1.5 JVM > (although they are pretty nice). Also no, IMHO this isn't really a good > enough reason to jump to Java - it *is* orders of magnitude faster at creating > objects, but that just brings it it the same ballpark as CF's query handling.
> Jaime
> On Feb 6, 2008 10:29 AM, Paul Marcotte <pmarco...@gmail.com> wrote: >> Hey all,
>> I started to echo what Jaime said and when I got to a couple of paragraphs, I >> figured I should just blog about it, since I think this is an important point >> (and was planning a post on it anyway). For the visually oriented folks in >> the crowd, there's a diagram with some boxes and lines.
>> In a nutshell, I've had a revelation in understanding how the view >> requirements heavily influence the model design.
>> Jaime, I hope you don't mind that I paraphrased you for the post. Your take >> on subject is spot on.
>> Paul
>> On Feb 5, 2008 1:30 PM, Jaime Metcher <jmetc...@gmail.com> wrote: >>> I'll just chip in with this thought as well - the architectural style of the >>> app makes a huge difference. If you keep your app close to the database, >>> use SQL (either directly or through Transfer) to do most of the heavy >>> lifting, use queries within your app for bulk data, and use transfer objects >>> for dealing with single objects or small collections of objects, CF + >>> Transfer scales very well. If you want to write a totally database agnostic >>> app and use Transfer to marshall arbitrary numbers of records between the >>> object model and the database, you'll run into performance problems very >>> quickly.
>>> So in terms of baseline performance tens of thousands of articles isn't >>> going to be a problem if you return single articles as transfer objects, but >>> use queries to list articles.
>>> To answer your question re size: I have an app with about 15,000 users where >>> the users/org units/managers are modelled as transfer objects. On the level >>> of a single request the pages which interact with transfer are not >>> perceptibly slower than pages that just use plain old cfquery. At the >>> server level, a single server copes with about 40 simultaneous users (plus >>> hosts lots of other apps) without breaking a sweat. Some informal load >>> testing indicates that this configuration will top out at about 10 >>> requests/second (which would equate to a couple of thousand users), but of >>> course that's very hardware dependent.
>>> Jaime
>>> On Feb 6, 2008 6:05 AM, Mark Mandel <mark.man...@gmail.com> wrote: >>>> Jeremy,
>>>> This is one of those million dollar questions ;o)
>>>> Scalability is very dependent on how you set up your system, what caching >>>> options you use, how you model your data, the size of the data that you >>>> pull in, how many objects you want to use at once.. the list goes on.
>>>> So to answer your question - it depends.
>>>> Maybe if you can outline some of the performance issues you are having (I >>>> believe that you mentioned it in another thread), we can work out solutions >>>> for them.
>>>> Regards,
>>>> Mark
>>>> On Feb 6, 2008 4:32 AM, jez_f <jeremy.fre...@gmail.com> wrote:
>>>>> Hi there
>>>>> I am at the early stages (still) of looking into migrating a legacy >>>>> code base to one of the CF OO frameworks. (Probably modelglue)
>>>>> We would like to use transfer in many places. But we are not sure >>>>> about how well it scales. We have a large website with tens of >>>>> thousands of articles, all of which are stored in the database. I am >>>>> wondering what sort of size systems people have built with transfer?
> I gotta mention the idea of an IBO here (or your own version of an object > iterator). Wrap the cfquery into a single business object with iterating > methods, a loadQuery() method and support for generic and custom getters and > setters. That way you can get the encapsulation benefits of an object and > the performance of a query. It also means that whether you have one > OrderItem or a hundred of them, if you want to have a getItemTotal() custom > method that multiplies the item unit price and item quantity, you can use > that same code for a single business object or for a collection of them > without worrying about performance.
> Best Wishes, > Peter
> On 2/5/08 11:25 PM, "Jaime Metcher" <jmetc...@gmail.com> wrote:
> Paul,
> The honour is all mine. The OO designer and the CF programmer in me are > still beating each other up over your conclusion, but that's a whole other > topic.
> I should add that the performance problems when using Transfer to marshall > large numbers of objects are not really Transfer's fault. You won't do > significantly better in any ColdFusion code. And no, CF8's object creation > speedups don't really change this conclusion, not even with the 1.5 JVM > (although they are pretty nice). Also no, IMHO this isn't really a good > enough reason to jump to Java - it *is* orders of magnitude faster at > creating objects, but that just brings it it the same ballpark as CF's query > handling.
> Jaime
> On Feb 6, 2008 10:29 AM, Paul Marcotte <pmarco...@gmail.com> wrote:
> Hey all,
> I started to echo what Jaime said and when I got to a couple of > paragraphs, I figured I should just blog about it, since I think this is an > important point (and was planning a post on it anyway). For the visually > oriented folks in the crowd, there's a diagram with some boxes and lines.
> In a nutshell, I've had a revelation in understanding how the view > requirements heavily influence the model design.
> Jaime, I hope you don't mind that I paraphrased you for the post. Your > take on subject is spot on.
> Paul
> On Feb 5, 2008 1:30 PM, Jaime Metcher <jmetc...@gmail.com> wrote:
> I'll just chip in with this thought as well - the architectural style of > the app makes a huge difference. If you keep your app close to the > database, use SQL (either directly or through Transfer) to do most of the > heavy lifting, use queries within your app for bulk data, and use transfer > objects for dealing with single objects or small collections of objects, CF > + Transfer scales very well. If you want to write a totally database > agnostic app and use Transfer to marshall arbitrary numbers of records > between the object model and the database, you'll run into performance > problems very quickly.
> So in terms of baseline performance tens of thousands of articles isn't > going to be a problem if you return single articles as transfer objects, but > use queries to list articles.
> To answer your question re size: I have an app with about 15,000 users > where the users/org units/managers are modelled as transfer objects. On the > level of a single request the pages which interact with transfer are not > perceptibly slower than pages that just use plain old cfquery. At the > server level, a single server copes with about 40 simultaneous users (plus > hosts lots of other apps) without breaking a sweat. Some informal load > testing indicates that this configuration will top out at about 10 > requests/second (which would equate to a couple of thousand users), but of > course that's very hardware dependent.
> Jaime
> On Feb 6, 2008 6:05 AM, Mark Mandel <mark.man...@gmail.com> wrote:
> Jeremy,
> This is one of those million dollar questions ;o)
> Scalability is very dependent on how you set up your system, what caching > options you use, how you model your data, the size of the data that you pull > in, how many objects you want to use at once.. the list goes on.
> So to answer your question - it depends.
> Maybe if you can outline some of the performance issues you are having (I > believe that you mentioned it in another thread), we can work out solutions > for them.
> Regards,
> Mark
> On Feb 6, 2008 4:32 AM, jez_f <jeremy.fre...@gmail.com> wrote:
> Hi there
> I am at the early stages (still) of looking into migrating a legacy > code base to one of the CF OO frameworks. (Probably modelglue)
> We would like to use transfer in many places. But we are not sure > about how well it scales. We have a large website with tens of > thousands of articles, all of which are stored in the database. I am > wondering what sort of size systems people have built with transfer?
Sure I do. I have a getAssociated() method on my IBOąs thatąll allow you to get an IBO containing 0..n associated objects based on the association relationships described in the business object XML.
Of course, if you have n-users, as n grows you have an n+1 query problem, so this approach would only be relevant for displaying a relatively small number of users each of which had many addresses. But thatąs a fundamental issue with a large number of objects each of which have a large number of to_many relationships to display in detail on a single screen. I have separate aggregate features which generate subqueries in the select if I just want to display a list of objects and aggregate information about their to_many children without having to run n+1 queries, and I have a data mapper that provides for left outer joins to has_one related objects if I want to display a long list of users with their (associated) bosses name using just one generated query.
Best Wishes, Peter
On 2/5/08 11:54 PM, "Mark Mandel" <mark.man...@gmail.com> wrote:
> But you don't get the benefits of object composition, do you Peter?
> Which is a pretty handy thing.
> Mark
> On Feb 6, 2008 3:48 PM, Peter Bell <pb...@systemsforge.com> wrote: >> I gotta mention the idea of an IBO here (or your own version of an object >> iterator). Wrap the cfquery into a single business object with iterating >> methods, a loadQuery() method and support for generic and custom getters and >> setters. That way you can get the encapsulation benefits of an object and the >> performance of a query. It also means that whether you have one OrderItem or >> a hundred of them, if you want to have a getItemTotal() custom method that >> multiplies the item unit price and item quantity, you can use that same code >> for a single business object or for a collection of them without worrying >> about performance.
>> Best Wishes, >> Peter
>> On 2/5/08 11:25 PM, "Jaime Metcher" <jmetc...@gmail.com> wrote:
>>> Paul,
>>> The honour is all mine. The OO designer and the CF programmer in me are >>> still beating each other up over your conclusion, but that's a whole other >>> topic.
>>> I should add that the performance problems when using Transfer to marshall >>> large numbers of objects are not really Transfer's fault. You won't do >>> significantly better in any ColdFusion code. And no, CF8's object creation >>> speedups don't really change this conclusion, not even with the 1.5 JVM >>> (although they are pretty nice). Also no, IMHO this isn't really a good >>> enough reason to jump to Java - it *is* orders of magnitude faster at >>> creating objects, but that just brings it it the same ballpark as CF's query >>> handling.
>>> Jaime
>>> On Feb 6, 2008 10:29 AM, Paul Marcotte <pmarco...@gmail.com> wrote: >>>> Hey all,
>>>> I started to echo what Jaime said and when I got to a couple of paragraphs, >>>> I figured I should just blog about it, since I think this is an important >>>> point (and was planning a post on it anyway). For the visually oriented >>>> folks in the crowd, there's a diagram with some boxes and lines.
>>>> In a nutshell, I've had a revelation in understanding how the view >>>> requirements heavily influence the model design.
>>>> Jaime, I hope you don't mind that I paraphrased you for the post. Your >>>> take on subject is spot on.
>>>> Paul
>>>> On Feb 5, 2008 1:30 PM, Jaime Metcher <jmetc...@gmail.com> wrote: >>>>> I'll just chip in with this thought as well - the architectural style of >>>>> the app makes a huge difference. If you keep your app close to the >>>>> database, use SQL (either directly or through Transfer) to do most of the >>>>> heavy lifting, use queries within your app for bulk data, and use transfer >>>>> objects for dealing with single objects or small collections of objects, >>>>> CF + Transfer scales very well. If you want to write a totally database >>>>> agnostic app and use Transfer to marshall arbitrary numbers of records >>>>> between the object model and the database, you'll run into performance >>>>> problems very quickly.
>>>>> So in terms of baseline performance tens of thousands of articles isn't >>>>> going to be a problem if you return single articles as transfer objects, >>>>> but use queries to list articles.
>>>>> To answer your question re size: I have an app with about 15,000 users >>>>> where the users/org units/managers are modelled as transfer objects. On >>>>> the level of a single request the pages which interact with transfer are >>>>> not perceptibly slower than pages that just use plain old cfquery. At the >>>>> server level, a single server copes with about 40 simultaneous users (plus >>>>> hosts lots of other apps) without breaking a sweat. Some informal load >>>>> testing indicates that this configuration will top out at about 10 >>>>> requests/second (which would equate to a couple of thousand users), but of >>>>> course that's very hardware dependent.
>>>>> Jaime
>>>>> On Feb 6, 2008 6:05 AM, Mark Mandel <mark.man...@gmail.com> wrote: >>>>>> Jeremy,
>>>>>> This is one of those million dollar questions ;o)
>>>>>> Scalability is very dependent on how you set up your system, what caching >>>>>> options you use, how you model your data, the size of the data that you >>>>>> pull in, how many objects you want to use at once.. the list goes on.
>>>>>> So to answer your question - it depends.
>>>>>> Maybe if you can outline some of the performance issues you are having (I >>>>>> believe that you mentioned it in another thread), we can work out >>>>>> solutions for them.
>>>>>> Regards,
>>>>>> Mark
>>>>>> On Feb 6, 2008 4:32 AM, jez_f <jeremy.fre...@gmail.com> wrote:
>>>>>>> Hi there
>>>>>>> I am at the early stages (still) of looking into migrating a legacy >>>>>>> code base to one of the CF OO frameworks. (Probably modelglue)
>>>>>>> We would like to use transfer in many places. But we are not sure >>>>>>> about how well it scales. We have a large website with tens of >>>>>>> thousands of articles, all of which are stored in the database. I am >>>>>>> wondering what sort of size systems people have built with transfer?
> Sure I do. I have a getAssociated() method on my IBO's that'll allow you > to get an IBO containing 0..n associated objects based on the association > relationships described in the business object XML.
> Of course, if you have n-users, as n grows you have an n+1 query problem, > so this approach would only be relevant for displaying a relatively small > number of users each of which had many addresses. But that's a fundamental > issue with a large number of objects each of which have a large number of > to_many relationships to display in detail on a single screen. I have > separate aggregate features which generate subqueries in the select if I > just want to display a list of objects and aggregate information about their > to_many children without having to run n+1 queries, and I have a data mapper > that provides for left outer joins to has_one related objects if I want to > display a long list of users with their (associated) bosses name using just > one generated query.
> Best Wishes, > Peter
> On 2/5/08 11:54 PM, "Mark Mandel" <mark.man...@gmail.com> wrote:
> But you don't get the benefits of object composition, do you Peter?
> Which is a pretty handy thing.
> Mark
> On Feb 6, 2008 3:48 PM, Peter Bell <pb...@systemsforge.com> wrote:
> I gotta mention the idea of an IBO here (or your own version of an object > iterator). Wrap the cfquery into a single business object with iterating > methods, a loadQuery() method and support for generic and custom getters and > setters. That way you can get the encapsulation benefits of an object and > the performance of a query. It also means that whether you have one > OrderItem or a hundred of them, if you want to have a getItemTotal() custom > method that multiplies the item unit price and item quantity, you can use > that same code for a single business object or for a collection of them > without worrying about performance.
> Best Wishes, > Peter
> On 2/5/08 11:25 PM, "Jaime Metcher" <jmetc...@gmail.com> wrote:
> Paul,
> The honour is all mine. The OO designer and the CF programmer in me are > still beating each other up over your conclusion, but that's a whole other > topic.
> I should add that the performance problems when using Transfer to marshall > large numbers of objects are not really Transfer's fault. You won't do > significantly better in any ColdFusion code. And no, CF8's object creation > speedups don't really change this conclusion, not even with the 1.5 JVM > (although they are pretty nice). Also no, IMHO this isn't really a good > enough reason to jump to Java - it *is* orders of magnitude faster at > creating objects, but that just brings it it the same ballpark as CF's query > handling.
> Jaime
> On Feb 6, 2008 10:29 AM, Paul Marcotte <pmarco...@gmail.com> wrote:
> Hey all,
> I started to echo what Jaime said and when I got to a couple of > paragraphs, I figured I should just blog about it, since I think this is an > important point (and was planning a post on it anyway). For the visually > oriented folks in the crowd, there's a diagram with some boxes and lines.
> In a nutshell, I've had a revelation in understanding how the view > requirements heavily influence the model design.
> Jaime, I hope you don't mind that I paraphrased you for the post. Your > take on subject is spot on.
> Paul
> On Feb 5, 2008 1:30 PM, Jaime Metcher <jmetc...@gmail.com> wrote:
> I'll just chip in with this thought as well - the architectural style of > the app makes a huge difference. If you keep your app close to the > database, use SQL (either directly or through Transfer) to do most of the > heavy lifting, use queries within your app for bulk data, and use transfer > objects for dealing with single objects or small collections of objects, CF > + Transfer scales very well. If you want to write a totally database > agnostic app and use Transfer to marshall arbitrary numbers of records > between the object model and the database, you'll run into performance > problems very quickly.
> So in terms of baseline performance tens of thousands of articles isn't > going to be a problem if you return single articles as transfer objects, but > use queries to list articles.
> To answer your question re size: I have an app with about 15,000 users > where the users/org units/managers are modelled as transfer objects. On the > level of a single request the pages which interact with transfer are not > perceptibly slower than pages that just use plain old cfquery. At the > server level, a single server copes with about 40 simultaneous users (plus > hosts lots of other apps) without breaking a sweat. Some informal load > testing indicates that this configuration will top out at about 10 > requests/second (which would equate to a couple of thousand users), but of > course that's very hardware dependent.
> Jaime
> On Feb 6, 2008 6:05 AM, Mark Mandel <mark.man...@gmail.com> wrote:
> Jeremy,
> This is one of those million dollar questions ;o)
> Scalability is very dependent on how you set up your system, what caching > options you use, how you model your data, the size of the data that you pull > in, how many objects you want to use at once.. the list goes on.
> So to answer your question - it depends.
> Maybe if you can outline some of the performance issues you are having (I > believe that you mentioned it in another thread), we can work out solutions > for them.
> Regards,
> Mark
> On Feb 6, 2008 4:32 AM, jez_f <jeremy.fre...@gmail.com> wrote:
> Hi there
> I am at the early stages (still) of looking into migrating a legacy > code base to one of the CF OO frameworks. (Probably modelglue)
> We would like to use transfer in many places. But we are not sure > about how well it scales. We have a large website with tens of > thousands of articles, all of which are stored in the database. I am > wondering what sort of size systems people have built with transfer?
Yup, but I donąt actually have a use case for displaying n-users along with multiple addresses for each on the same page, so I donąt have the n+1 query problem as I never use it that way. If I did, Iąd implement something different!
Best Wishes, Peter
On 2/6/08 12:20 AM, "Mark Mandel" <mark.man...@gmail.com> wrote:
> So for each user you iterator through, it's doing a query to get the > associated Address?
> so for 7 users, you are doing 8 queries? 1 user + 8 addresses? Is that right?
> Mark
> On Feb 6, 2008 4:08 PM, Peter Bell <pb...@systemsforge.com> wrote: >> Sure I do. I have a getAssociated() method on my IBO's that'll allow you to >> get an IBO containing 0..n associated objects based on the association >> relationships described in the business object XML.
>> Of course, if you have n-users, as n grows you have an n+1 query problem, so >> this approach would only be relevant for displaying a relatively small number >> of users each of which had many addresses. But that's a fundamental issue >> with a large number of objects each of which have a large number of to_many >> relationships to display in detail on a single screen. I have separate >> aggregate features which generate subqueries in the select if I just want to >> display a list of objects and aggregate information about their to_many >> children without having to run n+1 queries, and I have a data mapper that >> provides for left outer joins to has_one related objects if I want to display >> a long list of users with their (associated) bosses name using just one >> generated query.
>> Best Wishes, >> Peter
>> On 2/5/08 11:54 PM, "Mark Mandel" <mark.man...@gmail.com> wrote:
>>> But you don't get the benefits of object composition, do you Peter?
>>> Which is a pretty handy thing.
>>> Mark
>>> On Feb 6, 2008 3:48 PM, Peter Bell <pb...@systemsforge.com> wrote: >>>> I gotta mention the idea of an IBO here (or your own version of an object >>>> iterator). Wrap the cfquery into a single business object with iterating >>>> methods, a loadQuery() method and support for generic and custom getters >>>> and setters. That way you can get the encapsulation benefits of an object >>>> and the performance of a query. It also means that whether you have one >>>> OrderItem or a hundred of them, if you want to have a getItemTotal() custom >>>> method that multiplies the item unit price and item quantity, you can use >>>> that same code for a single business object or for a collection of them >>>> without worrying about performance.
>>>> Best Wishes, >>>> Peter
>>>> On 2/5/08 11:25 PM, "Jaime Metcher" <jmetc...@gmail.com> wrote:
>>>>> Paul,
>>>>> The honour is all mine. The OO designer and the CF programmer in me are >>>>> still beating each other up over your conclusion, but that's a whole other >>>>> topic.
>>>>> I should add that the performance problems when using Transfer to marshall >>>>> large numbers of objects are not really Transfer's fault. You won't do >>>>> significantly better in any ColdFusion code. And no, CF8's object >>>>> creation speedups don't really change this conclusion, not even with the >>>>> 1.5 JVM (although they are pretty nice). Also no, IMHO this isn't really >>>>> a good enough reason to jump to Java - it *is* orders of magnitude faster >>>>> at creating objects, but that just brings it it the same ballpark as CF's >>>>> query handling.
>>>>> Jaime
>>>>> On Feb 6, 2008 10:29 AM, Paul Marcotte <pmarco...@gmail.com> wrote: >>>>>> Hey all,
>>>>>> I started to echo what Jaime said and when I got to a couple of >>>>>> paragraphs, I figured I should just blog about it, since I think this is >>>>>> an important point (and was planning a post on it anyway). For the >>>>>> visually oriented folks in the crowd, there's a diagram with some boxes >>>>>> and lines.
>>>>>> In a nutshell, I've had a revelation in understanding how the view >>>>>> requirements heavily influence the model design.
>>>>>> Jaime, I hope you don't mind that I paraphrased you for the post. Your >>>>>> take on subject is spot on.
>>>>>> Paul
>>>>>> On Feb 5, 2008 1:30 PM, Jaime Metcher <jmetc...@gmail.com> wrote: >>>>>>> I'll just chip in with this thought as well - the architectural style of >>>>>>> the app makes a huge difference. If you keep your app close to the >>>>>>> database, use SQL (either directly or through Transfer) to do most of >>>>>>> the heavy lifting, use queries within your app for bulk data, and use >>>>>>> transfer objects for dealing with single objects or small collections of >>>>>>> objects, CF + Transfer scales very well. If you want to write a totally >>>>>>> database agnostic app and use Transfer to marshall arbitrary numbers of >>>>>>> records between the object model and the database, you'll run into >>>>>>> performance problems very quickly.
>>>>>>> So in terms of baseline performance tens of thousands of articles isn't >>>>>>> going to be a problem if you return single articles as transfer objects, >>>>>>> but use queries to list articles.
>>>>>>> To answer your question re size: I have an app with about 15,000 users >>>>>>> where the users/org units/managers are modelled as transfer objects. On >>>>>>> the level of a single request the pages which interact with transfer are >>>>>>> not perceptibly slower than pages that just use plain old cfquery. At >>>>>>> the server level, a single server copes with about 40 simultaneous users >>>>>>> (plus hosts lots of other apps) without breaking a sweat. Some informal >>>>>>> load testing indicates that this configuration will top out at about 10 >>>>>>> requests/second (which would equate to a couple of thousand users), but >>>>>>> of course that's very hardware dependent.
>>>>>>> Jaime
>>>>>>> On Feb 6, 2008 6:05 AM, Mark Mandel <mark.man...@gmail.com> wrote:
Jeremy,
This is one of those million dollar questions ;o)
Scalability is very dependent on how you set up your system, what caching options you use, how you model your data, the size of the data that you pull in, how many objects you want to use at once.. the list goes on.
So to answer your question - it depends.
Maybe if you can outline some of the performance issues you are having (I believe that you mentioned it in another thread), we can work out solutions for them.
Regards,
Mark
On Feb 6, 2008 4:32 AM, jez_f <jeremy.fre...@gmail.com> wrote:
Hi there
I am at the early stages (still) of looking into migrating a legacy code base to one of the CF OO frameworks. (Probably modelglue)
We would like to use transfer in many places. But we are not sure about how well it scales. We have a large website with tens of thousands of articles, all of which are stored in the database. I am wondering what sort of size systems people have built with transfer?
IBOs have always been a great way to iterate through a collection of
objects, however, I too see the problem with querying the database for
each iteration of an object. The only solution that I can think of is
some how performing a left outer join between the associations and
caching the query inside the IBO. Now when you iterate through the
collection the IBO can grab the corresponding record from the cached
query and inject the values into the object using the getter methods
on that object.
Only problem I see with this approach is if you try to cache a query
containing thousands of records.
From what I can tell, this approach is used in ActiveRecord and
DataMapper.
> Yup, but I donąt actually have a use case for displaying n-users along with
> multiple addresses for each on the same page, so I donąt have the n+1 query
> problem as I never use it that way. If I did, Iąd implement something
> different!
> Best Wishes,
> Peter
> On 2/6/08 12:20 AM, "Mark Mandel" <mark.man...@gmail.com> wrote:
> > So for each user you iterator through, it's doing a query to get the
> > associated Address?
> > so for 7 users, you are doing 8 queries? 1 user + 8 addresses? Is that right?
> > Mark
> > On Feb 6, 2008 4:08 PM, Peter Bell <pb...@systemsforge.com> wrote:
> >> Sure I do. I have a getAssociated() method on my IBO's that'll allow you to
> >> get an IBO containing 0..n associated objects based on the association
> >> relationships described in the business object XML.
> >> Of course, if you have n-users, as n grows you have an n+1 query problem, so
> >> this approach would only be relevant for displaying a relatively small number
> >> of users each of which had many addresses. But that's a fundamental issue
> >> with a large number of objects each of which have a large number of to_many
> >> relationships to display in detail on a single screen. I have separate
> >> aggregate features which generate subqueries in the select if I just want to
> >> display a list of objects and aggregate information about their to_many
> >> children without having to run n+1 queries, and I have a data mapper that
> >> provides for left outer joins to has_one related objects if I want to display
> >> a long list of users with their (associated) bosses name using just one
> >> generated query.
> >> Best Wishes,
> >> Peter
> >> On 2/5/08 11:54 PM, "Mark Mandel" <mark.man...@gmail.com> wrote:
> >>> But you don't get the benefits of object composition, do you Peter?
> >>> Which is a pretty handy thing.
> >>> Mark
> >>> On Feb 6, 2008 3:48 PM, Peter Bell <pb...@systemsforge.com> wrote:
> >>>> I gotta mention the idea of an IBO here (or your own version of an object
> >>>> iterator). Wrap the cfquery into a single business object with iterating
> >>>> methods, a loadQuery() method and support for generic and custom getters
> >>>> and setters. That way you can get the encapsulation benefits of an object
> >>>> and the performance of a query. It also means that whether you have one
> >>>> OrderItem or a hundred of them, if you want to have a getItemTotal() custom
> >>>> method that multiplies the item unit price and item quantity, you can use
> >>>> that same code for a single business object or for a collection of them
> >>>> without worrying about performance.
> >>>> Best Wishes,
> >>>> Peter
> >>>> On 2/5/08 11:25 PM, "Jaime Metcher" <jmetc...@gmail.com> wrote:
> >>>>> Paul,
> >>>>> The honour is all mine. The OO designer and the CF programmer in me are
> >>>>> still beating each other up over your conclusion, but that's a whole other
> >>>>> topic.
> >>>>> I should add that the performance problems when using Transfer to marshall
> >>>>> large numbers of objects are not really Transfer's fault. You won't do
> >>>>> significantly better in any ColdFusion code. And no, CF8's object
> >>>>> creation speedups don't really change this conclusion, not even with the
> >>>>> 1.5 JVM (although they are pretty nice). Also no, IMHO this isn't really
> >>>>> a good enough reason to jump to Java - it *is* orders of magnitude faster
> >>>>> at creating objects, but that just brings it it the same ballpark as CF's
> >>>>> query handling.
> >>>>> Jaime
> >>>>> On Feb 6, 2008 10:29 AM, Paul Marcotte <pmarco...@gmail.com> wrote:
> >>>>>> Hey all,
> >>>>>> I started to echo what Jaime said and when I got to a couple of
> >>>>>> paragraphs, I figured I should just blog about it, since I think this is
> >>>>>> an important point (and was planning a post on it anyway). For the
> >>>>>> visually oriented folks in the crowd, there's a diagram with some boxes
> >>>>>> and lines.
> >>>>>> In a nutshell, I've had a revelation in understanding how the view
> >>>>>> requirements heavily influence the model design.
> >>>>>> Jaime, I hope you don't mind that I paraphrased you for the post. Your
> >>>>>> take on subject is spot on.
> >>>>>> Paul
> >>>>>> On Feb 5, 2008 1:30 PM, Jaime Metcher <jmetc...@gmail.com> wrote:
> >>>>>>> I'll just chip in with this thought as well - the architectural style of
> >>>>>>> the app makes a huge difference. If you keep your app close to the
> >>>>>>> database, use SQL (either directly or through Transfer) to do most of
> >>>>>>> the heavy lifting, use queries within your app for bulk data, and use
> >>>>>>> transfer objects for dealing with single objects or small collections of
> >>>>>>> objects, CF + Transfer scales very well. If you want to write a totally
> >>>>>>> database agnostic app and use Transfer to marshall arbitrary numbers of
> >>>>>>> records between the object model and the database, you'll run into
> >>>>>>> performance problems very quickly.
> >>>>>>> So in terms of baseline performance tens of thousands of articles isn't
> >>>>>>> going to be a problem if you return single articles as transfer objects,
> >>>>>>> but use queries to list articles.
> >>>>>>> To answer your question re size: I have an app with about 15,000 users
> >>>>>>> where the users/org units/managers are modelled as transfer objects. On
> >>>>>>> the level of a single request the pages which interact with transfer are
> >>>>>>> not perceptibly slower than pages that just use plain old cfquery. At
> >>>>>>> the server level, a single server copes with about 40 simultaneous users
> >>>>>>> (plus hosts lots of other apps) without breaking a sweat. Some informal
> >>>>>>> load testing indicates that this configuration will top out at about 10
> >>>>>>> requests/second (which would equate to a couple of thousand users), but
> >>>>>>> of course that's very hardware dependent.
> >>>>>>> Jaime
> >>>>>>> On Feb 6, 2008 6:05 AM, Mark Mandel <mark.man...@gmail.com> wrote:
> Jeremy,
> This is one of those million dollar questions ;o)
> Scalability is very dependent on how you set up your system, what caching
> options you use, how you model your data, the size of the data that you pull
> in, how many objects you want to use at once.. the list goes on.
> So to answer your question - it depends.
> Maybe if you can outline some of the performance issues you are having (I
> believe that you mentioned it in another thread), we can work out solutions
> for them.
> Regards,
> Mark
> On Feb 6, 2008 4:32 AM, jez_f <jeremy.fre...@gmail.com> wrote:
> Hi there
> I am at the early stages (still) of looking into migrating a legacy
> code base to one of the CF OO frameworks. (Probably modelglue)
> We would like to use transfer in many places. But we are not sure
> about how well it scales. We have a large website with tens of
> thousands of articles, all of which are stored in the database. I am
> wondering what sort of size systems people have built with transfer?
The only use cases I regularly run into with a potential for n+1 queries are lists of objects with associated objects.
My solution is that my data mapper allows for auto-generating left outer joins for has-one relationships and subqueries in the select for has-many associations.
So if I want a list of users first names, last names, bosses title and total of auctions they have won, I just getByFilter with a PropertyNameList of FirstName,LastName,Boss.Title,AuctionsWon.Sum.Price and it'll create a single SQL query - with support for paging at the db or at the client level.
Best Wishes, Peter
On 2/12/08 8:30 AM, "tpetru...@gmail.com" <tpetru...@gmail.com> wrote:
> IBOs have always been a great way to iterate through a collection of > objects, however, I too see the problem with querying the database for > each iteration of an object. The only solution that I can think of is > some how performing a left outer join between the associations and > caching the query inside the IBO. Now when you iterate through the > collection the IBO can grab the corresponding record from the cached > query and inject the values into the object using the getter methods > on that object.
> Only problem I see with this approach is if you try to cache a query > containing thousands of records.
> From what I can tell, this approach is used in ActiveRecord and > DataMapper.
> On Feb 6, 4:45 am, Peter Bell <systemsfo...@gmail.com> wrote: >> Yup, but I donąt actually have a use case for displaying n-users along with >> multiple addresses for each on the same page, so I donąt have the n+1 query >> problem as I never use it that way. If I did, Iąd implement something >> different!
>> Best Wishes, >> Peter
>> On 2/6/08 12:20 AM, "Mark Mandel" <mark.man...@gmail.com> wrote:
>>> So for each user you iterator through, it's doing a query to get the >>> associated Address?
>>> so for 7 users, you are doing 8 queries? 1 user + 8 addresses? Is that >>> right?
>>> Mark
>>> On Feb 6, 2008 4:08 PM, Peter Bell <pb...@systemsforge.com> wrote: >>>> Sure I do. I have a getAssociated() method on my IBO's that'll allow you to >>>> get an IBO containing 0..n associated objects based on the association >>>> relationships described in the business object XML.
>>>> Of course, if you have n-users, as n grows you have an n+1 query problem, >>>> so >>>> this approach would only be relevant for displaying a relatively small >>>> number >>>> of users each of which had many addresses. But that's a fundamental issue >>>> with a large number of objects each of which have a large number of to_many >>>> relationships to display in detail on a single screen. I have separate >>>> aggregate features which generate subqueries in the select if I just want >>>> to >>>> display a list of objects and aggregate information about their to_many >>>> children without having to run n+1 queries, and I have a data mapper that >>>> provides for left outer joins to has_one related objects if I want to >>>> display >>>> a long list of users with their (associated) bosses name using just one >>>> generated query.
>>>> Best Wishes, >>>> Peter
>>>> On 2/5/08 11:54 PM, "Mark Mandel" <mark.man...@gmail.com> wrote:
>>>>> But you don't get the benefits of object composition, do you Peter?
>>>>> Which is a pretty handy thing.
>>>>> Mark
>>>>> On Feb 6, 2008 3:48 PM, Peter Bell <pb...@systemsforge.com> wrote: >>>>>> I gotta mention the idea of an IBO here (or your own version of an object >>>>>> iterator). Wrap the cfquery into a single business object with iterating >>>>>> methods, a loadQuery() method and support for generic and custom getters >>>>>> and setters. That way you can get the encapsulation benefits of an object >>>>>> and the performance of a query. It also means that whether you have one >>>>>> OrderItem or a hundred of them, if you want to have a getItemTotal() >>>>>> custom >>>>>> method that multiplies the item unit price and item quantity, you can use >>>>>> that same code for a single business object or for a collection of them >>>>>> without worrying about performance.
>>>>>> Best Wishes, >>>>>> Peter
>>>>>> On 2/5/08 11:25 PM, "Jaime Metcher" <jmetc...@gmail.com> wrote:
>>>>>>> Paul,
>>>>>>> The honour is all mine. The OO designer and the CF programmer in me are >>>>>>> still beating each other up over your conclusion, but that's a whole >>>>>>> other >>>>>>> topic.
>>>>>>> I should add that the performance problems when using Transfer to >>>>>>> marshall >>>>>>> large numbers of objects are not really Transfer's fault. You won't do >>>>>>> significantly better in any ColdFusion code. And no, CF8's object >>>>>>> creation speedups don't really change this conclusion, not even with the >>>>>>> 1.5 JVM (although they are pretty nice). Also no, IMHO this isn't >>>>>>> really >>>>>>> a good enough reason to jump to Java - it *is* orders of magnitude >>>>>>> faster >>>>>>> at creating objects, but that just brings it it the same ballpark as >>>>>>> CF's >>>>>>> query handling.
>>>>>>> Jaime
>>>>>>> On Feb 6, 2008 10:29 AM, Paul Marcotte <pmarco...@gmail.com> wrote: >>>>>>>> Hey all,
>>>>>>>> I started to echo what Jaime said and when I got to a couple of >>>>>>>> paragraphs, I figured I should just blog about it, since I think this >>>>>>>> is >>>>>>>> an important point (and was planning a post on it anyway). For the >>>>>>>> visually oriented folks in the crowd, there's a diagram with some boxes >>>>>>>> and lines.
>>>>>>>> In a nutshell, I've had a revelation in understanding how the view >>>>>>>> requirements heavily influence the model design.
>>>>>>>> Jaime, I hope you don't mind that I paraphrased you for the post. Your >>>>>>>> take on subject is spot on.
>>>>>>>> Paul
>>>>>>>> On Feb 5, 2008 1:30 PM, Jaime Metcher <jmetc...@gmail.com> wrote: >>>>>>>>> I'll just chip in with this thought as well - the architectural style >>>>>>>>> of >>>>>>>>> the app makes a huge difference. If you keep your app close to the >>>>>>>>> database, use SQL (either directly or through Transfer) to do most of >>>>>>>>> the heavy lifting, use queries within your app for bulk data, and use >>>>>>>>> transfer objects for dealing with single objects or small collections >>>>>>>>> of >>>>>>>>> objects, CF + Transfer scales very well. If you want to write a >>>>>>>>> totally >>>>>>>>> database agnostic app and use Transfer to marshall arbitrary numbers >>>>>>>>> of >>>>>>>>> records between the object model and the database, you'll run into >>>>>>>>> performance problems very quickly.
>>>>>>>>> So in terms of baseline performance tens of thousands of articles >>>>>>>>> isn't >>>>>>>>> going to be a problem if you return single articles as transfer >>>>>>>>> objects, >>>>>>>>> but use queries to list articles.
>>>>>>>>> To answer your question re size: I have an app with about 15,000 users >>>>>>>>> where the users/org units/managers are modelled as transfer objects. >>>>>>>>> On >>>>>>>>> the level of a single request the pages which interact with transfer >>>>>>>>> are >>>>>>>>> not perceptibly slower than pages that just use plain old cfquery. At >>>>>>>>> the server level, a single server copes with about 40 simultaneous >>>>>>>>> users >>>>>>>>> (plus hosts lots of other apps) without breaking a sweat. Some >>>>>>>>> informal >>>>>>>>> load testing indicates that this configuration will top out at about >>>>>>>>> 10 >>>>>>>>> requests/second (which would equate to a couple of thousand users), >>>>>>>>> but >>>>>>>>> of course that's very hardware dependent.
>>>>>>>>> Jaime
>>>>>>>>> On Feb 6, 2008 6:05 AM, Mark Mandel <mark.man...@gmail.com> wrote:
>> Jeremy,
>> This is one of those million dollar questions ;o)
>> Scalability is very dependent on how you set up your system, what caching >> options you use, how you model your data, the size of the data that you pull >> in, how many objects you want to use at once.. the list goes on.
>> So to answer your question - it depends.
>> Maybe if you can outline some of the performance issues you are having (I >> believe that you mentioned it in another thread), we can work out solutions >> for them.
>> Regards,
>> Mark
>> On Feb 6, 2008 4:32 AM, jez_f <jeremy.fre...@gmail.com> wrote:
>> Hi there
>> I am at the early stages (still) of looking into migrating a legacy >> code base to one of the CF OO frameworks. (Probably modelglue)
>> We would like to use transfer in many places. But we are not sure >> about how well it scales. We have a large website with tens of >> thousands of articles, all of which are stored in the database. I am >> wondering what sort of size systems people have built with transfer?
|
