Benefit to retrieving smaller sets of records per page?

[Daniel Olsson]

Good afternoon all,

Just wondering if there's a performance benefit on the service side when retrieving smaller sets of records rather than large/all at once.

What I mean specifically is I imagine I could retrieve all the data I want by specifying the records per page value to 10,000 (for retrieving unsubscribes as an example). However, I thought that this could be a little costly on the service itself and perhaps multiple requests of the default records per page would be better.

Then I start to wonder whether multiple requests is just as costly...

Anyone have any input on this?

Thanks again!

[bhayes]

I regularly use the 10,000 per page value to try and get the information I need as quickly as possible. I believe this is better all the way around. If you do multiple requests for a lesser amount then, you add the overhead of network requests across the internet as well multiple database queries for the StreamSend servers. (They do at least one query each request.)

One catch is that if you setup your script to get 1,000 records at a time and just repeat until there are no more records, you will always get all of the records. I've made the mistake of just asking for 10,000 and not setting up a loop. So as soon as there are more than 10,000 records my code needed to be updated. My general practice now is to do the 10,000 in a loop so I know it gets everything in a minimum of 2 requests. :)

[Daniel Olsson]

Thank you for the reply - I like what you're saying. I don't think we'll ever have more than 10,000 (maybe not any more than 1,000 to be honest) records per request.

I already have the query setup in a loop that will stop as soon as there are no more records on the final page it's reading.

I'll follow your advice, keep the loop in there and set the max per page to 10,000 - if there's the odd chance there's more than 10K per page, at least the loop will take care of it.

Thanks again!