followers/ids, cursoring vs. paging, and Charlie Sheen
43 views
Skip to first unread message
Craig Walls
Jul 22, 2011, 10:59:08 AM7/22/11
to twitter-deve...@googlegroups.com
I fully understand how followers/ids (or friends/ids) can be used with users/lookup to fetch profiles of users who follow or are friends of a user. Makes perfect sense, except for a bit of dissonance between the two resources.
If I use followers/ids without setting the cursor parameter, I might get all of a user's followers, but if the user has a lot of followers (Charlie Sheen as an extreme example), I get an error. Okay...the documentation warned me that might happen. So, I set cursor to -1 to get the first 5000 of Charlie's followers. But then users/lookup only lets me fetch 100 profiles at a time. So I have to chunk the 5000 IDs I get from followers/ids into groups of 100 and make 50 calls to users/lookup to get all 5000 profiles. Aside from the rate limit concerns, this takes quite some time to complete.
Okay, so maybe I shouldn't fetch 5000 profiles at a time. Maybe I should only fetch 100 at a time and page through them as/if needed. Well, that's fine and even desirable...but now I have two page-oriented things to keep track of: The cursor for the given set of 5000 IDs and another index into that 5000 for the 100 IDs I want to fetch. It's not a big deal, I suppose, but it wouldn't be necessary if I could get followers/ids to give me only 100 (or 50...or 25...or some number I specify) IDs at a time.
Honestly, I don't like cursoring...there, I've said it and I do feel better. It seems to leak out an internal detail through the API. But also, I'm left with less control over which set and how many entries I get. I prefer the page and count/per_page approach taken in other API resources (which, btw, why the inconsistency with count on some resources and per_page on others?).
What is recommended here? Keep track of the cursor and the index into that cursor for the 100 ID "page" I need? Do-able, I suppose, but it'd be so much easier if I could even get a cursor'd chunk of 100 IDs...or better yet paging.
If I use followers/ids without setting the cursor parameter, I might get all of a user's followers, but if the user has a lot of followers (Charlie Sheen as an extreme example), I get an error. Okay...the documentation warned me that might happen. So, I set cursor to -1 to get the first 5000 of Charlie's followers. But then users/lookup only lets me fetch 100 profiles at a time. So I have to chunk the 5000 IDs I get from followers/ids into groups of 100 and make 50 calls to users/lookup to get all 5000 profiles. Aside from the rate limit concerns, this takes quite some time to complete.
Okay, so maybe I shouldn't fetch 5000 profiles at a time. Maybe I should only fetch 100 at a time and page through them as/if needed. Well, that's fine and even desirable...but now I have two page-oriented things to keep track of: The cursor for the given set of 5000 IDs and another index into that 5000 for the 100 IDs I want to fetch. It's not a big deal, I suppose, but it wouldn't be necessary if I could get followers/ids to give me only 100 (or 50...or 25...or some number I specify) IDs at a time.
Honestly, I don't like cursoring...there, I've said it and I do feel better. It seems to leak out an internal detail through the API. But also, I'm left with less control over which set and how many entries I get. I prefer the page and count/per_page approach taken in other API resources (which, btw, why the inconsistency with count on some resources and per_page on others?).
What is recommended here? Keep track of the cursor and the index into that cursor for the 100 ID "page" I need? Do-able, I suppose, but it'd be so much easier if I could even get a cursor'd chunk of 100 IDs...or better yet paging.
Reply all
Reply to author
Forward
0 new messages