Although million follower accounts are rare, how to I design for a
million follower user logged into my application which users Social
If Barack Obama were to log into my application, it would take 566 API
calls to fetch his 2,828,782 followers, but I wouldn't have any left
after the 150 API calls to fetch his 747,127 friends.
Obviously, I'd like to work my way through the list a little bit each
hour. I'd like to store the cursor after 30 API calls, resume my
iteration over an hour later.
When do cursors expire? I assume they will still be valid an hour later,
but I've seen discussion on this group that says that they are opaque
and that they may change at some point. I suppose that when that time
goes, if my application is crawling a celebrity, it will not be able to
resume crawling with the cursor it stored an hour before.