next_cursor private?

[Ryan Williams]

I've recently been getting an error accessing the next_cursor

cursor = -1
ids = client.follower_ids({:cursor => cursor, :count => 5000})
puts page_ids.next_cursor

followers.rb:29:in `<main>':private method `next_cursor' called for #<Twitter::Cursor:0x007fde4b07e640> (NoMethodError)

I show as my Gem version: twitter (5.6.0)

In the code, https://github.com/sferik/twitter/blob/e14cc3391ebe8229184e9e83806c870df3baa24c/lib/twitter/cursor.rb, it was changed, moving the private line up on December 24th.

Is there a new suggested way to work with the cursor?

[sferik]

Twitter::Cursor#next_cursor was never documented as part of the public API and should never have been a public method. Twitter::Cursor is Enumerable and should be traversed with #each (or the other Enumerable methods). You should never be accessing the cursor position manually. It is relevant only to the implementation.

Here is an example of how this issue was resolved in someone else’s codebase. If you provide a more concrete example, I can help you resolve this issue in your code.

[Chris Finlayson]

I agree. I am getting the same error message when trying to implement this code which was posted in another thread: https://gist.github.com/ronhornbaker/7817176

The example you have here (https://github.com/sferik/twitter/issues/517), works for retrieving all followers. However, if I try to retrieve all Friends using the .to_a call, I am blocked by Twitter for exceeding the rate limit. This is solved by limiting the request to 200 friends, but I have not been able to successful cursor through multiple pages to retrieve all Friends. I am using v5 of the Twitter Gem. Thanks in advance for help.

[Chris Finlayson]

Sorry for the second post. I did some further digging. It seems that if you call client.friend_ids.to_a, the call works perfectly. I get a full list of friend IDs well over the 200 cap for client.friend.to_a without exceeding the rate limit. 

Also, surprisingly, if I try to access the User object of each friend I pull via client.friend (i.e. screen_name, location etc.), the limit drops to a max of 100 friends per request before twitter raises a rate limit exception.

[sferik]

On Monday, March 10, 2014 3:34:20 AM UTC+1, Chris Finlayson wrote:

Sorry for the second post. I did some further digging. It seems that if you call client.friend_ids.to_a, the call works perfectly. I get a full list of friend IDs well over the 200 cap for client.friend.to_a without exceeding the rate limit. 

Also, surprisingly, if I try to access the User object of each friend I pull via client.friend (i.e. screen_name, location etc.), the limit drops to a max of 100 friends per request before twitter raises a rate limit exception.

The behavior of Twitter::Cursor#to_a and Twitter::Cursor#each, with respect to rate limiting, should be identical, as these two methods share a single code path. Can you please provide a concrete example that demonstrates the behavior you’re seeing? I suspect there are some confounding factors.

I’ve forked the Gist you referenced and reduced the method down to 4 significant lines of code. I was able to remove all the rate limiting logic by using a combination of the Twitter::REST::Client#friends_ids and Twitter::REST::Client#users methods instead of Twitter::REST::Client#friends. This code would only need to be concerned about rate limiting for users who follow more than 18,000 people. For me, it ran in under 15 seconds.

[Sameer Siruguri]

If I understand the code correctly, then the slice size in the snippet below is not relevant as far as the rate limit is concerned, yes?

Regardless of the slice size, really what you need to know is the math on the Twitter Rate Limit chart...https://dev.twitter.com/rest/public/rate-limits - and the limits on the individual calls (e.g., /users/lookup will accept up to 100 IDs.)

The gem as far as I can make out only reflects these numbers.

Just thought I'd put this in here, in case someone stumbles on this post via the Internets.