User_Timeline - include_rts not working

410 views
Skip to first unread message

henasraf

unread,
Aug 19, 2010, 4:24:02 PM8/19/10
to Twitter Development Talk
Hey all, I'm using user_timeline to fetch the timeline of a user in my
app. Problem is, it doesn't include native Retweets, even though I've
set include_rts to true. The docs clearly say it should work, but it
doesn't. What could be the problem? Thanks in advance :)

Matt Harris

unread,
Aug 19, 2010, 4:34:33 PM8/19/10
to twitter-deve...@googlegroups.com
Hey,

Retweets should be included if they exist in the users timeline when you request them. For example if you request 20 Tweets of the timeline and one of those last 20 was a retweet - it will be included. If one of the last 20 Tweets wasn't a retweet it won't be included, even if include_rts is true.

Does this explain what is happening in your app?
If not could you give an example of a username which shows this problem.

Best,
Matt
--


Matt Harris
Developer Advocate, Twitter
http://twitter.com/themattharris

henasraf

unread,
Aug 21, 2010, 10:19:05 AM8/21/10
to Twitter Development Talk
Matt,
Any user I show in my app would not show retweets, even if they surely
exist. You can see for yourself at http://wosaic.net/twitguin (still
early stages, don't mind it being a bit lame), see any user such as
myself at http://wosaic.net/twitguin/user/henasraf ; you may go
through pages and see that no retweets ever show; they should be color
coded in orange. You can compare to http://twitter.com/henasraf to see
retweets that should show.

Thanks,
Hen

On Aug 19, 11:34 pm, Matt Harris <thematthar...@twitter.com> wrote:
> Hey,
>
> Retweets should be included if they exist in the users timeline when you
> request them. For example if you request 20 Tweets of the timeline and one
> of those last 20 was a retweet - it will be included. If one of the last 20
> Tweets wasn't a retweet it won't be included, even if include_rts is true.
>
> Does this explain what is happening in your app?
> If not could you give an example of a username which shows this problem.
>
> Best,
> Matt
>

Matt Harris

unread,
Aug 25, 2010, 2:30:48 PM8/25/10
to twitter-deve...@googlegroups.com
Hey Hen,

Are you still finding retweets are not showing up for you?

Matt

Taylor Singletary

unread,
Aug 30, 2010, 1:50:05 PM8/30/10
to twitter-deve...@googlegroups.com
Hi Timmerk,

Make sure you're using api.twitter.com as the host for all of your API requests. The proper API is not at twitter.com -- it's at api.twitter.com. Include_rts will not work against twitter.com (and one day you'll find all of your requests rejected.)

Also, please note that proper API routes include a version component. The call you're making should be:

Taylor

On Mon, Aug 30, 2010 at 10:43 AM, timmerk <tim...@gmail.com> wrote:
I'm having the same issue - you can see an example here:

http://twitter.com/statuses/user_timeline.json?screen_name=NuWaveGOV&count=4&include_rts=true

As you can see, there are no retweets included in the JSON.

Thanks!


On Aug 25, 2:30 pm, Matt Harris <thematthar...@twitter.com> wrote:
> Hey Hen,
>
> Are you still finding retweets are not showing up for you?
>
> Matt
>
>
>
>
>
> On Sat, Aug 21, 2010 at 7:19 AM, henasraf <doku...@gmail.com> wrote:
> > Matt,
> > Any user I show in my app would not show retweets, even if they surely
> > exist. You can see for yourself athttp://wosaic.net/twitguin(still
> > early stages, don't mind it being a bit lame), see any user such as
> > myself athttp://wosaic.net/twitguin/user/henasraf; you may go

> > through pages and see that no retweets ever show; they should be color
> > coded in orange. You can compare tohttp://twitter.com/henasrafto see

> > retweets that should show.
>
> > Thanks,
> > Hen
>
> > On Aug 19, 11:34 pm, Matt Harris <thematthar...@twitter.com> wrote:
> >> Hey,
>
> >> Retweets should be included if they exist in the users timeline when you
> >> request them. For example if you request 20 Tweets of the timeline and one
> >> of those last 20 was a retweet - it will be included. If one of the last 20
> >> Tweets wasn't a retweet it won't be included, even if include_rts is true.
>
> >> Does this explain what is happening in your app?
> >> If not could you give an example of a username which shows this problem.
>
> >> Best,
> >> Matt
>
> >> On Thu, Aug 19, 2010 at 1:24 PM, henasraf <doku...@gmail.com> wrote:
> >> > Hey all, I'm using user_timeline to fetch the timeline of a user in my
> >> > app. Problem is, it doesn't include native Retweets, even though I've
> >> > set include_rts to true. The docs clearly say it should work, but it
> >> > doesn't. What could be the problem? Thanks in advance :)
>
> >> --
>
> >> Matt Harris
> >> Developer Advocate, Twitterhttp://twitter.com/themattharris
>
> --
>
> Matt Harris
> Developer Advocate, Twitterhttp://twitter.com/themattharris- Hide quoted text -
>
> - Show quoted text -

--
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group: http://groups.google.com/group/twitter-development-talk?hl=en

timmerk

unread,
Aug 30, 2010, 1:43:49 PM8/30/10
to Twitter Development Talk
I'm having the same issue - you can see an example here:

http://twitter.com/statuses/user_timeline.json?screen_name=NuWaveGOV&count=4&include_rts=true

As you can see, there are no retweets included in the JSON.

Thanks!

On Aug 25, 2:30 pm, Matt Harris <thematthar...@twitter.com> wrote:
> Hey Hen,
>
> Are you still finding retweets are not showing up for you?
>
> Matt
>
>
>
>
>
> On Sat, Aug 21, 2010 at 7:19 AM, henasraf <doku...@gmail.com> wrote:
> > Matt,
> > Any user I show in my app would not show retweets, even if they surely
> > exist. You can see for yourself athttp://wosaic.net/twitguin(still
> > early stages, don't mind it being a bit lame), see any user such as
> > myself athttp://wosaic.net/twitguin/user/henasraf; you may go
> > through pages and see that no retweets ever show; they should be color
> > coded in orange. You can compare tohttp://twitter.com/henasrafto see
> > retweets that should show.
>
> > Thanks,
> > Hen
>
> > On Aug 19, 11:34 pm, Matt Harris <thematthar...@twitter.com> wrote:
> >> Hey,
>
> >> Retweets should be included if they exist in the users timeline when you
> >> request them. For example if you request 20 Tweets of the timeline and one
> >> of those last 20 was a retweet - it will be included. If one of the last 20
> >> Tweets wasn't a retweet it won't be included, even if include_rts is true.
>
> >> Does this explain what is happening in your app?
> >> If not could you give an example of a username which shows this problem.
>
> >> Best,
> >> Matt
>
> >> On Thu, Aug 19, 2010 at 1:24 PM, henasraf <doku...@gmail.com> wrote:
> >> > Hey all, I'm using user_timeline to fetch the timeline of a user in my
> >> > app. Problem is, it doesn't include native Retweets, even though I've
> >> > set include_rts to true. The docs clearly say it should work, but it
> >> > doesn't. What could be the problem? Thanks in advance :)
>
> >> --
>
> >> Matt Harris
> >> Developer Advocate, Twitterhttp://twitter.com/themattharris
>
> --
>
> Matt Harris

timmerk

unread,
Aug 30, 2010, 2:07:06 PM8/30/10
to Twitter Development Talk
Thanks Taylor! That worked.

On Aug 30, 1:50 pm, Taylor Singletary <taylorsinglet...@twitter.com>
wrote:
> Hi Timmerk,
>
> Make sure you're using api.twitter.com as the host for all of your API
> requests. The proper API is not at twitter.com -- it's at api.twitter.com.
> Include_rts will not work against twitter.com (and one day you'll find all
> of your requests rejected.)
>
> Also, please note that proper API routes include a version component. The
> call you're making should be:http://api.twitter.com/1/statuses/user_timeline.json?screen_name=NuWa...
>
> Taylor
>
>
>
> On Mon, Aug 30, 2010 at 10:43 AM, timmerk <timm...@gmail.com> wrote:
> > I'm having the same issue - you can see an example here:
>
> >http://twitter.com/statuses/user_timeline.json?screen_name=NuWaveGOV&...
> > > Developer Advocate, Twitterhttp://twitter.com/themattharris-Hide quoted
> > text -
>
> > > - Show quoted text -
>
> > --
> > Twitter developer documentation and resources:http://dev.twitter.com/doc
> > API updates via Twitter:http://twitter.com/twitterapi
> > Issues/Enhancements Tracker:
> >http://code.google.com/p/twitter-api/issues/list
> > Change your membership to this group:
> >http://groups.google.com/group/twitter-development-talk?hl=en- Hide quoted text -

timrnicholson

unread,
Sep 9, 2010, 10:30:35 AM9/9/10
to Twitter Development Talk
I'm properly calling api.twitter.com/1/statuses/user_timeline with
include_rts as true and retweets are not being included. The only
stream that I can get to return retweets (because it doesn't even
require the include_rts parameter) is the home_timeline.

For example, the following doesn't work:
http://api.twitter.com/1/statuses/user_timeline/RWW.xml?count=20&page=1&include_rts=true

The worst part about this is not just that the include_rts parameter
doesn't work for including retweets, the returned timeline doesn't
even have 20 records in it when there are retweets. In other words,
the API is not only ignoring the include_rts parameter but its not
even returning the right number of requested records when it thinks
include_rts is false.

This totally screws up my pagination methods because I'm getting less
than the expected number of records back, so my app thinks there's no
more timeline to be retrieved. I certainly don't want to just always
include a "next page" link when there really aren't any more records
left.

Also, the count parameter is being ignored in all of this as well, but
I've latched on to another thread about that.

On Aug 30, 12:50 pm, Taylor Singletary <taylorsinglet...@twitter.com>
wrote:
> Hi Timmerk,
>
> Make sure you're using api.twitter.com as the host for all of your API
> requests. The proper API is not at twitter.com -- it's at api.twitter.com.
> Include_rts will not work against twitter.com (and one day you'll find all
> of your requests rejected.)
>
> Also, please note that proper API routes include a version component. The
> call you're making should be:http://api.twitter.com/1/statuses/user_timeline.json?screen_name=NuWa...
>
> Taylor
>
>
>
> On Mon, Aug 30, 2010 at 10:43 AM, timmerk <timm...@gmail.com> wrote:
> > I'm having the same issue - you can see an example here:
>
> >http://twitter.com/statuses/user_timeline.json?screen_name=NuWaveGOV&...
> > > Developer Advocate, Twitterhttp://twitter.com/themattharris-Hide quoted
> > text -
>
> > > - Show quoted text -
>
> > --
> > Twitter developer documentation and resources:http://dev.twitter.com/doc
> > API updates via Twitter:http://twitter.com/twitterapi
> > Issues/Enhancements Tracker:
> >http://code.google.com/p/twitter-api/issues/list
> > Change your membership to this group:
> >http://groups.google.com/group/twitter-development-talk?hl=en- Hide quoted text -

Matt Harris

unread,
Sep 9, 2010, 11:06:33 AM9/9/10
to twitter-deve...@googlegroups.com
Hey Tim,

When I run:
curl "http://api.twitter.com/1/statuses/user_timeline/RWW.xml?count=20&page=1&include_rts=true"

I get the expected result of 20 statuses one retweet (from
@ConanOBrien). When you make the request are you running it like I am
or are you using a library?

To answer your query about the count parameter. The behavior is this:
we will try and return up to 'count' statuses in one request, where
count is equal to all statuses in the timeline before any filtering of
retweets. So, if you request 20 statuses without retweets, and 1 of
the statuses is a retweet, you will get 19 statuses back.

Hope that helps explain what is going on,
Matt

Mike Francisco

unread,
Sep 9, 2010, 5:41:40 PM9/9/10
to Twitter Development Talk, dangananl...@gmail.com

timrnicholson

unread,
Oct 4, 2010, 12:48:29 PM10/4/10
to Twitter Development Talk
Well, I thought i had this problem solved. I am using a library called
twitteroauth and it was stripping off the "&include_rts=true". I
finally figured out that include_rts needed to be placed into an
arguments array instead of being in the request URL.

So once I got the include_rts parm passed to Twitter everything was
working fine. Or at least it was in most cases. HOWEVER, I am seeing
now that certain requests are NOT returning the full count parameter
worth of records. There must be SOME filtering that the Twitter API is
doing AFTER it retrieves the count number worth of records. I have
this problem regardless of whether I use the home_timeline which is
supposed to always include everything or whether I use the
friends_timeline with the include_rts parameter.

Note that I am trying to request count = 25 records, not the default
count = 20 records. I can see that retweets are being included and I'm
still getting ust 24 records back in certain cases.

On Sep 9, 10:06 am, Matt Harris <thematthar...@twitter.com> wrote:
> Hey Tim,
>
> When I run:
>     curl "http://api.twitter.com/1/statuses/user_timeline/RWW.xml?count=20&page..."
>
> I get the expected result of 20 statuses one retweet (from
> @ConanOBrien). When you make the request are you running it like I am
> or are you using a library?
>
> To answer your query about thecountparameter. The behavior is this:
> we will try and return up to 'count' statuses in one request, wherecountis equal to all statuses in the timeline before any filtering of
> retweets. So, if you request 20 statuses without retweets, and 1 of
> the statuses is a retweet, you will get 19 statuses back.
>
> Hope that helps explain what is going on,
> Matt
>
>
>
>
>
> On Thu, Sep 9, 2010 at 7:30 AM, timrnicholson <timrnichol...@gmail.com> wrote:
> > I'm properly calling api.twitter.com/1/statuses/user_timeline with
> > include_rts as true and retweets arenotbeing included. The only
> > stream that I can get to return retweets (because it doesn't even
> > require the include_rtsparameter) is the home_timeline.
>
> > For example, the following doesn't work:
> >http://api.twitter.com/1/statuses/user_timeline/RWW.xml?count=20&page...
>
> > The worst part about this isnotjust that the include_rtsparameter
> > doesn't work for including retweets, the returned timeline doesn't
> > even have 20 records in it when there are retweets. In other words,
> > the API isnotonly ignoring the include_rtsparameterbut itsnot
> > even returning the right number of requested records when it thinks
> > include_rts is false.
>
> > This totally screws up my pagination methods because I'm getting less
> > than the expected number of records back, so my app thinks there's no
> > more timeline to be retrieved. I certainly don't want to just always
> > include a "next page" link when there really aren't any more records
> > left.
>
> > Also, thecountparameteris being ignored in all of this as well, but
> > I've latched on to another thread about that.
>
> > On Aug 30, 12:50 pm, Taylor Singletary <taylorsinglet...@twitter.com>
> > wrote:
> >> Hi Timmerk,
>
> >> Make sure you're using api.twitter.com as the host for all of your API
> >> requests. The proper API isnotat twitter.com -- it's at api.twitter.com.
> >> Include_rts willnotwork against twitter.com (and one day you'll find all
> >> of your requests rejected.)
>
> >> Also, please note that proper API routes include a version component. The
> >> call you're making should be:http://api.twitter.com/1/statuses/user_timeline.json?screen_name=NuWa...
>
> >> Taylor
>
> >> On Mon, Aug 30, 2010 at 10:43 AM, timmerk <timm...@gmail.com> wrote:
> >> > I'm having the same issue - you can see an example here:
>
> >> >http://twitter.com/statuses/user_timeline.json?screen_name=NuWaveGOV&...
>
> >> > As you can see, there are no retweets included in the JSON.
>
> >> > Thanks!
>
> >> > On Aug 25, 2:30 pm, Matt Harris <thematthar...@twitter.com> wrote:
> >> > > Hey Hen,
>
> >> > > Are you still finding retweets arenotshowing up for you?
>
> >> > > Matt
>
> >> > > On Sat, Aug 21, 2010 at 7:19 AM, henasraf <doku...@gmail.com> wrote:
> >> > > > Matt,
> >> > > > Any user I show in my app wouldnotshow retweets, even if they surely
> >> > > > exist. You can see for yourself athttp://wosaic.net/twitguin(still
> >> > > > early stages, don't mind it being a bit lame), see any user such as
> >> > > > myself athttp://wosaic.net/twitguin/user/henasraf;youmay go
> >> > > > through pages and see that no retweets ever show; they should be color
> >> > > > coded in orange. You can compare tohttp://twitter.com/henasraftosee
> >> > > > retweets that should show.
>
> >> > > > Thanks,
> >> > > > Hen
>
> >> > > > On Aug 19, 11:34 pm, Matt Harris <thematthar...@twitter.com> wrote:
> >> > > >> Hey,
>
> >> > > >> Retweets should be included if they exist in the users timeline when
> >> > you
> >> > > >> request them. For example if you request 20 Tweets of the timeline and
> >> > one
> >> > > >> of those last 20 was a retweet - it will be included. If one of the
> >> > last 20
> >> > > >> Tweets wasn't a retweet it won't be included, even if include_rts is
> >> > true.
>
> >> > > >> Does this explain what is happening in your app?
> >> > > >> Ifnotcould you give an example of a username which shows this
> >> > problem.
>
> >> > > >> Best,
> >> > > >> Matt
>
> >> > > >> On Thu, Aug 19, 2010 at 1:24 PM, henasraf <doku...@gmail.com> wrote:
> >> > > >> > Hey all, I'm using user_timeline to fetch the timeline of a user in
> >> > my
> >> > > >> > app. Problem is, it doesn't include native Retweets, even though
> >> > I've
> >> > > >> > set include_rts to true. The docs clearly say it should work, but it
> >> > > >> > doesn't. What could be the problem? Thanks in advance :)
>
> >> > > >> --
>
> >> > > >> Matt Harris
> >> > > >> Developer Advocate, Twitterhttp://twitter.com/themattharris
>
> >> > > --
>
> >> > > Matt Harris
> >> > > Developer Advocate, Twitterhttp://twitter.com/themattharris-Hidequoted
> >> > text -
>
> >> > > - Show quoted text -
>
> >> > --
> >> > Twitter developer documentation and resources:http://dev.twitter.com/doc
> >> > API updates via Twitter:http://twitter.com/twitterapi
> >> > Issues/Enhancements Tracker:
> >> >http://code.google.com/p/twitter-api/issues/list
> >> > Change your membership to this group:
> >> >http://groups.google.com/group/twitter-development-talk?hl=en-Hide quoted text -
>
> >> - Show quoted text -
>
> > --
> > Twitter developer documentation and resources:http://dev.twitter.com/doc
> > API updates via Twitter:http://twitter.com/twitterapi
> > Issues/Enhancements Tracker:http://code.google.com/p/twitter-api/issues/list
> > Change your membership to this group:http://groups.google.com/group/twitter-development-talk?hl=en
>
> --
>
> Matt Harris
> Developer Advocate, Twitterhttp://twitter.com/themattharris- Hide quoted text -

Michał Łowicki

unread,
Oct 4, 2010, 2:11:59 PM10/4/10
to twitter-deve...@googlegroups.com
I'm getting the same issue - http://groups.google.com/group/twitter-development-talk/browse_thread/thread/3c13f89ac3c5a945 .Still without any solution :(

2010/10/4 timrnicholson <timrni...@gmail.com>
Well, I thought i had this problem solvIed. I am using a library called



--
Pozdrawiam,
Michał Łowicki

timrnicholson

unread,
Oct 4, 2010, 2:28:36 PM10/4/10
to Twitter Development Talk
Further note that in testing, if I set the count parameter to 1 to
return a single record and page through the friends_timeline, I
actually get certain pages that return no data. i.e. I can page
through a single tweet at a time and still get some responses that are
empty. So the API is clearly first retrieving a record (or group of
records) and then applying some other filter to it. Its that other
filter that I need to figure out so that I can craft the request to
ensure that I always get back the requested number of records (except
when there are no more records).

I don't see any way to retrieve the TOTAL number of records for a
particular query either. If I could know for sure that a shortage of
returned records on a particular page was due to some filter and that
there truly are more records on subsequent pages, then I could at
least use this as a workaround.

On Oct 4, 11:48 am, timrnicholson <timrnichol...@gmail.com> wrote:
> Well, I thought i had this problem solved. I am using a library called
> twitteroauth and it was stripping off the "&include_rts=true". I
> finally figured out that include_rts needed to be placed into an
> arguments array instead of being in the request URL.
>
> So once I got the include_rts parm passed to Twitter everything wasworkingfine. Or at least it was in most cases. HOWEVER, I am seeing
> now that certain requests areNOTreturning the fullcountparameter
> worth of records. There must be SOME filtering that the Twitter API is
> doing AFTER it retrieves thecountnumber worth of records. I have
> this problem regardless of whether I use the home_timeline which is
> supposed to always include everything or whether I use the
> friends_timeline with the include_rtsparameter.
>
> Note that I am trying to requestcount= 25 records,notthe defaultcount= 20 records. I can see that retweets are being included and I'm
> > >> >http://groups.google.com/group/twitter-development-talk?hl=en-Hidequoted text -
>
> > >> - Show quoted text -
>
> > > --
> > > Twitter developer documentation and resources:http://dev.twitter.com/doc
> > > API updates via Twitter:http://twitter.com/twitterapi
> > > Issues/Enhancements Tracker:http://code.google.com/p/twitter-api/issues/list
> > > Change your membership to this group:http://groups.google.com/group/twitter-development-talk?hl=en
>
> > --
>
> > Matt Harris
> > Developer Advocate, Twitterhttp://twitter.com/themattharris-Hide quoted text -
>
> > - Show quoted text -- Hide quoted text -

timrnicholson

unread,
Oct 6, 2010, 9:56:58 AM10/6/10
to Twitter Development Talk
Ok, I think I may be closer to understanding what is not working or
works in a strange way. The returned data, when retweets are included
is of this structure. Each tweet that is a retweet, has the original
tweet information underneath it.

Tweet
Tweet
Retweet
Tweet

That's fine and all, except it appears that the original tweet is
being included in the count of tweets requested per page. In the above
example, this is considered 4 tweets being returned, not 3. If I
request 25 tweets and there is one retweet, then I'm really only
getting 24 tweets returned (plus the sub-tweet that is the original
tweet that had been retweeted).

So for now, I'm keeping a count of the number of retweets included in
each page and showing a link to go to the next page if #tweets +
#retweets > count parameter. At least this let's users keep paging
through the stream, but it still stinks that I can't get a full 25
tweets per page. What if a page was all retweets? Then the page would
show NO records.
> > > >> >http://groups.google.com/group/twitter-development-talk?hl=en-Hidequotedtext -
>
> > > >> - Show quoted text -
>
> > > > --
> > > > Twitter developer documentation and resources:http://dev.twitter.com/doc
> > > > API updates via Twitter:http://twitter.com/twitterapi
> > > > Issues/Enhancements Tracker:http://code.google.com/p/twitter-api/issues/list
> > > > Change your membership to this group:http://groups.google.com/group/twitter-development-talk?hl=en
>
> > > --
>
> > > Matt Harris
> > > Developer Advocate, Twitterhttp://twitter.com/themattharris-Hidequoted text -
>

dt

unread,
Dec 2, 2010, 1:30:10 PM12/2/10
to Twitter Development Talk
I'm having the same problem. This doesn't make much sense. It also
seems to still count deleted tweets, which seems like lousy behavior.
I assume they do this cut down on query paramters and thus server
overload, but it still stinks, especially since there is no user-
available programatic way to detect that this happening and thus
compensating for it.

Matt Harris

unread,
Dec 2, 2010, 2:55:05 PM12/2/10
to twitter-deve...@googlegroups.com
Hi,

Can you share the full URL you are requesting so I can take a closer look.
Thanks.

---
@themattharris

Developer Advocate, Twitter
http://twitter.com/themattharris



--
Reply all
Reply to author
Forward
0 new messages