the new retweet feature might break my app

7 views
Skip to first unread message

Yaniv Golan

unread,
Nov 11, 2009, 5:59:15 AM11/11/09
to Twitter Development Talk
Hi
I'm using twitter search API to search statuses with retweet
expressions (e.g RT VIA) with the link filter
i search the documentation over and over but i can't find any solution
to what will happen after the retweet will rollout completely (the
retweet expressions as we knew them will disappear and there is no
other API method to much my search criteria)
what should i do to get all the retweets that contains links?
thanks


Walter Smulders

unread,
Nov 11, 2009, 10:08:49 AM11/11/09
to Twitter Development Talk
With the retweet function the status text will still be prefixed with
RT

Yaniv Golan

unread,
Nov 11, 2009, 11:58:24 AM11/11/09
to Twitter Development Talk
oh...
that's cool :)
Thanks i really got wםrried for a sec

Yaniv Golan

unread,
Nov 11, 2009, 12:17:50 PM11/11/09
to Twitter Development Talk
and now i hear from someone who already has the new RT
that statuses aren't prefixed!
so are they or aren't they prefixed?
I'd love to get formal answer or reference to formal info on this

John Kalucki

unread,
Nov 11, 2009, 12:22:47 PM11/11/09
to Twitter Development Talk
Retweets do not modify the original text in any way. There is no RT to
search upon.

There is a feed of all public retweets on the Streaming API, but it is
not generally available. Instead, you can request a sample of all
statuses and filter for those that are retweets.

-John Kalucki
http://twitter.com/jkalucki
Services, Twitter Inc.

Yaniv Golan

unread,
Nov 11, 2009, 12:39:25 PM11/11/09
to Twitter Development Talk
Thanks John,
this means that my app won't work anymore!
streaming api makes my life very hard
hence i need to search for links and then extract them, this is very
resource demanding to do on the fly
while with search API i can search then extract and then search again
i could use queue system but then again i will lose all the real time
fun
will there be a filter in the search api for retweets like there is
for links?
that could solve all my problems




On Nov 11, 7:22 pm, John Kalucki <jkalu...@gmail.com> wrote:
> Retweets do not modify the original text in any way. There is no RT to
> search upon.
>
> There is a feed of all public retweets on the Streaming API, but it is
> not generally available. Instead, you can request a sample of all
> statuses and filter for those that are retweets.
>
> -John Kaluckihttp://twitter.com/jkalucki

John Kalucki

unread,
Nov 11, 2009, 12:48:36 PM11/11/09
to Twitter Development Talk
Search is aware of the need for a retweet operator, but the feature is
unscheduled and completely speculative.

In any case, Search will become less useful for this sort of
repetitive complete corpus search. If you need all of something, or a
sample of something, you should be moving to the Streaming API
wherever possible.

-John Kalucki
http://twitter.com/jkalucki
Services, Twitter Inc.



Walter Smulders

unread,
Nov 12, 2009, 5:50:36 AM11/12/09
to Twitter Development Talk
In the examples that are shown in the developers preview for the RT
api these where prefixed with RT, was this done on purpose or did this
change after the examples where made public?

On Nov 11, 6:48 pm, John Kalucki <jkalu...@gmail.com> wrote:
> Search is aware of the need for a retweet operator, but the feature is
> unscheduled and completely speculative.
>
> In any case, Search will become less useful for this sort of
> repetitive complete corpus search. If you need all of something, or a
> sample of something, you should be moving to the Streaming API
> wherever possible.
>

John Kalucki

unread,
Nov 12, 2009, 9:29:42 AM11/12/09
to Twitter Development Talk
The Retweet feature has many possible realizations. We've tried nearly
every possible combination of all the functional dimensions as the
feature evolved over many months. It's possible that what you saw was
based on a snapshot of the current state of the feature, and the
feature subsequently changed.

In any case, what was in production earlier this week is the best
indicator of what the feature will be moving forward. I'd also expect
the feature to continue to evolve and new features to sprout up.

Change breaks both expectations and code. It is inevitable. The
Platform team prioritizes mitigation efforts for third-party apps, but
a full mitigation isn't always possible. We can't gate all forward
progress on perfect mitigation, or we'd pretty much never be able to
ship a new feature. A bigger and better Twitter is ultimately better
for the ecosystem, despite the knocks along the way.

-John Kalucki
http://twitter.com/jkalucki
Services, Twitter Inc.


cadams500

unread,
Nov 12, 2009, 10:00:12 AM11/12/09
to Twitter Development Talk
> Change breaks both expectations and code. It is inevitable.
I agree with this statement. However, there seems to be some confusion
over whether retweets will still be prefixed with RT. If retweets are
no longer prefixed with RT, then I do not understand why that change
is being made. You could do everything you are doing with the retweet
API; however, still ensure that retweets are prefixed with RT.

My concern is that there is nothing to force users to upgrade their
twitter applications, there is nothing to force applications to use
the retweet API (although, I agree most probably will) and for an
indeterminate amount of time users will still "retweet" by prefixing
their tweets with RT. If the retweet API would simply still prefix
retweets with RT, at least retweets messages would "look" consistent
and searches that relied upon the de-factor starndard of RT
@<username> would still work as expected.

I am personally excited about the retweet api, and the ability to
unambiguously tie a retweet back to the original tweet message;
however, as mentioned above I'm concerned that retweets will no longer
be prefixed with 'RT'.

- Chris

On Nov 12, 8:29 am, John Kalucki <jkalu...@gmail.com> wrote:
> The Retweet feature has many possible realizations. We've tried nearly
> every possible combination of all the functional dimensions as the
> feature evolved over many months. It's possible that what you saw was
> based on a snapshot of the current state of the feature, and the
> feature subsequently changed.
>
> In any case, what was in production earlier this week is the best
> indicator of what the feature will be moving forward. I'd also expect
> the feature to continue to evolve and new features to sprout up.
>
> Change breaks both expectations and code. It is inevitable. The
> Platform team prioritizes mitigation efforts for third-party apps, but
> a full mitigation isn't always possible. We can't gate all forward
> progress on perfect mitigation, or we'd pretty much never be able to
> ship a new feature. A bigger and better Twitter is ultimately better
> for the ecosystem, despite the knocks along the way.
>

Cameron Kaiser

unread,
Nov 12, 2009, 11:00:56 AM11/12/09
to twitter-deve...@googlegroups.com
> My concern is that there is nothing to force users to upgrade their
> twitter applications, there is nothing to force applications to use
> the retweet API (although, I agree most probably will) and for an
> indeterminate amount of time users will still "retweet" by prefixing
> their tweets with RT.

Speaking for TTYtter only, while I'll support receiving retweets, I am
unhappy with the API as it currently exists and retweets received will
be canonized into the older format (and retweets sent will be done
programmatically in the older fashion instead of through the retweet
methods). I suspect there are other app authors who will also do something
similar.

--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * cka...@floodgap.com
-- Hell hath no fury like a bureaucrat scorned. -- Milton Freedman ------------

John Kalucki

unread,
Nov 12, 2009, 11:53:06 AM11/12/09
to Twitter Development Talk
I think I've added to the confusion. Sorry about making things worse.
I was coming from a strictly search viewpoint. But Retweet is not so
simple!

There are lots of different places that Tweets are rendered: Various
timelines, Search, Streaming, SMS, etc. etc. Various renderings and
search mechanisms have taken different approaches to ease the
transition to the new Retweet format. Some changes are temporary
patches until the feature is fully and finally rolled-out, and will
then subsequently go away. Others are expected to be permanent.

There's a lot of information, a lot of complexity and some transitory
behavior. Yes, I find it confusing too. And it shouldn't be so
confusing for what seems to be such a simple thing. But, once you
drill down, there is a lot of unavoidable complexity.

-John Kalucki
http://twitter.com/jkalucki
Services, Twitter Inc.

Dave Sherohman

unread,
Nov 13, 2009, 5:56:30 AM11/13/09
to twitter-deve...@googlegroups.com
On Thu, Nov 12, 2009 at 08:00:56AM -0800, Cameron Kaiser wrote:
> Speaking for TTYtter only, while I'll support receiving retweets, I am
> unhappy with the API as it currently exists and retweets received will
> be canonized into the older format (and retweets sent will be done
> programmatically in the older fashion instead of through the retweet
> methods). I suspect there are other app authors who will also do something
> similar.

I haven't looked closely at the RT API (it's not currently relevant to
FishTwits, so I figure I'll let it stabilize before concerning myself
with it), but would you mind sharing your issues with it, either here or
off-list if you think that would be more appropriate?

--
Dave Sherohman

cadams500

unread,
Nov 13, 2009, 7:54:38 AM11/13/09
to Twitter Development Talk
> I am unhappy with the API as it currently exists and retweets received will
> be canonized into the older format (and retweets sent will be done
> programmatically in the older fashion instead of through the retweet
> methods). I suspect there are other app authors who will also do something
> similar.

From our application's point of view, this is the scenario that I am
concerned about. I am afraid there are going to be so many different
versions of twitter applications floating around, some using the new
retweet API to retweet status, some not using the retweet API and just
issuing new tweets prefixed with RT.

Does twitter have plans to prevent status messages (just like they
prevent duplicates) that use the old retweet format and in effect,
force applications to upgrade to using the retweet api?

If not, then I do not see any short-term solution, other than
application developers who rely upon finding retweets for a tweet,
will have to now take into account both the new retweet api and the
old "search" method based upon the "RT @<username>" format. Which
obviously adds complexity.

Maybe I'm missing something? :)

On Nov 12, 10:00 am, Cameron Kaiser <spec...@floodgap.com> wrote:
> > My concern is that there is nothing to force users to upgrade their
> > twitter applications, there is nothing to force applications to use
> > the retweet API (although, I agree most probably will) and for an
> > indeterminate amount of time users will still "retweet" by prefixing
> > their tweets with RT.
>
> Speaking for TTYtter only, while I'll support receiving retweets, I am
> unhappy with the API as it currently exists and retweets received will
> be canonized into the older format (and retweets sent will be done
> programmatically in the older fashion instead of through the retweet
> methods). I suspect there are other app authors who will also do something
> similar.
>
> --
> ------------------------------------ personal:http://www.cameronkaiser.com/--
>   Cameron Kaiser * Floodgap Systems *www.floodgap.com* ckai...@floodgap.com

Cameron Kaiser

unread,
Nov 13, 2009, 8:40:14 AM11/13/09
to twitter-deve...@googlegroups.com
> Does twitter have plans to prevent status messages (just like they
> prevent duplicates) that use the old retweet format and in effect,
> force applications to upgrade to using the retweet api?

Ignoring how steamed people would be if that happened, I doubt it, because
there are plenty of ways to get around it effectively. There would just be
another de facto consensus established.

--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * cka...@floodgap.com

-- When in doubt, take a pawn. -- Mission: Impossible ("Crack-Up") ------------

Cameron Kaiser

unread,
Nov 13, 2009, 8:54:00 AM11/13/09
to twitter-deve...@googlegroups.com
> > Speaking for TTYtter only, while I'll support receiving retweets, I am
> > unhappy with the API as it currently exists and retweets received will
> > be canonized into the older format (and retweets sent will be done
> > programmatically in the older fashion instead of through the retweet
> > methods). I suspect there are other app authors who will also do something
> > similar.
>
> I haven't looked closely at the RT API (it's not currently relevant to
> FishTwits, so I figure I'll let it stabilize before concerning myself
> with it), but would you mind sharing your issues with it, either here or
> off-list if you think that would be more appropriate?

It isn't the API methods per se, it's the fact that (as others have pointed
out) there is no way to edit or mark up a tweet using the Retweet system as
it is currently designed. This is important to me personally, and certainly
to anyone posting with #saveretweets. Also, as implied by the fact that I
won't be supporting it in its current form, it's easy enough to continue to
post in the old manner (or come up with a new one), which dilutes its alleged
advantages in trackability and ignorability, and I've always considered
it more important to know who is doing the retweet than who is being
retweeted, because who the filter is tells me as much if not more than what
is being filtered through them.

These are just complaints about the design of the system, although in fairness
to Ev, he has acknowledged some of the deficiencies and has implied they will
be fixed in later versions (cf.

http://evhead.com/2009/11/why-retweet-works-way-it-does.html

). But I won't be supporting posting through it in its current form.

--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * cka...@floodgap.com

-- BOND THEME NOW PLAYING: "Thunderball" --------------------------------------

stephane

unread,
Nov 13, 2009, 3:24:32 PM11/13/09
to Twitter Development Talk
Thing is the new RT api simply does not express the same thing the
classic RT @xxx is :
It's a subset of a big set of meanings : [ "Like" , "Forward" ,
"Comment", "Thanks", "Emphasis", "Reply" ] (and I'm sure there are a
lot more uses)

The rich semantic of classic RTs make them sometime difficult to
analyze through simple algorithms but is a no-brainer for a human.

Twitter (or Ev) thinks that they needed to disambiguate the RT
concept. I'm really not sure about that on the user perspective ...
though I see the advantages for the platform. They probably decided to
chose the most frequent use (ie : "Like").

It's a really complex issue, I would not have addressed this way
personally (we went through the exact same "brainstormings" when
working on Yokway) but I'm curious to see where the new RT API leads
to :)

So probably we will keep on seeing classic RTs in the feed, I'll
probably keep on RTing full text as I often need to say more /
something else than "I Like it" to my (small) audience.

As a developer, I'll try my best to be able to support both as long as
possible.

Stephane
http://www.twazzup.com
>   Cameron Kaiser * Floodgap Systems *www.floodgap.com* ckai...@floodgap.com

M. Edward (Ed) Borasky

unread,
Dec 2, 2009, 10:31:02 PM12/2/09
to Twitter Development Talk
I've been looking at data acquired from the "sample" feed. In those
tweets, I've discovered something, and I wanted to run it by Twitter
and see if this is an official, documented API feature.

When a tweet is a retweet using the retweet button on the web page,
there is an extra key-value pair in a returned tweet. I've been
handling them as JSON objects (in Perl), but I suspect other formats
do the same. The key is "retweeted_status", and the value is an
*entire* object containing the original tweet that was retweeted. If a
tweet was not a retweet, this extra key-value pair isn't there.

Questions:

1. Is this official, documented behavior, or did I discover it on my
own?
2. Is it likely to change?
3. Why is the *entire* original tweet given as the value? Wouldn't it
be enough to give the tweet ID and let the app go hunt for it if
necessary?

John Kalucki

unread,
Dec 2, 2009, 10:36:05 PM12/2/09
to twitter-deve...@googlegroups.com
This is documented, supported and subject to as much change or stasis as any other Twitter feature.

The entire tweet is given to avoid an extra round-trip in rendering timelines. Many our results are denormalized in this way, as a fully normalized schema delivered via an Internet service would be impractical from a display latency standpoint.


-John Kalucki
http://twitter.com/jkalucki
Services, Twitter Inc.


M. Edward (Ed) Borasky

unread,
Dec 3, 2009, 4:07:24 AM12/3/09
to Twitter Development Talk


On Dec 2, 7:36 pm, John Kalucki <j...@twitter.com> wrote:
> This is documented, supported and subject to as much change or stasis as any
> other Twitter feature.
>
> The entire tweet is given to avoid an extra round-trip in rendering
> timelines. Many our results are denormalized in this way, as a fully
> normalized schema delivered via an Internet service would be impractical
> from a display latency standpoint.

Thanks!! That should solve my coding problems.
Reply all
Reply to author
Forward
0 new messages