Feedback needed: changes to JSON responses

[Alex Payne]

It's come to my attention that there are some potential issues with
how JSON responses are handled in the Twitter API. I need your
collective help in figuring out a good timetable for changes to be
released.

First off, there's a potential security issue with the ability to have
user-defined callbacks around our JSON responses (sometimes referred
to as "JSONP"). An attacker could construct a page with a malicious
callback that calls Twitter API methods that return potentially
private data and store that data on his server. A user would have to
have created a Basic Auth session to the Twitter API in her browser,
which is fairly unlikely, but this is still concerning. My proposal
is to disable user-specified callbacks for all methods that could
return private data. I'd like to push that change by the end of the
week, but I need to know if that's enough time for all of you to push
new releases.

Secondly, there's another security issue in older versions of Firefox
that allows attackers to redefine the Array prototype to malicious
ends. The standard workaround for this is to return all responses as
hashes with a fixed root element, ex:

{"statuses": [...] }

This would be a more far-reaching change, as it breaks any and all
applications that consume JSON from the Twitter API. It's also a much
lower-priority security issue. What do you all think is a reasonable
time frame for making this change?

Thanks much for your feedback. Please, if you have friends working
with the Twitter API who use JSON, ask them to weigh in on this. I'd
like to plan a smooth, painless transition for these issues.

--
Alex Payne - API Lead, Twitter, Inc.
http://twitter.com/al3x

[Cameron Kaiser]

> Secondly, there's another security issue in older versions of Firefox
> that allows attackers to redefine the Array prototype to malicious
> ends. The standard workaround for this is to return all responses as
> hashes with a fixed root element, ex:
>
> {"statuses": [...] }
>
> This would be a more far-reaching change, as it breaks any and all
> applications that consume JSON from the Twitter API. It's also a much
> lower-priority security issue. What do you all think is a reasonable
> time frame for making this change?

I'd like at least 90 days myself. I don't really need that length of time
for my little app but given the handwringing over friends_timeline, it's
going to need to be at least that long for all the people who will come
out of the woodwork.

--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * cka...@floodgap.com
-- LOAD"STANDARD DISCLAIMER",8,1 ----------------------------------------------

[Mike Scott]

I'd like at least 90 days myself. I don't really need that length of time
for my little app but given the handwringing over friends_timeline, it's
going to need to be at least that long for all the people who will come
out of the woodwork.

90 Days is just way too long, especially for something that is a low-security risk. For a pure feature change that'd be fine, but for something that poses a security risk... Thats just unreasonable.

I'm happy with a 30 day deadline. Thats plenty of time in my book. Just a general though out loud here Alex: Maybe it would be a good idea to write up a policy for this regarding timing which will give developers some feedback as to how often they should be checking for updates. Also perhaps creating a low-volume list that is announce only for this sort of information is a need/requirement too.

[Adam Cloud]

A week should be plenty of time to make a small change (at least it's small for me for how my code is managed), but to allow everyone who might be affected to see this message, more time would be appropriate.

Security risks to the site itself should be changed immediately, what if in the weeks time between the change someone exploited this hole?

Lets paint a scenario (granted i dont know the depth of the security risk), but by "private data" i assumed you meant someone's account pedigree info. What if someone was able to manipulate the hole you mention to return password hashes and email login? The attacker then uses the email to log into their email, and the person was able to access someone's bank account (yes bank websites have all kinds of new security features, but if the person hasn't logged in in a while they couldn't have changed over to the newer more secure method of using that bank website). Attacker ends up being caught and we find out he got the information to go on this little adventure through a security hole in twitter. The LAST thing any of us want is a lawsuit against twitter.

A week of having my program not work, versus the potential of twitter being down indefinitely? I would rather deal with complaints of my program not working temporarily. :)

[Cameron Kaiser]

> A week should be plenty of time to make a small change (at least it's small
> for me for how my code is managed), but to allow everyone who might be
> affected to see this message, more time would be appropriate.
> Security risks to the site itself should be changed immediately, what if in
> the weeks time between the change someone exploited this hole?

With all due respect, security holes do need to be closed and dealt with in
a timely fashion, but this is a security hole in a now deprecated version
of JavaScript and a hole that by fixing it will render large portions of the
user community off-line immediately. People here have short memories:
remember all the screaming that occurred for weeks when friends_timeline got
modified? Most of that screaming came from people who did NOT have any way
of changing their twitter badges and the like, and claimed to have no idea
that the change was going forward.

I suggested ninety days with frequent list reminders (YOUR APP WILL NOT WORK
IF IT USES JSON) because it took nearly that long for the static to die down
and some very common uses such as blog plugins may be slower to update. A
post to dev.twitter.com (YOUR APP WILL NOT WORK IF IT USES JSON) should catch
the rest of the people who don't watch the list. Maybe a shorter period would
also work but I think a week is way too short.

> Lets paint a scenario (granted i dont know the depth of the security risk),
> but by "private data" i assumed you meant someone's account pedigree info.
> What if someone was able to manipulate the hole you mention to return
> password hashes and email login?

As far as I can tell, Alex is referring to this:

http://directwebremoting.org/blog/joe/2007/03/05/json_is_not_as_safe_as_people_think_it_is.html

This is clearly bad but not catastrophic. It lets you get to potentially
protected data, which is clearly bad, but

> into their email, and the person was able to access someone's bank account
> (yes bank websites have all kinds of new security features, but if the
> person hasn't logged in in a while they couldn't have changed over to the
> newer more secure method of using that bank website). Attacker ends up being
> caught and we find out he got the information to go on this little adventure
> through a security hole in twitter. The LAST thing any of us want is a
> lawsuit against twitter.

anyone who is posting bank information as tweets, even protected ones, is
also clearly stupid. Most people understand that the anonymity that the
padlock provides is not designed to be airtight.

--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * cka...@floodgap.com

-- Furious activity is no substitute for understanding. -- H. H. Williams -----

[Adam Cloud]

I was painting a picture. Nobody said anything about tweeting bank information, but logins are commonly shared with a person's email. And that same person's email can have emailed statements from a persons bank, or any number of emails giving the attacker direction of what he can access.

I was simply painting a worst case scenario.

If it was known depreciated, then I'll assume steps are already in progress to move your code to the newer supported methods.

You know my opinion on the matter. Since he said collective on finding a reasonable timetable, i felt i had a right to express what would be an appropriate timetable for myself.

[Alex Payne]

Okay, so right now I'm hearing somewhere between one and two months.
Can I get some more voices?

--

[jstrellner]

30 days would be more than enough. A hard date with a time period
(e.g. 10/1/08 between 1am and 2am, PST) would need to be set, and you
guys would need to make sure that it doesn't get deployed before or
after the window.

As long as everyone has ample time to read about the issue and
understand what they are going to have to change, as well as know
exactly when it will happen, there shouldn't be too many issues.

I vote for 30 days out.

-Joel

On Sep 16, 10:41 am, "Alex Payne" <a...@twitter.com> wrote:
> Okay, so right now I'm hearing somewhere between one and two months.
> Can I get some more voices?
>
>
>
> On Tue, Sep 16, 2008 at 8:05 AM, Adam Cloud <cloudy...@gmail.com> wrote:
> > I was painting a picture. Nobody said anything about tweeting bank
> > information, but logins are commonly shared with a person's email. And that
> > same person's email can have emailed statements from a persons bank, or any
> > number of emails giving the attacker direction of what he can access.
>
> > I was simply painting a worst case scenario.
>
> > If it was known depreciated, then I'll assume steps are already in progress
> > to move your code to the newer supported methods.
>
> > You know my opinion on the matter. Since he said collective on finding a
> > reasonable timetable, i felt i had a right to express what would be an
> > appropriate timetable for myself.
>

> > On Tue, Sep 16, 2008 at 8:57 AM, Cameron Kaiser <spec...@floodgap.com>

> >>  http://directwebremoting.org/blog/joe/2007/03/05/json_is_not_as_safe_...

>
> >> This is clearly bad but not catastrophic. It lets you get to potentially
> >> protected data, which is clearly bad, but
>
> >> > into their email, and the person was able to access someone's bank
> >> > account
> >> > (yes bank websites have all kinds of new security features, but if the
> >> > person hasn't logged in in a while they couldn't have changed over to
> >> > the
> >> > newer more secure method of using that bank website). Attacker ends up
> >> > being
> >> > caught and we find out he got the information to go on this little
> >> > adventure
> >> > through a security hole in twitter. The LAST thing any of us want is a
> >> > lawsuit against twitter.
>
> >> anyone who is posting bank information as tweets, even protected ones, is
> >> also clearly stupid. Most people understand that the anonymity that the
> >> padlock provides is not designed to be airtight.
>
> >> --
> >> ------------------------------------ personal:
> >>http://www.cameronkaiser.com/--
> >>  Cameron Kaiser * Floodgap Systems *www.floodgap.com*

> >> ckai...@floodgap.com

[Rob Iles]

Easy for me to say "30 days should be sufficient" as my app is still very much in private Beta, with no real userbase.....I guess some of the guys with large deployed userbases would have more valuable contributions to make to this discussion than I.

Regards,

Rob

2008/9/16 jstrellner <jstre...@urltrends.com>

--
Rob Iles

RMIDevelopment

Web: http://www.rob-iles.co.uk/rmidevelopment
Twitter: http://twitter.com/Rob_Iles
Skype: rob_iles
12seconds: http://12seconds.tv/channel/rmi

[jazzychad]

This may be a silly question b/c I don't know where the line blurs
between the Twitter API and the Twitter Search API, but would these
changes also affect the Twitter Search JSON callback mechanism?
-Chad

On Sep 16, 6:44 pm, "Rob Iles" <rob.i...@gmail.com> wrote:
> Easy for me to say "30 days should be sufficient" as my app is still very
> much in private Beta, with no real userbase.....I guess some of the guys
> with large deployed userbases would have more valuable contributions to make
> to this discussion than I.
>
> Regards,
>
> Rob
>

> 2008/9/16 jstrellner <jstrell...@urltrends.com>

[Alex Payne]

They don't, because the Search API doesn't return any private data.

[Colby Palmer]

30 days would work fine for me. Thanks.

Colby Palmer
http://itweet.net

[Kazuho Okui]

Is this time frame already started? If the time frame is 30 days, when
is the deadline?

Also, is there any example JSON data available? Do you have a list of
APIs which will be changed?

Thanks,
Kazuho

[Alex Payne]

I'll be announcing the deadline and such shortly in a separate thread.

[Marco Kaiser]

A time frame of 30 days seems to be sufficient to allow all API developers to catch up on this, I agree. 

For server based services using the Twitter API, a given "switch deadline" from old to new format might work well, but please consider that this is not so true for all the desktop apps / widgets / plugins etc. out there that don't have a single point where the change can be made, but need to distribute new versions to possibly tens of thousands of users. 

I know that it's hard to solve this problem if you want to provide the new format at the same endpoint, but could you please make sure you are communicating the change not only to developers, but also to end users, so that they are aware what probably causes their apps to malfunction? Not every developer has a chance to reach out to his users directly and tell them about it, so he would have to deal with a lot of support questions and angry users...

Thanks!

2008/9/23 Alex Payne <al...@twitter.com>

[Alex Payne]

The resolution of this discussion can be found in this new thread:
http://groups.google.com/group/twitter-development-talk/browse_thread/thread/165061da1d61c0b