Issues 419 [1] and 474 [2] are very popular, in the painful kind of way. The defects report that methods returning user objects (see users/show for an example [3]) are returning incorrect or invalid values for the <following> element.
The fix for this inconsistency is in fact non trivial [4]. The problem lies within the interaction of the application logic, caching layer and database design. The persistent data behind <following> and <notification> values are separate from the user data in our architecture, so to keep these elements valid in cache alongside user objects adds a large amount of complexity.
Developers made it obvious that these data are a priority and we want to ensure they available. We also want to guarantee they are accurate and that performance remains good. Given the problems explained above, we are going to be making a number of changes to the API so that you can rely on the <following> or <notification> data.
Deprecations: The following elements are to be removed from all returned user objects returned by the API:
1) <following> 2) <notifications>
This deprecation will not occur until we finish the following:
Additions: To continue to provide access to this data we will be creating a new method:
Issue 532 [4] outlines the need to perform a mutual following lookup. We will use a method similar to that described in this issue to deliver <following>, <followedby>, <notification> and <pending> (in the case of protected users) data with a single call.
We realize this change will cause an increase in API usage for some applications. Therefore we are going to increase the default API rate limit across the board. This should help absorb some of the costs for applications attempting to do interesting things with social graph data. The number will be somewhere between 101 and 200 calls but we still need to look at growth projections and current hardware capacity before settling on a definite number.
We plan to begin work on this relatively soon with the fix coming in a few weeks. We do not have a planned ship date at this time but will communicate specifics with developers as they are determined. We anticipate the new number of calls and a documented schema for the new method will be made available before the new method ships. Please watch this thread and @twitterapi for the incremental details.
I'll admit I'm a little disappointed that the info won't be part of
the user objects anymore (will have to rethink some of my planned
features... ie. won't be able to dynamically show/hide the dm button
next to tweets if it means I need an additional api call for each
user) instead relying on another api call, but I'm sure you guys have
your reasons.
Thanks for the advanced warning :)
On May 12, 7:18 am, Doug Williams <d...@twitter.com> wrote:
> Issues 419 [1] and 474 [2] are very popular, in the painful kind of way. The
> defects report that methods returning user objects (see users/show for an
> example [3]) are returning incorrect or invalid values for the <following>
> element.
> The fix for this inconsistency is in fact non trivial [4]. The problem lies
> within the interaction of the application logic, caching layer and database
> design. The persistent data behind <following> and <notification> values are
> separate from the user data in our architecture, so to keep these elements
> valid in cache alongside user objects adds a large amount of complexity.
> Developers made it obvious that these data are a priority and we want to
> ensure they available. We also want to guarantee they are accurate and that
> performance remains good. Given the problems explained above, we are going
> to be making a number of changes to the API so that you can rely on the
> <following> or <notification> data.
> Deprecations:
> The following elements are to be removed from all returned user objects
> returned by the API:
> 1) <following>
> 2) <notifications>
> This deprecation will not occur until we finish the following:
> Additions:
> To continue to provide access to this data we will be creating a new method:
> Issue 532 [4] outlines the need to perform a mutual following lookup. We
> will use a method similar to that described in this issue to deliver
> <following>, <followedby>, <notification> and <pending> (in the case of
> protected users) data with a single call.
> We realize this change will cause an increase in API usage for some
> applications. Therefore we are going to increase the default API rate limit
> across the board. This should help absorb some of the costs for applications
> attempting to do interesting things with social graph data. The number will
> be somewhere between 101 and 200 calls but we still need to look at growth
> projections and current hardware capacity before settling on a definite
> number.
> We plan to begin work on this relatively soon with the fix coming in a few
> weeks. We do not have a planned ship date at this time but will communicate
> specifics with developers as they are determined. We anticipate the new
> number of calls and a documented schema for the new method will be made
> available before the new method ships. Please watch this thread and
> @twitterapi for the incremental details.
To be blunt this is very unsatisfactory. Once again you guys are not
being at all cognisant of the requirements of mobile Twitter client
apps. These face much bigger problems than just the rate limit. They
are constrained by physical limitations such as battery life, latency
and bandwidth. And they also have to take account of carrier data
charges. Every time something in your API requires an additional
method call you are making life difficult for us mobile app developers
who are trying to deliver a quality Twitter client to our users (who
are also your users!).
What annoys me too is that whenever a mobile-specific issue is raised
your stock response is "handle that in a proxy". Guys, that's just not
good enough. The World is going mobile and the continuing development
of your API needs to take account of this.
Very unhappy about this!
On May 11, 10:18 pm, Doug Williams <d...@twitter.com> wrote:
> Issues 419 [1] and 474 [2] are very popular, in the painful kind of way. The
> defects report that methods returning user objects (see users/show for an
> example [3]) are returning incorrect or invalid values for the <following>
> element.
> The fix for this inconsistency is in fact non trivial [4]. The problem lies
> within the interaction of the application logic, caching layer and database
> design. The persistent data behind <following> and <notification> values are
> separate from the user data in our architecture, so to keep these elements
> valid in cache alongside user objects adds a large amount of complexity.
> Developers made it obvious that these data are a priority and we want to
> ensure they available. We also want to guarantee they are accurate and that
> performance remains good. Given the problems explained above, we are going
> to be making a number of changes to the API so that you can rely on the
> <following> or <notification> data.
> Deprecations:
> The following elements are to be removed from all returned user objects
> returned by the API:
> 1) <following>
> 2) <notifications>
> This deprecation will not occur until we finish the following:
> Additions:
> To continue to provide access to this data we will be creating a new method:
> Issue 532 [4] outlines the need to perform a mutual following lookup. We
> will use a method similar to that described in this issue to deliver
> <following>, <followedby>, <notification> and <pending> (in the case of
> protected users) data with a single call.
> We realize this change will cause an increase in API usage for some
> applications. Therefore we are going to increase the default API rate limit
> across the board. This should help absorb some of the costs for applications
> attempting to do interesting things with social graph data. The number will
> be somewhere between 101 and 200 calls but we still need to look at growth
> projections and current hardware capacity before settling on a definite
> number.
> We plan to begin work on this relatively soon with the fix coming in a few
> weeks. We do not have a planned ship date at this time but will communicate
> specifics with developers as they are determined. We anticipate the new
> number of calls and a documented schema for the new method will be made
> available before the new method ships. Please watch this thread and
> @twitterapi for the incremental details.
David, As with any solution there are compromises (the normal big three are time, resources, quality of service) and while this change may make your particular use of the API more difficult, it is not only important but also necessary given our architecture and growth. The API provides Twitter data in a format that is consistent with our strengths. It is up to the consuming application to make the data we freely provide useful in its independent context. This decoupling of data and application allows us to focus on data delivery while the developer attends to user experience. We aim to maximize performance for board array of use-cases and while at the same time minimizing operational and maintenance costs.
There are many very successful mobile applications that run a proxy to get around the resource/time trade-off that this deprecation creates. If you are mobile heavy, it is suggested you do the same. A proxy is highly recommended for iPhone apps because it insulates the application changes in the Twitter API with the App Store acceptance delay.
If anyone has an open source Twitter API proxy, please start another thread so mobile developers like David do not have to reinvent the wheel. In fact, there should be a FOSS project for mobile devs to rely on -- I've got a couple ideas to contribute. Again, please start a thread (and link back here) if you have code or interest in starting a proxy project.
On Mon, May 11, 2009 at 4:38 PM, Dave Mc <davidmccorm...@gmail.com> wrote:
> To be blunt this is very unsatisfactory. Once again you guys are not > being at all cognisant of the requirements of mobile Twitter client > apps. These face much bigger problems than just the rate limit. They > are constrained by physical limitations such as battery life, latency > and bandwidth. And they also have to take account of carrier data > charges. Every time something in your API requires an additional > method call you are making life difficult for us mobile app developers > who are trying to deliver a quality Twitter client to our users (who > are also your users!).
> What annoys me too is that whenever a mobile-specific issue is raised > your stock response is "handle that in a proxy". Guys, that's just not > good enough. The World is going mobile and the continuing development > of your API needs to take account of this.
> Very unhappy about this!
> On May 11, 10:18 pm, Doug Williams <d...@twitter.com> wrote: > > Issues 419 [1] and 474 [2] are very popular, in the painful kind of way. > The > > defects report that methods returning user objects (see users/show for an > > example [3]) are returning incorrect or invalid values for the > <following> > > element.
> > The fix for this inconsistency is in fact non trivial [4]. The problem > lies > > within the interaction of the application logic, caching layer and > database > > design. The persistent data behind <following> and <notification> values > are > > separate from the user data in our architecture, so to keep these > elements > > valid in cache alongside user objects adds a large amount of complexity.
> > Developers made it obvious that these data are a priority and we want to > > ensure they available. We also want to guarantee they are accurate and > that > > performance remains good. Given the problems explained above, we are > going > > to be making a number of changes to the API so that you can rely on the > > <following> or <notification> data.
> > Deprecations: > > The following elements are to be removed from all returned user objects > > returned by the API:
> > 1) <following> > > 2) <notifications>
> > This deprecation will not occur until we finish the following:
> > Additions: > > To continue to provide access to this data we will be creating a new > method:
> > Issue 532 [4] outlines the need to perform a mutual following lookup. We > > will use a method similar to that described in this issue to deliver > > <following>, <followedby>, <notification> and <pending> (in the case of > > protected users) data with a single call.
> > We realize this change will cause an increase in API usage for some > > applications. Therefore we are going to increase the default API rate > limit > > across the board. This should help absorb some of the costs for > applications > > attempting to do interesting things with social graph data. The number > will > > be somewhere between 101 and 200 calls but we still need to look at > growth > > projections and current hardware capacity before settling on a definite > > number.
> > We plan to begin work on this relatively soon with the fix coming in a > few > > weeks. We do not have a planned ship date at this time but will > communicate > > specifics with developers as they are determined. We anticipate the new > > number of calls and a documented schema for the new method will be made > > available before the new method ships. Please watch this thread and > > @twitterapi for the incremental details.
Something to keep in mind is that when switching from a computer to a mobile device you are losing power/features for mobility. You have to expect the same loss in functionality.
On Mon, May 11, 2009 at 18:38, Dave Mc <davidmccorm...@gmail.com> wrote:
> To be blunt this is very unsatisfactory. Once again you guys are not > being at all cognisant of the requirements of mobile Twitter client > apps. These face much bigger problems than just the rate limit. They > are constrained by physical limitations such as battery life, latency > and bandwidth. And they also have to take account of carrier data > charges. Every time something in your API requires an additional > method call you are making life difficult for us mobile app developers > who are trying to deliver a quality Twitter client to our users (who > are also your users!).
> What annoys me too is that whenever a mobile-specific issue is raised > your stock response is "handle that in a proxy". Guys, that's just not > good enough. The World is going mobile and the continuing development > of your API needs to take account of this.
> Very unhappy about this!
> On May 11, 10:18 pm, Doug Williams <d...@twitter.com> wrote: > > Issues 419 [1] and 474 [2] are very popular, in the painful kind of way. > The > > defects report that methods returning user objects (see users/show for an > > example [3]) are returning incorrect or invalid values for the > <following> > > element.
> > The fix for this inconsistency is in fact non trivial [4]. The problem > lies > > within the interaction of the application logic, caching layer and > database > > design. The persistent data behind <following> and <notification> values > are > > separate from the user data in our architecture, so to keep these > elements > > valid in cache alongside user objects adds a large amount of complexity.
> > Developers made it obvious that these data are a priority and we want to > > ensure they available. We also want to guarantee they are accurate and > that > > performance remains good. Given the problems explained above, we are > going > > to be making a number of changes to the API so that you can rely on the > > <following> or <notification> data.
> > Deprecations: > > The following elements are to be removed from all returned user objects > > returned by the API:
> > 1) <following> > > 2) <notifications>
> > This deprecation will not occur until we finish the following:
> > Additions: > > To continue to provide access to this data we will be creating a new > method:
> > Issue 532 [4] outlines the need to perform a mutual following lookup. We > > will use a method similar to that described in this issue to deliver > > <following>, <followedby>, <notification> and <pending> (in the case of > > protected users) data with a single call.
> > We realize this change will cause an increase in API usage for some > > applications. Therefore we are going to increase the default API rate > limit > > across the board. This should help absorb some of the costs for > applications > > attempting to do interesting things with social graph data. The number > will > > be somewhere between 101 and 200 calls but we still need to look at > growth > > projections and current hardware capacity before settling on a definite > > number.
> > We plan to begin work on this relatively soon with the fix coming in a > few > > weeks. We do not have a planned ship date at this time but will > communicate > > specifics with developers as they are determined. We anticipate the new > > number of calls and a documented schema for the new method will be made > > available before the new method ships. Please watch this thread and > > @twitterapi for the incremental details.
At least we are not expecting this bug to be fixed. So we will have to
go with a peripheral API call. I would have really love to get this in
the same stanza however because, as Dave said very *loudly*, that
makes life much easier for us mobile developers.
Now we will have to wrap this in a soft transactional layer to ensure
we get all the proper data for a user.
But if this is you final answer than we must go with that...
> David,
> As with any solution there are compromises (the normal big three are time,
> resources, quality of service) and while this change may make your
> particular use of the API more difficult, it is not only important but also
> necessary given our architecture and growth. The API provides Twitter data
> in a format that is consistent with our strengths. It is up to the consuming
> application to make the data we freely provide useful in its independent
> context. This decoupling of data and application allows us to focus on data
> delivery while the developer attends to user experience. We aim to maximize
> performance for board array of use-cases and while at the same time
> minimizing operational and maintenance costs.
> There are many very successful mobile applications that run a proxy to get
> around the resource/time trade-off that this deprecation creates. If you are
> mobile heavy, it is suggested you do the same. A proxy is highly recommended
> for iPhone apps because it insulates the application changes in the Twitter
> API with the App Store acceptance delay.
> If anyone has an open source Twitter API proxy, please start another thread
> so mobile developers like David do not have to reinvent the wheel. In fact,
> there should be a FOSS project for mobile devs to rely on -- I've got a
> couple ideas to contribute. Again, please start a thread (and link back
> here) if you have code or interest in starting a proxy project.
> On Mon, May 11, 2009 at 4:38 PM, Dave Mc <davidmccorm...@gmail.com> wrote:
> > To be blunt this is very unsatisfactory. Once again you guys are not
> > being at all cognisant of the requirements of mobile Twitter client
> > apps. These face much bigger problems than just the rate limit. They
> > are constrained by physical limitations such as battery life, latency
> > and bandwidth. And they also have to take account of carrier data
> > charges. Every time something in your API requires an additional
> > method call you are making life difficult for us mobile app developers
> > who are trying to deliver a quality Twitter client to our users (who
> > are also your users!).
> > What annoys me too is that whenever a mobile-specific issue is raised
> > your stock response is "handle that in a proxy". Guys, that's just not
> > good enough. The World is going mobile and the continuing development
> > of your API needs to take account of this.
> > Very unhappy about this!
> > On May 11, 10:18 pm, Doug Williams <d...@twitter.com> wrote:
> > > Issues 419 [1] and 474 [2] are very popular, in the painful kind of way.
> > The
> > > defects report that methods returning user objects (see users/show for an
> > > example [3]) are returning incorrect or invalid values for the
> > <following>
> > > element.
> > > The fix for this inconsistency is in fact non trivial [4]. The problem
> > lies
> > > within the interaction of the application logic, caching layer and
> > database
> > > design. The persistent data behind <following> and <notification> values
> > are
> > > separate from the user data in our architecture, so to keep these
> > elements
> > > valid in cache alongside user objects adds a large amount of complexity.
> > > Developers made it obvious that these data are a priority and we want to
> > > ensure they available. We also want to guarantee they are accurate and
> > that
> > > performance remains good. Given the problems explained above, we are
> > going
> > > to be making a number of changes to the API so that you can rely on the
> > > <following> or <notification> data.
> > > Deprecations:
> > > The following elements are to be removed from all returned user objects
> > > returned by the API:
> > > 1) <following>
> > > 2) <notifications>
> > > This deprecation will not occur until we finish the following:
> > > Additions:
> > > To continue to provide access to this data we will be creating a new
> > method:
> > > Issue 532 [4] outlines the need to perform a mutual following lookup. We
> > > will use a method similar to that described in this issue to deliver
> > > <following>, <followedby>, <notification> and <pending> (in the case of
> > > protected users) data with a single call.
> > > We realize this change will cause an increase in API usage for some
> > > applications. Therefore we are going to increase the default API rate
> > limit
> > > across the board. This should help absorb some of the costs for
> > applications
> > > attempting to do interesting things with social graph data. The number
> > will
> > > be somewhere between 101 and 200 calls but we still need to look at
> > growth
> > > projections and current hardware capacity before settling on a definite
> > > number.
> > > We plan to begin work on this relatively soon with the fix coming in a
> > few
> > > weeks. We do not have a planned ship date at this time but will
> > communicate
> > > specifics with developers as they are determined. We anticipate the new
> > > number of calls and a documented schema for the new method will be made
> > > available before the new method ships. Please watch this thread and
> > > @twitterapi for the incremental details.
I appreciate your taking the time to respond to my criticism. I
understand where you're coming from and if I was you I would probably
take the same view. I'm slow to go down the proxy route because this
would require server resources and, like most Twitter clients, my app
is non-commercial. Another issue with use of a proxy is that
Enterprise-managed mobile devices (e.g. BlackBerry devices that use a
BES) are often restricted to a white list of HTTP endpoints. It is
much easier to persuade an admin to allow twitter.com:80 than
proxy.dodgy.com:80.
Hey, here's a cheeky suggestion for Twitter: provide free VM-based
hosting for mobile proxies...
Regards,
Dave.
On May 12, 1:14 am, Doug Williams <d...@twitter.com> wrote:
> David,
> As with any solution there are compromises (the normal big three are time,
> resources, quality of service) and while this change may make your
> particular use of the API more difficult, it is not only important but also
> necessary given our architecture and growth. The API provides Twitter data
> in a format that is consistent with our strengths. It is up to the consuming
> application to make the data we freely provide useful in its independent
> context. This decoupling of data and application allows us to focus on data
> delivery while the developer attends to user experience. We aim to maximize
> performance for board array of use-cases and while at the same time
> minimizing operational and maintenance costs.
> There are many very successful mobile applications that run a proxy to get
> around the resource/time trade-off that this deprecation creates. If you are
> mobile heavy, it is suggested you do the same. A proxy is highly recommended
> for iPhone apps because it insulates the application changes in the Twitter
> API with the App Store acceptance delay.
> If anyone has an open source Twitter API proxy, please start another thread
> so mobile developers like David do not have to reinvent the wheel. In fact,
> there should be a FOSS project for mobile devs to rely on -- I've got a
> couple ideas to contribute. Again, please start a thread (and link back
> here) if you have code or interest in starting a proxy project.
> On Mon, May 11, 2009 at 4:38 PM, Dave Mc <davidmccorm...@gmail.com> wrote:
> > To be blunt this is very unsatisfactory. Once again you guys are not
> > being at all cognisant of the requirements of mobile Twitter client
> > apps. These face much bigger problems than just the rate limit. They
> > are constrained by physical limitations such as battery life, latency
> > and bandwidth. And they also have to take account of carrier data
> > charges. Every time something in your API requires an additional
> > method call you are making life difficult for us mobile app developers
> > who are trying to deliver a quality Twitter client to our users (who
> > are also your users!).
> > What annoys me too is that whenever a mobile-specific issue is raised
> > your stock response is "handle that in a proxy". Guys, that's just not
> > good enough. The World is going mobile and the continuing development
> > of your API needs to take account of this.
> > Very unhappy about this!
> > On May 11, 10:18 pm, Doug Williams <d...@twitter.com> wrote:
> > > Issues 419 [1] and 474 [2] are very popular, in the painful kind of way.
> > The
> > > defects report that methods returning user objects (see users/show for an
> > > example [3]) are returning incorrect or invalid values for the
> > <following>
> > > element.
> > > The fix for this inconsistency is in fact non trivial [4]. The problem
> > lies
> > > within the interaction of the application logic, caching layer and
> > database
> > > design. The persistent data behind <following> and <notification> values
> > are
> > > separate from the user data in our architecture, so to keep these
> > elements
> > > valid in cache alongside user objects adds a large amount of complexity.
> > > Developers made it obvious that these data are a priority and we want to
> > > ensure they available. We also want to guarantee they are accurate and
> > that
> > > performance remains good. Given the problems explained above, we are
> > going
> > > to be making a number of changes to the API so that you can rely on the
> > > <following> or <notification> data.
> > > Deprecations:
> > > The following elements are to be removed from all returned user objects
> > > returned by the API:
> > > 1) <following>
> > > 2) <notifications>
> > > This deprecation will not occur until we finish the following:
> > > Additions:
> > > To continue to provide access to this data we will be creating a new
> > method:
> > > Issue 532 [4] outlines the need to perform a mutual following lookup. We
> > > will use a method similar to that described in this issue to deliver
> > > <following>, <followedby>, <notification> and <pending> (in the case of
> > > protected users) data with a single call.
> > > We realize this change will cause an increase in API usage for some
> > > applications. Therefore we are going to increase the default API rate
> > limit
> > > across the board. This should help absorb some of the costs for
> > applications
> > > attempting to do interesting things with social graph data. The number
> > will
> > > be somewhere between 101 and 200 calls but we still need to look at
> > growth
> > > projections and current hardware capacity before settling on a definite
> > > number.
> > > We plan to begin work on this relatively soon with the fix coming in a
> > few
> > > weeks. We do not have a planned ship date at this time but will
> > communicate
> > > specifics with developers as they are determined. We anticipate the new
> > > number of calls and a documented schema for the new method will be made
> > > available before the new method ships. Please watch this thread and
> > > @twitterapi for the incremental details.
Can I suggest that you add that this value is unused and will be
removed to the API documentation. I have spent quite a while writing
code which I found out wasnt working cos the values are random and
unreliable and then searched around to eventually find this post and
find out it is a problem that wont be fixed. Thanks.
> I believe that "following" is supposed to indicate of the
> authenticating user is following the requested user, but even if it's
> the other way around, it seems wrong. Am I missing something, though?
> On May 11, 5:18 pm, Doug Williams <d...@twitter.com> wrote:
> > Issues 419 [1] and 474 [2] are very popular, in the painful kind of way. The
> > defects report that methods returning user objects (see users/show for an
> > example [3]) are returning incorrect or invalid values for the <following>
> > element.
> > The fix for this inconsistency is in fact non trivial [4]. The problem lies
> > within the interaction of the application logic, caching layer and database
> > design. The persistent data behind <following> and <notification> values are
> > separate from the user data in our architecture, so to keep these elements
> > valid in cache alongside user objects adds a large amount of complexity.
> > Developers made it obvious that these data are a priority and we want to
> > ensure they available. We also want to guarantee they are accurate and that
> > performance remains good. Given the problems explained above, we are going
> > to be making a number of changes to the API so that you can rely on the
> > <following> or <notification> data.
> > Deprecations:
> > The following elements are to be removed from all returned user objects
> > returned by the API:
> > 1) <following>
> > 2) <notifications>
> > This deprecation will not occur until we finish the following:
> > Additions:
> > To continue to provide access to this data we will be creating a new method:
> > Issue 532 [4] outlines the need to perform a mutual following lookup. We
> > will use a method similar to that described in this issue to deliver
> > <following>, <followedby>, <notification> and <pending> (in the case of
> > protected users) data with a single call.
> > We realize this change will cause an increase in API usage for some
> > applications. Therefore we are going to increase the default API rate limit
> > across the board. This should help absorb some of the costs for applications
> > attempting to do interesting things with social graph data. The number will
> > be somewhere between 101 and 200 calls but we still need to look at growth
> > projections and current hardware capacity before settling on a definite
> > number.
> > We plan to begin work on this relatively soon with the fix coming in a few
> > weeks. We do not have a planned ship date at this time but will communicate
> > specifics with developers as they are determined. We anticipate the new
> > number of calls and a documented schema for the new method will be made
> > available before the new method ships. Please watch this thread and
> > @twitterapi for the incremental details.
|
