First off, thank you all for making the Twitter API the success it is. Our API is responsible for a staggering amount of daily traffic; many times all of our web, IM, and SMS traffic put together. That's great news.
The bad news is that we have to take steps against those users that are abusing the API to ensure its continued availability. Starting early next week we'll be deploying some simple API rate limiting. Every authenticated user gets 1440 (one thousand four hundred and forty) requests per 24 hour period, starting from their first request.
To put this in perspective, that's enough to make one authenticated request to the Twitter API every minute, all day, every day. Requests to API methods that don't require authentication are not rate limited because they're easily cached, and thus not a big load on our servers.
If, for some terrifying reason, you're concerned about needing more than 1440 requests per day, you have two options:
1. Start using Jabber/XMPP to receive updates and direct messages. We'll be writing up more information on this shortly, but a number of developers are already using it for bots and other projects.
2. Email me privately and ask to be put on a white list. Please include the screen name of the user you're authenticating with and an explanation of why you need to talk to the Twitter API more than once per minute.
I'm happy to answer any questions you may have about our API rate limiting. We're just trying to make sure that our service is highly available for legitimate uses and highly unavailable to abusers. Thanks for your understanding.
This is one of the problems I was concerned about in the early days of twitter API.
Using XMPP is a solution but it does requires Twitter to maintain a reliable jabber connection. Sadly, from experience polling from Jabber, I see a very high "drop rates" while receiving message from XMPP. Would twitter provide a more reliable XMPP interface or alt, one that is specifically for API developers?
I hope these issues are resolved before the rate limiting is put into operation (incidentally, when?)
> First off, thank you all for making the Twitter API the success it is. > Our API is responsible for a staggering amount of daily traffic; many > times all of our web, IM, and SMS traffic put together. That's great > news.
> The bad news is that we have to take steps against those users that > are abusing the API to ensure its continued availability. Starting > early next week we'll be deploying some simple API rate limiting. > Every authenticated user gets 1440 (one thousand four hundred and > forty) requests per 24 hour period, starting from their first request.
> To put this in perspective, that's enough to make one authenticated > request to the Twitter API every minute, all day, every day. Requests > to API methods that don't require authentication are not rate limited > because they're easily cached, and thus not a big load on our servers.
> If, for some terrifying reason, you're concerned about needing more > than 1440 requests per day, you have two options:
> 1. Start using Jabber/XMPP to receive updates and direct messages. > We'll be writing up more information on this shortly, but a number of > developers are already using it for bots and other projects.
> 2. Email me privately and ask to be put on a white list. Please > include the screen name of the user you're authenticating with and an > explanation of why you need to talk to the Twitter API more than once > per minute.
> I'm happy to answer any questions you may have about our API rate > limiting. We're just trying to make sure that our service is highly > available for legitimate uses and highly unavailable to abusers. > Thanks for your understanding.
> First off, thank you all for making the Twitter API the success it is. > Our API is responsible for a staggering amount of daily traffic; many > times all of our web, IM, and SMS traffic put together. That's great > news.
> The bad news is that we have to take steps against those users that > are abusing the API to ensure its continued availability. Starting > early next week we'll be deploying some simple API rate limiting. > Every authenticated user gets 1440 (one thousand four hundred and > forty) requests per 24 hour period, starting from their first request.
> To put this in perspective, that's enough to make one authenticated > request to the Twitter API every minute, all day, every day. Requests > to API methods that don't require authentication are not rate limited > because they're easily cached, and thus not a big load on our servers.
> If, for some terrifying reason, you're concerned about needing more > than 1440 requests per day, you have two options:
> 1. Start using Jabber/XMPP to receive updates and direct messages. > We'll be writing up more information on this shortly, but a number of > developers are already using it for bots and other projects.
> 2. Email me privately and ask to be put on a white list. Please > include the screen name of the user you're authenticating with and an > explanation of why you need to talk to the Twitter API more than once > per minute.
> I'm happy to answer any questions you may have about our API rate > limiting. We're just trying to make sure that our service is highly > available for legitimate uses and highly unavailable to abusers. > Thanks for your understanding.
> First off, thank you all for making the Twitter API the success it is. > Our API is responsible for a staggering amount of daily traffic; many > times all of our web, IM, and SMS traffic put together. That's great > news.
> The bad news is that we have to take steps against those users that > are abusing the API to ensure its continued availability. Starting > early next week we'll be deploying some simple API rate limiting. > Every authenticated user gets 1440 (one thousand four hundred and > forty) requests per 24 hour period, starting from their first request.
> To put this in perspective, that's enough to make one authenticated > request to the Twitter API every minute, all day, every day. Requests > to API methods that don't require authentication are not rate limited > because they're easily cached, and thus not a big load on our servers.
> If, for some terrifying reason, you're concerned about needing more > than 1440 requests per day, you have two options:
> 1. Start using Jabber/XMPP to receive updates and direct messages. > We'll be writing up more information on this shortly, but a number of > developers are already using it for bots and other projects.
> 2. Email me privately and ask to be put on a white list. Please > include the screen name of the user you're authenticating with and an > explanation of why you need to talk to the Twitter API more than once > per minute.
> I'm happy to answer any questions you may have about our API rate > limiting. We're just trying to make sure that our service is highly > available for legitimate uses and highly unavailable to abusers. > Thanks for your understanding.
> The bad news is that we have to take steps against those users that > are abusing the API to ensure its continued availability.
Is it people who are just hitting the servers over and over? Or ... ?
> Starting > early next week we'll be deploying some simple API rate limiting. > Every authenticated user gets 1440 (one thousand four hundred and > forty) requests per 24 hour period, starting from their first request.
I think this is a good idea, but I can see one population getting dinged inadvertently -- if they leave, for example, a client running on one computer and then go and start up another one somewhere else, they could easily hit that number and more if they forget. I, too, am guilty of that, but people may wonder why Twitterrific suddenly stopped working and start blaming clients.
-- ------------------------------------ personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- Everywhere is within walking distance if you have the time. ----------------
On Jun 23, 7:27 am, Cameron Kaiser <spec...@floodgap.com> wrote:
> [...] > I think this is a good idea, but I can see one population getting > dinged inadvertently -- if they leave, for example, a client running on > one computer and then go and start up another one somewhere else, they > could easily hit that number and more if they forget. I, too, am guilty > of that, but people may wonder why Twitterrific suddenly stopped working > and start blaming clients.
Maybe that encourages developers of Twitter clients to implement some intelligence in the client, e.g. stall hitting the server as long as the user is idle or the screensaver is running.
> The bad news is that we have to take steps against those users that > are abusing the API to ensure its continued availability. Starting > early next week we'll be deploying some simple API rate limiting. > Every authenticated user gets 1440 (one thousand four hundred and > forty) requests per 24 hour period, starting from their first request.
The objection to releasing this was that it would have a big impact on the database, but by limiting the amount of requests over a given time would surely be a way to enable this functionality without overloading the database servers, not?
> > I think this is a good idea, but I can see one population getting > > dinged inadvertently -- if they leave, for example, a client running on > > one computer and then go and start up another one somewhere else, they > > could easily hit that number and more if they forget. I, too, am guilty > > of that, but people may wonder why Twitterrific suddenly stopped working > > and start blaming clients.
> Maybe that encourages developers of Twitter clients to implement some > intelligence in the client, e.g. stall hitting the server as long as > the user is idle or the screensaver is running.
This would be very hard to do in a web browser instance or running remotely.
Someone suggested ratelimiting over a shorter period of time rather than over a whole day. I think I agree with that because if you accidentally screw up under the currently proposed ratelimiting arrangement, you're toast on Twitter for the remainder of the 24 hour period. Restricting people to, say, 60 requests an hour would mean you're only down for *that* count.
This would also mean that checking DMs and regular posts would count as two, using the count up faster than usual.
I still want to reiterate that this is a good idea and no doubt necessary -- as long as people had a chance to modify their behaviour to stay under the limit, and that the message returned was something greppable or at least machine grokkable.
Alternatively, could something be added to the API to query "accesses left"? As long as it doesn't count to the total, of course :)
-- ------------------------------------ personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- I like being single. I'm always there when I need me. -- Art Leo -----------
> > > I think this is a good idea, but I can see one population getting > > > dinged inadvertently -- if they leave, for example, a client running on > > > one computer and then go and start up another one somewhere else, they > > > could easily hit that number and more if they forget. I, too, am guilty > > > of that, but people may wonder why Twitterrific suddenly stopped working > > > and start blaming clients.
> > Maybe that encourages developers of Twitter clients to implement some > > intelligence in the client, e.g. stall hitting the server as long as > > the user is idle or the screensaver is running.
> This would be very hard to do in a web browser instance or running remotely.
> Someone suggested ratelimiting over a shorter period of time rather than over > a whole day. I think I agree with that because if you accidentally screw up > under the currently proposed ratelimiting arrangement, you're toast on Twitter > for the remainder of the 24 hour period. Restricting people to, say, 60 > requests an hour would mean you're only down for *that* count.
> This would also mean that checking DMs and regular posts would count as two, > using the count up faster than usual.
> I still want to reiterate that this is a good idea and no doubt necessary -- > as long as people had a chance to modify their behaviour to stay under the > limit, and that the message returned was something greppable or at least > machine grokkable.
> Alternatively, could something be added to the API to query "accesses left"? > As long as it doesn't count to the total, of course :)
> -- > ------------------------------------ personal: http://www.cameronkaiser.com/ -- > Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com > -- I like being single. I'm always there when I need me. -- Art Leo -----------
> I like the 60 requests an hour idea. Going with that instead.
> On 6/23/07, Cameron Kaiser <spec...@floodgap.com> wrote:
> > > > I think this is a good idea, but I can see one population getting > > > > dinged inadvertently -- if they leave, for example, a client running on > > > > one computer and then go and start up another one somewhere else, they > > > > could easily hit that number and more if they forget. I, too, am guilty > > > > of that, but people may wonder why Twitterrific suddenly stopped working > > > > and start blaming clients.
> > > Maybe that encourages developers of Twitter clients to implement some > > > intelligence in the client, e.g. stall hitting the server as long as > > > the user is idle or the screensaver is running.
> > This would be very hard to do in a web browser instance or running remotely.
> > Someone suggested ratelimiting over a shorter period of time rather than over > > a whole day. I think I agree with that because if you accidentally screw up > > under the currently proposed ratelimiting arrangement, you're toast on Twitter > > for the remainder of the 24 hour period. Restricting people to, say, 60 > > requests an hour would mean you're only down for *that* count.
> > This would also mean that checking DMs and regular posts would count as two, > > using the count up faster than usual.
> > I still want to reiterate that this is a good idea and no doubt necessary -- > > as long as people had a chance to modify their behaviour to stay under the > > limit, and that the message returned was something greppable or at least > > machine grokkable.
> > Alternatively, could something be added to the API to query "accesses left"? > > As long as it doesn't count to the total, of course :)
> > -- > > ------------------------------------ personal:http://www.cameronkaiser.com/-- > > Cameron Kaiser * Floodgap Systems *www.floodgap.com* ckai...@floodgap.com > > -- I like being single. I'm always there when I need me. -- Art Leo -----------
> Looks like TwitterTroll.com is going to take a hit.... :(
> -Brad
> On Jun 25, 7:02 pm, "Alex Payne" <a...@al3x.net> wrote:
> > I like the 60 requests an hour idea. Going with that instead.
> > On 6/23/07, Cameron Kaiser <spec...@floodgap.com> wrote:
> > > > > I think this is a good idea, but I can see one population getting > > > > > dinged inadvertently -- if they leave, for example, a client running on > > > > > one computer and then go and start up another one somewhere else, they > > > > > could easily hit that number and more if they forget. I, too, am guilty > > > > > of that, but people may wonder why Twitterrific suddenly stopped working > > > > > and start blaming clients.
> > > > Maybe that encourages developers of Twitter clients to implement some > > > > intelligence in the client, e.g. stall hitting the server as long as > > > > the user is idle or the screensaver is running.
> > > This would be very hard to do in a web browser instance or running remotely.
> > > Someone suggested ratelimiting over a shorter period of time rather than over > > > a whole day. I think I agree with that because if you accidentally screw up > > > under the currently proposed ratelimiting arrangement, you're toast on Twitter > > > for the remainder of the 24 hour period. Restricting people to, say, 60 > > > requests an hour would mean you're only down for *that* count.
> > > This would also mean that checking DMs and regular posts would count as two, > > > using the count up faster than usual.
> > > I still want to reiterate that this is a good idea and no doubt necessary -- > > > as long as people had a chance to modify their behaviour to stay under the > > > limit, and that the message returned was something greppable or at least > > > machine grokkable.
> > > Alternatively, could something be added to the API to query "accesses left"? > > > As long as it doesn't count to the total, of course :)
> > > -- > > > ------------------------------------ personal:http://www.cameronkaiser.com/-- > > > Cameron Kaiser * Floodgap Systems *www.floodgap.com*ckai...@floodgap.com > > > -- I like being single. I'm always there when I need me. -- Art Leo -----------
> I like the 60 requests an hour idea. Going with that instead.
The only other question I have (dunno about others) is if this applies to hitting personal timeline or all authenticated actions (the initial announcement implied the latter). Since there are I'm sure many people with their Twitterrifics and TTYtters set to one-hit-a-minnit, posting at the beginning of a particular hour would mean that they would not be able to get updates at the end proportional to their activity.
I see a couple solutions to this:
- Make the recommended interval, say, two minutes, and keep the 60 req/hr. Reinforce this with two minute caching so people get no benefit from banging the timeline more rapidly. This means that most people will hit it 30 times an hour, and thus have 30 tweets to make in that time.
On the other hand, this will need some way of letting the user know how much "time" they have left with some sort of API call (which hopefully will not count towards the 60).
- Make tweets "free" (and only hits to view their timeline will actually cost). I support this option, and I imagine that timeline hits are by far a worse offender to load than actual tweets sent; I know there's a side benefit in muzzling spammy tweeters but I think this will annoy more than it will delight.
Naturally neither solution is exclusive ... both could be implemented simultaneously.
-- ------------------------------------ personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- Absence of evidence is not evidence of absence. -- Carl Sagan --------------
I'm a bit concerned about this. I have a bot that responds (as close to real time as possible) to direct messages, at the minute we run a script every 15 seconds which pulls in the new direct messages and sends whatever number of appropriate responses there are. Currently we're doing this with HTTP requests and as you can tell, we could be doing anything between 4 (to get the direct messages) and any larger number of requests a minute (to send whatever responses to new requests).
We tried moving to being a Jabber bot recently and had immense difficulty, in fact I even posted to the group and got this response from yourself:
"If you'd like some official word: we're working on Jabber and delivery stability, but it's an uphill battle. For that reason, Jabber/XMPP is not presently a supported platform for third-party development. I'm hoping this changes real soon :) "
> This is one of the problems I was concerned about in the early days of > twitter API.
> Using XMPP is a solution but it does requires Twitter to maintain a > reliable jabber connection. Sadly, from experience polling from > Jabber, I see a very high "drop rates" while receiving message from > XMPP. Would twitter provide a more reliable XMPP interface or alt, one > that is specifically for API developers?
> I hope these issues are resolved before the rate limiting is put into > operation (incidentally, when?)
> > First off, thank you all for making the Twitter API the success it is. > > Our API is responsible for a staggering amount of daily traffic; many > > times all of our web, IM, and SMS traffic put together. That's great > > news.
> > The bad news is that we have to take steps against those users that > > are abusing the API to ensure its continued availability. Starting > > early next week we'll be deploying some simple API rate limiting. > > Every authenticated user gets 1440 (one thousand four hundred and > > forty) requests per 24 hour period, starting from their first request.
> > To put this in perspective, that's enough to make one authenticated > > request to the Twitter API every minute, all day, every day. Requests > > to API methods that don't require authentication are not rate limited > > because they're easily cached, and thus not a big load on our servers.
> > If, for some terrifying reason, you're concerned about needing more > > than 1440 requests per day, you have two options:
> > 1. Start using Jabber/XMPP to receive updates and direct messages. > > We'll be writing up more information on this shortly, but a number of > > developers are already using it for bots and other projects.
> > 2. Email me privately and ask to be put on a white list. Please > > include the screen name of the user you're authenticating with and an > > explanation of why you need to talk to the Twitter API more than once > > per minute.
> > I'm happy to answer any questions you may have about our API rate > > limiting. We're just trying to make sure that our service is highly > > available for legitimate uses and highly unavailable to abusers. > > Thanks for your understanding.
The rate limiting is currently disabled while we field requests from users like you. Jabber stability has improved greatly since June 1st, and we'll be writing up a document on how to best make use of it for development purposes.
If you contact me off-list, I can place you on a white list until we've sorted out a better strategy for your particular application's API needs.
On 6/26/07, Colm McMullan <colm.mcmul...@gmail.com> wrote:
> I'm a bit concerned about this. I have a bot that responds (as close > to real time as possible) to direct messages, at the minute we run a > script every 15 seconds which pulls in the new direct messages and > sends whatever number of appropriate responses there are. Currently > we're doing this with HTTP requests and as you can tell, we could be > doing anything between 4 (to get the direct messages) and any larger > number of requests a minute (to send whatever responses to new > requests).
> We tried moving to being a Jabber bot recently and had immense > difficulty, in fact I even posted to the group and got this response > from yourself:
> "If you'd like some official word: we're working on Jabber and > delivery > stability, but it's an uphill battle. For that reason, Jabber/XMPP is > not presently a supported platform for third-party development. I'm > hoping this changes real soon :) "
> On Jun 23, 3:53 am, "James Seng" <j...@seng.sg> wrote: > > Hi Alex,
> > This is one of the problems I was concerned about in the early days of > > twitter API.
> > Using XMPP is a solution but it does requires Twitter to maintain a > > reliable jabber connection. Sadly, from experience polling from > > Jabber, I see a very high "drop rates" while receiving message from > > XMPP. Would twitter provide a more reliable XMPP interface or alt, one > > that is specifically for API developers?
> > I hope these issues are resolved before the rate limiting is put into > > operation (incidentally, when?)
> > > First off, thank you all for making the Twitter API the success it is. > > > Our API is responsible for a staggering amount of daily traffic; many > > > times all of our web, IM, and SMS traffic put together. That's great > > > news.
> > > The bad news is that we have to take steps against those users that > > > are abusing the API to ensure its continued availability. Starting > > > early next week we'll be deploying some simple API rate limiting. > > > Every authenticated user gets 1440 (one thousand four hundred and > > > forty) requests per 24 hour period, starting from their first request.
> > > To put this in perspective, that's enough to make one authenticated > > > request to the Twitter API every minute, all day, every day. Requests > > > to API methods that don't require authentication are not rate limited > > > because they're easily cached, and thus not a big load on our servers.
> > > If, for some terrifying reason, you're concerned about needing more > > > than 1440 requests per day, you have two options:
> > > 1. Start using Jabber/XMPP to receive updates and direct messages. > > > We'll be writing up more information on this shortly, but a number of > > > developers are already using it for bots and other projects.
> > > 2. Email me privately and ask to be put on a white list. Please > > > include the screen name of the user you're authenticating with and an > > > explanation of why you need to talk to the Twitter API more than once > > > per minute.
> > > I'm happy to answer any questions you may have about our API rate > > > limiting. We're just trying to make sure that our service is highly > > > available for legitimate uses and highly unavailable to abusers. > > > Thanks for your understanding.
> > I like the 60 requests an hour idea. Going with that instead.
> The only other question I have (dunno about others) is if this applies to > hitting personal timeline or all authenticated actions (the initial > announcement implied the latter). Since there are I'm sure many people > with their Twitterrifics and TTYtters set to one-hit-a-minnit, posting at > the beginning of a particular hour would mean that they would not be able > to get updates at the end proportional to their activity.
> I see a couple solutions to this:
> - Make the recommended interval, say, two minutes, and keep the 60 req/hr. > Reinforce this with two minute caching so people get no benefit from > banging the timeline more rapidly. This means that most people will hit it > 30 times an hour, and thus have 30 tweets to make in that time.
> On the other hand, this will need some way of letting the user know how > much "time" they have left with some sort of API call (which hopefully will > not count towards the 60).
> - Make tweets "free" (and only hits to view their timeline will actually > cost). I support this option, and I imagine that timeline hits are by far > a worse offender to load than actual tweets sent; I know there's a side > benefit in muzzling spammy tweeters but I think this will annoy more than > it will delight.
> Naturally neither solution is exclusive ... both could be implemented > simultaneously.
> -- > ------------------------------------ personal: http://www.cameronkaiser.com/ -- > Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com > -- Absence of evidence is not evidence of absence. -- Carl Sagan --------------
On 26 Jun, 11:02, "Alex Payne" <a...@al3x.net> wrote:
> I like the 60 requests an hour idea. Going with that instead.
I think the problem with 60/hour as opposed to 1440/24h is that most people sleep or are away from a computer during the 24hours. A typical working day I may be logged into twitter on two computers, both updating... on my mac with Twitterrific and Twitbin on my pc. I would much rather use my 1440 to keep updated during the hours I'm awake and online then to be 'wasting' the allocation for the hours I'm not.
My own twitter utility (a plugin for Yahoo! Messenger that keeps your status in sync as well as notify of friends updates http://www.wackyb.co.nz/Yahoo_Messenger_Twitter_Sync_Plugin ) makes two requests.. once for the friends (to notify of the latest tweets from your friends) and again for your own latest tweet (to update and sync your status). What would make this easier would be having a method that retrieves your latest friends tweets AND your own last tweet.
As you can see... for the 8 or so hours I may be using my computers and logged into the twitter service it's possible that with Twitterrific, Twitbin and plugins running that even with them all set to a low interval it would exceed that 1/min limit but would still be within the 1440/24h
> > I like the 60 requests an hour idea. Going with that instead.
> I think the problem with 60/hour as opposed to 1440/24h is that most > people sleep or are away from a computer during the 24hours.
This is a valid criticism, but my original objection to the 1440/24h is that if you accidentally blow it, you're dead for the whole 24 hours.
-- ------------------------------------ personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- What an incredible thing we did. -- R. J. Mical, Commodore-Amiga -----------
> > > I like the 60 requests an hour idea. Going with that instead.
> > I think the problem with 60/hour as opposed to 1440/24h is that most > > people sleep or are away from a computer during the 24hours.
> This is a valid criticism, but my original objection to the 1440/24h is > that if you accidentally blow it, you're dead for the whole 24 hours.
> -- > ------------------------------------ personal: > http://www.cameronkaiser.com/ -- > Cameron Kaiser * Floodgap Systems * www.floodgap.com * > ckai...@floodgap.com > -- What an incredible thing we did. -- R. J. Mical, Commodore-Amiga > -----------
> I agree with Andy. I understand the fear of "blowing it", but that will > probably happen on a small percentage of days.
> I'd much rather be able to give users better response times during peak > hours, and slow things down when traffic tends to be low.
However, the 1440/24h doesn't achieve this either. Yes, it lets you spread the access throughout the day, but it also lets everyone else do it, so peak hours will still have peak activity since people have access to burn and no penalty to do so.
-- ------------------------------------ personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- Homestar has a web site? -- Strong Bad -------------------------------------
Well there are two different issues here: what is best for Twitter and what is best for developers and users.
Alex's first proposal was 1440/24h, so it seems that they don't have a problem with that.
Users will have a better experience with bots that tend to answer more quickly. If Twitter can handle the load, we should let each developer tailor their service to be as enjoyable for users as possible.
> > I agree with Andy. I understand the fear of "blowing it", but that will > > probably happen on a small percentage of days.
> > I'd much rather be able to give users better response times during peak > > hours, and slow things down when traffic tends to be low.
> However, the 1440/24h doesn't achieve this either. Yes, it lets you spread > the access throughout the day, but it also lets everyone else do it, so > peak hours will still have peak activity since people have access to burn > and no penalty to do so.
> -- > ------------------------------------ personal: > http://www.cameronkaiser.com/ -- > Cameron Kaiser * Floodgap Systems * www.floodgap.com * > ckai...@floodgap.com > -- Homestar has a web site? -- Strong Bad > -------------------------------------