A hole in twitter

[Jonathan Hoyt]

Monday when I woke up freedns.afraid.org was down. The first thing I
did was jump on twitter and do a search for freedns. Sure enough,
other users were reporting it. Two hours later I searched twitter
again because I noticed freedns was back up, and sure enough, 45min
previous someone had tweeted that it was working again.

If I could have "followed" that twitter search and been notified via
sms as soon as there were more search results for freedns, I would
have known a lot sooner that all my websites were back up and running.

So what do you guys think about a twitter service/user that you can
follow, and you can @ or dm a search term to it, and it dm's you back
when there are new search results for that term? Here, let me explain
work flow:

1. I want to know on my laptop, or phone, or wherever, as soon as
there is a new mention on twitter about "Natalie Portman"
2. I tweet "@notifiy_me Natalie Portman"
3. @notify_me dm's me as soon as there are any new tweets about
Natalie Poortman, and since I have @notify_me set to sms me, I get the
notification immediately

Does this sounds like something useful? Or am I just a crackpot?

Jon Hoyt

[Joe Van Overberghe]

That could be dangerous... "@notify_me Micheal Jackson" :-P

I could see uses for it, but you would definitely need some rate controls in there.

[Jonathan Hoyt]

I was thinking if there were more than a few results in 60 seconds, it would just dm you a link to the twitter search itself :-) The more we research the less we think we can do it. There are just too many constraints on the twitter api and twitter search api usage.

We kind of need it for a site we are doing tho, so if someone knows of a way, let us know :-)

Jon

[John Nunemaker]

You can get your app IP white listed and you get up to 10k searches a day or something. I've thought about building a service like this. Sucky thing is keeping followers/following synced up with who is *actually* using the service. If they get too out of whack, you can't follow any more people until you get them back in line. And alas the only way to send a DM is through a mutual relationship.

[Steve Smith]

Interestingly enough if you use Oauth, you can use the users account to send a DM to themself. Just an idea. 

Sent from my iPhone

[John Nunemaker]

Stellar idea. Although I don't know if you can dm yourself. 

[Steve Smith]

In fact you *can* DM yourself.  And that's not a euphemism. 

[John Nunemaker]

Huh. Didn't know that. Interesting.

[Daniel Parker]

Theoretically, we could use gnip to follow a twitter search, and get a much higher ping frequency, or even have them post the updates to us. And, we can supposedly follow up to 25,000 searches for free. I say theoretically, because I tried to set it up and couldn't get it to work.

We could also do the SMS ourselves simply via email to people's carriers. Or just make it an email service with a little javascript helper control to generate your phone's SMS email address from your carrier and number.

Would posting to "yourself" via OAuth circumvent the frequency limiting?

- daniel parker -

"You have granted me life and steadfast love, and your care has preserved my spirit." Job 10:12
"The LORD is my chosen portion and my cup . . . indeed, I have a beautiful inheritance." Psalm 16:5-6
"Give what you can ... take nothing back!"

[Jonathan Hoyt]

You told me the api was limited by IP, not by username

Jon

[Daniel Parker]

http://apiwiki.twitter.com/Twitter-REST-API-Method%3A-direct_messages%C2%A0new

Direct Messages are NOT rate-limited!

So I think we can handle the whole thing using gnip to follow the searches (up to 25,000 of them before we'd have to pay $1K/month), and then posting DM as fast as we get search updates.

- daniel parker -

"You have granted me life and steadfast love, and your care has preserved my spirit." Job 10:12
"The LORD is my chosen portion and my cup . . . indeed, I have a beautiful inheritance." Psalm 16:5-6
"Give what you can ... take nothing back!"

[Daniel Parker]

Some limits are still imposed: http://help.twitter.com/forums/10711/entries/15364

• 250 total direct messages per day

I think that's for sending, and per user. That would mean we'd have to use the OAuth method and have the user DM themself - and only up to 250/day.

Some of the limits are a little vague...

- daniel parker -

[Jon Warner]

Originally there was the possibility built into twitter itself for
that... I remember having it set up for "goshen" at one point. I think
they removed that functionality, though.. But while searching for how
to do that, I found this information:

You can follow a feed a search results page like so
http://search.twitter.com/search?q=freedns

You can also apparently do this in a client with tweetdeck and twirl.
I think spaz does this as well, come to think of it.
http://searchengineland.com/how-to-track-keyword-based-tweets-16519

There is also a service called twitterfeed that lets you turn a feed
into a twitter user
www.twitterfeed.com


So it seems like a need that is partially filled You would need to
make it easier to use or more attractive. Maybe something that lets
you do twitterfeed, but autocreate the user account. Maybe it does
that, I didn't look into it too deeply.

And I'm curious why twitter removed that functionality in the first
place, maybe too much processing or schema changes or something. does
anyone else remember what I'm talking about? Alec, I think you were
doing the same thing with 'Goshen'.

--
--
Jonathan Warner
Developer
574-534-6654

[Alec Hipshear]

You're talking about the track feature, right?

I had texted 'track goshen' to Twitter, and anytime someone mentioned "goshen" it would text me.  That got more than a little overwhelming very quickly.  I can't imagine what "track chicago" would have been like.  It didn't DM you specifically, it came through like a normal Tweet.

This is pretty much what you guys are talking about, right?  Twitter must have canceled it due to bandwidth considerations - just tracking goshen probably doubled my incoming tweets.

On Thu, Jul 9, 2009 at 9:27 AM, Alec Hipshear <alechi...@gmail.com> wrote:

You're talking about the track feature, right?

I had texted 'track goshen' to Twitter, and anytime someone mentioned "goshen" it would text me.  That got more than a little overwhelming very quickly.  I can't imagine what "track chicago" would have been like.  It didn't DM you specifically, it came through like a normal Tweet.

This is pretty much what you guys are talking about, right?  Twitter must have canceled it due to bandwidth considerations - just tracking goshen probably doubled my incoming tweets.

[John Nunemaker]

They removed it for scaling reasons. It was a big reverse index or
something that worked for a while but blew up when they started
growing. I loved track.

[Daniel Parker]

Yeah, that's right, they had that feature didn't they? So they cancelled it? I guess that's what we're up against. But I think we have a solution:

• Watch twitter searches via gnip (up to 25,000 searches at a time)

• PM yourself via OAuth whenever an update to your watched searches appears

Right now this solution should work for our isolated case (watching just one search that is relevant for all our users) by using the regular twitter search API to watch the one search. Some of our users won't have twitter accounts too, so I'll probably be emailing them or emailing their carrier SMS email.

The service I'm creating is http://twitter.com/geofreecycle. It will be a hosted listings site, but twitters all listings and also watches the twitter search for listings properly twittered by users. All listings are geocoded by location, and this whole search question is because we need a way to allow people to receive SMS/email updates only from nearby freecycle listings. It's in development right now, but it's most of the way there. (Jon and I did a one-day hackathon yesterday.) If you like freecycling, follow @geofreecycle and at some point I'll have it follow you back and PM you when it's ready for use.

Jon Hoyt's original question stemmed off of this need, but twitter might not allow it the scaling it would need to watch thousands of searches and PM everybody. Perhaps we could still do it using gnip, and at least email, possibly PM or raw SMS.

- daniel parker -

[William H. Harle Jr.]

http://mashable.com/2009/07/10/tidytweet/f

This was just posted and seems to have a lot of functionality you are
looking for. It's only in private beta now, but still.

Regards,

William H. Harle Jr.
Owner, Web Developer
90 Percent Gravity LLC
856.655.2769
http://90percentgravity.com