Connection with Streaming API drops off after ~5 mins

[Cian Murphy]

I am using python-twitter with the Twitter Streaming API to try and collect tweets from the UK which are about a particular topic. The loop I'm using is:

while True:
    try:
        for item in api.GetStreamFilter(locations = UK_BB, stall_warnings = True):
            tweet = parse_tweet(item)
            if relevant(tweet):
                save_tweet(tweet)
    except Exception as err:
        logging.exception("Error at %s:", datetime.datetime.now().strftime("%A, %d. %B %Y %I:%M%p"))

This process works fine for about five minutes, saving relevant tweets. However after about five minutes, every time, data stops coming from the API. I don't receive any exceptions or any unusual messages from the stream. I don't have more than one connection open to the stream. The API credentials seem to be fine and there's nothing wrong with my connection. Restarting the connection will make it work for another five minutes, but then the problem will kick in again.

Does anyone have any idea what might be going wrong here?

[bear]

During the time right before the drop are their any tweets being
received? I'm trying to think if it is the python socket lib doing a
timeout or if it is the Twitter side doing it from a lack of data
flowing?

We may have to send a keep-alive somehow. I'm also wondering if we
need to look at how other streaming twitter libs do things because I'm
worried that the streaming part may be too simple. :/

> --
> You received this message because you are subscribed to the Google Groups
> "python-twitter" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to python-twitte...@googlegroups.com.
> To post to this group, send email to python-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/python-twitter.
> For more options, visit https://groups.google.com/d/optout.



--
Bear

be...@andyet.com (email)
bea...@gmail.com (email)
be...@bear.im (xmpp, email)
http://bear.im

PGP Fingerprint = 9996 719F 973D B11B E111 D770 9331 E822 40B3 CD29

[Cian Murphy]

There is a very steady flow of tweets before it drops since I'm receiving all UK tweets. But I tried implementing the same code in tweepy and have run into the same problem, so it looks like this isn't a problem with python-twitter! I guess I'll have to keep looking - possibly a network problem or something with the credentials.

Thanks,

Cian

[mark.s...@4dpipeline.com]

Any luck Cian, or is just restarting every 5 mins the only solution ?