How are you retrieving your users' tweets?

[Mark Armendariz]

I'm working on 3 Twitter apps. For all of them, I have been following
the users when they sign up and then have a cronjob check the friends
timeline every minute. This has been working pretty well thus far
with relatively low membership, but I can see it getting hairy as the
membership grows. I'm sure that after x users, the friends timeline
will start to miss messages (beyond the 20 message threshold).

I figure an XMPP client would resolve this, but it seems that's going
to be down for a while. Would making a user archive request (with
since param) for every user be too much? I definitely don't want to
contribute to server load, but I have to make sure to aggregate all
our users' tweets.

How is everyone else going about this?

Thanks!!