is there a problem with the "since" parameter on friends_timelime, or behaviour change?
dean.j.robinson
changed the code in Hahlo since the weekend, and it had been working
fine up until now.
Problem is, friends timeline loads, upon first auto-refresh it returns
the most recent tweet (which had already been returned), this
continues (every 60 seoncds) until a new tweet appears, then that new
tweet is repeated etc etc. (Hope that makes sense).
Basically its like the api (when using the since param) is returning
all tweets after the 'since' time PLUS the tweet with the same time.
In other words looks like its doing >= instead of >
I don't know whether other people have had similar problems with other
api apps such as twitteriffic, twhirl etc, but I've seen a few people
mention it for hahlo today.
Cheers
Dean
Andy Price
wrote:
> all tweets after the 'since' time PLUS the tweet with the same time.
> In other words looks like its doing >= instead of >
>
> I don't know whether other people have had similar problems with other
> api apps such as twitteriffic, twhirl etc, but I've seen a few people
> mention it for hahlo today.
in Twyt for a while and it has recently started to show the message
matching the date passed in as the since parameter instead of only
returning the messages posted *since* that timestamp. Seems to me like
someone replaced a > with a >=
dean.j.robinson
crazy.
I've put a temporary fix in Hahlo to just skip the first tweet thats
returned by the auto refresh, its not perfect by any means, and
preferable if I didn't have to do it long term.
Mike Scott
If you've got a big issue with it then resolve it yourself by simply adding one second to the time?
Phyllis Diller - "Never go to bed mad. Stay up and fight."
dean.j.robinson
the behaviour of the parameter has changed. If it has that is fine, I
will modify my code, but I just want to find out whether is staying
like this or not first.
On Jun 7, 1:32 pm, "Mike Scott" <mic...@gmail.com> wrote:
> I can see reasoning for both methods, effectively the problem that could be
> foreseen is that the milliseconds between your request and the last tweet
> for that second could be different, meaning that despite the tweets and the
> last request having the same time in seconds, they could differ.
>
> If you've got a big issue with it then resolve it yourself by simply adding
> one second to the time?
>
Andrew Price
> I can see reasoning for both methods, effectively the problem that could be
> foreseen is that the milliseconds between your request and the last tweet
> for that second could be different, meaning that despite the tweets and the
> last request having the same time in seconds, they could differ.
nearest second meaning that any millisecond component should be zero.
Also your theory doesn't address the fact that the Twitter API has
clearly changed its behaviour recently. It's a simple conversation:
Client: Can I have the messages listed since:<timestamp> please?
Server: OK here are the messages where their timestamps is greater
than your since parameter.
So I don't see how the milliseconds field could affect the behaviour,
unless Twitter stores the timestamps with milliseconds for some silly
reason and actually takes them into account now when doing the
timestamp comparison. Maybe if the API spoke milliseconds to clients
it would make sense to take milliseconds into account but it doesn't -
it returns timestamps to the nearest second.
> If you've got a big issue with it then resolve it yourself by simply adding
> one second to the time?
documentation, I'm not going to compensate for breakage in the Twitter
API by changing the behaviour of my client program. Adding one second
to the time will quite obviously create rare yet possible black holes
where messages posted within one second will seem to disappear.
edl...@gmail.com
wrote:
> crazy.
>
> I've put a temporary fix in Hahlo to just skip the first tweet thats
> returned by the auto refresh, its not perfect by any means, and
> preferable if I didn't have to do it long term.
if a new tweet is a duplicate, it's discarded.
Ed
Kee Hinckley
> This doesn't make sense. The request always sends the timestamp to the
> nearest second meaning that any millisecond component should be zero.
> Also your theory doesn't address the fact that the Twitter API has
> clearly changed its behaviour recently. It's a simple conversation:
Clearly Twitter should not have changed the behavior without
notification.
However, >= is the only mechanism which is guaranteed not to drop
messages; it doesn't matter if we're talking milliseconds or seconds,
there is always going to be the possibility of two messages with the
same date stamp. So long as two messages can arrive in the same clock
instant, but one before, and the other after, your query, time
checking is not going to work reliability. Your only options are to
risk getting the same message twice, or risk dropping a message.
As with a number of other argu////discussions on the list--this is
another difference between a system that uses queues or inboxes, and
one which uses database queries.
For this reason, the only time we call "since" is when we want a broad
swath of messages (e.g. everything since yesterday)". If we really
want "the most recent messages I have seen" we store the highest id
seen for each API call for a given user, and ask for ids greater than
that. Since that mechanism can potentially return the same message
via two different API calls, we still check for and toss duplicate
messages.
An interesting side discussion is whether any of the functionality
offered by a pure-database solution is in fact interesting to anyone.
(E.g. add a friend, get all of their old messages). If not, a queue
system would presumably offer a number of efficiencies.
Evan Weaver
to revert to the old behavior? I think the race conditions are
probably too rare to be worth worrying about.
Evan
--
Evan Weaver
dean.j.robinson
With the old behaviour of the since parameter it only returned tweets
that were "newer" than the date/time passed through. If there were no
new tweets it returned the http status 302 and (I think) an empty
array which was great (I preferred it like this). Now its returning
the "newer" tweets and the one that matches the date/time that was
passed through.
I personally don't see the point in returning the tweet that matches
the passed through timestamp, but thats just me. If this is how its
going to work in the future I'll make the necessary change to Hahlo to
cater for it, just looking for a definitely answer one way or the
other.
cheers
Dean
Robert Fischer
losing a tweet over repeating tweets, since most users won't notice missing the occasional tweet,
but will definitely notice the occasional repeating tweet.
~~ Robert.
Ed Finkler
<robert....@smokejumperit.com> wrote:
>
> I'd much prefer the since parameter to mean >, not >=. I suspect most applications would favor
> losing a tweet over repeating tweets, since most users won't notice missing the occasional tweet,
> but will definitely notice the occasional repeating tweet.
Depends on the app, I think. I suspect desktop client users, and any
folks who aren't *expecting* to miss some content (like users with a
relatively small # of friends, which is the majority of Twitter
users), will be unhappy to lose a message. In such cases, I think it's
better to assume dupes will come through and ditch the ones you
already have.
--
Ed Finkler
http://funkatron.com
AIM: funka7ron
ICQ: 3922133
Skype: funka7ron
Robert Fischer
equally small. This issue mainly effects people with huge numbers of followers.
I don't know about other people, but I will out-and-out abandon the "since" feature if it is decided
to show dupes. If I have to check for dupes anyway, I'm going to have to start tracking ids. And
if I'm having to tracking ids anyway, then what's the point of tracking dates, too?
It sounds like the "since" feature was a clever idea that's flat-out broke.
~~ Robert.
Ed Finkler
<robert....@smokejumperit.com> wrote:
>
> If you have a relatively small number of friends, your odds of encountering this race condition are
> equally small. This issue mainly effects people with huge numbers of followers.
>
> I don't know about other people, but I will out-and-out abandon the "since" feature if it is decided
> to show dupes. If I have to check for dupes anyway, I'm going to have to start tracking ids. And
> if I'm having to tracking ids anyway, then what's the point of tracking dates, too?
Good point, and I don't use "since" at all. I've thought about it just
to lower transfer sizes/Twitter load, but I doubt Spaz makes much diff
there 8)
> It sounds like the "since" feature was a clever idea that's flat-out broke.
Well, let's not go over the top here. I think it's still a good idea.
Maybe just needs a bit o' tweak.
Robert Fischer
>
> Well, let's not go over the top here. I think it's still a good idea.
> Maybe just needs a bit o' tweak.
>
this catch-22.
Actually, if you were to abandon timestamps and have "since" mean "since id", that'd get you the
same functionality without the race condition.
~~ Robert.
Ed Finkler
<robert....@smokejumperit.com> wrote:
> What tweak, though? Unless tweets are guarantied to have unique timestamps, you're going to be in
> this catch-22.
Perhaps not such a clever idea, then? 8)
> Actually, if you were to abandon timestamps and have "since" mean "since id", that'd get you the
> same functionality without the race condition.
There ya go.
Cameron Kaiser
> "since id", that'd get you the same functionality without the race
> condition.
And that's the better approach.
--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * cka...@floodgap.com
-- Optimization hinders evolution. --------------------------------------------
Cameron Kaiser
> to lower transfer sizes/Twitter load, but I doubt Spaz makes much diff
> there 8)
TTYtter does use since with a last ID#. There's a bit of glitchiness which
is mostly overcome with an ID database to keep "stuttering" to a minimum.
--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * cka...@floodgap.com
-- The only thing to fear is fearlessness -- R. E. M. -------------------------
Robert Fischer
Got a link to TTYtter?
~~ Robert.
Cameron Kaiser
Surely; it's since_id.
http://twitter.com/statuses/friends_timeline.json?since_id=x
> Got a link to TTYtter?
http://www.floodgap.com/software/ttytter/
Hopefully a new version this week. Waiting to see what happens to rate limits
*cough*al3x?*cough*.
--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * cka...@floodgap.com
-- "I adore my Commodore 64" --------------------------------------------------
Cameron Kaiser
>
> Surely; it's since_id.
>
> http://twitter.com/statuses/friends_timeline.json?since_id=x
Bleh. This doesn't seem to work for friends_timeline anymore. It *is*
documented for public_timeline, however, so I defer on this one with
apologies.
--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * cka...@floodgap.com
-- Why was I born with such contemporaries? -- Oscar Wilde --------------------
Evan Weaver
Our first priority was capacity (witness today's uptime). Regressions
in the API are unintentional and will be fixed. In addition, every
resource should eventually get both "since" and "since_id" parameters.
We would prefer to drop the "since" (the timestamp one) and only use
"since_id", but it seems like a lot of people use the timestamp
"since".
Evan
--
Evan Weaver
Cameron Kaiser
> Our first priority was capacity (witness today's uptime). Regressions
> in the API are unintentional and will be fixed.
Whew! Thought I was going nuts there for a second. Thanks for the reply. :)
--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * cka...@floodgap.com
-- BOND THEME NOW PLAYING: "For Your Eyes Only" -------------------------------
dean.j.robinson
stops the duplicates. So long as if/when you decide to drop 'since'
that you give us developers sufficient notice to allow us to make the
necessary changes.
thanks for the updates Evan.
cheers
Dean
On Jun 10, 2:11 pm, Cameron Kaiser <spec...@floodgap.com> wrote:
> > We are in the process of unifying a lot of the timeline handling code.
> > Our first priority was capacity (witness today's uptime). Regressions
> > in the API are unintentional and will be fixed.
>
> Whew! Thought I was going nuts there for a second. Thanks for the reply. :)
>
> --
> ------------------------------------ personal:http://www.cameronkaiser.com/--
maineone
too rare to be worth worrying about".
race condition will not happen,
it will within 15 minutes. My friends and I are all actively
updating our status and there were 21 tweets in since my last
check and the last two came in at the same second. That's really not
so unlikely.
The bigger issue is that if you are willing to accept, by design that
Tweets can be dropped, then
the system becomes unreliable and untrustworthy. Especially if there
is no good reason for losing data.
the Since_ID parameter solves the problem.
Andrew Price
> We are in the process of unifying a lot of the timeline handling code.
> Our first priority was capacity (witness today's uptime). Regressions
> in the API are unintentional and will be fixed. In addition, every
> resource should eventually get both "since" and "since_id" parameters.
fellow. Thanks for clearing it up.
> We would prefer to drop the "since" (the timestamp one) and only use
> "since_id", but it seems like a lot of people use the timestamp
> "since".
replaced with since_id and made a lot better and user friendly so
that's fine for me. Consistency is good.
Have a nice day.
Kee Hinckley
> Having a since_id in the friends timeline will make me one happy
> fellow. Thanks for clearing it up.
Agreed, but in the meantime you can simply implement the functionality
yourself at a lower level, and know that you're code will keep working
when Twitter makes the change.
# 20 most recent IN LAST 24 HOURS
friends_timeline => {
_url => '/statuses/friends_timeline',
_type => 'GET',
_auth => REQUIRED,
_cost => 1,
_class => 'update',
id => OPTIONAL|OBSOLETE,
since => OPTIONAL,
page => OPTIONAL|UNAVAILABLE,
since_id => OPTIONAL|EXTENSION,
},
I added since_id as an extension. The code in the library API checks
everything returned and makes sure nothing is older (or equal to :-)
the since_id, and tosses the rest. Voila, it's supported via the API
until Twitter has a chance to fix it.
Evan Weaver
I'd like to remove the 24 hour constraint, too, btw. This would be a
permanent change. Instead it would be the same as web, which is
currently 1 week but soon will be upped to 1 month.
In general we're trying to converge web and API functionality, which
will include pagination at some point in the API methods.
Evan
--
Evan Weaver