Ruby Wrapper

[Jonathan Rudenberg]

Hey Guys,

Not sure if anyone else uses Ruby, but I just whipped up a quick'n'dirty
wrapper that might be useful: https://gist.github.com/762761

Can anyone confirm that whether the adherence seconds needs to be added
or subtracted from the scheduled seconds?

Cheers,

Jonathan

[swf...@gmail.com]

It would be really good to have the answer to Jonathon's question.   

m

I'm in the UK at the moment, so can't do field experiments with my app:

http://ottawa-transpo.appspot.com/

I've done a bit more work on this, decided that for phone use the simple text interface is best, but you can still get the map.   Currently, if it knows your location it shows you all stops within 500m, or you can enter your favourite stop number.

If anyone can use this to answer Jonathon's question about adherence that would be great.

It would also be good to get some resolution to the problem of devices on the buses having their own clocks which are not synced.   Any further progress on that front?

John

[Don Kelly]

On Sun, Jan 2, 2011 at 14:49, Jonathan Rudenberg <jona...@titanous.com> wrote:
> Can anyone confirm that whether the adherence seconds needs to be added or
> subtracted from the scheduled seconds?
>

My field testing suggests that negative "adherence" represents
"lateness". Therefore subtraction if you're telling the user an
absolute arrival time.

Even following this rule, I still see "arriving in -5 minutes"
sometimes. This /might/ be due to octranspo's clock skew, so I'd like
confirmation of this from the octranspo implementors.

--
karfai [AT] gmail.com
http://www.strangeware.ca
http://blog.strangeware.ca

[Dave Gallant]

can we get this confirmed?

from the minimal field testing I've done, I thought it was reversed.

Dave

--
Move and the Way will open.

[Eric Lussier]

Hi John,

Excellent app! The map link is very handy and you have the most important information displayed.

I was just sitting here watching the times for a bus that passes infront of my house and it seems like you discard the entry as soon as the scheduled time passes, not when the bus passes. Is it that the API call no longer returns info for that entry? I may be mistaken as the times get pretty jumpy when it's down to the last 10s of seconds. In any case, one very handy feature would be to leave the last two passed busses in the list. Perhaps just grey out the data and empty the ETA field but leave the late/early field. Knowing you've just missed a bus is useful, especially if it passed early.

If you're in the UK and want to do some testing, perhaps you can use one of the city's web cams to spot buses. I'm not sure how real-time those cams are, but perhaps it could be handy. Just a thought.

http://traffic.ottawa.ca/

I'm heading out to work today with this app in hand, so I'll do some testing while i'm on Rideau and report back.

Keep it up!

[swf...@gmail.com]

Hi Eric,

Thanks for the feedback and ideas.

I'm just using the data I get from the live data API calls.   It is those calls that don't return information once a bus gets past its scheduled arrival time.   There was some debate on the list about this.   The claim was that the API returns stuff up to scheduled arrival time +/- the adherence seconds, but that is clearly not the case -- too bad as that would give a way to know whether adherence has to be added or subtracted.

Really, it would be good if the API returned info for a few minutes after the bus has passed -- especially given there is also some uncertainty about the times being reported by buses due to clock sync issues.

I could try caching info, but that will get messy and not help with the first call the app gets for a stop -- best fix is to fix the API.

Meanwhile, I've been polling stop 3000 every 90 seconds to collect some adherence data.   My theory was that buses ought to be late more often than early (since they should not pass stops >before< scheduled time so being late is inevitable.   So far the data has a slight bias to negative values, with a mean of -64 seconds.   If my theory is correct, negative means late -- which means my code has it the wrong way round :(

John

[Eric Lussier]

Not sure if you were doing that polling over the last week, but something to consider is that the buses (and streets) were ridiculously empty last week. I've never seen drivers deliberately drive slowly before, so perhaps the next weeks data will show different results once people get back to work.

Another thing I noticed. My bus came early this morning and after I got on, it just sat there for 30s of so. I refreshed the page and the ETA came up as -20sec and the entry disappeared once we got goin. So perhaps the negative ETA numbers are indicating wait time before the bus moves on??

Eric

[Darren Kipp]

On one of the super snowy days in December I was able to easily conclude
that positive numbers mean the bus is late and negative numbers mean early.

I followed a bus I was on yesterday using a stop further down the
route. The driver seemed to either stop or drive slow any time the
number went over about -60 or so... Less than 60 seconds and it seemed
to be business as usual.

[swf...@gmail.com]

On Mon, Jan 3, 2011 at 5:28 PM, Eric Lussier <ericl...@gmail.com> wrote:

Not sure if you were doing that polling over the last week, but something to consider is that the buses (and streets) were ridiculously empty last week. I've never seen drivers deliberately drive slowly before, so perhaps the next weeks data will show different results once people get back to work.

Thanks, I only started monitoring it today, but I suspect you are right and it will get more interesting tomorrow.

 

Another thing I noticed. My bus came early this morning and after I got on, it just sat there for 30s of so. I refreshed the page and the ETA came up as -20sec and the entry disappeared once we got goin. So perhaps the negative ETA numbers are indicating wait time before the bus moves on??

I think this is what I'd expect.  The ETA being -20seconds means the bus should have arrived 20 seconds ago (which it had).   After the 30 second wait it will have passed the scheduled time and so would be dropped from the data source.

Negative ETA's in my app should mean the bus is already there, or if you are really unlucky it has already left ;

John 

[swf...@gmail.com]

Thanks Darren, that is excellent information -- just what I needed.

John

[Eric Lussier]

This morning was incredibly even quieter than last week. It was a ghost town in the downtown core.

[Don Kelly]

On Mon, Jan 3, 2011 at 12:37, Darren Kipp <darre...@gmail.com> wrote:
> On one of the super snowy days in December I was able to easily conclude
> that positive numbers mean the bus is late and negative numbers mean early.
>

I don't know that we can completely solve this issue via observation.
Whatever clock that octranspo uses for these timings doesn't seem to
be in sync with the NTP and carrier sync'd clocks I've been using for
field testing. Best I can observe, their time source is a minute or
two later than mine. This means that either a positive or negative
adherence could place the final time between my source and their's.
Therefore an "early" adherence (which ever sign is valid) could appear
"late" by the reckoning of the device I'm using.

Who ever has contact with the implementors should ask them which
direction the sign indicates in the feed. This way, we'll know for
certain.

For now, I'm going with Darren's conclusions since a snowy day
/should/ make the bus "late" from both time perspectives.

Don/

[Nair, Biju]

Hi Jonathan,

 

Adherence seconds needs to be added to the scheduled seconds.

 

Adherence seconds is negative when the bus is early, and positive when its late.

 

Biju

 

---

From: open-data-ottawa-...@googlegroups.com on behalf of Jonathan Rudenberg
Sent: Sun 02/01/2011 2:49 PM
To: open-data-ottawa-...@googlegroups.com
Subject: Ruby Wrapper

This e-mail originates from the City of Ottawa e-mail system. Any
distribution, use or copying of this e-mail or the information it
contains by other than the intended recipient(s) is unauthorized.
If you are not the intended recipient, please notify me at the
telephone number shown above or by return e-mail and delete
this communication and any copy immediately. Thank you.

Le pr�sent courriel a �t� exp�di� par le syst�me de courriels de
la Ville d'Ottawa. Toute distribution, utilisation ou
reproduction du courriel ou des renseignements qui s'y trouvent
par une personne autre que son destinataire pr�vu est interdite.
Si vous avez re�u le message par erreur, veuillez m'en aviser par
t�l�phone (au num�ro pr�cit�) ou par courriel, puis supprimer
sans d�lai la version originale de la communication ainsi que
toutes ses copies. Je vous remercie de votre collaboration.

[Nair, Biju]

Hi John,

 

This problem was reported earlier I believe by Darren and has been noted. It will be fixed in a future release of the API (can't promise when... ).

 

Please note that the data is received at the OC transpo servers every 2 minutes from a bus, so you can expect some lag time.

 

The rule you have to follow is : schedule time + adherence time = expected arrival time.

 

 

Biju

 

 

---

From: open-data-ottawa-...@googlegroups.com on behalf of swf...@gmail.com
Sent: Mon 03/01/2011 8:36 AM
To: open-data-ottawa-...@googlegroups.com
Subject: Re: Ruby Wrapper

[Nair, Biju]

Hello Don,

 

We will include another api call (as was suggested on the group) to return the server time. You can use that to offset the time on your devices.

 

Cheers

 

Biju

 

---

From: open-data-ottawa-...@googlegroups.com on behalf of Don Kelly
Sent: Mon 03/01/2011 4:43 PM

To: open-data-ottawa-...@googlegroups.com
Subject: Re: Ruby Wrapper

[Don Kelly]

On Tue, Jan 4, 2011 at 11:19, Nair, Biju <Biju...@ottawa.ca> wrote:
> We will include another api call (as was suggested on the group) to return
> the server time. You can use that to offset the time on your devices.
>

That'd be awesome, thanks. There was some question that the actual
devices on the buses might not be in sync with this time source. Will
this new API produce the "official time" at octranspo? That is, will
it be in sync with the other octranspo services like the planner on
the website?

[Michael Richardson]

>>>>> "Biju" == Biju Nair <Nair> writes:
Biju> Please note that the data is received at the OC transpo
Biju> servers every 2 minutes from a bus, so you can expect some lag
Biju> time.

I try hard not to be pedantic, but I think it's important in this case.

Based upon what I have read, the buses reports a location every 2
minutes. Some of this, I think from what I've read, is based upon the
capacity of the data network to accept the data at that time.
I'm unclear if it reports it's current position or a position sometime
(somewhere) previous to when it reports.

How often do the buses sample their GPS data?
Is it synchronous or asynchronous to the reporting process?

That means that there will up to 2 minutes of jitter.
Sometimes, the buses will report their position exactly when they
measure it, and sometimes it will be delayed by up to 2 minutes.
That's jitter. Ones can hope it is stochastic around delay of 1 minute,
but I'll bet it's more complex.

lag (or latency) is a consistent amount of delay.
(Network Time Protocol, for instance, can set the clock on your computer
to within microseconds of the NRC atomic clock, if it can get an
accurate estimate of the latency of the network path.)

From what I've seen (I'm mostly an observer), there are few timestamps
on the data, specifically missing is:
- the current time on the bus (is it synchronized to something?)
- the time at which the data was received

--
] He who is tired of Weird Al is tired of life! | firewalls [
] Michael Richardson, Sandelman Software Works, Ottawa, ON |net architect[
] m...@sandelman.ottawa.on.ca http://www.sandelman.ottawa.on.ca/ |device driver[
Kyoto Plus: watch the video <http://www.youtube.com/watch?v=kzx1ycLXQSE>
then sign the petition.

[Edward Ocampo-Gooding]

[Apologies for how screwed up the top-bottom posting of these email are.]

I'm also curious why time syncro is such a problem given that the time signal given by the GPS device is crazy accurate. Isn't having an incredibly accurate clock the fundamental idea with GPS?

[Michael Richardson]

>>>>> "Edward" == Edward Ocampo-Gooding <edwa...@gmail.com> writes:
Edward> I'm also curious why time syncro is such a problem given
Edward> that the time signal given by the GPS device is crazy
Edward> accurate. Isn't having an incredibly accurate clock the
Edward> fundamental idea with GPS?

I think, it's not about knowing the precise time that the bus did the
reading ("I'm at the corner of Bank and Laurier"). It's about the
latency that it takes for the info to reach the user. If it takes too
long for the information about the location of the bus to arrive to the
user, then the uncertainty about the location is too high.

[swf...@gmail.com]

On Tue, Jan 11, 2011 at 5:57 PM, Michael Richardson <m...@sandelman.ca> wrote:

>>>>> "Edward" == Edward Ocampo-Gooding <edwa...@gmail.com> writes:
   Edward> I'm also curious why time syncro is such a problem given
   Edward> that the time signal given by the GPS device is crazy
   Edward> accurate. Isn't having an incredibly accurate clock the
   Edward> fundamental idea with GPS?

I think, it's not about knowing the precise time that the bus did the
reading ("I'm at the corner of Bank and Laurier").  It's about the
latency that it takes for the info to reach the user.  If it takes too
long for the information about the location of the bus to arrive to the
user, then the uncertainty about the location is too high.

I don't think this is so bad in general.     What most bus users care about is not so much the actual location of the bus but how late/early it is running.

The change in late/early is unlikely to be significant over the course of a few minutes.   Worst case the bus is stopped during the entire time, either because it is having to wait due to being ahead of schedule (the bus user isn't going to be too worried about that, it will still arrive on schedule) or because it is stopped in traffic -- this one is just a bit unfortunate, but hopefully not too common.

If OCtranspo want to get fancy they could track the last two readings for each bus and calculate the rate of change in the lateness and adjust the figure by this rate of change multiplied by the time since the last reading.    Certainly it would be good to have rate of change estimates and the actual times of the readings along with the readings.

John

[Michael Richardson]

>>>>> "swfiua" == swfiua <swf...@gmail.com> writes:
Edward> I'm also curious why time syncro is such a problem given
Edward> that the time signal given by the GPS device is crazy
Edward> accurate. Isn't having an incredibly accurate clock the
Edward> fundamental idea with GPS?

>> I think, it's not about knowing the precise time that the bus did
>> the reading ("I'm at the corner of Bank and Laurier"). It's
>> about the latency that it takes for the info to reach the user.
>> If it takes too long for the information about the location of
>> the bus to arrive to the user, then the uncertainty about the
>> location is too high.

swfiua> I don't think this is so bad in general. What most bus
swfiua> users care about is not so much the actual location of the
swfiua> bus but how late/early it is running.

I regularly care to know if a bus has passed a stop I can not see.
There are numerous places where I have a choice of walking one block
west (for one set of buses), or one block north (for another set of
buses). If you can tell me that my preferred bus has already passed
the stop, I'll make a difference choice.

This is truly down to intervals of less than 30s.

swfiua> The change in late/early is unlikely to be significant over
swfiua> the course of a few minutes. Worst case the bus is stopped
swfiua> during the entire time, either because it is having to wait
swfiua> due to being ahead of schedule (the bus user isn't going to
swfiua> be too worried about that, it will still arrive on schedule)
swfiua> or because it is stopped in traffic -- this one is just a
swfiua> bit unfortunate, but hopefully not too common.

Buses run early regularly. It's epidemic.
It was only in the past 3 years that OCTranspo management stopped saying
that "5 minutes early" was "on schedule"

swfiua> If OCtranspo want to get fancy they could track the last two
swfiua> readings for each bus and calculate the rate of change in
swfiua> the lateness and adjust the figure by this rate of change
swfiua> multiplied by the time since the last reading. Certainly it
swfiua> would be good to have rate of change estimates and the
swfiua> actual times of the readings along with the readings.

Or more complex things based that vary by time of day, location, traffic
conditions, etc.

[Sean Kibbee]

I don't think latency via network slowdown is going to be a problem. As it stands now bus feeds can, and often are, 20 minutes or more old so a second or two isn't going to matter. I also don't believe we should be talking about getting "to the millisecond" syncrinization time data. We should keep our request for additional data elements as simple as possible, for now.

Here is what I propose we request:

What we need is two additional elements added to the feed - trip Id (to compare against the GSF data) and timestamp (when the bus sent the data or when it was received by the server).

These two elements will allow for much greater integrity of the applications that rely on the data.

I also propose we chat about this over a few bubblies in the comming weeks. Woud be great to put some faces to the email addresses and domian names.

Sean

[Dave Schindler]

I have a few suggestions for improvements too.

I'd like to see a way to request GPS info for individual stops at
large bus stations (like 2B vs 2C at Hurdman, for instance). And the O-
Train...At either end of the O-Train, the stops share the same stop
number as the nearest bus stop, but you only get bus data back from
the GPS feed for those stop numbers - no train data.

More precisely, I'd like to be able to ask for data by stop_id
(instead of stop_code/number).

I'd also like to see the bug where buses that are late disappear from
the GPS feed when their scheduled arrival at the stop passes. It
should be when the sum of adherence and scheduled time passes (ie.
when they've actually passed the stop).

And lastly, when the <gps> value is false, instead of returning fake
values for lat/long and adherence, could it return the last known
values?

Dave

[Don Kelly]

Alot of this info is in the gtfs feed for octranspo. All that is needed is a trip id (stop + bus intersection at a given time) to cross reference the two data sets.

Don/

[Dave Schindler]

Don Kelly said "Alot of this info is in the gtfs feed for octranspo".

By 'gtfs feed' do you mean the gps feed at http://octranspo.opendataottawa.ca/stop/? If so, then can you explain? I was under the impression that you couldn't ask it for gps data by stop_id, only stop_code. And the other points I brought up are bugs in the gps feed that make the data unreliable. Why not ask if they can be addressed?

Dave

[Thomas Cort]

> By 'gtfs feed' do you mean the gps feed athttp://octranspo.opendataottawa.ca/stop/?

GTFS => General Transit Feed Specification
It's a file format for transit data (stops, routes, schedules, etc).

The GTFS file can be downloaded here:
http://www.ottawa.ca/online_services/opendata/info/transit_schedule_en.html
http://www.octranspo1.com/files/google_transit.zip

[Don Kelly]

Yup, this is what I meant. Thanks.

Unfortunately, there's no cross referencing id b/w the two data sets.
Currently, I've had to use the bus# + arrival time to map back into
the static schedule (gtfs) data.