new feed?

[jeffm...@gmail.com]

Any new feeds in the works?

[Kevin Branigan]

Yep, we just recently made the switch from mysql to a custom built C
app generating the stop times and storing them in ram so I am
rewriting the feed generator that I was originally using - this is the
reason for the delay.

Give me another week and I should one for you, thanks for the
interest!

[Kevin Branigan]

It's up! Hopefully the GTFS is what you wanted - if you need an SQL
file let me know

[George Talusan]

We probably wouldn't mind the SQL db _and_ C library!

[Kieran Huggins]

Hey George - I think "library" was a bit of a misnomer :-) It's
actually just a bunch of methods that are tied to a crap load of other
things we're using, most of which are on their way out. We'll
definitely pass the code your way as soon as we've distilled it into
something useful.

Kevin's going to write some SQL code to work with the new generator
thingy - will post it here when he's done.

Cheers,
Kieran

On Jun 30, 8:08 am, George Talusan <george.talu...@gmail.com> wrote:
> We probably wouldn't mind the SQL db _and_ C library!
>

[William Lachance]

IMO, the best strategy would be to just write a generic GTFS->SQL
converter (or at least sample code) and be done with it. :) Seems like
this is a common problem people have.

Will

2009/6/30 Kieran Huggins <kieran....@gmail.com>:

--
William Lachance
wrl...@gmail.com

[Kieran Huggins]

there's this one (but I haven't tried it): http://github.com/sbma44/py-gtfs-mysql/tree/master

I think Kevin's chief concern is the size of our GTFS - he had to
rewrite the google validator so it didn't choke on the TTC feed.

Maybe he'll write a C converter or something and we'll push it up to
github.

On Jun 30, 9:15 am, William Lachance <wrl...@gmail.com> wrote:
> IMO, the best strategy would be to just write a generic GTFS->SQL
> converter (or at least sample code) and be done with it. :) Seems like
> this is a common problem people have.
>
> Will
>

> 2009/6/30 Kieran Huggins <kieran.hugg...@gmail.com>:

[William Lachance]

That's mainly a problem with Google's TransitFeed library, which
insists on loading the entire thing into a bunch of heavy weight
in-memory data structures. This is arguably necessary for the primary
task of the library (validating GTFS), but you can skip that step if
you just want to do a straight conversion. What did Kevin change about
the google validator to make it work on the TTC feed?

The tool you linked to seems to just stream the feed in/out line by
line, so it probably performs much better on the memory/speed front.

Will

2009/6/30 Kieran Huggins <kieran....@gmail.com>:
>

--
William Lachance
wrl...@gmail.com

[Kieran Huggins]

Not exactly sure what he did - he'll likely chime in when he wakes up.

Yeah, honestly I haven't even looked at the GTFS->SQL code, but it
sounds promising as long as it does stream parsing. If anyone tries it
and reports back that would be pretty sweet ;-)

On Jun 30, 9:25 am, William Lachance <wrl...@gmail.com> wrote:
> That's mainly a problem with Google's TransitFeed library, which
> insists on loading the entire thing into a bunch of heavy weight
> in-memory data structures. This is arguably necessary for the primary
> task of the library (validating GTFS), but you can skip that step if
> you just want to do a straight conversion. What did Kevin change about
> the google validator to make it work on the TTC feed?
>
> The tool you linked to seems to just stream the feed in/out line by
> line, so it probably performs much better on the memory/speed front.
>
> Will
>

> 2009/6/30 Kieran Huggins <kieran.hugg...@gmail.com>:

[Joe Hughes]

If you guys have changes that improve the performance of TransitFeed,
it would certainly be worth feeding them back to the project so that
others could benefit from them. Incidentally, Tom Brown did a bunch
of work in the past year to improve the scalability of the library by
refactoring so that some of the memory management could be done by
SQLite, so it might be worth taking a look at the latest.

Cheers,
Joe

[Kevin Branigan]

Hey guys,

The only change I made to the TransitFeed python script was removing
the "Check for stops that might represent the same location" test.
With 10k bus stops it was just taking far too long.
With the MyTTC dataset we can have up to 8 bus stops at each
intersection (each compass direction and nearside/farside) so it would
end up producing 20-30 thousand warnings.

I'll produce a generated SQL dump from the same C code as I wouldn't
want to use a GTFS->SQL converter on our db because of some extra
fields we have.
It would be helpful for other agencies though. I could whip something
up pretty quickly in C but I don't want to promise anything - I'll try
to look into it after producing the SQL dump

[Joe Hughes]

Thanks for the details, Kevin. It might be worth adding a validator
flag to disable that check. Is the large numbers of warnings because
you don't have exact geocodes for the stops on all the corners?

Joe