Logic behind how to build gtfs realtime vehicle position

[Vasile Catana]

Hello, 

I managed to run the transit time it is a great project!!!

The issue : When pressing on "Maps for selected route" I am able to see only one trolley for each route and after few seconds another trolley pops up and the older one disappears. 

The way I am sending the protobuf is the following: I am taking all vehicles locations and I am building the gtfs realtime and sending it(The vehicles are unique). 

I guess I have to send in a different way the protobuf but I don't find any information to help me.  

I have attached an protobuf in human readable text. 

[Óg Crudden]

HI Vasile,

I see from the proto.txt that you do not have a trip_id for each vehicle? Is this information available from the source of the AVL data?

Cheers,

Sean.

--
www.thetransitclock.org
---
You received this message because you are subscribed to the Google Groups "TheTransitClock" group.
To unsubscribe from this group and stop receiving emails from it, send an email to thetransitclo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/thetransitclock/51700354-8ce7-4b7e-90e9-4b4aa11d487en%40googlegroups.com.

[Vasile Catana]

This is the information I build from vehicles locations(location, speed, direction). The transitclock should assign the trip id, is not? 

[Óg Crudden]

Hi Vasile,

It can but this is not the ideal method. It is better if the trip assignment comes from the vehicle location feed. If you want it to do this the auto assigner needs to be enabled.

Also, if you look in the vehicle events table in the database you can get more information on what is going on.

Cheers,

Sean.

[Vasile Catana]

How to enable auto assigner? Do you mean "auto assigner" to auto assign the trip id ? Also what can be the problem that there is one maximum vehicle for each route, I see like there are more vehicles for each route but they pop up and disappear randomly? 

[Óg Crudden]

Hi Vasile,

Adding these entries to the transitclock.properties file will turn on the auto assigner. 

transitclock.autoBlockAssigner.autoAssignerEnabled=true
transitclock.autoBlockAssigner.ignoreAvlAssignments=false

If you look in the vehicle events table or (for more detail) in the logs you will get more detail on what is happening to each vehicle.

Thanks,

Sean.

[Vasile Catana]

I have tried to change the config and it did not work

here is a screen capture.. https://youtu.be/CYLntqlpmv4 of what is the problem. I am suspecting the should show all vehicles assigned to that route.

[Óg Crudden]

Hi Vasile,

Have you had a look in the vehicle events table?

Thanks,

Sean.

[Vasile Catana]

Hello, I have not, I have some issues connecting to the database. I have the output from the transitclock api output(vehicle location (gtfs realtime vehicle position)

You received this message because you are subscribed to a topic in the Google Groups "TheTransitClock" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/thetransitclock/QCJmKxu1wjg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to thetransitclo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/thetransitclock/10a22c18-c013-459b-a8ee-07c2729fb7e7n%40googlegroups.com.

[Alex Railean]

Hi everyone,

I am trying to replicate Vasile's setup, and under his guidance I am able to replicate his results. Having looked around a bit, I can provide answers to questions Sean asked earlier, plus add some extra details.

1. the auto-assigned feature is enabled, per instructions given on March 22nd

2. the file core.log is continuously updated with entries like `Assigning vehicleId=X to blockId=Y but vehicleId=Z already assigned to that block so removing assignment from vehicleId=Z. Making vehicleId=Z unpredictable.` [1]

3. the database is continuously written to, for example if I run `select count(*) from avlreports;` I see that the number of records is getting bigger and bigger. The table `predictions` also grows continuously, while the number of records in `predictionevents` is constant (or it is updated very infrequently).

The GTFS feed itself seems to be right, as I used this tool [2] to visualize it, and I was able to observe that the dots are moving in the expected directions at every tick of the feed.

At this point, I'd like to ask for advice with respect to the error in #2, and a clarification - what to look for in the database, and in which table?

I thank you forward for your feedback,

Alex

[1] here's a full excerpt with actual values:

====================================================AvlProcessor processing AvlReport [vehicleId=0001028, time=04-12-2021 20:34:08.000 EEST, location=[47.02140, 28.83720], speed=2.8m/s, heading=314.0 deg, source=GTFS-rt, assignmentId=8, assignmentType=ROUTE_ID, licensePlate=2171]

Assigning vehicleId=0001028 to blockId=bd. Traian - Parcul 'La Izvor' but vehicleId=0000328 already assigned to that block so removing assignment from vehicleId=0000328.

Making vehicleId=0000328 unpredictable. Assigning vehicleId=0001028 to blockId=bd. Traian - Parcul 'La Izvor' but vehicleId=0000328 already assigned to that block so removing assignment from vehicleId=0000328.

vehicleId=0001028 matched to routeId=8. Vehicle is now predictable. Match=TemporalMatch [temporalDifference=-114.4 sec (late), avlTime=04-12-2021 20:34:08.000 EEST, blockId=bd. Traian - Parcul 'La Izvor', tripIndex=273, gtfsStopSeq=11, stopPathIndex=11, segmentIndex=0, isLayover=false, distanceToSegment=12.16m, distanceAlongSegment=202.67m, distanceAlongStopPath=202.67m, atStop=null, trip=Trip [tripId=8_0_back_138, tripShortName=bd. Traian - Parcul 'La Izvor', tripPatternId=1363586642_to_461062058_999a9748, tripIndexInBlock=273, startTime=19:12:00, endTime=20:47:00, headsign="Loop to Bariera Sculeni", directionId=1, routeId=8, routeShortName=8, serviceId=WORKWEEK, blockId=bd. Traian - Parcul 'La Izvor', shapeId=null]]

For vehicleId=0001028 There was no previous match so in theory could estimate arrivals/departures for the beginning of the assignment. But the vehicle is being reassigned to blockId=bd. Traian - Parcul 'La Izvor' which probably means that vehicle already had arrivals/departures for the stops. Therefore not estimating arrivals/departures for the early stops.

 

[2] https://github.com/OneBusAway/onebusaway-gtfs-realtime-visualizer/wiki

[Vicente Pérez]

Hi Alex,

The assignment to some block is made according to some rules, and of course it depends on the position and  time. 

Do you have the gtfs file in order to have a look at the structure?. Is the gtfs built under frequency or schedule bases? 

Best regards 

Vicente Pérez 

--
www.thetransitclock.org
---
You received this message because you are subscribed to the Google Groups "TheTransitClock" group.
To unsubscribe from this group and stop receiving emails from it, send an email to thetransitclo...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/thetransitclock/e1d7b410-cd69-4d74-9247-53f4517605d8n%40googlegroups.com.

[Alex Railean]

Hi Vicente, and thanks for the quick reaction.

You can find the static GTFS data set here: https://github.com/roataway/gtfs-data/tree/master/GTFS_static

>  Is the gtfs built under frequency or schedule bases? 

My understanding is that it uses a fixed schedule.

Alex

[Alex Railean]

p.s. if it helps, you might want to look at the GTFS realtime feed itself. It is currently exposed at https://gps.dekart.com/tmpgtfs/get_data

[Vicente Pérez]

Hi Alex,

Can you take a look at TheTransitClock trip table and please let me know if the trips you see are the same than the gtfs file?. You might find more than one ConfigRevision, just take in consideration the active one.

Other way is to get the information is to go to web app/api/block. You should have a list of blocks.

I don't recall by heart the TheTransitClock parsing of gtfs, but for sure Onebusaway have the character "_" reserved. So I recommend to change it. For example "-" .

What it might be happening is that you are reading only one trip from gtfs file because of the separator character. 

Best regards 

Vicente 

--
www.thetransitclock.org
---
You received this message because you are subscribed to the Google Groups "TheTransitClock" group.
To unsubscribe from this group and stop receiving emails from it, send an email to thetransitclo...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/thetransitclock/f832262d-6ad9-49ba-9b64-a2635794b6b8n%40googlegroups.com.

[Alex Railean]

This might be it! I did a search-and-replace to swap the underscores for dashes and the results look promising. Unfortunately it is past midnight in our part of the planet and public transport is offline, so I cannot try it with a fully blown data stream. I will try it out tomorrow and report back my findings.

Thanks again for your help,

Alex

[Alex Railean]

Hi,

After checking it in the morning with all the vehicles on the road - the problem is still there. What I thought I saw last night was just a misinterpretation of a person high on "hopium" :-)

I wiped everything out and started from a clean slate, here are my observations:

- The trip names in the `trips` table coincide with those that I see in `trips.txt` (and they all use dashes instead of underscores)

- If I retrieve a list of blocks via web app/api/block, I see the same trip names as above

- After manually inspecting the trips for a particular route (returned via the API) I did not find anything unusual. There is a single configRevision [as expected] and they're all ordered chronologically, there is no obvious issue.

My questions so far are:

1. What other things should I look into?

2. Could this be the result of erratic traffic? In very soft terms, our public transport is more on the "chaotic" end of the spectrum. Even though we obtained the schedule data from the transport authority, I would not be surprised if the drivers do not adhere to it. Would this be a possible cause?

My current hypothesis is that the problem is actually in the feed itself, because the trip_id of each FeedEntity is not a value like `30-0-front-12` (what I see in trips.txt and in the `trips` table), but the route_id (e.g. "30"). So, all the trips in the feed have the same ID, therefore the system probably takes the last feed entity that had the given ID and shows it on the map. The vehicles are jumping around every now and then, depending on which one happened to be the last one in the list containing that ID.

The experimental code that produces the feed [1] is not written by me,  I am still trying to wrap my mind around it and I am not confident I fully understand the logic behind it. Therefore my reasoning could be wrong.

Alex

[1] github.com/roataway/bridge-gtfs-realtime/blob/cleanup/app.py#L71

[Alex Railean]

Hi

 

The ideal feed should include the tripid in order to associate the location with the trip. If you have only  the route id an algorithm will search the most probable one.

Unfortunately it is not possible, because of the "chaotic element" in our public transport, that I mentioned earlier. So we're counting on Transitclock's magic for now. Until reforms are rolled out throughout the city (e.g., dedicated bus lanes), deterministic schedules that are respected remain a dream.

It might happen, because of delays vehicles, some incorrect gps data (as heading), that one or more automatic assignment fail or two vehicles match the same trip at the same time. 

How sensitive is it to deviations? Are we talking about seconds, minutes? Or bigger delays, of ~15 minutes?

The heading information in our feed is far from perfect. The raw coordinates we get from the vehicles' GPS units are matched using an API from OSRM (open source routing machine), and the post-processed coordinates look very good, but the heading information is taken "as is" from the raw data. How sensitive is the Transitclock to noise in the heading data? If this is a major problem, we'll filter it somehow to get more realistic values.

Is there a smoothing algorithm that is known to work well in such cases, that you could recommend?

 

Please let me know if you find anything bad in the feed.

I haven't found anything that looks wrong to me, here's an example of an entry:

trip {

  route_id: "22"

}

position {

  latitude: 47.01692

  longitude: 28.843153

  bearing: 345.0

  speed: 0.5555556

}

timestamp: 1618425352

vehicle {

  id: "0000192"

  label: "1307"

}

It doesn't contain detailed trip info, but everything else seems to be fine.

Alex

p.s. since you replied directly to my mailbox, I'm appending your original message, for the benefit of archaeologists browsing the archives of this mailing list.

------

The ideal feed should include the tripid in order to associate the location with the trip. If you havebonly  the route id an algorithm will search the most probable one. It might happen, because of delays vehicles, some incorrect gps data (as heading), that one or more automatic assignment fail or two vehicles match the same trip at the same time. 

I will try to find some time to run your configuration (lately I don't have much), but anyway take a look at the location and, if possible, to add trip assignment to the feed.

Please let me know if you find anything bad in the feed.

[Alex Railean]

Hi,

Is there any news about this matter?

In the meantime, I have compared our feed with other ones I've found on the Internet, which are known to work well with TransitClock - I haven't found anything suspiciously different. The telemetry looks similar, the identifiers are in order. The only difference is the absence of trip information.

What other troubleshooting steps could we try?

Alex