reverse engineering the db navigator format
601 views
Skip to first unread message
j.sc...@email.de
Jan 20, 2014, 5:53:03 PM1/20/14
to open...@googlegroups.com
Hi,
I had a look into how DB Navigator works because there exists no DB Navigator for my mobile phone which runs Debian GNU/Linux. To this end I snooped the traffic from the DB Navigator android app and made the following discoveries:
On startup, the application asks for the location of some services on http://reiseauskunft.bahn.de/addons/dbmobile.cfg which yields something along the lines of:
One can search for connections by sending the following as a https POST request to https://reiseauskunft.bahn.de/bin/query.exe/en
most fields are self-explanatory. The SID and ZID specify the requested start and finish of the trip and when urldecoded have the following format:
I believe you already found most of these values for stations through the decoding of plan data. So it should be no problem to formulate any such request between any two stations.
Btw, the user agent is set to:
You can execute above request easily by using curl:
Notice the invocation of zcat because the content is gzipped. The resulting plan.out is similar to files which can be found on the hafas CDs but again a bit different because it contains a selection of possible trips. I made an analysis of this data three years ago and came up with a reasonably good interpretation for most values. I didnt publish anything back then because my code was in a very bad state and soon I was overwhelmed by some real life issues. This time (today) I went back to this because I need an application that gives me timetable information for my phone. Since I dont know whether or not I will find time to develop this further this time, I thought instead of keeping everything to myself again, I should first share it with you guys :)
Here are some more nice things:
You can create a "persönlicher Fahrplan" which downloads not only the information for the given time and date but for the whole year! The resulting data format matches the one on the hafas CD because this time the result is not individual trips but individual train schedules out of which the DB Navigator constructs trips by itself. You can request a "persönlicher Fahrplan" like this:
Send a https POST request to http://persoenlicherfahrplan.bahn.de/bin/pf/query-p2w.exe/en with the following data:
Values are mostly self explanatory and SID and ZID are as above. You get back a number of cookies and the following content:
This tells you the estimated time you have to wait until your fahrplan is ready and the url under which you check for its availability. So lets try that out and lets send a POST request to http://persoenlicherfahrplan.bahn.de/bin/pf/query-p2w.exe/en with the following data:
We eventually get back this:
Which is again a url that we have to retrieve. After doing yet another POST request to http://persoenlicherfahrplan.bahn.de/bin/pf/query-p2w.exe/BerlinBremen.pln with content:
We finally get back gzipped plan data. I didnt try yet but it looks extremely similar to what is on the CD so openPlanB should be able to parse it.
Another nice source of information is a query which allows to look up which trains are currently late and by how much. Try doing a POST request to https://reiseauskunft.bahn.de/bin/bhftafel.exe/en with the following DATA:
The values are luckily self explanatory again and the train is even given as a human readable name! You get back something like this:
And you see that depDelay and arrDelay allow to find out by how much time trains are supposedly delayed.
Maybe this helps somebody.
cheers, josch
I had a look into how DB Navigator works because there exists no DB Navigator for my mobile phone which runs Debian GNU/Linux. To this end I snooped the traffic from the DB Navigator android app and made the following discoveries:
On startup, the application asks for the location of some services on http://reiseauskunft.bahn.de/addons/dbmobile.cfg which yields something along the lines of:
ESUITE=YES
serverKey=NULL
QUERY=https://reiseauskunft.bahn.de/bin
QUERYSECURE=https://reiseauskunft.bahn.de/bin
QUERYP2W=http://persoenlicherfahrplan.bahn.de/bin/pf
DEEPLINKMOBIL=https://mobile.bahn.de/bin/mobil
PUSHMOBIL=https://mobile.bahn.de/bin/mobil/query.exe/
ESUITEPUSH=https://fahrkarten.bahn.de/mobile/st/st.post?sc=vu
AUSLAND=YES
FACHKONFIG=http://reiseauskunft.bahn.de/addons/fachkonfig.cfg
ESUITESERVICES=https://fahrkarten.bahn.de/mobile/dbc/xs.go?
ESUITENEUREG=https://fahrkarten.bahn.de/mobile/st/st.post?sc=rg
ESUITEBAHNBONUS=https://fahrkarten.bahn.de/mobile/st/st.post?sc=bbpu
ESUITEPREISFINDERSUCHE=https://ps.bahn.de/preissuche/preissuche/psc_service.go?&service=pscangebotsuche
ESUITEPREISFINDERDETAILS=https://ps.bahn.de/psmobile/ps/st.post?sc=details
ESUITECAPTCHA=http://captcha.bahn.de/cgi-bin/scramble-standalone.pl?
CAPTCHA=YES
CMSOUTDATED=http://www.bahn.de/m/view/{SPRACHE}/hilfe/apps/dbnavigator_{OS}_{RELEASE}.shtml
CMSHILFE=http://www.bahn.de/m/view/{SPRACHE}/hilfe/apps/dbnavigator_{OS}_{RELEASE}.shtml
CMSINFO=http://www.bahn.de/m/view/{SPRACHE}/apps/uebersicht.shtml
CMSIMPRESSUM=http://www.bahn.de/m/view/{SPRACHE}/home/kontakt/impressum.shtml
CMSRECHTLICHES=http://www.bahn.de/m/view/{SPRACHE}/home/agb/uebersicht.shtml
TRACKING=https://st.bahn.de/b/ss/dbbahnmob/5/
LAGEPLAN=http://railnavigator.bahn.de/rnav/lageplan
BIKECAR=db-mapplet.hafas.de/bin/stops-and-bikes
One can search for connections by sending the following as a https POST request to https://reiseauskunft.bahn.de/bin/query.exe/en
start=1&clientType=ANDROID&androidversion=10.50.p00.04&time=03%3a28&hcount=1&clientSystem=Android19&date=20.01.2014&REQ0JourneyProduct_prod_list_1=11111111111111&ignoreMinuteRound=yes&h2g-direct=11&existOptimizePrice=1&timeSel=depart&SID=A%3d1%40O%3dBERLIN%40X%3d13386987%40Y%3d52520501%40U%3d80%40L%3d008096003%40B%3d1%40p%3d1389725366%40&ZID=A%3d4%40O%3dKarlsruhe%40X%3d8395942%40Y%3d48994950%40U%3d106%40L%3d870000001%40B%3d1%40p%3d1389762001%40&clientDevice=google_sdk&htype=google_sdk&optimize=0
most fields are self-explanatory. The SID and ZID specify the requested start and finish of the trip and when urldecoded have the following format:
A=1@O=BERLIN@X=13386987@Y=52520501@U=80@L=008096003@B=1@p=1389725366@
I believe you already found most of these values for stations through the decoding of plan data. So it should be no problem to formulate any such request between any two stations.
Btw, the user agent is set to:
Dalvik/1.6.0 (Linux; U; Android 4.4.2; google_sdk Build/KK);DBNavigator/10500000/Android_4.4.2
You can execute above request easily by using curl:
echo "start=1&clientType=ANDROID&androidversion=10.50.p00.04&time=03%3a28&hcount=1&clientSystem=Android19&date=20.01.2014&REQ0JourneyProduct_prod_list_1=11111111111111&ignoreMinuteRound=yes&h2g-direct=11&existOptimizePrice=1&timeSel=depart&SID=A%3d1%40O%3dBERLIN%40X%3d13386987%40Y%3d52520501%40U%3d80%40L%3d008096003%40B%3d1%40p%3d1389725366%40&ZID=A%3d4%40O%3dKarlsruhe%40X%3d8395942%40Y%3d48994950%40U%3d106%40L%3d870000001%40B%3d1%40p%3d1389762001%40&clientDevice=google_sdk&htype=google_sdk&optimize=0" | curl -vvv --data @- --user-agent "Dalvik/1.6.0 (Linux; U; Android 4.4.2; google_sdk Build/KK);DBNavigator/10500000/Android_4.4.2" https://reiseauskunft.bahn.de/bin/query.exe/en | zcat > plan.out
Notice the invocation of zcat because the content is gzipped. The resulting plan.out is similar to files which can be found on the hafas CDs but again a bit different because it contains a selection of possible trips. I made an analysis of this data three years ago and came up with a reasonably good interpretation for most values. I didnt publish anything back then because my code was in a very bad state and soon I was overwhelmed by some real life issues. This time (today) I went back to this because I need an application that gives me timetable information for my phone. Since I dont know whether or not I will find time to develop this further this time, I thought instead of keeping everything to myself again, I should first share it with you guys :)
Here are some more nice things:
You can create a "persönlicher Fahrplan" which downloads not only the information for the given time and date but for the whole year! The resulting data format matches the one on the hafas CD because this time the result is not individual trips but individual train schedules out of which the DB Navigator constructs trips by itself. You can request a "persönlicher Fahrplan" like this:
Send a https POST request to http://persoenlicherfahrplan.bahn.de/bin/pf/query-p2w.exe/en with the following data:
spmo=1&start=1&pp=20&clientType=ANDROID&androidversion=10.50.p00.04&hcount=1&REQ0PttH2GDirectFlags=10&p2wCreateMaps=1&clientSystem=Android19&REQ0JourneyProduct_prod_list_1=11111111111111&L=vs_javapln&output=pln&SID=A%3d1%40O%3dBERLIN%40X%3d13386987%40Y%3d52520501%40U%3d80%40L%3d008096003%40B%3d1%40p%3d1389725366%40&ZID=A%3d1%40O%3dBremen%20Hbf%40X%3d8813833%40Y%3d53083477%40U%3d80%40L%3d008000050%40B%3d1%40p%3d1389725366%40&p2wIVRoute=1&clientDevice=google_sdk&htype=google_sdk
Values are mostly self explanatory and SID and ZID are as above. You get back a number of cookies and the following content:
This tells you the estimated time you have to wait until your fahrplan is ready and the url under which you check for its availability. So lets try that out and lets send a POST request to http://persoenlicherfahrplan.bahn.de/bin/pf/query-p2w.exe/en with the following data:
L=vs_javapln&id=ywyc.ekhc.gm47.yk88
We eventually get back this:
error=0
depLocation=Berlin
arrLocation=Bremen Hbf
url=http://persoenlicherfahrplan.bahn.de/bin/pf/query-p2w.exe/BerlinBremen.pln?pathinfo=/en&dwn=yw/ywycekhcgm47yk88_1_1.pln
Which is again a url that we have to retrieve. After doing yet another POST request to http://persoenlicherfahrplan.bahn.de/bin/pf/query-p2w.exe/BerlinBremen.pln with content:
pathinfo=/en&dwn=yw/ywycekhcgm47yk88_1_1.pln
We finally get back gzipped plan data. I didnt try yet but it looks extremely similar to what is on the CD so openPlanB should be able to parse it.
Another nice source of information is a query which allows to look up which trains are currently late and by how much. Try doing a POST request to https://reiseauskunft.bahn.de/bin/bhftafel.exe/en with the following DATA:
start=yes&clientType=ANDROID&androidversion=10.50.p00.04&time=19%3a46&hcount=2&productsFilter=11111111111111&input=%238011160&clientSystem=Android19&date=20.01.2014&REQTrain_name=ICE%20%20832&L=vs_java3&sTI=1&dirInput=8000050&clientDevice=google_sdk&htype=google_sdk&boardType=dep
The values are luckily self explanatory again and the train is even given as a human readable name! You get back something like this:
<St evaId="8010255" name="Berlin Ostbahnhof" depTime="19:35" depDelay="+0" ddelay="0" platform="7 D-G " x="13434567" y="52510972" />
<St evaId="8011160" name="Berlin Hbf" depTime="19:46" depDelay="+0" ddelay="0" platform="13 D-G " x="13369548" y="52525589" />
<St evaId="8010404" name="Berlin-Spandau" depTime="20:01" depDelay="+0" ddelay="0" platform="4 D-G " x="13197530" y="52534470" />
<St evaId="8006552" name="Wolfsburg Hbf" arrTime="20:54" arrDelay="+0" adelay="0" depTime="20:56" depDelay="+0" ddelay="0" platform="1 " x="10788197" y="52429494" />
<St evaId="8000152" name="Hannover Hbf" arrTime="21:28" arrDelay="" adelay="" depTime="21:41" depDelay="" ddelay="" platform="12 " x="9741016" y="52376763" />
<St evaId="8000279" name="Nienburg(Weser)" arrTime="22:09" arrDelay="" adelay="" depTime="22:11" depDelay="" ddelay="" platform="2 " x="9217754" y="52645568" />
<St evaId="8006076" name="Verden(Aller)" arrTime="22:26" arrDelay="" adelay="" depTime="22:28" depDelay="" ddelay="" platform="2 " x="9237854" y="52920818" />
<St evaId="8000050" name="Bremen Hbf" arrTime="22:48" arrDelay="" adelay="" platform="6 " x="8813833" y="53083477" />
And you see that depDelay and arrDelay allow to find out by how much time trains are supposedly delayed.
Maybe this helps somebody.
cheers, josch
Michael Kreil
Jan 20, 2014, 7:02:32 PM1/20/14
to open...@googlegroups.com
Great work! :)
You should also have a look at "public transport enabler": https://code.google.com/p/public-transport-enabler/
They reverse engineered a lot of these interfaces, including DB. "Unfortunately" in Java - but great for Android.
Michael
j.sc...@email.de
Jan 21, 2014, 3:49:12 AM1/21/14
to open...@googlegroups.com
Hi Michael,
On Tuesday, January 21, 2014 1:02:32 AM UTC+1, Michael Kreil wrote:
You should also have a look at "public transport enabler": https://code.google.com/p/public-transport-enabler/They reverse engineered a lot of these interfaces, including DB. "Unfortunately" in Java - but great for Android.
Awesome! Indeed they already reversed most of it :)
It is indeed "unfortunately" in java. It turnrs out that half of its build dependencies are not packaged for debian (for example ksoap2-android) and installing the other half already required 554 MB of disk space...
The project seems to be active and support many transport providers so I do not want to just reimplement it in yet another languages out of some NIH syndrome. I just wish that its choice of programming language and build system would not make it so hard on Debian :(
Interestingly its AbstractHafasProvider includes an implementation for an xml based query return which is though never used anywhere else. Am I misreading the code or does the hafas server also spit out xml as an alternative to their proprietary binary format?
Michael Kreil
Jan 21, 2014, 6:28:43 AM1/21/14
to open...@googlegroups.com
Interestingly its AbstractHafasProvider includes an implementation for an xml based query return which is though never used anywhere else. Am I misreading the code or does the hafas server also spit out xml as an alternative to their proprietary binary format?
As far as I can remember ... Hafas servers can provide different interfaces like HTML, WAP, XML, JSON, binary stuff, ... These interface are switch on/off when needed. e.g. a transport organization want's to publish a smartphone app, that uses XML.
So basically: Some Hafas-Server uses XML, binary, both or maybe neither.
That's why every provider has an abstract interface description in:
Reply all
Reply to author
Forward
0 new messages