GPS module and parsing NMEA strings

[Nelson Neves]

Hi,

I've been testing a GPS module with bifferboard and OpenWrt system
(not much time available to explore it as I would like - still
incomplete):
http://sites.google.com/site/bifferboard/Home/howto/connect-to-a-serial-gps

I've tested gpsd package but it doesn't work very well with this
module, also my 1st time with this package... It compiles and runs but
I'm not able to get it to parse the GPS NMEA data!

Well, I will probably try my luck with C code, getting an application
communicating with GPS module from serial port.
My question, does anybody know some simple C routines/lib to parse
NMEA strings? Or even a C string parser that would help me with this
job? I'm a bit green when it comes to Linux development (don't know
what is available to use).

Just asking before re-inventing the wheel ;)

Regards,

Nelson.

[bifferos]

I don't know anything about NMEA, however for serial port parsing, you
could always try the Python + Pexpect combination that I was using for
bb_upload.py. That will run on Bifferboard with USB stick.
Presumably picking up the data is a case of
1) Wait for some marker string
2) Read the data after the marker string.

Expect (and therefore Pexpect) can do that kind of thing, and it may
be simpler than messing about with C.

In C I suspect you use the select() call to wait for data from the
port (if you don't want 100% CPU usage while waiting), buffer it up
and repeatedly scan it, fiddly stuff to get right even for more
advanced C programmers, and not a good first C project IMHO!

cheers,
Biff.

On Jul 21, 11:11 pm, Nelson Neves <nelson.s.ne...@gmail.com> wrote:
> Hi,
>
> I've been testing a GPS module with bifferboard and OpenWrt system
> (not much time available to explore it as I would like - still

> incomplete):http://sites.google.com/site/bifferboard/Home/howto/connect-to-a-seri...

[Nelson Neves]

Thanks Biff,

I think that I will go with your suggestion! Will give it a try with Python an Pexpect!

Regards,

Nelson.

[Razvan Dragomirescu]

You could also try this: http://dmh2000.com/nmea/nmeap.shtml 

I haven't tried it myself but it claims to be portable and make minimal use of external libraries. 

Might be worth a try. 

Razvan

Sent from my iPhone

[Nelson Neves]

Thanks Razvan,

it looks pretty clean! I've found lots of C code to parse NMEA but didn't found the code to be very developer friendly!

Since I'm C guy (windoez for the time being :P ) I think I will give it a try!

Regards,

Nelson.

[David Prieto]

If you want to give python a try... there's a python GPS daemon and library to parse NMEA:

http://code.google.com/p/python-gpsd/

--
Un saludo,
David Prieto

[Sunspot]

My version using a PIC for interest!
http://www.sunspot.co.uk/Projects/SWEEX/telepresence/GPS_to_rs232.html

On Jul 21, 11:11 pm, Nelson Neves <nelson.s.ne...@gmail.com> wrote:

> Hi,
>
> I've been testing a GPS module with bifferboard and OpenWrt system
> (not much time available to explore it as I would like - still

> incomplete):http://sites.google.com/site/bifferboard/Home/howto/connect-to-a-seri...

[Nelson Neves]

Thanks David,

really like that part "... allowing multiple applications access to GPS data.".

It would be also a good possibility ;)

Regards,

Nelson.

[Nelson Neves]

Hi Graham,

it should be easy to parse the $GPGGA NMEA string using only command line tools, and that was my initial idea!
For now I will be testing some of suggested solutions, try to get the max out of the GPS.

Thanks for the 'howto'!

OK, picking up previous suggestions I have at the moment:

1) Python + Pexpect to connect to Serial Port and search for specific NMEA strings

2) NMEA Portable C code: http://dmh2000.com/nmea/nmeap.shtml

3) Python-gpsd: http://code.google.com/p/python-gpsd/

4) Parse $GPGGA NMEA string directly from command line (very limited info, but quite easy)

I think this should be sufficient to keep me busy for a while ;)

Thank you all,

Nelson.

[Nelson Neves]

I've being testing point 2) - NMEA Portable C code: http://dmh2000.com/nmea/nmeap.shtml - and it looks quite good!
This will generate a lib that will be statically linked with the binary. Final demo code will only have 14Kb, very good!

Also making another test, C code to communicate with BB Serial Port, my objective is to create an autonomous application that will get data from GPS, parse it and serve it! (Isn't this what gpsd do? OK, but if I'm able to do it on C code with the minimum amount of code I will also be able to port this solution to the BB On-Boad-Flash system)
At the moment I'm just compiling the code from my Desktop with Linux Ubuntu and also generating a static binary! Will have to check later on how to use dietlibc do shrink this a bit!

Well, still doing some more tests before getting all the pieces together!

Regards,

Nelson.

[lvh]

Hi!

I'm writing a positioning (currently GPS only) library for Twisted,
the Python asynchronous event loop framework. I'm not entirely sure
yet if the Bifferboard is powerful enough to run it (neither Python
nor Twisted are particularly lightweight, but apparently some people
have had success with running Python), but I plan on finding out. I'm
probably also going to buy that same GPS module as you have now.

While writing this lib, particularly the gpsd part of it, I've come in
contact with the gpsd devs (particularly esr). If you want, I can try
to get gpsd working on it. Of particular help would be a dump of a lot
of sentences, assuming it's a sentence parsing problem.

I really can't imagine that gpsd won't parse sentences that a much
more simplistic parser would, but hey, you have a bug report and I
have time (well, actually, no I don't) to fix it. I'll try to get it
to reproduce with a plain fake gps and perhaps even in Qemu to see
what the actual problem is. Personally, I think it's more likely to be
an unrelated gpsd problem rather than an actual sentence parsing
problem, but I'd love to be proven wrong :-)

I'm also writing my own NMEA parser for Twisted. Not because I
couldn't get gpsd to work, but because gpsd doesn't work on terrible
platforms such as Windows Mobile. It'd obviously be cool to check if
it chokes on these sentences or not :-)

I plan on buying one (bifferboard + gps module) myself, but I just
know that I'd spend unhealthy amounts of time on it, which I can't do
just yet (should be okay come September though! :-)).


thanks in advance
Laurens

[Nelson Neves]

Hi Laurens,

From my short tests, I was able to determine that gpsd was trying to connect to the gps module by looping trough different serial port speeds (baudrates), sort of scanning serial port to determine the appropriated speed (or the speed that gps module was communicating)! I've noticed that when it was reaching the gps module baudrate it didn't recognize it, so I tested something different! I had a tip to use gpsctl (its included with gpsd but in openwrt it wasn't available into Bifferboard, requires to be manually copied) to force baudrate but still no luck, I had some strange messages from gpsd regarding parsing problems! Was getting a bit frustrated and so left gpsd to test something different!

Following the above tips, I've moved on into C code and used this highly portable C lib to parse NMEA code and I can tell you that it worked pretty well!
http://dmh2000.com/nmea/nmeap.shtml

After getting my hands into Linux C code I got a bit excited and tried to do all the parsing myself! I'm still in the middle of it but its looking pretty good!
I'm now able to connect to serial port, fetch data and send it to a parsing routine, all in the same C application, and so I can use the data to serve my needs.
At this point I already have access to NMEA GPGGA and GPRMC data from my C app.

Once I've finished this I will share it and post the C code and binary in the wiki, but still have some work to do! I'm still compiling the binary from my desktop with 'make -static' and the binary is getting around 570kb (mainly because of the static link). The advantage is that I can run this from Bifferboard with OpenWrt or with the alternative OS like BB OnFlashSystem (butt will required a 4MB/8MB Flash Bifferboard to be able to include binary into flash). Still need to get this compiled with dietlibc trying to shrink the binary to values around 20 to 50 kb.

Regards,

Nelson.

[Laurens Van Houtven]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ah, so as I suspected, it's probably more of a connection detection
problem than really parsing the sentences (ie gpsd doesn't even get to
see the sentences properly probably).

I'm going to buy a Bifferboard + GPS module and see where we end up :-)

thanks for your reply
Laurens

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Use GnuPG with Firefox : http://getfiregpg.org (Version: 0.7.8)

iEYEARECAAYFAkp9+YUACgkQT5v5zGkvKT7zqwCgxFmOu2VAYX1T1TO3+4ZxpezY
hhUAn0MBHt1HxBPvkIWUScDWKOY9A/4e
=ZiFr
-----END PGP SIGNATURE-----

[bifferos]

On Aug 8, 3:03 pm, lvh <l...@laurensvh.be> wrote:
> Hi!
>
> I'm writing a positioning (currently GPS only) library for Twisted,
> the Python asynchronous event loop framework. I'm not entirely sure
> yet if the Bifferboard is powerful enough to run it (neither Python
> nor Twisted are particularly lightweight, but apparently some people
> have had success with running Python), but I plan on finding out. I'm

My early experiments with Python on a router with 16MB DRAM indicated
that it will run, and you can launch about 5-6 copies of the
interpreter before it runs out of memory. On Bifferboard this should
leave a good 16MB DRAM for Twisted. I think it might be enough,
although from what I remember Twisted is composed of a lot of
functionality, so it probably depends which modules are being imported
for your application.

regards,
Biff.

[Nelson Neves]

Ah, so as I suspected, it's probably more of a connection detection
problem than really parsing the sentences (ie gpsd doesn't even get to
see the sentences properly probably).

 It really points that way! There is another issue that may interfere with this (I have only noticed it now -  still have this to solve), the gps module may reset it's configuration when detects that serial communications speed is not appropriated, and so will get binary data mode (I think, still haven't found the default mode and speed of this module) - more details on this latter!

Maybe when gpsd is in autoscan mode, it may leave the gsp module confused .... just an idea, really need to get this sorted out!

I'm going to buy a Bifferboard + GPS module and see where we end up :-)

thanks for your reply
Laurens

Will be glad to have some extra feedback!

Regards,

Nelson.

[Nelson Neves]

Just updated the wiki page with some new stuff!

I've decided to make some tests with C code and got myself my 1st GPS
Parser application! It's quite simple an not optimized!
Also managed to use dietlibc alternative / highly_optimized_libraries
and got myself a 25Kb binary, hurray!

There is still lot of work to do with this GPS module, just sharing
for others to use!

http://sites.google.com/site/bifferboard/Home/howto/connect-to-a-serial-gps

Regards,

Nelson.

ps: will be on vacations for 2 weeks, so very limited internet access.

[Nelson Neves]

I decided to make some more tests and have created a 2nd application that will server the GPS data through a TCP/IP interface (emulating a light http server). Have successfully created such application but I'm having some problems sharing the data from my 1st application to this new one!

I've used  the Shared Memory object mechanism so that the 1st app could share the parsed data with the 2nd!

1st: gpsparser will get serial ASCII data from GPS module and will convert that data into internal structure. This data is being shared by using 'Shared Memory' object, allowing the 2nd app. to have 'direct' access to it.

2nd: gpsparserhttpserver will wait for http requests to get access to Shared Memory object and retrieve the updated gps data.

So far so good!

Note: shared memory objects requires to be linked with librt library.

When compiling it normally I have no issues, just need to had -lrt to include librt library:
>gcc -lrt gpsparser.c -o gpsparser

The problem is when I try to compile it and forcing to statically link librt:
>gcc -static -lrt gpsparser.c -o gpsparser

I will get this error:
/tmp/ccixRVX7.o: In function `main':
gpsparser.c:(.text+0x899): undefined reference to `shm_open'
gpsparser.c:(.text+0x8d0): undefined reference to `shm_open'
collect2: ld returned 1 exit status

I really need to statically link librt (only this lib) because I will force it to use dietlibc to shrink/optimize the final binary! Since dietlibc will force internally everything to be statically linked I will get the same error!

Any ideas ? Google is not cooperating with me :( ... TIA.

[bifferos]

Hi,

I haven't used shared memory before, but does this also require some
kind of syncronisation mechanism or is that taken care of by the
library? If I was doing this myself, I'd be inclined to use a couple
of FIFOs for two-way communication between the two processes - this
doesn't require any library support, since you're just reading/writing
files, so is perhaps more likely to work in the dietlibc case.

If you want to go the shmem route, you are probably better off
perusing (or posting to) the dietlibc mailing list. It seems the
developer is (or has been) quite helpful to people with problems like
that.

Just curious - why do you need two applications in the first place?

cheers,
Biff.

[Razvan Dragomirescu]

Hi Nelson,

 

If I remember correctly, you can also link to shared libraries by name, something like

 

gcc -static -o test test.c /usr/lib/librt.a

 

Try this and see if it works, it did for me for another library.

 

Razvan

--
Razvan Dragomirescu
Chief Technology Officer
Cayenne Graphics SRL

[Nelson Neves]

Hi Biff,

the way I'm using Shared Memory  I don't need to worry about synchronization, 1st app will write to it and 2nd will read, also have a lock variable (also available from shmem area)  that I will use when updating data.

I wanted to use only one application for the all process but I have this problem:
Getting data from GPS module requires that my main process loops to fetch serial port data and parse it to save the converted data into internal structures. To implement the http interface I used some code that will create a tcp/ip socket and that is where I get my problem, the socket will lock the process and wait for requests and it will only unlock when it gets a request! My 1st idea was to use a thread, but really don't know if it's possible just with C in Linux, so decided to go trough my school stuff and found this Shared Memory objects stuff and decided to give it a try.

I based my tcp/ip code on this (just an example - instead of fetching directories I will get my gps data from shmem area): http://www.cs.utah.edu/~swalton/listings/sockets/programs/part2/chap6/html-ls-server.c

Using FIFOs will also lock my process right ?

Also though to use fork to get a parent / child process but I don't think that they will share the same memory zone! I'm a bit rusty with this old school techniques, lol ....  windowz development let's your brain in really bad shape! :P


I followed Razvan tip and it worked under normal compilation:
>gcc gpsparser.c -o gpsparser /usr/lib/librt.a
and also
>gcc -static gpsparser.c -o gpsparser /usr/lib/librt.a

But using diet will fail
>diet gcc gpsparser.c -o gpsparser /usr/lib/librt.a

Will ask for dietlibc help, at least I'm now able to do it statically and test both apps.
Thanks guys!

Regards,

Nelson.

[bifferos]

On Aug 30, 1:13 pm, Nelson Neves <nelson.s.ne...@gmail.com> wrote:
> a request! My 1st idea was to use a thread, but really don't know if it's
> possible just with C in Linux

It's definitely possible - there must be loads of tutorials around, a
quick Google reveals: http://www.ibm.com/developerworks/library/l-pthred.html

cheers,
Biff.

[Nelson Neves]

Me and my old habit of trying to do things the harder way!
Thanks Biff, threads really look pretty easy to implement!

I'll give it a try, it's also a good opportunity to learn some new stuff ;)

Regards,

Nelson.

[Nelson Neves]

Just to update the work-in-progress:

I've managed to get the C code for the thread example to get compiled under dietlibc, hurray!

It seems that threads should be the way and not Shared Memory!

Once again, thank you guys for your tips!

Regards,

Nelson.