Getting Kplex data into a Python App

[Chris Jones]

Hi, apologies if this a really dumb question, but this is a whole new world for me. ...

I have a RPi that is working with Kplex and as a Wifi server; At the moment kplex takes data from NMEA on Serial (USB) network and sends out on WLAN0 and that works fine to apps on iphone/ipad etc. 

Now I would like to write a Python app on another RPi to do some performance calculations and cleaver displays using the NMEA data, so I need to pick up the NMEA data from my existing RPi and get it into the pyton code on the second RPi.

I would like to know what is the best approach and where can I get the necessary documentation.

I am thinking that I should be able to open a socket from my Python app that would connect to my RPi on port 10110, but at that point I run out of understanding.

- do I need to do anything to get the data or does kplex keep pushing it out?

- does the data just come in the NMEA sentence format in text ?

- are there any modules that have any NMEA class definitions alraedy set up?

- and anything else that will make it all easier ...

Thanks in advance for any help

Cheers

Chris

[Keith Young]

 

I am thinking that I should be able to open a socket from my Python app that would connect to my RPi on port 10110, but at that point I run out of understanding.

- do I need to do anything to get the data or does kplex keep pushing it out?

- does the data just come in the NMEA sentence format in text ?

- are there any modules that have any NMEA class definitions alraedy set up?

Sadly I'm not familiar with python (hopefully someone else here is) but I can give you the general principles.

On a basic level it's pretty straight forward. On pi#2, pen a TCP connection to pI#1, tcp port 10110 then read from that connection.  If getting your data via a class library as you probably would be in a high level language make sure there's no buffering involved: You don't want receipt of sentences to be delayed whilst a buffer fills up.  How you wait for data to arrive will depend on your programming style/approach and the language you use.

Yep, the data are just in standard NMEA-0183 format, ASCII characters terminated by <CR><LF> (i.e. 0xd0xa).  I've written some stuff about writing apps for processing it here.

For the last part I apologise again: no python experience.

One alternative you may consider: If network programming is blocking the development of your app but you're happy with text processing, do a Mk 1. version using another instance of kplex on pi#2 to do the network heavy lifting.  Options would include:

1. invoke kplex on the command line, output to stdout, pipe that into your program.  So if your program is "myprog", try something like:

kplex -f- tcp:address=pi1,port=10110,direction=in,persist=yes file:direction=out | myprog

-f- says ignore any config file, pi1 is the server's hostname (or IP address), persist=yes tells kplex to reconnect if the connection drops.  Standard out is the default output for a "file" interface. mode=client is default for a tcp interface

2. Create a fifo with mkfifo (e.g. "mkfifo /var/myfifo" : obviously filename is arbitrary but I'll use that in this example).  Run an instance of kplex in the background (e.g. from boot) reading from the network and writing to the fifo.  kplex.conf might look something like this (Note: off the top of my head and not tested so could contain errors :-)

[tcp]

address=pi1

port=10110

persist=yes

direction=in

[file]

filename=/var/myfifo

direction=out

persist=yes

Now all you have to do is have your program read from the fifo.  Well, in theory :-)  BIG WARNING with these examples.  when kplex writes to a "file" interface as here (and *only* a file interface) its default is to use a UNIX end of line sequence, i.e. just a newline (0xa) rather than the windows/nmea carriage-return-newline.  If you want you can change this behaviour with the "eol=rn" option for file interfaces but it's normally easier to process text that's just newline terminated and if you go with the suggested parsing rules in the link above it doesn't matter either way.

Sorry for lack of python help.  Let me know if you need clarification on this

 

[Chris Jones]

Keith - thanks for amazingly quick and full reply. So far everything has worked in my desk top set up.

However, I have been down at my boat this evening and had an Airmar WX200 on my masthead commissioned which is very exciting ... BUT, I cant get the Actisense NGT-1 working on the RPi. I know it is not directly related to Kplex, but I am wondering if there is any experience out there on RPi and Actisense NGT-1; I have got the usbserial and fdti-sio modules being loaded, but the RPi seem sresolute in not recognisng that it should set up as USB Serial link and so I dont get a ttyUSB0 to play with!

Thanks

Chris

[Keith Young]

Not got one of these myself but did you have to load ftdi_sio manually? Could be the vendor/product id not being recognised?  This guy seems to have had the same problem and got it working:

http://knilluz.buurnet.nl/?p=1173

[Chris Jones]

Yep - I added the module in /etc/modules which at least seemed to get the software available, I am hoping the actisense config file will persude RPi to link the module with the hardware - I'll have a chance to try out in the morning and let you all know the outcome.

Thanks

C

[Chris Jones]

I finally got my Actisense working with RPi -- it was all to do with getting the FDTI driver set up properly;

Below is the info from CANBOAT that was the info that got it all working ... 

Device drivers

On Linux you can use the standard FTDI serial driver. It used to be a bit difficult to enable this driver for a device that uses a USB device PID that the driver did not know about.

The best way with a new kernel is to add new device ID after the module is loaded. This allows the driver to function for multiple FTDI device IDs, so it will also work if you have other FTDI devices in the system.

# - Detect any Actisense device. This adds a new USB ID to the FTDI driver, so is always safe.

#   The driver will autoenumerate the bus after this and report the new device (see dmesg)

modprobe ftdi_sio

echo 0403 d9aa > /sys/bus/usb-serial/drivers/ftdi_sio/new_id

A less flexible way to tell the driver about the Actisense device ID is to load it at module load. This might make the driver not function for other devices:

sudo echo 'ftdi_sio vendor=0x0403 product=0xd9aa' >>/etc/modules

and after reboot it shows up:

You can get persistent naming for the NGT-1 USB device with the following udev rule:

pi@raspberrypi:~$ cat /etc/udev/rules.d/99-my-usb-device.rules 

SUBSYSTEM=="tty", ACTION=="add", ENV{ID_SERIAL}=="Actisense_NGT-1_1FD34", NAME="actisense"

The serial number above is unique, so you can find out information about your device to base the udev rule on with

udevadm info -q all -n /dev/ttyUSB0

HOWEVER - I have not yet managed to get the vendor and product details to work from the modules file - so I will need to get another way of executing the commands automatically. Nor have I tried the udev configuration.

... more updates to follow - C

[joachim bakke]

First of all, there is no need for a second pi for this reason only. The pi itself is capable of many processes at once. I have an rpi2 with kplex, access point, python calculations, AIS receiver on a RTL dongle, MYSQL db for race and polar data etc. 

To your initial question if that has not been solved yet, there is a library called pynmea2 (https://github.com/Knio/pynmea2) which is a good basis for manipulating nmea, if that is what you want to do. See my working code here: https://github.com/joabakk/Scripts/blob/master/python/tagpty.py 

This is a fairly extensive code to start with, but some of the other simpler examples should work as well. This specific example takes the pty stream, finds the relevant (for me) values and compares and saves polar data to a mysql base. I may clean this up with more explanation if I find the time...

Joachim

[Chris Jones]

Interesting - much that will help me, thank you; We are racing and the first thing I am planning to do is display the interpolated target boat speed using the optimum polar data from North and I am planning to use a red/green bar graph to indicate if I am above or below the target true wind angle (F1 steering wheel style) and that is why I am planning to use a second RPi out in the cockpit. (still got to figure out how to keep it dry!

Once I have this infrastructure working, I am planning a number of other nodes, to get nav data to helm (lay lines using bar graph, wind cycle, in/out of 3 boat zone ... the possibilities are pretty endless!)

How have you linked your RPi to tell if the engine is on? (I am using i2c pins for displays, but nothing else on GPIO)

C

[joachim bakke]

Sounds great, I would love to hear more!

I am taking the acc output from the engine control (key switch) with voltege division resistors to indicate that the engine is on. I had planned on measuring RPM, but this is easier.

My approach is to have tablets and phones as screens, with a dynamic polar chart (design polars and polar values added dynamically) with a moving dot to see the current performance, and with target tack and reach lines.I started off with this: http://jsfiddle.net/joabakk/avt3mrt3/27/ and added the dynamic things later.