Raspberry Pi, Kplex and a USR-TCP232-410 Serial to Ethernet Module

[Declan Barry]

Hi folks..

I currently run a Windows 7 workstation with a USR-TCP232-410 serial to Ethernet module along with AISDispatcher to feed AIS data to MarineTraffic and AISHub

I'd now like to streamline the installation by using a Raspberry Pi to supply the feeds to Marinetraffic and AISHub but am having a bit of a problem configuring Kplex to read the information from my MXA5000 AIS receiver via a USR-TCP232-410 serial to Ethernet module.

Is it possible to configure Kplex to read the info from the USR-TCP232-410 IP Address?

I've tried a number of configuration combinations but am having no success.

I do have a serial to USB cable on its way but I'd rather be able to use the Ethernet module across my network rather than limiting the AIS receiver to the Raspberry pi.

Any assistance greatly appreciated.

Declan

[Keith Young]

Is it possible to configure Kplex to read the info from the USR-TCP232-410 IP Address?

Should be.  First make sure the serial-to-ethernet device is in tcp server mode.  Check the "local port" it's using in the setup.  A look at the manual suggests that the default is 23 which is the reserved port for telnet.  You could change that to 10110 which is the default for NMEA-0183-over-TCP or leave it as is: doesn't really matter.  From the raspberry pi check you can access the server:

telnet <server ip address> <server port>

e.g.

telnet 192.168.1.123 10110

Can't remember if telnet is installed by default on raspbian.  If not, install it: it's good for debugging (or skip to the config below and just see if it works)

See NMEA strings?  Good it's working. If not see if you can telnet to the IP address and port from a command window in windows.  If you can maybe it's a networking setup issue with the pi

Once it's working, the appropriate interface in the kplex configuration file would be:

[tcp]

address=<server address>

port=<server port>

persist=yes

e.g.:

[tcp]

address=192.168.1.123

port=23

persist=yes

If you have set up the serial to ethernet device to use port 10110 you don't need the "port" line (10110 is the default)

Let us know how that goes.  I'd advise trying the beta release 1.3.2rc1: The "persist" option could crash the interface in 1.3.1.

[Declan Barry]

Hi Keith..

Many thanks for the reply.

The following is where I am at:

1. I've checked the serial-to-ethernet device and it is set as tcp server mode. I've also changed the port to 10110

2. I installed telnet to my Raspberry Pi (AISPi) and am able to see nmea data.

3. I've installed your latest version of Kplex and added the configuration as below

[tcp]

address=172.16.10.254

port=10110

persist=yes

Whilst I can telnet directly (using telnet 172.16.10.254 10110) to the serial-to-Ethernet device from the Raspberry Pi  and see the nmea data, how do I tell that data is being received by Kplex on the Raspberry Pi in order to output it to other sources?

Regards

Declan

[Keith Young]

Whilst I can telnet directly (using telnet 172.16.10.254 10110) to the serial-to-Ethernet device from the Raspberry Pi  and see the nmea data, how do I tell that data is being received by Kplex on the Raspberry Pi in order to output it to other sources?

Depends how you're using kplex to distribute the data.  If you're running kplex as a tcp server you can telnet to the port it's listening on from a command shell in windows and see if data is coming through.  My usual way would be:

- stop kplex (if it's running)

- run it on the command line with one argument, i.e. "kplex file:"

This will use kplex's normal configuration file but add extra interfaces which appear on the command line, in this case "file:" is a file interface with no arguments which will therefore be a bi-directional interface using standard input and output (i.e. the terminal).  You should see everything flowing into kplex in your window.

- If you see data being printed, happy days.  Kill kplex using ctrl-C

- restart kplex in the background using the start script (if that's how you normally do it)

- If you see nothing more investigation is needed

BIG NOTE: If you're going to be using marine traffic you're probably going to be using the "preamble" option.  I've just been notified of a bug in 1.3.2rc1 with persistent connections using the "preamble" option.  This doesn't apply in this simple case here but do be aware that this bug exists, that I think it's now fixed (code pushed to git) but I've yet to make new deb packages.

[Declan Barry]

Hi Keith.

I'm slowly but surely getting somewhere.

This is my current kplex.conf file:

_______________________________________________________________

#Serial to Ethernet module

[tcp]

address=172.16.10.254

port=10110

persist=yes

#Marinetraffic

[tcp]

address=5.9.207.224

port=5986

persist=yes

direction=yes

[tcp]

mode=server

port=10110

direction=out

[broadcast]

direction=out

device=eth0

___________________________________________

When I shut the kplex program down and run the command:

kplex file:

It starts to display my nmea data so I can assume that the first part is working fine from the serial-to-Ethernet module

In the second part where I've tried setting up the MarineTraffic side of things as per the instructions (using MarineTraffic test port) they are receiving nothing from the Raspberry Pi

When I telnet to the Raspberry pi iP address 172.16.10.251 Port 10110 it also shows me nmea data so I assume the third part is working

My problem at the moment seems to be the Marine Traffic bit.

Declan

[Keith Young]

#Marinetraffic

[tcp]

address=5.9.207.224

port=5986

persist=yes

direction=yes

"direction=yes"?  I'm guessing that this is a typo in your post rather than the config file or kplex *should* fail to initialise.  Some thoughts on debugging this:

1. Confirm (using the previous method) that you're actually getting AIS sentences rather than just GPS (I'm guessing you've done that)

2. Name the marine traffic interface by putting something like "name=mtraffic" into the interface declaration stanza, then enable debugging assuming you're using 1.3.2rc1 or 1.3.2rc2 (now available, which fixes the bug I referenced before).  Either run kplex on the command line with the "-d3" switch or edit the startup script to add "-d3" onto the kplex startup command line.  Debugging output goes to the terminal if running kplex in the foreground or syslog if in the background.  Look for evidence that the connection is being established and then if it's being dropped and not reconnecting

3. Check the connection does get established with :

"netstat -an |grep 5.9.207.224" (yes purists, I know those dots are wildcards :-): You should see a line with "ESTABLISHED" at the end of it.

4. If all else fails there's good 'ol tcpdump but all else hasn't failed yet :-)

[Declan Barry]

lol!!

Sorry.. Type on my behalf.

Strangely, after retyping the config file, everything is now working. The Raspberry Pi is quite happily Reading the data from the AIS receiver and supplying AIS data to my local version of Shipplotter and MarineTraffic. Just about to add in AISHub.

Many thanks for you assistance. Hopefully others on the forum may be able to make use of my experiences. All  I need to do now is move the Raspberry Pi up into the roof space beside the AIS Receiver and I can monitor it remotely.

Regards

Declan

[Declan Barry]

Hi Keith..

Just one other little niggle that is tripping me up.

I issued the command:

sudo update-rc.d kplex defaults

To run kplex at startup and when I reboot I can see the system has started but, I find I have to stop the service then restart it again in order for the program to run correctly.  This is demonstrated when I reboot the Pi and test if any data is being outputted to Shipplotter.

Shipplotter on the Windows PC tells me it isn't able to connect but if I stop and start the service on the Pi and then ask Shippotter to connect, away it goes and displays data.

Not sure what I have done wrong. Could it be that kplex is starting up before the network has been fully initiated?

Declan

[Keith Young]

Could it be that kplex is starting up before the network has been fully initiated?

Interesting.  What OS and version are you running?

The init script has a dependency on networking so it shouldn't start before the network is available.  However I've been running an older version of Raspbian on my pi to compile for the lowest common denominator.  Do the recent versions now use systemd?  I'll have to have a look at how systemd deals with dependencies in legacy scripts.

Meantime you might be able to get a better handle on the problem by sticking "-d3" onto the command line for kplex in the init script, then looking in the log files to see what's going wrong on startup

[Teppo Kurki]

  Do the recent versions now use systemd? 

They do. https://www.debian.org/releases/stable/amd64/release-notes/ch-information.en.html#systemd-upgrade-default-init-system

FWIW here's a systemd unit file  template for running a node app: https://github.com/tkurki/marinepi-provisioning/blob/master/roles/node-app/templates/node_app_systemd_script.j2

https://fedoramagazine.org/systemd-unit-dependencies-and-order/

[Declan Barry]

Hi Keith..

I'm running the latest version of Raspbian Jesse Kernel Version 4.1 - (Full desktop image based on Debian Jessie)

what version are you running on your Pi?

Declan

[Keith Young]

what version are you running on your Pi?

"Wheezy".  (why can't these debian/ubuntu people just use numbers?)

Thanks to Teppo for confirming that Jessie has moved to systemd. According to my research  systemd-sysv-generator *should* take care of dependencies for legacy ini scripts, honouring dependencies on $network.  So even though I should do something about making the install process test the boot environment and install a unit file or an init script as appropriate the init script *should* work ok on a systemd system.

Any joy with the debugging output to tell us what is failing?

[Declan Barry]

Hi Keith..

I'm not familiar with Linux as such. Just learning.. Where do I put the '-d3'?

in that in the main script /etc/init.d/kplex file?

I've also installed a Wheezy version (think it is a May 2015 image) and the same thing happens. Must be something I've doing.  It starts the kplex program okay as I can see it in the main boot screen as started successfully but when I do a tcpdump, I get no output but as soon as I stop|start kplex, tcpdump works fine.

Declan

[Keith Young]

in that in the main script /etc/init.d/kplex file?

Yes. Change this line:

start-stop-daemon --start --quiet --oknodo --exec $DAEMON --startas $DAEMON --chuid $RUN_AS_USER -- -f $KPLEXCONF -o mode=background

to this:

start-stop-daemon --start --quiet --oknodo --exec $DAEMON --startas $DAEMON --chuid $RUN_AS_USER -- -f $KPLEXCONF -d3 -o mode=background 

Note to self: should make that easier to do...

[Declan Barry]

Cheers Keith..

Will give that a try in the morning when I get in to work. Incidentally, where does the log file be kept? Apologies for my lack of programming knowledge. New to all this.

Declan

[Declan Barry]

Hi Keith..

Added the -d3 into the kplex, restarted the Raspberry Pi and looked at the log file.

from what I can see, it looks like kplex is failing at initialising the network interface.  Below is the reference in the log:

Apr 12 08:30:28 AISPi kplex[527]: Starting kplex multiplexer: kplexkplex DEBUG: Using config file /etc/kplex.conf

Apr 12 08:30:28 AISPi systemd[1]: Starting Terminate Plymouth Boot Screen...

Apr 12 08:30:28 AISPi kplex[527]: .

Apr 12 08:30:28 AISPi kplex[527]: kplex DEBUG: kplex starting, config file /etc/kplex.conf

Apr 12 08:30:28 AISPi kplex[527]: Failed to open tcp connection for 172.16.10.254/10110

Apr 12 08:30:28 AISPi kplex[527]: Failed to initialize Interface _tcp-id1

Apr 12 08:30:28 AISPi systemd[1]: Started LSB: Start kplex NMEA 0183 multiplexer.

Apr 12 08:30:28 AISPi systemd[1]: Started Wait for Plymouth Boot Screen to Quit.

Apr 12 08:30:28 AISPi ntpd[543]: ntpd 4.2...@1.2349-o Mon Nov 2 04:29:47 UTC 2015 (1)

Apr 12 08:30:28 AISPi systemd[1]: Started Terminate Plymouth Boot Screen.

When I issue the commands to stop and start the service, I get the following in the log.

Apr 12 08:35:20 AISPi systemd[1]: Starting LSB: Start kplex NMEA 0183 multiplexer...

Apr 12 08:35:20 AISPi kplex[902]: Starting kplex multiplexer: kplexkplex DEBUG: Using config file /etc/kplex.conf

Apr 12 08:35:20 AISPi kplex[902]: .

Apr 12 08:35:20 AISPi systemd[1]: Started LSB: Start kplex NMEA 0183 multiplexer.

Apr 12 08:35:20 AISPi kplex[902]: kplex DEBUG: kplex starting, config file /etc/kplex.conf

Apr 12 08:35:20 AISPi kplex[902]: kplex DEBUG: _tcp-id1: initialised

Apr 12 08:35:20 AISPi kplex[902]: kplex DEBUG: _tcp-id2: initialised

Apr 12 08:35:20 AISPi kplex[902]: kplex DEBUG: _tcp-id3: initialised

and the system works fine.

Declan

On Monday, 11 April 2016 18:17:44 UTC+1, Keith Young wrote:

[Keith Young]

Will give that a try in the morning when I get in to work. Incidentally, where does the log file be kept? Apologies for my lack of programming knowledge. New to all this.

When run in the background as it is from the init script kplex logs to the system log (facility DAEMON).  Where that goes depends on your syslog configuration but on debian-based systems /var/log/daemon.log  and /var/log/syslog are two places to look (can't remember exactly how raspbian is set up)

[Keith Young]

Apr 12 08:30:28 AISPi kplex[527]: kplex DEBUG: kplex starting, config file /etc/kplex.conf

Apr 12 08:30:28 AISPi kplex[527]: Failed to open tcp connection for 172.16.10.254/10110

Apr 12 08:30:28 AISPi kplex[527]: Failed to initialize Interface _tcp-id1

 

Workaround is to add this to the tcp interface declaration:

persist=fromstart

That will make kplex retry the connection if it doesn't connect initially but I'm curious to know why it isn't starting.  Let me think about that and post later.  172.16.10.254 is the address of serial to ethernet device, yes?  Is the pi configured with dhcp and if so, what is handing out addresses and is it only the pi which is being power cycled?

[Declan Barry]

Yes. 172.16.10.254 is the IP address of the Serial to Ethernet device. The Pi is configured with DHCP and is receiving the IP address from a Mac Mini on the network which handles all IP and DNS addressing.

The Pi is the only device that is getting recycled.

As I say, kplex works okay if I stop and restart the kplex service. from my limited knowledge, whatever is happening it looks like kplex needs to wait until the network has fully initialised before it works properly. I stopped it from auto starting then rebooted the system. (tested that kplex service wasn't running 'service --status-all' ) I waited for the network to fully initialise the issued the command:

 sudo service kplex start

Did a tcpdump of the marinetraffic test port that I have set up in the kplex.conf file and all works fine. nmea date being displayed.

Also tried the persist=fromstart

It didn't work

Baffled!

Declan

[Keith Young]

Also tried the persist=fromstart

That's disturbing if it didn't work correctly.  Could you post the contents of the log after "persist=fromstart" has been added to the config?

Here's something else to try  In the init script change:

    start)

        log_daemon_msg "Starting kplex multiplexer" "kplex"

to:

    start)

        log_daemon_msg "Starting kplex multiplexer" "kplex"

        (date; ifconfig -a ; netstat -rn --inet) >> /var/log/kplexlog 2>&1

That will hopefully tell us the state of the interfaces when kplex starts (saved in a file called /var/log/kplexlog)

[Declan Barry]

Hi Keith..

This is the kplexlog file when pi rebooted. As you can see, the interface starts up with the 169.x.x.x. IP Range as the Pi hasn't fully connected to the DHCP Server. No tcpdump data

Wed 13 Apr 09:58:22 UTC 2016

eth0 Link encap:Ethernet HWaddr b8:27:eb:be:d6:3b

inet addr:169.254.238.120 Bcast:169.254.255.255 Mask:255.255.0.0

inet6 addr: fe80::db71:a826:4655:d101/64 Scope:Link

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:0 errors:0 dropped:0 overruns:0 frame:0

TX packets:26 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:0 (0.0 B) TX bytes:5078 (4.9 KiB)

lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0

inet6 addr: ::1/128 Scope:Host

UP LOOPBACK RUNNING MTU:65536 Metric:1

RX packets:0 errors:0 dropped:0 overruns:0 frame:0

TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:0

RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)　

Kernel IP routing table

Destination Gateway Genmask Flags MSS Window irtt Iface

169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0

This is the second part of the same log AFTER the Pi has connected to the DHCP Server and I have stopped and restarted the kplex service. tcpdump data displayed.

Wed 13 Apr 10:04:33 UTC 2016

eth0 Link encap:Ethernet HWaddr b8:27:eb:be:d6:3b

inet addr:172.16.10.251 Bcast:172.16.255.255 Mask:255.255.0.0

inet6 addr: fe80::db71:a826:4655:d101/64 Scope:Link

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:37783 errors:0 dropped:851 overruns:0 frame:0

TX packets:122 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:11309286 (10.7 MiB) TX bytes:15919 (15.5 KiB)

lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0

inet6 addr: ::1/128 Scope:Host

UP LOOPBACK RUNNING MTU:65536 Metric:1

RX packets:136 errors:0 dropped:0 overruns:0 frame:0

TX packets:136 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:0

RX bytes:11472 (11.2 KiB) TX bytes:11472 (11.2 KiB)

Kernel IP routing table

Destination Gateway Genmask Flags MSS Window irtt Iface

0.0.0.0 172.16.0.1 0.0.0.0 UG 0 0 0 eth0

172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0

[Keith Young]

This is the kplexlog file when pi rebooted. As you can see, the interface starts up with the 169.x.x.x. IP Range as the Pi hasn't fully connected to the DHCP Server. No tcpdump data

Great: Now we know that dhcp is slow for some reason.  Don't forget to remove those lines form the init file :-)

Some investigation suggested that the LSB header $network in the init script *should* get translated to systemd's network-online.target which should mean that the network is not only initialised but available.  But clearly it isn't working like that.

There's a couple of suggestions in the above link for using NetworkManager or systemd-networkd to wait for the network to be "online".  The suggestions it makes for software developers other than listen addresses to bind to are linux-specific rather than POSIX/cross-platform.

There was this comment on a stack exchange question but not sure that it's relevant:

http://unix.stackexchange.com/questions/209832/debian-systemd-network-online-target-not-working

Give me a day to come up with a systemd unit file to try.

Meantime though "persist=fromstart" *should* have worked.  What did the log files say?  If it failed it should have given a reason

[Declan Barry]

Sorry Keith..

Meant to get back to you on that one. Which tcp section do I put that under?

Declan

[Keith Young]

Meant to get back to you on that one. Which tcp section do I put that under?

"persist=fromstart"?  The interface declaration which was failing.  i.e. instead of:

[tcp]

address=172.16.10.254

port=10110

persist=yes

 use...

[tcp]

address=172.16.10.254

port=10110

persist=fromstart

[Keith Young]

Teppo Kurki has suggested using a dhcp client script:

> Start kplex with a dhcpd hook?

> 

> https://github.com/tkurki/marinepi-provisioning/blob/master/roles/l2tp_vpn_server/tasks/main.yml#L40

See:

http://linux.die.net/man/8/dhclient-script

These are scripts which the dhcp client calls when it changes state so that you can perform actions when you gain an address, rebind, release an address etc.  An appropriate script to address this problem would restart kplex when your network interface gains an address.

This would work very well in this situation and would undoubtedly solve the problem in hand in the most reliable way but I'd like to find out (a) why "persist=fromstart" isn't working (or if it is) and (b) what we can do with a systemd unit file as that might be a more generic solution. Sorry for the delay on the latter: I'm having a few hardware issues...

[Declan Barry]

Hi Keith..

No apologies necessary. Its me who probably should be apologising for breaking your system. LOL!!

I've put the persist=fromstart into the kplex.conf but what should I be looking for?

I've attached the daemon.log for you to have a look at to see if you can see anything as Linux is relatively new to me.. I still have the -d3 debug code in the init script

Declan

[Keith Young]

Ah.  You have two tcp connections: I forgot about the marinetraffic one.  I'm presuming that still has "persist=yes" in the config?  Change that to "persist=fromstart" too, and any other *outbound* (i.e. to a remote server) tcp connections kplex is making, have a "persist=fromstart" statement

[Declan Barry]

Hi Keith..

Sorry for delay getting back to you. I've made thechanges and it seem that the 'persist=fromstart' is doing the job. I rebooted the Pi and waited for the network to establish itself (using ifconfig) and once I confirmed that the network was up and running, I did a tcpdump and hey presto, it was displaying data WITHOUT me having to stop and restart kplex.

At the moment, I am sending data to both Marinetraffic and AISHub and haven't been pinged by email to say that data upload has failed.

Declan

[Keith Young]

Thanks for the update Declan.  Glad to hear that's working.  I still need to sort out that unit file though (sorry: been out sailing of all things :-)

[Declan Barry]

No problem. Lifeboat training and assessments had me over the past few days.

Declan

On Tuesday, 19 April 2016 16:13:04 UTC+1, Keith Young wrote: