Isoblue 2.0 (repost)

[Aaron Ault]

I am posting this email with permission from Henry Northern:

Hi,

I am an arable farmer (and ex-research scientist) from the UK and have been following the development of Isoblue with interest on the website and in the google group for a couple of years. I really appreciate the hard work you guys have put into this project and the fact that you have open sourced the information.

I am considering building a device to record yield data from our Claas combine harvester this harvest which is now only a few weeks away. I have read Yang's posts with the hardware parts list and feel fairly confident putting a device together, however, I have been able to find less information about the software side of things. Is there a tutorial or some information somewhere giving some details of how to set up the software on an Isoblue 2.0 to get it logging CAN frames? I have had a quick look through the github repo but it's not obvious to me exactly what packages I would need to install/how to install the isoblue code. Initially I am happy to not worry about the cellular connection and just to log data to a SSD. Also do you have an order of magnitude estimate of the data acquisition rate to expect from a combine harvester? Just so I know how quickly my SSD will fill up!

Many thanks,

Henry Northern

[Yang Wang]

Hello Henry,

I will answer your questions one by one:

however, I have been able to find less information about the software side of things. Is there a tutorial or some information somewhere giving some details of how to set up the software on an Isoblue 2.0 to get it logging CAN frames?

The software setup (not the code themselves) for the whole ISOBlue system is a little complicated. The ISOBlue software uses a Bitbake build environment. Currently, I use a manufacturer-supplied (Toradex) build environment with additions of ISOBlue needed patches and software packages. There are still a bit of work for me to do to make the software deployment easier for everybody by writing down documentations as well as getting a straightforward bitbake building environment setup for ISOBlue. It will be there but it is not there yet.

The quickest solution is this: I will share a flashable image that you could flash your board with all the ISOBlue2 stuff and play with it. The image is the same as the one that all deployed ISOBlue2s are using but with different ISOBlue2 identifications. After flashing the image (before rebooting or anything), you might need to temporarily disable can-watchdog.service. This systemd service monitors CAN activities and suspend the ISOBlue once the activity is absent. If the system suspends, the only way to wake it up is via CAN activities. The intention is to automate the machine data collection process since the ISOBlue2 can suspend and be woken up automatically.

Once your ISOBlue2 is running, it should log all CAN frames locally onto the SSD. You probably don't need to be worried about it sharing your data online since your board does not have a cell module. There is one thing you need to verify though: you need to somehow download kafka_can_consumer.py from the repo (use v1.1-dev branch, and it might be better if you clone the whole repo instead of just a file since the script uses a schema that resides in one folder above it) and run it to see if the CAN frame loggers are running correctly. If you see a stream of timestamp, PGNs, and data payload, then the ISOBlue2 is definitely logging.

Remember this image is fully loaded, meaning that it has all the libraries need and is trying to log GPS data, misc. debug information along with systemd services to start/restart pretty much all the programs.  If you just want to set up ISOBlue to log CAN frames without all the other stuff running, it is definitely achievable given that you have some experience in embedded systems, git, and autotools. You can use the following as a very brief tutorial and some pointers:

1) Take a Toradex imx6 board that is flashed with latest manufacturer's image, make sure to follow this to setup a native compilation environment.

2) Install and set up JDK (required by Kafka).

3) Install python-kafka and avro's python binding.

3) There are 3 ways that you could set your board as a CAN frame logger:

1. If you want to save some time and have a quick prototype, you could just install can-utils and run candump as a logger and redirect the candump output to a file.
2. I would say take a look at this documentation, write a bare minimum program, make it and run it for logging CAN frames.
3. This way uses my program; it derives from the bare minimum with the addition of librdkafka for message distribution and avro-c for data serialization. If you are interested in using my program as well as the data logging scheme, you would need to install these two libraries first. Just a quick explanation on the data logging scheme, the current ISOBlue2s run an instance of Kafka broker natively. The logger programs log the data first, uses avro to serialize the data (mainly to reduce the data size) and then produces onto a Kafka topic managed by the broker. Each Kafka topic can be viewed as an append-only data queue. Details on Kafka can be found here. After you have some background on this, please proceed to step 4.

4) Download Kafka onto your board and run zookeeper and broker. Tutorials can be found here.

5) Go to the isoblue2 repo, switch to the v1.1-dev branch, and go to the producer directory. Run configure and make. The executable you are interested in should be in the kafka_can_log folder. Run it with the right flags along with the kafka_can_consumer.py script in the consumer directory, you should be able to see ISOBUS messages if the board is correctly connected to the ISOBUS on the machine.

The isoblue2 repo also contains some handy udev rules. For instance, 70-isoblue-can.rules gives aliases to CAN interfaces and 80-wakeoncan.rules set up wake on CAN feature automatically during boot.

Also do you have an order of magnitude estimate of the data acquisition rate to expect from a combine harvester? Just so I know how quickly my SSD will fill up!

We were worried about this too initially. However, for one experiment that we ran on one of the Case combines, for the length of 60 days, all logged data (CAN frames, GPS data, and misc) occupy approximately 70GB. It is not that much but the size might depend on the model type of the machine.

Hopefully this helps. The tutorial might be quite brief and let me know if you have any questions. I will post the link to the image tomorrow.

Regards,

Yang

[Rifo Turs]

Hello Aaron,

Have you kept in touch with Henry Northern?  I was wondering if he was able to interpret Claas combine harvester data.

thanks a lot
rifo

12 Temmuz 2017 Çarşamba 06:14:40 UTC+3 tarihinde Aaron Ault yazdı:

[Aaron Ault]

I have not heard anything on that particular project: if you have access to one and can record a CAN stream, we're starting work on an open source tool to extract any known parameters from a growing list of known things and we can add it to that.

Thanks!

Aaron

--
You received this message because you are subscribed to the Google Groups "ISOBlue" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isoblue+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rifo Turs]

okey, I am trying to get access to anything that has an isobus debug interface. My friend who had access to the equipment has no longer has that :(
I will record and upload it to the database.

is it possible to tell about the latest list of known things? Are there any additions to the list of messages in pysobus repository?

Farmobile seems to have a much larger list of vendors but don't know if they share it.

13 Ekim 2017 Cuma 17:40:36 UTC+3 tarihinde Aaron Ault yazdı: