Streaming the output of Flir Lepton on Raspberry Pi

[owen]

Hi, 
I connected a Flir Lepton with a breakout board to the raspberry pi and installed LeptonModule according to the guide here:
https://groupgets.com/blog/posts/8-inst ... berry-pi-2
It all works well, and I would like to thank the creators very much.

At the moment, the output goes to a window on the pi display, which I can view remotely using vnc/xrdp.
I would like to stream the output to another linux pc on the same local network. I was thinking netcat on the pi and netcat+mplayer/vlc on the pc. 
I tried on the pi:
./raspberrypi_video | nc <pc_ip> <port>

and on the pc:
nc -l -p <port> | mplayer (I tried several options here but nothing worked)

I use a similar setup to stream raspivid from pi camera module, but I can't make it work with the Lepton.

I have seen this post, but it does say no streaming.
https://groups.google.com/forum/#!newtopic/flir-lepton/flir-lepton/FF7y0NKrMpU

I have posted the same question on the raspberry pi forum.

Any help would be much appreciated.

Thanks

[Kurt Kiefer]

Since that application doesn't output anything on the stdout stream, netcat isn't going to do anything. You'll need to write a program that outputs to stdout in a format that could be consumed by mplayer (without netcat in the equation at all).

If you're willing to do the programming work, the most straightforward way to get this done is to write your raw frames to a v4l2loopback device, then you can probably read from v4l2loopback with netcat. At the very least, you will be able to read from v4l2loopback with gstreamer, and rebroadcast that way.

Now (and full disclosure, I work at GroupGets, and wrote a good bit of the firmware for this board :), if you want something that will work out of the box, I recommend the PureThermal 1 FLIR Lepton dev board. Its USB video output will give you a v4l2 video stream on Linux that can be rebroadcast in standard ways.

Kurt

[owen]

Hi Kurt,

Thanks for your reply.

Programming as suggested is beyond my capabilities I'm afraid.

Is there any way I could get the dev board in the UK?

Many thanks

Owen

[Kurt Kiefer]

Yes, GroupGets will ship anywhere. That page has links for purchase on the store and a group buy for reduced price.

Based on a quick search it looks like netcat will work with V4L:

server: cat /dev/video0 | nc <laptop ip> <port>
client: nc -l -p <port> | mplayer -- -

In the end, there's going to be a lot of ways to stream the output (using gstreamer, mplayer, or VLC), so pretty much take your pick! The readme for the UVC capture examples git has some one-liners for basic capture via gstreamer:

https://github.com/groupgets/purethermal1-uvc-capture

[Marty Sullivan]

Hello, If you look under software in LeptonModule, you should see a simple program I wrote to use v4l2loopback to create a /dev/video device for the Lepton using the original breakout board. (v4l2lepton)

You will need to install the v4l2loopback kernel module to your Pi, which can prove a bit difficult. Use the rpi-source utility (search google) to install the kernel headers for Raspbian, then you can compile v4l2loopback and use modprobe to activate. 

Once v4l2loopback is installed, you can simply run v4l2lepton and it will then be streaming to the device you specify.

Obviously, it is much easier to just get the new Dev board, but if you can't get your hands on one quickly, this will work.

[Kurt Kiefer]

Oh didn't know that was there -- thanks for pointing it out! For my money, this may be the best way to get video out of the breakout on Linux.

The great thing with v4l2 is that it's a standard interface, so it doesn't matter what's providing the stream -- classic breakout or the PT1 -- it's all going to be consumed the same way.

[owen]

Thanks a lot for your help.

I will try to implement Marty's solution in the next few days and hopefully save the money on buying the new dev board :)

[Ron]

Hello Marty,

 

unfortunately I'm still struggling to get the v4l2loopback running. Before I list what I've done please note that I'm

totally new to this topic and therefore some of my questions may sound stupid:

 

1) I hopefully installed the correct kernel headers following this guide https://www.anujsehgal.com/2015/05/12/raspberry-pi-kernel-source.htm

2) I installed v4l2loopback-dkms via

# aptitude install v4l2loopback-dkms

 

How can I determine if the first two steps were successful?

 

3) If I understood this correctly the next step is going to the directory LeptonModule/software/v4l2lepton and there exectute

# make

 This should give me a file named v4l2lepton.ko which is the kernel module? Because I did not get such a file, so I guess I'm stuck in this step.

4) Then I dont know what the next step might be. You said modprobe should be used to activate, but I guess I have to

    install the kernel module  before using modprobe?!
5) Last step I guess is executing

./v4l2lepton /dev/videoX

     videoX would be what I specified with modprobe before?

I hope Marty or someone else can help me with this.

Thank you very much in advance!

 

Ron

[Robby Balona]

Hi Ron

The process to get v4l2lepton running is so badly documented and frankly not really worth the time I spend on it ... You land up with a 80 x 60 image that is so crappy and not worth doing anything with. But here is my process(what I can remember)

Remove totally ever bit of  v4l2loopback-dkms

Go here and follow the "Kernel Building process".. pretty easy .. copy and paste. Use the "Local Building" and make sure you have about 2 day and at least 300 meg space. This will build the modules that you need.

https://www.raspberrypi.org/documentation/linux/kernel/building.md

Once this is done reboot.

Go here and download the zip and unzip it anywhere. Make sure you sudo everything

When done and complied run  "sudo modprobe v4l2loopback"

You should have a /dev/videoX.

Go into the  LeptonModule/software/v4l2lepton the folder and read the README on how to run it...

Hopefully it will work for you.

Again the images are very very crappy. I use Motion to stream it and the increase the size of the webpage by about 20. Was a bit disappointed about this because its a lot of money to pay for the module and the QR front end does a better job, but has no way to stream images from it.

Hope it helps

Robby

[Ron]

Hi Robby,

wow thank you so much for your help! I will try this :)

But one question is left:

Once this is done reboot.

Go here and download the zip and unzip it anywhere. Make sure you sudo everything

What do you mean by "here"? I suppose you forgot to link it?

Again thank you very much!

Ron

[Marty Sullivan]

Hi guys,

I built v4l2loopback from source (the dkms package does not work at all from what I can tell). You need to download the rpi-source script so you can install the RPi kernel headers easily. Then you can compile v4l2loopback normally, copy the .ko file to your /lib/modules and then you can do a modprobe to enable v4l2loopback.

Robby, I'm not sure what happened that it gave you such terrible quality. v4l2loopback simply copies the frames that come directly from the Lepton, there should not be any quality issues that arise from using it. The 80x60 image comes from the Lepton itself. The most ideal way to use the thermal image is for sensing or overlaying the thermal image on a higher resolution normal image of the same scene. The Lepton is not designed for high resolution thermal imaging.

[Robby Balona]

Ops sorry here

https://github.com/umlaeute/v4l2loopback

Thanks

Robby

[Ron]

Hi,

thank you so much for your support and your efforts! I finally got the V4L2Loopback running

and can view the thermal image with VLC Media Player on my raspberry pi 2.

The next step is to display the image in Matlab Simulink using the raspberry support package,

but I dont know if its even possible to do this.

My raspberry is connected to the host computer via ethernet cable and I can successfully ping
the raspberry, thats why I guess the setup should work like this. In the matlab raspberry tutorial they're

using an ethernet-to-USB dongle, but I dont know why an ethernet cable would be inappropriate.

Unfortunately the SDL Video display block is only showing some kind of test image that is

moving a bit or flickering/jittering, sorry it's hard for me to find the right words to describe (see 

below) when the simulink file is running. I'm not quite sure if the pixel format (RGB) is correct.

The other option is YCbCr 4:2:2. Maybe it is necessary to do a colour space conversion in

matlab before displaying it? How can I determine what kind of pixel format is coming from

the raspberry via the V4L2loopback in /dev/video0? Maybe also the sample time isn't

correct at all?!

Thank you very much in advance.

Ron 

[Ron]

Does anyone have an idea? :( Any help would be very appreciated.

[Kurt Kiefer]

While I haven't used this Lepton v4l2loopback device, in general to list the native formats available on a V4L2 device, you can do the following:

v4l2-ctl -d /dev/video0 --list-formats

[owen]

Hi everyone,

It has been a while since I started this topic, but I am still trying to stream the output from a Lepton connected to a raspberry pi running raspbian over a local network.

I have now purchased a Pure Thermal 1 but I still cannot get it to stream. I am quite certain this is due to lack of knowledge on my part.

I have tried many ways but to no avail.

For example, a simple solution could have been:

On the pi connected to the PT1:

sudo su

cat /dev/video0 | nc -u 192.168.1.xxx 8000 

and on the target:

nc -lu -p 8000 > a.mjpg | mplayer -cache 32 -vo gl -demuxer lavf a.mjpg

but i get /dev/vidoe0 invalid argument.

This works with the pi camera using raspivid as detailed here https://www.raspberrypi.org/forums/viewtopic.php?f=26&t=7557

I have also tried gstreamer (based on the example here https://github.com/groupgets/purethermal1-uvc-capture) and other examples I have found for streaming video from a webcam using various programs but none of them have worked.

Any help would be much appreciated.

Thank you,

Owen

[Kurt Kiefer]

Owen,

I ran across this blog post that I mentioned in another thread: http://uavmatrix.com/Blog/38

The guy uses gstreamer to set up an RTP stream to his ground station. Maybe you'll find some help in it.

Kurt

[아쇼크]

Hi Kurt,

I am newbie n this...i am developing an application using Lepton camera (SPI interface) while connected with OrangepiZero Board. Please help me.

Since the lepton module is connected through SPI...i can only check the video output from camera from Gstreamer. I can check the video streaming of USB camera through gstreamer but not of lepton camera. Could you please help me to configure gstreamer for lepton module. Thanks 

BR

/Ash

[Magnús Pétur Bjarnason Obinah]

Hi Owen,

Did you ever get this working for you?

I also need to do pretty much the same thing, except (I didn't see this mentioned in your posts) I need to be able to control the pallette and temp.range on the stream.

What software were you using to control your camera/stream?

- Magnús