Our minimal Pi-Lepton application [Red square no more!]

[Frank Swenton]

We had a lot of problems with red squares and the camera basically crashing (needing a Pi reboot), then decided to create a minimal working example avoiding the SDK---and now it works perfectly!  No extra resistors, no extra scripts or GPIO calls (just the standard setup for the Lepton with a Pi).

We've included it as something to look at for anyone having problems---it was a relief to have something that consistently works!

This little Qt project runs a live RGB feed and, at the click of a button, saves a snapshot as an RGB JPEG and also saves the raw temperature data into a binary file.  #define's allow you to run it on a laptop without a Lepton (with simulated data) or to trace the Lepton data packets on the PI (see Readme.txt in the .zip file).

(You'll need Qt installed---run qmake first to get your makefile, or just open the project on a PC, then make)

Cheers!

[sa...@pureengineering.com]

Nice work frank, thanks for letting me post this to the git hub site. 

here is the url for anyone interested. 

https://github.com/PureEngineering/LeptonModule/tree/master/raspberrypi_qt

[Dale jason]

Thank you for the new code.  I got it to work for me but only if I defined DEBUG_LEPTON to be true.  Otherwise I get a red screen and a stderror message saying ERROR transfering SPI packet.

I isolated it to the Lepton_thread.h file.  If the line (48) that defines the sequence used by debug is not declared even if no one uses it as when debug is false then the ERROR transferring SPI packet message will happen.  Strangely if I comment out the debug defines around that line and also change the name from "sequence" to "sequencex"  that  will work with DEBUG_LEPTON false.  So it must be some sort of complier/stl bug.  Obviously "sequencex" is never referenced but using list to define it somehow make the code work.

[Frank Swenton]

Hmm...very strange!  This was straight out of the box, qmake'd, and built?  I'll look at the code...definitely not an issue on our Pi, so there must be some difference in configuration.

[Dale jason]

Yes qmake and then make.  However I eventually switched to using Qt creator to edit compile and run and that is when I found my solution of instantiating the sequence list variable even with debug turned off.  I know it makes little sense as to why it fails.  The ioctl to read the camera data returns -1 otherwise.  

Dale 

[Steve Gulick]

try putting  "sudo" in front of ""raspberrypi_video".  Then it will work. I had the same "symptoms" before - Steve

[Dale jason]

Sudo does not work with raspberrypi_video on my system at least.  When I do that I get a window without any buttons and error messages stating that it has a bad pixmap.  Today raspberypi_video without sudo started giving me the read screen even with my debug fix.  Yet my python version worked.  So since the python version uses spi speed of 18000000 I tried that.  Then raspberrypi_video started working again and this time without my sequence fix.  Hmmm.  So next I went to the original raspberrypi_video that uses the SDK and changed the speed from 10000000 to 18000000 and now it too works with out getting the red square.  Maybe....

Dale

[Leo Richer]

Hello,
I compiled your soft well and I can launch it. But il have a RED screen, and the Snapshot button. The original Raspberry-video work well, but not your program.

Do you have any idea ?

Leo

[Mark Ricklick]

Leo, 

I had the same problem moving from the original Raspberry_video to this one. I moved the CE1 to CE0, and it worked awesome. 

I have a noob question though, is there a straight forward way to read the data from the raw?.bin files? I'd eventually like to move the temperature data into Octave. (Sorry for a likely simple problem, I'm fumbling through this so that we can try to mount these into our wind tunnel). 

Thanks and awesome work!

Mark

[Mark Ricklick]

Sorry, I take back my original question. 

In Octave I used: 

fopen("raw1.bin")

a=fread("raw1.bin", "ushort");

And that seemed to work. 

[Alexander Hupfer]

Does this give you actual temperature data? I thought this is only possible in the telemetry mode which is undocumented?

[Frank Swenton]

Well, it gives you unitless 14-bit unsigned integers...in principle, you can poll the camera for its temperature and take some images of externally-known temperatures to try to fit temperatures to (camera temp, pixel value) pairs...but this program just gives the raw values. We're looking into the fitting ourselves to turn these into more meaningful numbers, but that's still in process.

[Mark Ricklick]

Frank, I'm in the process of doing this as well (should hopefully have some preliminary results today that I'd be happy to share). 

One question I have though (again being an experimentalist and not a SW guy).

The documentation states that without the radiometry mode enabled, the output is relative to the camera's temperature. I've seen some other posts that can extract this. Your code seems to work the best for me, but because it is not using the sdk, do you have an easy way to extract these values? Ideally I'd like to add it to the raw file, as say 2 more lines of data. I was thinking of modifying the mainwindow.cpp and lepton_i2c codes to read from the sdk. Am I going about this in the right way? 

Thanks,

Mark

[Leo Richer]

Thank you Mark !

It works well !

[Frank Swenton]

Here's an updated version that should use the SDK to poll the camera for its temperature (in degrees Kelvin*100) and record that as the first word of the .bin file---thanks to Mark for poking at this and finding the spots that still had it in PC-testing mode.  Caveat: this hasn't been completely run on a Pi+Lepton Module yet, but it was modified from a copy that was.

Also included is the source for a little program (intended for Mac/PC/Linux, not the Pi---you'll want to get the "Community" Qt Creator 5.x from http://qt-project.org if you don't already have it) that lets you drag the .bin files to it to view pixel values and see the camera temperature at the time of the image capture.

Please let me know if you run into any issues!

Frank 

[InSoon Park]

Thanks for great job,

when I build at PC, it echos below error message.

Any advice would be appreciated.

C:\Users\is\Documents\Project\92. thermalcam\source\RawViewer>\qt\tools\mingw491

_32\bin\mingw32-make

C:/qt/tools/mingw491_32/bin/mingw32-make -f Makefile.Release

mingw32-make[1]: Entering directory 'C:/Users/is/Documents/Project/92. thermalca

m/source/RawViewer'

g++ -c -pipe -fno-keep-inline-dllexport -O2 -frtti -Wall -Wextra -fexceptions -m

threads -DUNICODE -DQT_NO_DEBUG -DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_CORE_LIB -DQT

_NEEDS_QMAIN -I'.' -I'C:/qt/5.4/mingw491_32/include' -I'C:/qt/5.4/mingw491_32/in

clude/QtWidgets' -I'C:/qt/5.4/mingw491_32/include/QtGui' -I'C:/qt/5.4/mingw491_3

2/include/QtCore' -I'release' -I'C:/qt/5.4/mingw491_32/mkspecs/win32-g++'  -o re

lease/main.o main.cpp

g++: error: CreateProcess: No such file or directory

Makefile.Release:461: recipe for target 'release/main.o' failed

mingw32-make[1]: *** [release/main.o] Error 1

mingw32-make[1]: Leaving directory 'C:/Users/is/Documents/Project/92. thermalcam

/source/RawViewer'

makefile:34: recipe for target 'release' failed

mingw32-make: *** [release] Error 2

C:\Users\is\Documents\Project\92. thermalcam\source\RawViewer>

[Frank Swenton]

A little update on the viewer---it was set to take only files ending in "-raw.bin" (as in the RawCapture code), but that probably isn't sensible---the attached update takes drops of any ".bin" file of size 9604 or 9606.

Cheers,

Frank

[must...@umich.edu]

So I've been debugging this for a few weeks, it seemed to be my best bet for the project I'm doing.

My goal is to be able to automatically and reliably capture lepton images on a timer for review later, either in seconds or every so many frames. So far, I have successfully tested the lepton_video many times, and the red square issue occurs in almost  a third of seemingly identical tests. Your program claims to get rid of that which is why it's very important to me.

As of right now, my issue is the "packet reset counter hitting 750" (and keeping a red screen), and I believe this is due to a failure of communication, though none of the other debug lines have triggered. I have the SPI and I2C kernels loaded and wiring is correct, and I'm using CL0 (so I have SPI_dev pointing to 0.0). 

Does anyone please have tips in regard to connection troubleshooting? I am fairly proficient in coding and generally understand most of what the program is doing.

[Mark Ricklick]

Hey, 

Just wanted to post some recent results I was able to get with Frank's code. 

2 images attached compare the Lepton against a FLIR 600 Series image viewing an oblique impinging jet. You can see the Lepton matches pretty well. 

I did a rough calibration of the Lepton, taking images of a piece of copper painted black, and mounted against a heater. 

My first calibration turned out the following linear relationship: y = 0.05872x - 479.22999

Where, Y is the target temperature - the camera temperature. All in Fahrenheit. 

Or Target (F) =  0.05872x - 479.22999 + Camera (F). 

I ran my calibration from about room temp to about 140F. I used two type E thermocouples to read the block temperature. 

Thought you all would like that. 

[Avarachan Cherian]

Hi ,

Just curious, have you looked into Radiometry? According to the datasheet, this should have eliminated the need to consider camera temperature, right?

[Mark Ricklick]

When I first read the spec sheet, that's what I was hoping. There are other posts here that state that it is disabled, and only experimental at this point. I am by no means proficient at programming, so that is a little beyond my capabilities.

[InSoon Park]

Hi,

While executing this application about 30minutes, below error issues continuously.

Packet reset counter hit 750

Any idea for solve this problem?

[captainmoron]

Hi.

I had the same error. The problem here is a troubled connection with the camera. I solved that problem as follows:

In LeptonThread.cpp, Line 20 change LeptonThread::device="dev/spidev0.0" to LeptonThread::device="dev/spidev0.1"

After that, I had no troubles anymore.

Hope, that helps
Regards, Peter

[captainmoron]

My previous posting was refering to the post of must...@umich.edu

"As of right now, my issue is the "packet reset counter hitting 750" (and keeping a red screen), and I believe this is due to a failure of communication, though none of the other debug lines have triggered. I have the SPI and I2C kernels loaded and wiring is correct, and I'm using CL0 (so I have SPI_dev pointing to 0.0). "

Regards.
Peter

[Klayt Morfoot]

Go into the LeptonThread.cpp file in the raspberrypi_qt folder. Scroll down to the code

if (iRow < FrameHeight)  {

    if (++reset >= 750)  {

          qDebug() << "Packet reset counter hit 750";

Change both 750's to 10000.

Worked for me. In the LeptonThread.cpp file within the raspberrypi_video folder, it is commented that 750 is an arbitrary selection "since there should never be 750 "null" packets between two valid transmissions at the current poll rate. By polling faster, developers may easily exceed this count, and the down period may be flagged as a loss of sync". But hey, it worked to make it 10000 for me and the video is fine. I didn't change anything else.

[Klayt Morfoot]

Actually, it worked for a few go's, then it gave "packet reset counter hit 10000". So, back to the beginning. I know that 'fix' doesn't actually fix what's going wrong, but it did work at first haha

[Jake]

Hi Frank, does this work on Pi3? I have tried but get just a red screen.

Cheers

Jake

[Kai Wang]

Yes, it works for me on Raspberry pi 3

On Friday, June 19, 2015 at 9:39:02 AM UTC-5, Frank Swenton wrote:

We had a lot of problems with red squares and the camera basically crashing (needing a Pi reboot), then decided to create a minimal working example avoiding the SDK---and now it works perfectly!  No extra resistors, no extra scripts or GPIO calls (just the standard setup for the Lepton with a Pi).

We've included it as something to look at for anyone having problems---it was a relief to have something that consistently works!

This little Qt project runs a live RGB feed and, at the click of a button, saves a snapshot as an RGB JPEG and also saves the raw temperature data into a binary file.  #define's allow you to run it on a laptop without a Lepton (with simulated data) or to trace the Lepton data packets on the PI (see Readme.txt in the .zip file).

(You'll need Qt installed---run qmake first to get your makefile, or just open the project on a PC, then make)

Cheers!

[Jose Antonio Sanchez Valero]

Mark Ricklick's solution works fine for me. Using this aplication on Raspberry pi3 I had the red square. Then I changed CE1 to Ce0 and the camera started streaming at the moment. Thank you for the effort.

Regards.

[mike dois]

i got this : https://www.youtube.com/watch?v=q6WhSGf0mSM

how can i improve this video feed ?

i am using RPiZero and analog video through 5.8ghz FPV already

[mike dois]

i have found that the lepton sensor i got is a version 3 with larger resolution, 160x120 of array format

http://www.flir.com/uploadedFiles/OEM/Products/LWIR-Cameras/Lepton/Lepton-3-Engineering-Datasheet.pdf

so i think there is an way to make this mods into your code Frank Swenton !

[pwic...@bsmschool.org]

I'm having the same issue, so I can verify that it's not your wireless video. Thanks for the hint, though. I also have the Lepton 3 and the buffering errors I've been receiving make a ton of sense now.

I'll update if I find a solution

[pwic...@bsmschool.org]

GroupGets has a lepton3-dev branch to their pylepton code that works for me: https://github.com/groupgets/pylepton/tree/lepton3-dev

[Tony]

I installed Qt 5 from Qt 4 on Raspberian and ensured that Qt5 was default. However, when I go into the Lepton-RawViewer directory and run qmake && make I get this error message:

Any ideas?

[John Owens]

I get the Error transferring SPI packet, but I only used the qmake && make command to build it, is that correct?

On Tuesday, June 23, 2015 at 2:47:02 PM UTC-4, Frank Swenton wrote:

Hmm...very strange!  This was straight out of the box, qmake'd, and built?  I'll look at the code...definitely not an issue on our Pi, so there must be some difference in configuration.

[John Owens]

On Tuesday, December 12, 2017 at 8:05:54 AM UTC-5, John Owens wrote:

I get the Error transferring SPI packet, but I only used the qmake && make command to build it, is that correct?

Like someone else earlier in the thread. It would only work when I changed the Debug from False to True, but I would prefer not to be constantly be generating debug output. 

[Henrik Cox]

Hi all,

I've been trying to set up a Lepton Thermal Camera (breakout v1.4) with both a raspberry pi 3 and a zero W. I started with this wiring and setup guide and have been troubleshooting with all the suggestions mentioned in this thread with no luck.

Running sudo ./raspberrypi_video 

   - Black pop up window with red square in the upper left corner. 'Perform FCC' button works and causes the Lepton camera's shutter to close and open. 

Running sudo ./raspberrypi_capture [with output file name]

   - Message 'spi mode: 0   bits per word: 8   max speed: 16000000 Hz (16000KHz)', terminal hangs on a blank command line. Ctrl-c to quit

Running sudo ./raspberrypi_overlay [with options such as '-a 155']

   - terminal hangs on a blank command line (pi camera IS connected for this one

Running sudo ./raspberrypi_qt

   - All-red window pops up, 'snapshot' creates and rgb#.jpeg file, error message in terminal 'Error transferring SPI packet'

To troubleshoot I have:

- Carefully removed and put back the Lepton module from its breakout board.

- Placed the CS pin in #24 (CE0) and #26 (CE1) and matched it correspondingly for spidev in the LeptonThread.cpp script (0.0 for CE0, 0.1 for CE1)

- From an answer in this thread, the wiring guide I've followed is allegedly false. So I experimented my full pin layout between these two patterns (found on ):

     - [SPI0] MOSI: #19, MISO: #21, SCLK: #23, CE0: #24, CE1: #26

     - [SPI1] MOSI: #38, MISO: #35, SCLK: #40, CE0: #12, CE1: #11

- Set DEBUG_LEPTON to true in the Lepton_thread.h script, then back to false but commented out the debug defines surrounding line 48, and changed 'sequence' to 'sequencex'

- Changed the SPI speed to 18000000 from 10000000

- Checked the voltage through the Lepton breakout board while plugged in to make sure it isn't fried (its shutter also goes off every few mins).

In addition to LeptonModule (C/C++) I have also tried the same with pylepton (Python) with identical results.

Any insights and advice are appreciated in advance

[Henrik Cox]

Downloading Frank's files, qmaking && making, and running sudo ./raspberrypi_video, I get an all-red window, rgb#.jpg file created in folder when clicking "Snapshot" and "Error transferring SPI packet" OR 'Packet reset counter hit 750" shows on a hanging command line in terminal. Have tried troubleshooting with all the same ways as my reply above, inclusing changing spidev0.0 to spidev0.1, chainging CS pin between #24 and #26, and changing DEBUG_LEPTON to true.

[Hanif Izzudin]

I will say full thank you to you because, with this code, is absolutely make me happiest person in the world, hehehe...

But, there is little problem in my project (Lepton 2.5 with board V1.4 + Raspberry Pi 3 B)

my schematic is

- CS = SPI0 CS0 (Pin 24)            # Use spidev0.0

- MOSI = SPI0 MOSI (Pin 19)

- MISO = SPI0 MISO (Pin 21)

- CLK = SPI0 SCLK (Pin 23)

- GND = GND (Pin 14)

- VIN = 3.3V PWR (Pin 17)

- SDA = I2C1 SDA (Pin 3)

- SCL = I2C1 SCL (Pin 5)

According with the suggestion above, i make some change in ....

* LeptonThread.cpp

- const char *LeptonThread::device = "/dev/spidev0.0";   # (can change in spidev0.1)

- unsigned int LeptonThread::speed = 18000000;            # (from 16000000)

- qDebug() << resets << "reset," << errors << "errors";    # (before this line is commented)

* LeptonThread.h

- #define DEBUG_LEPTON true                                       # (before this line is false)

when i run the Lepton-CaptureRaw, the result is good like this, but... 

but..., sometimes when i try 10 times run this i can video it the thermal video and when i press the button "Snapsot" it will give me the .jpg picture (1-rgb.jpg, 2-rgb.jpg) and .bin format (1-raw.bin, 2-raw.bin) that this raw format will show a temperature value (i'm not yet with this) but 2 times will fail. 

The error is "Error transfering SPI packet" and the layer full of red. Can you help me with this? why this happen? 

[Deivydas Dante]

Hey Henrik,

Did you manage to solve the problem? Which lepton are you using? I get exactly the same error as you and I tried all the possibilities already as you did. This left me really frustrated. 

Regards,

Deivydas

[Jean-Marc Fournier]

Hello Frank

I use the following configuration:

- Lepton 2.5

- Dev kit v2.0 (the one with 20 pins)

- Raspberry Pi 3B+

- Raspbian OS

I tried without success everything (almost) on the web and lastly your solution: our_video.zip

Test facts:

- qt4 is properly installed and compilation works (just 3 warnings when DEBUG_LEPTON is true)

- after compilation (qmake && make) for the first run I get "Packet reset counter hits 750"

- further run attempts return "Error transferring SPI packets

- the window is full of red and the snapshot button appears

- if I set to 'false' the flag "HAVE_LEPTON", it works and I can snapshot an image (the colored pattern)

- SPI CS is programmed to "/dev/spidev0.0" and SPI_CS from the dev kit is connected to RPI GPIO 24 (SPI_CE0_N)

- MISO is as well connected (not sure we need that): from dev kit SPI_MISO to RPI GPIO 21 (SPI_MISO)

I cannot get it work.

What do you suggest ?

Thank you for your help.

JM

[Kazem Gholibeigian]

It seems very crazy, but when I changed the jumper wires with the short ones(shortest), I did not get this error anymore!

Just change the jumper wires!