adding CMUcam PIXY to Arducopter

[Antoine Lecestre]

Hi Guys,
I've been working for a few days on implementation of CMUcam PIXY to Ardupilot.
I have created AP_PIXY library which is available in my GIthub (https://github.com/2Zit/AP_Pixy) and based on the original Pixy's arduino library (https://github.com/charmedlabs/pixy/tree/master/arduino).
My objectif is to use the Pixy as an "optical flow" to allow a copter to hover precisely over a target, to land precisely on it or event to recognise color codes and do actions accordingly.
The library works well when I compile the "AP_PIXY_test.pde" example to my APM2.5 through the arduino IDE for Ardupilot. My problem is when I try to add it to the main code. When I just include all the files there is no problem and the APM works fine. But when I try to define a variable in the UserVariables.h such as : "static AP_OpticalFlow_PIXY Pixy_Flow" the APM doesn't work. It compiles with no errors but the APM doesn't restart, no lights are ON exept the "Alive" one. I've uncomment the "#define USERHOOK_VARIABLES UserVariables.h" in AP_Config.h and disabled the optical flow through "#define OPTFLOW DISABLED" . Nothing else was changed.

what am I suppose to do to make it work? What are my errors?
Thanks,
Antonie

more info on pixy here: https://www.kickstarter.com/projects/254449872/pixy-cmucam5-a-fast-easy-to-use-vision-sensor

[Ben Nizette]

Two possibilities spring to mind: First, the APM is out of memory.  I would have thought you'd get compile-time errors here but perhaps not.  When you run the code without the variable defined, how much free RAM is reported? How big is that data structure?

The other possibility is simply a bug in your code, most likely in the constructor of the object.  You do say that the test sketch works though, so it'd have to be something to do with the particular build (e.g. buffer overrun or other memory corruption that might stamp on empty space in the test sketch but hit something important in the full build).

Ben.

--
You received this message because you are subscribed to the Google Groups "drones-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to drones-discus...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Randy Mackay]

Antoine,

 

     That sounds like the flash space is over the APM2 limit.  I think you’ll need to try disabling some more features through the APM_Config.h to free up enough space.  By the way the Arduino IDE incorrectly reports how large the firmware can be.  In fact, if it’s over 145,800 bytes it won’t work.

 

     What you’re working on looks really interesting and seems like you’ve made good progress already.

 

     I think the PIXY does a lot of the processing for you so you *may* be able to get it to work with the APM.  When it comes to doing custom image recognition type stuff, I think a good number of us on the dev team are moving towards the Pixhawk + Odroid.  http://dev.ardupilot.com/wiki/odroid-via-mavlink/.  So at the moment I’m working on trying to get the copter to find and pop balloons for the Sparkfun AVC 2014 competition.  Anyway, best of luck.

 

-Randy

[Antoine Lecestre]

Ben, Randy, 

Thank you for your reponse. You were right, it was a memory problem. To much memory allocation for Pixy's minimum array size. It was not necessary to disable more features through the  APM_Config.h and now APM and PIXY are communicating as they should.

It's now time for debug!

@Randy,  I've also moved to Pixhawk + odroid or UDOO quad + OpenCV on one of my Rover. But I also want to add Pixy to ArduPilot for it's light weight, low price and "plug and play" solution. I know this is not the best device, but for some really simple applications such as color codes recognition or even precision landing, it coul be perfect. Many people could benefit from not having to install opencv on a OS, with custom camera connection or other things. I want an I2C or SPI device that you just plug, you anable it through MP and your ready to go. 

By the way I'm also working on special a special of_RTL that work as a classic RTL but use PIXY for a precision landing on a specific marker.

More to come!

Antoine

[Thomas Stone]

@Antoine,

That sounds really neat! Have you had any success with using Pixy for this project?

I created an infrared tracking kit that has helped me do target-tracking with Pixy. 

bit.ly/IR-LOCK

Best,

Thomas

[Randy Mackay]

Antoine, Thomas,

 

     We definitely want precision landing in arducopter.  The Pixy+IR LED method and/or the Odroid+Webcam method would be great.  For the Odroid+webcam method I think the balloon-popper stuff would be a decent starting point.

 

     I’m very happy to help in any way I can which includes advice, adding controls to arducopter, etc, email me at rmac...@yahoo.com or skype me at rmackay9 if you need more immediate help or want to talk about how it can be done.

 

     I think the boat has sailed on the APM2 though so best to focus on the Pixhawk and eventually the Linux boards (Pixhawk Fire – which doesn’t actually exist in any reasonable quantity yet).  To integrate the Pixy into the Pixhawk is a little tricky because it involves getting your hands dirty in the PX4Firmware layer but it’s still do-able.

 

-Randy

--

[Thomas Stone]

Randy, thanks for reaching out. I sent you an e-mail.

Best,

Thomas

[Antoine Lecestre]

@Thomas,

Yes, I had success with using Pixy with APM trough my AP_Pixy library and I2C. Finaly I moved to a more complex solution with much more computing power and space for developpement... I will be comunicating on the results in a couple of weeks

I'm also interested on how you modified the Pixy firmware to detect the white spot on the image created by the IR filter. I suppose it is no more hsv... 

@Randy

thanks for your help. Your balloon-popper was indeed a good start for what I developped. 

[Thomas Stone]

@Antoine

That's great! So you got it to work with APM 2.5/2.6? 

Yes, I am working with simple 'white blob' detection, rather than color detection. The filtering is the key component that makes this method work reliably ... I look forward to the release of your results. 

Best,

Thomas

[James McGregor-Macdonald]

Hi Thomas,

I'm currently working on an undergraduate research project using Pixy with APM, so I thought you might be interested in hearing about it and might possibly be able to help us too! My project partner and I are developing a control system for a quadcopter to fly with a slung-load from one point to another while minimizing the payload oscillations. Pixy will be used for detecting the position of the payload (so that we can calculate its approximate angle), as well as targets on the ground for the quad to position itself over. At the moment we are trying to get Pixy communicating with the APM, with not much luck. We have tried using your AP_Pixy library, and we're continually getting "Connection Error". We've tried to identify the source of this, and the error seems to be coming from reading the I2C port within the getWord function. I noticed that you said you now have APM and Pixy communicating as they should, and I was wondering if your library has been significantly updated since you first posted it, or if you have any ideas about what could be going wrong?

Thanks,

James

[Thomas Stone]

Hi James, 

That sounds cool! Actually, a researcher down the hall did his masters thesis on that topic (w/ helicopters). 

Regarding the APM/Pixy integration, you will want to talk to @Antoine. I am considering Pixhawk integration (w/ ArduCopter), but that is future work. 

If you want to talk more about Pixy target detection, you can shoot me an e-mail or just check out my page (bit.ly/IR-LOCK) (thomas a t irlock d o t com).

Best,

Thomas

[James McGregor-Macdonald]

Haha you're right, thanks Thomas. I'll just copy-paste this message to Antoine. Cheers!

[James McGregor-Macdonald]

Hi Antoine,

I actually sent this message to Thomas as you can see below, though it was meant to be sent to you. Here it is again anyway:

I'm currently working on an undergraduate research project using Pixy with APM, so I thought you might be interested in hearing about it and might possibly be able to help us too! My project partner and I are developing a control system for a quadcopter to fly with a slung-load from one point to another while minimizing the payload oscillations. Pixy will be used for detecting the position of the payload (so that we can calculate its approximate angle), as well as targets on the ground for the quad to position itself over. At the moment we are trying to get Pixy communicating with the APM, with not much luck. We have tried using your AP_Pixy library, and we're continually getting "Connection Error". We've tried to identify the source of this, and the error seems to be coming from reading the I2C port within the getWord function. I noticed that you said you now have APM and Pixy communicating as they should, and I was wondering if your library has been significantly updated since you first posted it, or if you have any ideas about what could be going wrong? If you do have a working AP_Pixy library would you be able to upload it?

Thanks,

James

On Monday, 1 September 2014 19:06:28 UTC+12, Antoine Lecestre wrote:

[Antoine Lecestre]

Hi James,

cool project! First, the library I published had a problem with memory allocation for Pixy. There was not enough space left for APM to run. I updated the library on my GIthub (https://github.com/2Zit/AP_Pixy)  so it should work fine now. As for "Connection Error", I created this message to indicate an I2c connnection error. That's why you have this message each time getWord() is called. have you change the "data out port" through pixymon? I2c is not enabled by default on pixy. You have to set data out port to 1 to enable I2C (follow this wiki). In addition, you have to verify that the address set with the #define in the begining of AP_Pixy.h correspond to the one set in pixymon. If you have not change it, it should be good, but it's worth a check. 

I hope it will help you.

Antoine

[James McGregor-Macdonald]

Yeah we double checked the data out port and the I2C address and it all seems in order, so we're wondering why it isn't working if it did for you :/ In regards to the connection error, we've isolated it to within the receiveByte function which is in the I2C driver within AP_HAL_AVR, and it's waiting for an interrupt but timing out - the weird thing is we've tried it on the board with the standalone Pixy library by itself and it worked, so we know that the connection isn't faulty. If you have any more advice it would be greatly appreciated. 

Thanks for the feedback so far though!!

--

[Mark Middleton]

Hi Antoine,

When I try to verify the AP_PIXY_test using the modified Arduino compiler for APM, I am getting:
C:\GIT\ardupilot\libraries\AP_Pixy\AP_OpticalFlow_PIXY.cpp: In member function 'virtual void AP_OpticalFlow_PIXY::update()':
C:\GIT\ardupilot\libraries\AP_Pixy\AP_OpticalFlow_PIXY.cpp:85:5: error: expected ',' or ';' before 'uint16_t'
     uint16_t min_dist_from_center_pixels = sqrt( pow((PIXY_PIXELS_RESOLUTION_X / (float)2.0), 2.0) + pow((PIXY_PIXELS_RESOLUTION_Y / (float)2.0), 2.0));
     ^

Even after adding the missing semicolon on line 85, I'm still having troubles compiling.
Did you happen to have a more updated solution or library to share?

Thanks for all of your efforts,

- mark

[楊尚儒]

Hello my name is Sam. I am working on a project my own. I am attempting to program a 4 propeller arducopilot with an ardupilot2.72 module. It is equipped with a pixy CMUcam5 that can be used to detect color. My program, once executed, will have the camera constantly look for a landing target. Once the camera detects the landing target, it will override remote signals for pitch and roll in order to maintain constant position over landing target. The only signal that is left allowed from the remote will be throttle control. Do you have any advice on how to approach? Any coding suggestions? Thank you.

My email:4a13...@stust.edu.tw

      or:samnp...@yahoo.com.tw

Mark Middleton於 2014年11月24日星期一 UTC+8上午7時41分26秒寫道：

==南臺科技大學==

[Thomas Stone]

Hi Sam,

We have done some work with infrared beacon sensing using a modified Pixy (IR-LOCK, www.irlock.com). You can find the preliminary results here and discussion here:

https://groups.google.com/forum/#!topic/drones-discuss/jjS8J7nLte0

There is a driver for IR-LOCK that enables it to be used with Pixhawk. This driver may also work with the standard Pixy sensor, but I am not sure. 

Best,

Thomas

--
You received this message because you are subscribed to a topic in the Google Groups "drones-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/drones-discuss/c0uyd4QYPQk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to drones-discus...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Thomas Stone

Thoma...@gmail.com

tho...@irlock.com

[are rief]

Thomas Stone, i have read and explore your ir-lock. all i know is we need to connect the pixy camera with pixhawk by using i2c cable. but can you teach me how to open and edit code and reprogram it in mission planner? or you use other software to edit all the source code??

-arif 

[tho...@irlock.com]

If you need connection instructions regarding IR-LOCK, you should start here (see Tutorial tab). 

http://irlock.com

I am not sure which code you are asking about. If referring to Pixy sensor code, you should probably start here:

http://cmucam.org/boards/9/topics/7109

Best,

Thomas