Automated Landing

[Sunil Shah]

Hello drones-discuss, thought I'd share a project that we were working on at the end of last year.

A couple of fellow students and I implemented a method published in 2001 that allows for accurate localisation based on a fixed landing pattern - our code is on GitHub here, final report here and final presentation here.

This was for a couple of CS classes on robotics and computer vision last semester at UC Berkeley and as part of a Master's project to do with drones (we're being advised by Brandon at 3DR).

Implementation is in c++ for the two ROS nodes we created. We re-used roscopter for control of our ArduPilot, although we ended up forking it to provide better instructions on how to compile and run it (forked repository is here). We ran this all on a Beaglebone Black running Ubuntu, connected via USB to a Logitech C920 webcam, wi-fi dongle and the ArduPilot.

The pose estimation node uses OpenCV to perform thresholding and corner detection. We implemented a very basic state machine and proportional controller to communicate with roscopter (sending RC control signals) that didn't work too well. I suspect this was due to the poor performance on-board - roscopter alone takes 40% of the load when idling and that gives us ~ 1.8 frames per second. 

Performance aside, we get very accurate pose estimates of the multi-rotor relative to the landing station - to less than 5cm deviation from the actual pose (tested in the lab) and less than 0.5 degrees of yaw error.

Sadly we weren't able to get it landing fully autonomously (the wind was gusting 30+mph when we went to test it). Some videos of the pose estimation in action:

http://www.youtube.com/watch?v=9SVWvjeoO1o

http://www.youtube.com/watch?v=gFiCEo77A1Q

Some of the other problems we faced, besides poor performance (= sluggish controller) was the physical size of the our landing pad. Once we went below a certain height (~ 1 metre in absolute terms but ~ 2 metres when flying, because of camera shake) we would fail to recognise the landing pad because it would fall out of the field of view of our camera. An obvious solution to this is to the shrink the pad itself. 

There are no immediate plans to continue this particular project but I hope that this effort and our code may be useful to others! Happy to answer any questions about our findings.

Cheers,

Sunil

--

p: +1 510 229 2304

e: sunil...@cantab.net

w: http://www.geekonabicycle.co.uk

[Robert Lefebvre]

Interesting work, thanks for sharing.

I wonder if the problem of not being able to see the target when you get too close could be solved by having a target within a target?  When you get close, use the smaller target?

--
You received this message because you are subscribed to the Google Groups "drones-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to drones-discus...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Randy Mackay]

Sunil,

 

     That’s interesting.  This year’s AVC includes a challenge to find and destroy large red balloons so I was thinking of trying something similar to what you’ve done to horizontally find and fly towards the balloons.  I’ll have a peek at what you’ve done.

 

     Instead of putting all the control logic into the external board (I’m using Raspberry Pi instead of a Beaglebone), I’m planning to make the external board just provide the sensed position in the image of the target.  The pixhawk will then be responsible for taking that position-in-image info and combining it with the copter’s attitude and position info to come up with an absolute position estimate (i.e. lat, lon, alt).  The position control logic (using gps + accels) will remain in the pixhawk.  I think this will move the lag to affect only the absolute position estimate instead of leaving it up in the position controller.

 

     You’re at least the 2^nd person I’ve heard mention using ROS to control ardupilot/arducopter.  I suspect we should build out a wiki page similar to the recent RaspberryPi one I added a few days ago.

                     http://dev.ardupilot.com/wiki/raspberry-pi-via-mavlink/

 

-Randy

--

[Sunil Shah]

Robert, 

That was an approach we considered but ran out of time to implement. Using QR codes to demark corners was an interesting idea - you'd just need two corners at each 'size' to give you enough information to orient the UAV. Placing concentric QR corners within each size would work well.

In fact, something like libkoki markers might be more invariant to image noise and to being detected at height (it has lower entropy than standard QR codes but still manages to encode 36 bits of information in each marker).

Randy,

That sounds interesting, I'd be happy to contribute our findings to a wiki page. This approach works well because we know the size of the landing pad which might be difficult to replicate with a balloon. Are you considering using stereo cameras to try and calculate the distance to balloons?

Cheers,

Sunil

[Randy Mackay]

Sunil,

 

     Using stereo cameras sounds pretty fancy.  I hadn’t thought of that.

 

     In the competition rules they say that the balloons will be about 10feet (3+ meters) so I am hoping that we will be able to guess their distance based on the size.

[john...@gmail.com]

Randy,

As long as the balloon has a known size and a simple shape and unison color with a clear separation from the background you will get a good size/distance estimation. The accuracy of the estimation will get better as you get closer (provided the camera can still see the entire balloon).

I would first try this approach. If it turns out to be robust enough, it will require reasonably low resources (embedded.ARM in the 1-200mhz class).

1. Convert the camera input to HSV color space for more accurate separation of the target color range and generate a black and white representation of the object
2. Run a lowpass filter on the b&w image to remove small specks
2. Do a simple box search to enclose the object, and calculate the center
3. Add a Kalman filter if needed to make the solution more robust

Alternatively if you want a more robust solution, look for the largest HOG cluster that match the balloon signature. But this approach will require more resources.

- JAB

[Randy Mackay]

JAB,

 

     Thanks for the advice!  The first few I can do, the last one (Kalman Filter) is probably beyond my abilities but I’ll do some maths to project its location and filter that location so it doesn’t move around too much.

 

-Randy

[Al B]

Hi Randy,

Have you made any progress implementing this?

> " I’m planning to make the external board just provide the sensed position in the image of the target."

Won't that be similar to what the IR Lock folks are already doing with the PIXI board instead of RPi or Beaglebone?

[Randy Mackay]

Al,

 

     I haven’t progressed with the precision landing library much in the past few days so it’s still just a skeleton.

https://github.com/rmackay9/rmackay9-ardupilot/tree/precland1

 

     I’m not planning on personally making any hardware, this software library within arducopter is just so arducopter works better with external sensors like the IR-Lock and Daniel Nugent’s Odroid based solution.

 

-Randy

--

You received this message because you are subscribed to the Google Groups "drones-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to drones-discus...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.