Onboard Optical flow and altitude issues with EKF_GPS_TYPE = 3

[Julien BERAUD]

Hi everyone,

I have implemented a version of the PX4 optical flow algorithm and integrated it in APM for the Bebop.

All the processing is done inside APM. It is mant to work on the Bebop with its embedded camera sensor, but since it is using v4l2, it can be adapted to any linux board with a usb camera plugged on it.

You can find the source code here  if you are interested though there are issues to be solved and code cleaning before trying to merge in master.

https://github.com/jberaud/ardupilot/tree/dev/opticalflow

In order to use the optical flow in Loiter mode, I have enabled optical flow :

FLOW_ENABLE = 1

and set 

EKF_GPS_TYPE = 3

The problem is, that even in alt hold mode, I have a lot of trouble with the altitude when EKF_GPS_TYPE is set to 3. I don't understand why but you can find a log attached, altitude_pb.BIN

Paul, maybe you'll be able to tell me what's wrong. The Quad is going up like crazy directly after taking off.

Apart from that the performances seem ok, as you can see in log file opticalflow_bebop.BIN, comparing OF.flowX and IMU.GyrX, thtough I don't have a PX4 optical flow to compare with.

Julien

[Randy Mackay]

Julien,

 

     That is really cool and a really great contribution.

 

     I’ll give my bebop a fly with master and see if my AltHold is working ok.

 

-Randy

--
You received this message because you are subscribed to the Google Groups "drones-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to drones-discus...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Paul Riseborough]

Julien,

A couple of major issues with the optical flow log

1) The body rates in the OF message look like copies of the flow rates. That will not work with the EKF. It needs gyro rates accumulated over the same time interval as the optical flow measurements.

2) The EKF optical flow debug log (EKF5) shows that the measure range EKF5.meaRng is flatlined at zero. Without a range finder measurement, the EKF will not go into optical flow mode.

With the altitude log your setup has large accelerometer biases but your barometer is low noise so I recommend that you perform an accel cal and reduce the value of EKF_ALT_NOISE or improved height accuracy. You could reduce EKF_ALT_NOISE from 2.0 to 1.0 and then try 0.5

Regards,

-Paul

[Julien BERAUD]

Thanks for your answers Paul

A couple of major issues with the optical flow log

1) The body rates in the OF message look like copies of the flow rates. That will not work with the EKF. It needs gyro rates accumulated over the same time interval as the optical flow measurements.

It is due to my misreading of the other optical flow drivers. I have everything available to do that so I'll fix it.

2) The EKF optical flow debug log (EKF5) shows that the measure range EKF5.meaRng is flatlined at zero. Without a range finder measurement, the EKF will not go into optical flow mode.

One of my collegue is currently working on the Rangefinder and when his work is stabilized, I'll try to fly with it. There is still something I don't get about the necessity of having a rangefinder compared to a Baro. I assume it is a lot more accurate but it becomes useless when it is not perpendicular to the ground.

With the altitude log your setup has large accelerometer biases but your barometer is low noise so I recommend that you perform an accel cal and reduce the value of EKF_ALT_NOISE or improved height accuracy. You could reduce EKF_ALT_NOISE from 2.0 to 1.0 and then try 0.5

I'll try that too.

Julien

[Randy Mackay]

Julien,

 

     Re the range finder and vehicle angle compensation, it depends upon the beam width of the sensor.  So a very narrow beam (like a lidar light which is 5deg or maybe less) should be compensated for but for the maxbotix sonar which has a beam width of more like 30degrees it’s not necessary.  At the moment in copter we have a #define which allows turning on the tilt compensation but clearly this isn’t really correct.  Really we should have either a parameter to turn it on/off or maybe we can add a method to each sensor driver to specify whether it should be compensated for or not.  So for example, if we’re using the Maxbotix sonar we never would, if we’re using the lidar light we always would.  It’s a little unclear what we would do in cases where we’re not sure what device is being used (i.e. the analog and pwm range finder drivers could be used for a maxbotix sonar but normally they’re used for the lightware SF02, SF10 sensors).

 

-Randy

--

[Julien BERAUD]

Thanks Randy,

But I still don't understand why it is necessary to run optical flow.

Julien

[Randy Mackay]

Julien,

 

     I think we could try assuming that the terrain altitude is the same as the alt-above-home but we’d have some problems if the two became different because of baro drift or the vehicle actually flies over something that’s higher.  The difference between say 7m and 8m probably wouldn’t matter much but the difference between 50cm and 1.5m might upset the EKF’s position estimate.

 

     If I wanted to give this a try, I’d probably temporarily hack the sonar lib so that we can pass in the current_loc.alt to it and then fly it and see what happens.

[Julien BERAUD]

Hi Again,

I have fixed the body rates, here is a new log, still no RangeFinder, I'll fix that but I might want to push the optical flow before the RangeFinder even if the optical flow Loiter won't be usable yet.

Cheers,

Julien

[Paul Riseborough]

Julien,

Those rates look OK. Did you implement the default method in the PX4Flow library or the the method?  Both work OK, but I found that with the PX4Flow hardware, the KLT method worked better when the image was out of focus or contrast was low. it will be interesting to see the results you get with the BeeBop hardware.

-Paul

[Julien BERAUD]

Paul,

I thought that KLT was the default method. Basically I implemented the exact code with the exact current params of the PX4 Flow.

It basically calculates for each block, the distance that gives the minimum distance with the original block, up to subpixel, and makes a mean of all the values that have been found over the different blocks.

Julien

[Alex G]

Hi,

i am very interested on the source of this code. Do you still have it? It seems that the git link/repo is down. Have you developed anything more on optical flow and altitude for bebop with apm? 

Thank you

[Tom Pittenger]

Looks like it's been merged into master

--
You received this message because you are subscribed to the Google Groups "drones-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to drones-discuss+unsubscribe@googlegroups.com.

[Julien BERAUD]

It has indeed been merged into master. The current code is not fully functionnal, I need to make some modifications. It basically works but has some issues.
A student at ETH Zurich has also implemented another algo using opencv, and I'll be working
with him to get this merged.

Julien

To unsubscribe from this group and stop receiving emails from it, send an email to drones-discus...@googlegroups.com.

[Alex G]

Thank you very much for your response. I finally found on git the different branches. does the bebop-fixes work with optical flow and altitude? are they working with mavlink?

Thank you for your great work!!

--
You received this message because you are subscribed to a topic in the Google Groups "drones-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/drones-discuss/l7NssP3LuMQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to drones-discuss+unsubscribe@googlegroups.com.

[Julien BERAUD]

my bebop-fixes branch does not solve the optical flow issues. I will try to solve them at some point but later.

To unsubscribe from this group and all its topics, send an email to drones-discus...@googlegroups.com.