Future plans for APM development

[Andrew Tridgell]

Craig asked me if I could put together something that gives a bit of a
road-map of future plans for the software side of APM. Road-maps are
tricky things to get right for open source software projects as the
rapidly changing nature of the code and the types of volunteer
contributions we get are hard to predict, so please consider this
road-map only a very rough guide to what I personally think would be
good things to work on over the next year or two.

I should also point out that I am most heavily involved in the
APM:Plane fixed wing code and the APM:Rover ground vehicle code. I'm
sure Randy, Leonard, Robert and others working on the APM:Copter code
have their own plans for that part of the code.

The items below are in no particular order, and most of them don't
have any sort of definite time frame. If you want one of these to
happen soon then jump in and help!

Library refactoring
-------------------

A general theme in APM development over the last couple of years has
been moving code from the individual vehicle directories to a common
library. The motivation behind this work is to give more consistent
behaviour between vehicle types, and also to reduce the time it takes
to maintain the code. A lot of code has moved into common libraries in
over the last couple of years, but there is still more to go. Perhaps
the biggest remaining pieces are:

- mission handling (see AP_Mission below)
- common MAVLink handling (see libraries/GCS_MAVLink/GCS_Common.cpp
for a start on that)
- common sensor startup (maybe we need an AP_CoreSensors library?)

AP_Mission
----------

The AP_Mission library was started by Brandon Jones to move all of the
mission handling code out of the Copter, Plane and Rover code into a
common library. We've taken a long time to get this integrated into
the git master branch, but it now looks like it will be integrated
soon as Randy has been doing a lot of work on it. That will be another
big piece of common code.

Extended Kalman Filter
----------------------

A major effort is underway at the moment to integrate the EKF created
by Paul Riseborough into APM. It is in git master now, and is working
very well for fixed wing. Thanks to some great collaboration between
Peter Plischka, Tom Coyle and Paul Riseborough it looks like we may
get the EKF tuned well for copters and rovers soon too.

Once it is well tuned and tested we can look at enabling it by
default. The end result should be a good improvement in position and
attitude accuracy, and greater robustness.


Dual-sensors
------------

We are now starting to take advantage of multiple sensors of the same
type on Pixhawk. The first efforts are with the accelerometers, using
the different sampling rates of the MPU6000 and LSM303D to reduce the
effects of accelerometer aliasing and reduce noise. The gyros are also
being averaged where possible. This has been integrated with DCM, and
Paul Riseborough has done a more sophisticated weighting between
accelerometers for the EKF code. We have some flight logs where the
result is a really dramatic improvement in position and attitude
accuracy - coping with extreme aliasing that would otherwise crash the
aircraft.

We also have the beginnings of dual-GPS support in git master. Right
now we are just logging the 2nd GPS, but I expect we will start to
fuse in both GPS sensors into the EKF in the near future, and start to
see some real position and inertial improvements from the 2nd GPS.

Differential GPS and Piksi
--------------------------

Differential GPS has huge potential for improving positioning
accuracy, especially in applications such as swarming where relative
positioning is the priority. We have some (very!) experimental code to
do RTCMv2 based differential GPS working, and Niels Joubert is working
on a driver for the new Piksi GPS which should offer us the ability to
do full RTK positioning.

I think the integration of the Piksi with APM has huge potential, and
I'm really looking forward to seeing the first results from that work!

Sonar/Lidar
-----------

While still on sensors another area that needs some work is on
distance sensors such as sonar and lidar. We need to get more I2C
sonars supported in all 3 vehicle types and start to add code to use
sonars for landing in planes.

The new Lidar-lite from Pulsed Light looks very promising for plane
landing assistance as well. The 40m range and high accuracy seems
ideal, and I think it could make precision landings on fixed-wing
aircraft much better. It also has great potential for copters and
rovers!

PXF and Linux based APM
-----------------------

On the hardware front a very exciting development is the effort that
Philip Rowse is leading on the PXF Fire cape for the BeagleBoneBlack
running embedded Linux. This effort has the potential to completely
revolutionise how we use APM, as it will give us a single board
running APM that has gigaHz class CPU speed and gigabyte level of
memory. See the video from LCA'2014 for more information.

I'd also like to mention the collaboration on the BBB with new groups
such as BeaglePilot. I think supporting a Linux based platform has the
potential to bring in a whole lot of new developers from the embedded
Linux work to our community which would be great to see. The
BeaglePilot developers have applied for a Google SoC project working
on the BBB port of APM and I'm looking forward to assisting them with
that work.

I'm also expecting that this work will lead to ports of APM to other
Linux based platforms, opening up the code to a wide range of
interesting new boards. The low price and rapid development of
embedded Linux boards means we can start to think about running
imaging and mapping algorithms within APM that we could never dream of
running on the existing boards.

Imaging
-------

I think one of the big future developments with APM is going to be
integration of vision algorithms. The "red balloon" in this years
Sparkfun AVC will be a good motivator to start on that, and Randy is
already thinking about how to solve that problem with a RaspberryPi
talking to a Pixhawk, using OpenCV and possibly a MAVProxy module.

What I expect we will see over the next couple of years is more and
more effort on imaging systems, driven by cheap embedded Linux boxes
like the RaspberryPi, the Odroid-U3 and of course the BeagleBoneBlack!
Some of these systems will run APM directly on the Linux board
(starting with the BBB) while others will use a "companion computer".

I think we'll see people doing SLAM (Simultaneous Location and
Mapping), vision based landing systems, more sophisticated optical
flow systems, sensing of other vehicles (maybe for sense and avoid?)
and numerous other applications.

We've already done some of this with the CanberraUAV imaging system on
APM:Plane, but I think it is going to become much more common.

ROS Integration
---------------

I'd really like to see better integration with ROS, The Robotics
Operating System. The ROS system has a rich set of planning, mapping
and vision code that we should be able to take advantage of in
APM. I'd like to see better ROS<->MAVLink integration and some nice
demos of using ROS to do interesting things like SLAM and object
tracking.

Optical Flow
------------

More immediately it would be great to properly support the PX4 Flow
optical flow board for better ground speed/altitude/heading
support. It is a very interesting sensor that is already available, we
just need to do some integration work to take advantage of its
capabilities. Paul Riseborough has already started looking at it, but
it is early days!

Terrain Following
-----------------

One of my personal interests is building terrain following into APM,
based on both stored terrain height data (probably SRTM initially) and
on sensors like the Lidar-lite. I'd like to use the large storage
space we have on the SD card of the Pixhawk to allow us to pre-load
areas of SRTM data and then allow missions to be specified as AGL
altitudes. This would make it possible for a RTL in rough terrain to
automatically follow the terrain on the way home, avoiding an
embarrassing crash!

Inter-drone Interactions
------------------------

I think we are going to see a lot more effort going into inter-drone
interactions in the next year or two. This has already started with
work on supporting multiple aircraft in DroidPlanner and work by Luke
Hovington on multi-point support in SiK based radios. The API work by
Kevin Hester will be a big part of this, but I also think we'll see
direct MAVLink comms between aircraft for things like swarming.

Fixed-wing Landing
------------------

Automatic landing with APM:Plane is still quite primitive. It works,
but involves a lot of setup by the user beforehand, and is not as
reliable as it should be. Michael Day has been doing some great work
on improving this and I'm really hopeful that automatic landing will
improve a lot over the next year. Having better height sensors will
help, but we also need path planning code and code that plans an
approach to a runway based on wind, runway direction and known
obstacles.

Scripting
---------

Work on scripting of APM has been bubbling along for some time now. We
already support in-aircraft Python scripting via MAVProxy on an
embedded Linux box, but we have not yet achieved general purpose
scripting for APM at the level we should have it. Kevin Hester has
been doing some great preparatory work on scripting and I'm hoping we
will settle on a scripting language and API fairly soon.

Fixed-wing vector flight and trick mode
---------------------------------------

Paul Riseborough and I have been talking about adding true 3D flight
modes for APM:Plane for a while, supporting things like prop hang,
knife-edge and maybe even things like rolling loops with the right
aircraft. The EKF work and the new attitude controllers are a step in
that direction but there is a lot more to do. I'm looking forward to
doing some real aerobatic display flights with APM:Plane some day!

Default parms
-------------

One thing I would like to see us get soon is a reliable way for ground
stations to ask for the correct default value for any MAVLink
parameters. This would allow users to more easily see exactly what
parameters they have changed and revert any experiments with
parameters that didn't work out.

I think doing that will involve two things. First we need to create a
vehicle-type parameter default override table which would replace the
current ad-hoc way that copter, plane and rover override parameter
defaults in libraries. A table much like the AP_Param::ConversionInfo
table would instead hold all vehicle type overrides.

The second part of this is MAVLink protocol additions to allow the GCS
to quickly query the default value of any parameter, or possibly of
all parameters. The GCS can then present the parameters with GUI
elements to indicate that a parameter has been changed from its
default value.

Rover reverse
-------------

Rovers are unusual in that they can go backwards and forwards. We now
support going backwards in manually controlled modes, and we support
pivot turning on skid steering rovers, but we really need some smarts
to make the rover use reverse when it gets stuck on an obstacle in
auto mode. It shouldn't be a lot of work to add, so maybe on one of
our Sunday Rover dev calls soon we'll get it done!

SITL on Windows
---------------

The SITL simulator has been a huge boon for APM development but as yet
it doesn't run on Windows except by installing a virtual Linux
box. I'd love to get that fixed, so that everyone can fire up a
virtual rover, plane or copter whenever they want to and explore the
capabilities of APM.

Auto-tune for planes and rovers
-------------------------------

The auto-tune work by Leonard and Randy in copter has been fantastic
and has really helped users get great results. We need to build a
similar auto-tuning system for planes and rovers. I think we should
support both offline logfile based tuning and in-motion tuning. We
don't yet know exactly how this will work, but reading the copter code
would be a good start!

More sensor types
-----------------

There are all sorts of other sensor types that I'd like to see support
for in APM. For example, on rovers we should support wheel encoders
for accurate movement sensing, and on planes we should support RPM
sensors so you know when your petrol engine has stopped, and on all
vehicle types we should support more types of fuel and battery
sensors, including fuel flow, secondary battery and charge sensors for
solar chargers.

Boat support
------------

The APM:Rover code already supports boats to some extent, but I'd like
to see better boat support, especially for navigation in strong
currents and better steering code that takes into account the
differences between water and ground steering. A number of people are
looking at using APM for long distance boat trips, plus a APM based
boat with a camera would be a great way to explore a waterway.

High Altitude Planes
--------------------

Quite a few people have already taken APM:Plane to high altitudes in
balloons, with the record being over 100k feet above the ground with
automatic flight home. Paul and I have added code to make APM behave
better at high altitudes where low air pressure changes aerodynamics a
lot, but as we get more log files there is sure to be more to do. The
LOHAN project from the Register offers a great opportunity to get some
interesting logs!

Low rate telemetry
------------------

Our telemetry code is all oriented towards continuous connections at
the moment. For some projects, such as crossing oceans with a boat, it
makes more sense to use very low rate telemetry, perhaps as low as a
few hundred bytes a day. I think we should prototype this using SMS
messages on a GSM modem, then adapt it to satellite modems. I'd like
to see an AP_Telemetry library which takes the current MAVLink stream
as input and supports outputting low rate telemetry.

This same AP_Telemetry library could also be used to provide a nice
interface for HoTT and FrSky telemetry protocols from APM which
several users have asked for.

Hybrid Vehicles
---------------

Interesting hybrid vehicles are coming out quite often now, with
hybrid rover-copter and copter-plane both available. I wouldn't be
surprised if we saw a rover-copter-plane-boat combo in a single
vehicle soon!

It would be nice to support these in APM, but that will be a lot of
work. We first need to continue our work on library integration so
that we can start to think about merging the vehicle types. It is a
longer term research project, but would be a lot of fun to get going!

Antenna Tracker
---------------

A good closed loop antenna tracker has been missing from APM for a
long time. I created a basic sketch for it in Tools/AntennaTracker a
while ago, and since then Matt Ridley and Mike McCauley have extended
it a lot. The recent work by Mike has added SITL support and I hope
that this project will become the de-facto antenna tracking system for
APM, allowing for really accurate closed loop antenna tracking with
very high gain antennas. We plan to use this for OBC'2014, so we need
to get it running well soon!

Failsafe Passthru
-----------------

A significant missing feature from APM:Plane is support for failsafe
pass-thru of controls by PX4IO if the FMU ever fails. The PX4IO
firmware has support for it as long as you use a mixer, so what we
need to add to APM is on-the-fly mixer creation based on APM
parameters.

ADS-B sense and avoid
---------------------

I'd like to see ADS-B receivers integrated with APM for sense and
avoid, allowing APM:Plane to avoid any unexpected full sized
aircraft. This is a key feature for use of APM in any national
airspace system. This will most likely initially happen for Linux
based APM systems with as the PX4 Fire, but could also be done with a
companion computer or ground station assistance.

Windows mav tools
-----------------

We have a lot of very useful log analysis tools that run on Linux and
to some extent on MacOS, but we really need to make these easy to use
on Windows too. I think the easiest approach will be to use
pyinstaller to create windows executables and Linus Penzlien is going
to have a look at that soon. We already have some of the mav tools
running on Windows via pyinstaller, but we need docs and good
packaging to make it accessible to all users.

Continued PX4 upstream merging
------------------------------

A less glamorous but very important piece of work is the merging of
upstream features from the PX4/Firmware and PX4/NuttX repository into
the APM versions of those repositories. This is quite time consuming
because of the testing involved, but brings with it new features and
important bug fixes. For example, right now SBUS2 support is in the
PX4 code but not in APM, and it shouldn't be a big task to merge that
support across.

What have I missed?
-------------------

I'm sure I've missed lots of things that are coming in APM. Reply with
your favourie future feature!

Cheers, Tridge

[Kevin Hester]

Hi Tridge,

I think that's a great list!

Other things I can think of...

Better web services integration

----

I think the 'mega droneshare' work item of the API work will provide much of this. Over time I think that for things like log viewing/analysis moving much of that into the web allows all the GCSs to share some common behavior - and let users 'self help' by having an easy way of sharing flights/projects/problems.  Also, live flight monitoring (and in some special cases control) will just be cool. ;-)

Better support for wifi telemetry

----

Once we have something like the BBB it becomes easy to consider wifi for our telemetry link (rather than the SiK radios).  Of course this only works for short range applications (<80 m), but it has some nice benefits: cheap high bitrate radio included in the vehicle, no special radio needed on the GCS, support for N vehicles talking to one GCS machine (i.e. super cool swarming etc...), live video over the same radio as used for telemetry, etc...  (I did a little experiment with a CC3000 on a PX4, but wifi really makes the most sense from a COGS standpoint on a linuxish CPU) 

3G telemetry

---

One thing I'd like to work on after release 0 & 1 of the API is out is making the little 3G adapter widget 'just work' out of the box for even a non dev. (This has a lot of overlap with your low bitrate telemetry item - so I might be able to take that one) 

--
You received this message because you are subscribed to the Google Groups "drones-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to drones-discus...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Brendan O'Reilly]

Picking up on the Linux and Wifi topics, I'd love to be able to browse to my vehicle using any web browser and get a full GCS experience.

[Gary McCray]

Super list Tridge really happy to see so much great stuff on it.

One of the things I am starting to work on is integration of Pixhawk as a part of an interactive multiprocessor system.

I have now got an Odroid U3 and my intention is to use it for IO and environment sensor processing in my 2 wheel robot using ROS.

I got their very nice little IO interface board which includes both an I2C interfaced GPIO chip and a small Atmel Arduino processor set up for IO / A/D and PWM.

I will also be interfacing a serial port based multi channel electronic potentiometer or D/A.

This will let me drive my robots hall sensor based brushless bicycle wheels and use the hall sensors for wheel position feedback.

Basically the thought is to not compromise or modify significantly the very capable code already running in the PixXXX box but to primarily use the mavlink interface to interact 

with the Odroid and let the Odroid take final responsibility for control using ROS.

This might be a bit sluggish for something that flies or is trying to win a race, but it might be a very good solution for a non-time critical application such as this.

Sort of the bottom line is I think it might be a good idea to consider the use of the APM - PixXXX as a component in a true multi-computer system.

I believe this sort of approach can reduce the required software development necessary to implement novel capabilities and can provide increased flexibility.

The built in on board Linux development system in the Odroid U3 does seem to make it easier to program and modify and its multi-core architecture seems very well designed for resolving competing real time tasks.

When we get to 3D point clouds, SLAM and visual object recognition, you really need a multi-computer approach in any case.

I know we are developing the BeagleBone Black / Pixhawk-Fire, but I also think the BBB is a good candidate for using as a co-processor with a PixHawk.

This isn't really so much of something to add to the list as it is simply a alternate way of looking at it.

Best Regards,

Gary

[Gary Mortimer]

Gosh, looks amazing exciting times

[Thomas Bashford]

I'm most interested in the additional sensors (and onboard processing power) for aerial vehicles; reliable sense-and-avoid capabilities will unlock a new level of autonomy.

-Thomas

On Tuesday, March 4, 2014 7:49:16 PM UTC-8, Andrew Tridgell wrote:

[Paul Riseborough]

Increasing the robustness of our code through the use of Monte-Carlo simulations is something to have a think about. Too often it is users that encounter a combination that results in loss of a vehicle, and when introducing new features, it is very easy to break something else that remains undetected due to the extremely large combination of parameters and configuration available.

A realistic flight simulation with representative actuator and sensor models, running a Monte-Carlo analysis would be a good investment. It could be left to run continuously from master, with failures captured for analysis as they come up. This way, when a new feature is added to master, by the time it becomes a release, the chances of a user encountering a previously undetected crash combination is reduced.

This will become more important as we start to add sensors and more complex algorithms to take advantage of them. If we don't move to a more comprehensive  test regime, the crash rate for end users could go up, not down.

[Jon Farhat]

Tridge

Thank you for this punch list of future APM goals.   The one point that strikes home for us, regarding real integration with controlled airspace is your point about ADS-B receivers for sense and avoid.  

"I'd like to see ADS-B receivers integrated with APM for sense and 

avoid, allowing APM:Plane to avoid any unexpected full sized 

aircraft. This is a key feature for use of APM in any national 

airspace system. This will most likely initially happen for Linux 

based APM systems with as the PX4 Fire, but could also be done with a 

companion computer or ground station assistance. "

May we add to this punch list, the need for ADS-B Out, as a real step forward towards integration of drones with humans in the sky.   Communication with ATC tests now are amazing, (not in US) but if the drone community took the leap to provide ADS-B Out long before the FAA 2020 (over 10K ft.) mandate for all aircraft.  

Thanks again

j.

[Doug Walmsley]

FYI.  ADS-B is required for all aircraft in Class A, B, C & E (Above 10,000 ft MSL but not below 2,500 ft AGL). 

How many DIY drones will be allowed above 500ft let alone have the additional weight and cost of a Transmitter (estimated cost $5,000)?

IMHO,  maybe development should be for Mavlink messages should be sent down to GPS and the broadcasted out to world by some means such as a Transmitter or via Internet injection.

I like the idea,  but much discussion should be considered on how this is to be implemented.

Doug Walmsley

--

You received this message because you are subscribed to the Google Groups "drones-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to drones-discus...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Jon Farhat]

First, you are correct about ADS-B and 10,000.  I’m not saying we have to ‘comply’ I’m saying it’s a great opportunity for us to advance the argument to integrate drones flying 500 and under for other aircraft like say, an R44 or corporate helicopter. 

IF drones will be allowed to be used in cities and within airspace, possibly under helicopter charts in LA for instance, then ADS-B would be a slam dunk for the FAA right now. 

Yes, even for the so called do it yourself drones, but APM is used for some serious commercial work, and piloting some very expensive ships now. 

Just a way for those of us in the sky to ‘know where a drone was’.   I was lucky to avoid one near vny.

You received this message because you are subscribed to a topic in the Google Groups "drones-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/drones-discuss/vsSn7g3PSno/unsubscribe.
To unsubscribe from this group and all its topics, send an email to drones-discus...@googlegroups.com.

[Jon Farhat]

A number of drone applications are in front of FAA that seek waivers to fly drones only as high as their water towers.  We put lights on towers and note them on charts.  Drone flight paths are being registered, not just the right to use them in commercial apps.  I know many of these are Mavlink based, APM mostly. 

The resistance seems relaxed if ADS-B Out were present for other aircraft since the nav and position lights aren’t usually visible on UAVs.  

Just curious what the time table was for Tridge’s point.  Not selling anything here. 

[Jon Farhat]

FYI...

http://hackaday.com/2014/08/25/piaware-automated-airliner-tracking-on-the-raspberry-pi/

[Justin Martin]

 Hey Tridge, those are some great ideas!

Right off the bat, RPM should be fairly easy to integrate. Castle Creations EDGE series of ESC already have an AUX cable that is capable of providing this information. Infact I ahve looked around and asked for it in the past. It would be very useful for efficient airframes reaching for efficiency. Also, it could help with multi's (if you could somehow integrate 4+), to help detect an issue, prior to a mishap.

https://groups.google.com/forum/#!searchin/drones-discuss/castle/drones-discuss/RqUdrlhH6K4/41FLumU63SUJ

[Justin Martin]

I should emphasize that when I say "should be easy" I am talking from exactly ZWRO coding experience, and have no idea how much work this would take... From an outsiders point of view, it looks seemingly easy.

[Robert Lefebvre]

Is it an open protocol?

On 27 March 2015 at 18:31, Justin Martin <jwmfl...@gmail.com> wrote:

I should emphasize that when I say "should be easy" I am talking from exactly ZWRO coding experience, and have no idea how much work this would take... From an outsiders point of view, it looks seemingly easy.

[Justin Martin]

Should be! It is designed to work with most telemetry rx on the market.

http://www.castlecreations.com/support/documents/edge_guide.pdf

[Michael du Breuil]

With regards to castle link I'd like to nominate that even more interesting then reading the AUX wire would be utilizing the castle live link protocol (it also works with almost all of the castle ESCs rather then just the EDGE series). Documentation on it is available here: http://www.castlecreations.com/products/castle-link-live.html and they provide a nice breakout board that allows the use of I2C/SPI/UART insead of using the over the throttle wire protocol. The advantage of live link over the aux wire interface is that you now get access to battery voltage, voltage ripple, current consumption, output power, RPM, BEC Voltage, BEC Current, and temperature, and while providing this information allows the AUX wire to be used as a throttle safety key.