Custom MAVProxy Interface for 13 Drone Swarm

[Paul]

I'm working on some research (masters thesis) that requires me simultaneously control 13 drones (identical quads running Pixhawks with ArduCopter firmware and 3DR radio telemetry sets each) from a single computer. I'm controlling everything from a central computer as a first step. After that I'll upload the processing to each individual drone.

The way this is going to work is that all the swarming logic would be handled on the ground by a computer that would then send commands to the drones in guided mode to go to certain GPS positions. The process would be as such -- keep in mind that a computer would be sending all these commands:

• Arm drones
• Tell drones to takeoff to different altitudes and hold
• With drones in guided mode, tell drones to go to specific position and then match altitude
• Repeat based on algorithm used (ie, hexagonal cells, follow the leader, maximum coverage, etc).

I have developed a mockup for a simple interface that would be used to monitor and control the operation. Note that this is not meant to take the place of qGroundControl or Mission Planner, etc. This is a very specialized application that is not for the common hobbyist due to the danger that it can pose if not handled properly. It is purposefully minimal because the whole goal of this is for the algorithm to do the controlling and the human would be there solely for safety overrides and monitoring. Also most people will not have 13 identical drones to work with. 

Currently, I'm looking at using MAVProxy in conjunction with droneapi to do perform these tasks:

1. I need to break away from the terminal when using MAVProxy so that the connecting can be done using my interface instead of the terminal.
2. I need to modify droneapi so that it can control multiple vehicles.

How can I use only the parts of MAVProxy that I need? I'm new to Python so I'm working on learning the intricacies of the language by looking at the source of these packages. I know MAVProxy relies heavily on "MPModule" class so I'm trying to figure out how I can use just what I need from the whole thing. I've been able to connect without using terminal by overriding the "argparse" defaults but I feel like that's a band-aide type of fix. Essentially I need to be able to do these commands:

• Arm
• Takeoff
• Change modes 
• Goto (GPS Position)
• Land
• Implement geofence

This is all the functionality I need out of this. I want to be able to call these commands from within a script. 

I would greatly appreciate someone pointing me in the proper direction when it comes to doing this. 

Last notes: I am fully aware of the dangers of what I'm doing and I'm working on implementing many safeguards. This is a controlled research environment so proper care is taken in all aspects of the project. Also, everything I do will be fully documented and published so it will hopefully help other people working on a similar project.

If you are interested in hearing more about my research you are also welcome to contact me privately.

[benni...@gmail.com]

Hi Paul,

So MAVProxy/DroneAPI is an environment in which you write your code, not something that you call from your own code. As such I don’t really know what you mean when you say you’ve been able to connect without using the terminal by overriding argpase defaults. The flow for this would be to write your code entirely in Python as a DroneAPI module. You’d then add the “module load droneapi…” and “api start myprogram.py” lines to your mavproxy init script. Run 13 copies of MAVProxy with different connections and you’re good.

It’s this last bit that’s going to be the pain for you in a multi-agent system as you then have a many-to-many comms infrastructure that doesn’t fit with the centralized interface you want. I guess you’ve found that as well which is why you’re talking about MPModule. If you aren’t going to use MAVProxy as above, then I wouldn’t use it at all.  It sounds like you’re trying to use MAVProxy more as a library to issue commands: In fact there’s a proper library for that, mavlink. You can refer to the MAVProxy code to see which messages these are for each operation (look for the calls of the form (master.mav.<message_name>_send() ).

The alternative would be to run your 13 instances of MAVProxy with a small droneapi script that simply listens on a socket for actions and implements them and have your standalone program issue these actions in some form that’s convenient. That would isolate you from the low level mavlink stuff I suppose.

Another option if you’re familiar with ROS is to use that along with one of the many ROS/mavlink bridges.

Perhaps Kevin can kick in with a better architecture using DroneAPI? Note that in the future there is intent to add multi-vehicle awareness to MAVProxy so that DroneAPI can request vehicles other than vehicle 0 but that’s no small task.

Ben.

--
You received this message because you are subscribed to the Google Groups "drones-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to drones-discus...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Víctor MV]

Hey Paul,

I support Ben's comments you shold look into ROS. Here you have some information about the some ROS packages to do so. You'll probably need a companion computer running roscore.

I'm interested in the multidrone interface you are setting up. Is it web-based? Under what kind of license are you planning to release your work?

[Paul]

I think you are correct in your analysis. I am able to connect to multiple drones by simply opening multiple instances of MAVProxy but I was hoping to abstract that with a GUI. I agree that creating something myself using the Mavlink messages would be a better option -- That was my first course of action but I wasn't really sure what to look for in the MAVProxy code but now that you've pointed that out, I have more direction in that regard.

Also, I am familiar with ROS (I've used it with ground robots before) so that might be an option -- I wasn't aware of the Mavlink integration. 

Thanks for the input! 

[benni...@gmail.com]

> I think you are correct in your analysis. I am able to connect to multiple drones

> by simply opening multiple instances of MAVProxy but I was hoping to abstract

> that with a GUI.

If you set up DroneAPI scripts to listen on a socket for commands (i.e. don’t use the MAVProxy CLI to issue them) then everything’s trivially scriptable, you can still run a single thing and have every piece pop in to place.

But yes, mavlink or ROS/mavlink is probably a better bet until DroneAPI gets native multi-vehicle support.

[Paul]

Victor, 

Thanks for the info about ROS. I'll definitely take a look at it. 

The interface I'm making is going to run is going to be browser-based since that has the most compatibility (so something like Mavelous) . My current plan, though, is to purposefully make it in such a way that it's not really usable on a small screen or mobile device because I don't want anyone controlling multiple drones from a phone without a proper ground station, etc. But that's just a personal decision right now.

As far as releasing the code for the GUI, I don't think that will be a problem once I publish everything but I'll have to check with my school about their policies since I'm building this as part of my thesis. 

[Víctor MV]

Thanks for the answers Paul.

From our side (OSRF) there's growing interest in providing a formal bridge for drones and MAVLink. We like the idea of introducing native ROS support for ardupilot (basically having another thread with lower priority than the current one publishing and subscribing to ROS topics using the API).

Keep us posted about your progresses. Looking quite good!

--
You received this message because you are subscribed to a topic in the Google Groups "drones-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/drones-discuss/D-NJ_sTBDM8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to drones-discus...@googlegroups.com.

[Paul]

Thanks! I'll definitely try to have regular updates. Progress should be pretty fast since I'm aiming to have initial results (something publishable) in December.

[Randy Mackay]

     Very interesting thing you’re working on Paul.

 

     Some random points from me:

·         I think you should be able to control multiple vehicles right now using mission planner.

o   Use the Beta mission planner and press Ctrl-X to pop-up a selector of which vehicle you wish to control

o   Reflash the 3DR radios with the RFD900 software which supports multiple vehicles on the same network.  This will mean you can get away with just one 3dr radio on the ground station computer

o   Use ArduCopter 3.2 (the release candidate is available through the Mission Planner’s beta firmwares link).  AC3.1.5 is missing the check of the target on the set-mode command so all vehicles would change to the same mode at the same time.

·         I’d be disappointed if you had to use ROS.  Tridge has discussed adding multivehicle support to MAVProxy.  DroneAPI already seems to have at least some of the structure to support this.  At least for the red-balloon-finder project there seemed to be an index associated with the vehicle.  I think if you run into troubles we should ask Tridge and Kevin Hester to resolve them in MAVProxy and DroneAPI.

 

-Randy

--

You received this message because you are subscribed to the Google Groups "drones-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to drones-discus...@googlegroups.com.

[David Pawlak]

Randy,

This link seems quite complete:

https://github.com/RFDesign/SiK

Does the repository indicated therein also have the code for multi-vehicle support? Or is this just the "regular code"?

[Paul]

Randy,

I'm avoiding using the swarm function of Mission Planner because that's not the goal of what I'm doing. qGroundControl also has multi drone support by default. Both of these solutions, though, are manual and tied to a GUI. What I'm trying to do is have the swarming happening based on a script-based algorithm, not something you manually click. My GUI would simply supplement those scripts but is not needed.

I would be very interested in learning more about the RFD900 firmware. Do you have any details on how to flash the 3DR radios with it? Also do you know how it supports multiple vehicles -- essentially how would I know which vehicle I'm talking to from within my script?

[Michael Day]

Howdy Paul,

The radio is going to be the long pole in your tent.  In your case, the many-to-many firmware is very important (unless you want to mange 13 radios at your GCS).  I talked to one of the firmware devs for SiK around last March and at that time they had a branch on github that could support around 3 or 4 nodes at a time on a link that all used the same channel -- that would bring you down to 4 or so radios at your GCS if you still want to go that route.  The repo I was pointed to is here:

https://github.com/RFDesign/SiK/tree/SiK_Multipoint

don't know how far it's come in the last 6 months, but to get much beyond 5 nodes would be quite challenging. 

My team has also been working towards getting a swarm flying and we have been doing something different: using a Wi-Fi mesh network.  I've cc-ed our network expert, Michael Clement, on this if you want to get details.

Good luck,

Michael

[Bill Bonney]

Hello,

AP2 and QGroundcontrol have multidrone support. And GUI which looks very similar to what you have. You can extend it creating a Tool Widget to do what you require. The current widget called ‘Unmanned Systems’ could even be modified to better suit your needs as template (and it needs updating in anycase)

I'm not sure what you mean by ‘script-alogorithms’ since i would expect this just means algorithms. Are these running on the drones or on the GCS?

The GCS we have currently are not really ‘controlling’ anything like position actively, they are just instructing the drone to do a command, or take this list of commands, and follow-it. it's generally very hands off type on control.

The advantage is that if you integrate a solution using AP2, you get all the vehicle set-up stuff and parameter config for free, as it handles multiple drones, you can select which one you are using and configure away.

If you need more details to asses out a potential solution using AP2, send me an email and I will try and help out.

Bill

[Paul]

Michael,

I've thought about using a WiFi mesh network but since I want to test my drones to at least 500m radius (or whatever distance I can get away with before the signal attenuates too much) apart, I decided to just stick with my 915 MHz radios for now. At this point in development I don't mind managing 13 radios at my control side (I have already designed and 3D printed a harness that holds all the radios and wires in a neat array). Right now I'm not doing a "true swarm" since all the processing is in a central location and each individual drone is not making its own decisions (well it terms of the overall formation -- the drones still control emergency protocols and safeguards). 

I also actually do have XBee radios for each drone even though they are currently not being utilized. 

The most important thing with the algorithm I'm developing is that my swarm needs to re-arrange itself dynamically based on some set criteria and the order of the drones is critically important. That means that if one drone in the swarm is identified as "5" and then it disconnects/fails, drone "6" would then become drone "5" and drone "7" becomes drone "6", etc. This means the drones have to be completely modular and cannot be hard-tied to anyone one receiver, etc. I should be able to bring in a new drone, plug it in, and have it fit in seamlessly into the rest of the swarm based on it's number. My algorithm is using a highly specific geometric pattern that is very much sequential number based. This is the reason I am using a centralized "brain" for the time being. If a drone every loses touch with the central brain for an extended amount of time it will simply drop out of the sequence (causing the rest of the drones' numbers to shift, thus re-arranging themselves) and return to home.

[Paul]

Bill,

I guess I can try to explain a little bit more of what I'm doing. Essentially, the drones are the final physical output of some theoretical computations. So imagine a network (say an Artificial Neural Network) where each drone represents a node. Each drone then has a number of weighted parameters which are constantly changing and are being monitored by the central processing unit. My program, or script, would be running on a computer and doing all the calculations based on the data its getting from the drones and the only output from the program would be something like "drone 1 to 54.345345,23.34234" in order to have the whole swarm conform to the new calculated scenario. So the inputs would be the location of each drone and my weighted parameters, and the output would be the new location for that particular drone. This process runs continuously.

I hope that helps clarify what I'm doing a little bit. 

[David Pawlak]

As far as I can tell from my tests, the 3DR radios already have collision detection. You can have multiple radios running with defferent SYS IDs without conflict. The problem is, the SYS ID is hardware programmed. Each radio has a transparant link, you can put whatever you want over the link and it spits out the other end, but ONLY radios hardware programmed to the Tx SYS ID will receive the data. So the SYS ID isn't included in the "user's" message.

The updated firmware is presumably what gives you the ability to change SYS IDs on the fly. Does it change the existing hardware SYS ID on the fly, or does it superimpose a sub-id? Does that indtroduce hardware reconfiguration delays?   In the end, what is the max bandwidth one can expect? As I understand it, MAVLink is not all that dense, but to get more than 10 units interchanging information you'd have to have a density of somewhere less than 8%.

Then the radio works on several frequencies. Does that mean that you can put "X" units (determined by the above density) on each frequency, or does the radio hop between the available frequencies as a common practice? If you can utilize independant frequencies individually, does the modified software permit also frequency changes on the fly (to assure complete configuration flexibility)?

[benni...@gmail.com]

OK so this is how I understand it (I’ve got some of these radios but haven’t actually turned on the multipoint bit yet, that’s future work so CMIIW):

Even in multipoint, all radios have the same Network ID. This governs what set of frequencies the radios hop over, and in what order, same behavior as for a p2p.

To enable multipoint, you must set each radio to its own node ID, with the base as 0 (and set node count > 0). The base station should set the destination ID before each packet by AT command. This might take some time, so the way I intended to do it was to transmit everything broadcast (dest=65535) and use the MAVLink System ID parameter to demux.

I’m not entirely sure how many extra bits on the air the multipoint addressing takes up, I’m guessing just the 16 bit address, perhaps there needs to be another flags byte in there somewhere, donno. There should be no resynchronization overhead or anything like there would be if you kept changing the base station’s Network ID (rather than Dest Node ID).  I don’t know whether the extra few bits have a practical impact on the on-air throughput, quantitative results welcome!

HTH.

Ben.

--

[David Pawlak]

OK thanks. That clears up a lot.

So we have the MAV_LINK system ID cleaned up in rev 3.2, so will we actually require multi-point to be enabled for that? We could just use one Network ID and change the MAVLink ID in software. No?

Is broadcast only available in multi-point?

[Paul]

Okay so I've made a graphic detailing what I think we've discussed so far concerning multipoint with 3DR radios. Take a look and let me know if the information is accurate. I relied mainly on the RFD900 Datasheet.

[benni...@gmail.com]

Hi David,

> So we have the MAV_LINK system ID cleaned up in rev 3.2, so will we actually require multi-point to be enabled for that?

For that to do what? The mavlink system ID is arbitrary unless there is more than one device on the wireless network. With RFD900 that latter point means the multipoint FW I guess. Of course, if you’re using zigbee modules, wifi etc then the system ID is (or can be) the primary way to differentiate drones

> We could just use one Network ID and change the MAVLink ID in software. No?

Sure

> Is broadcast only available in multi-point?

Sorry I don’t know what you mean by this. Broadcast sends to all radios in a network (same network ID) and for there to be more than 2 radios on a network (and make broadcast different from any other transmission) you need the multipoint features enabled..

[benni...@gmail.com]

Seems legit.

Perhaps worth pointing out for the sake of completeness that I was a little unclear in my original email: The base station doesn’t have to be ID 0 but one radio does have to be, and that radio has to be in range of every other [1]. For your app setting base=0 makes the most sense, for a redundant gcs setup, setting the airbourne radio to 0 makes more sense.  As I say, just for completeness, in case others read this thread in the future 😊

Ben.

[1] ignoring the “sync all” flag

--

[David Pawlak]

Sorry Ben, more clearly:

It comes down to trying to understand MAVLink more clearly. I wondered about the ability to utilize MAVLink ID for multi-point demux without installing the multi-link firmware.

For instance, in the present systen there are two IDs, a network ID which is fixed "in hardware", and a MAVLink ID which can be set in MAVLink. Correct?

The MAVLink ID rides over the individual Nework IDs, and is controlled by the python scripts.

I wasn't sure if the broadcast capability was related to the radios' Network ID or the software MAVLink ID. Since there is a MAVLINK ID it is possible

to transmit various channels  and demux on the basis of the MAVLink ID, all on one Network ID. Given that more than one device can have access, one can assume that there may also be a broadcast feature at that level. My question was about clarifying that.

Another question. If a broadcast mode is not available at the MAVLink ID level, implementing one should be possible, right? No restrictions to being able to modify to receive either a specific MAVLink ID OR (-1)?

It becomes obvious of course that the system is always limited to multipoint ONLY on a given Network ID this way. A restriction that I can manage, however not as useful for Paul in this case.

[benni...@gmail.com]

Hi David, OK understand!

> I wondered about the ability to utilize MAVLink ID for multi-point demux without installing the multi-link firmware.

Sure. For example, if you’re using a different radio such as xbee, wifi etc.  The avionics don’t have any idea what type of radio is connected, whether it’s capable of multipoint etc. It listens for its mavlink ID always. The point is that if you’ve got RFD900s without the multipoint fw then there’s nothing to demux (but the ID still exists as a mechanism and must be correctly matched between vehicle and GCS).

> I wasn't sure if the broadcast capability was related to the radios' Network ID or the software MAVLink ID

I’m referring to network broadcast. I’m not aware of a mechanism at the mavlink system ID level (but I’ve never looked)

> Another question. If a broadcast mode is not available at the MAVLink ID level, implementing one should be possible, right? No restrictions to being able to modify to receive either a specific MAVLink ID OR (-1)?

I guess so… What’s the use case? If you’re having thoughts like “it’d be really neat to send a single ARM command and have 13 quads spool up” then be a bit careful. I personally wouldn’t do that, the risk of having a colleague in the next lab doing bench tests of a “spare” UAV, me arming it accidentally as part of the swarm and chaos ensuing is too high.

[Paul]

Okay so I went through the source of the various libraries and was able to find the exact commands I need to send to do what I want -- that means that I can get away with using only pymavlink. I'm going to test out the commands this week. The only tricky part will be the takeoff -- I'm thinking I might put it in "auto" mode and then give it a simple mission to go to a certain altitude initially and then take over from there. 

As far as telemetry, I decided to just stick with using individual radios and not mess with multipoint at the moment. I've designed a harness that takes care of the mess of having so many radios -- see the image below. Surprisingly, the close proximity of the radios hasn't caused any major issues with signal loss.

 

Once I confirm the commands are working I'll then work on integrating them into the GUI. I plan on using Flask to integrate with my interface.

On Friday, October 31, 2014 12:23:39 PM UTC-4, Paul wrote:

[nickza...@gmail.com]

Paul, did you ever have any success?

[Muscles Inwood]

Hi,

Hope you're doing great. Can you please connect with me ? I am working on the same problem. My email ID is muscle...@gmail.com

Sincerely,

[Joel Ratner]

Hi Paul - whatever happened with this effort?

[Peng Cheng]

Paul, any success?

Do you have a github URI that we can look into, maybe we can help.

[Paul Bupe Jr]

Hey guys sorry for the major absence and lack of replies - work has kept me busy. I can't release the full source code since the school now owns it but I can certainly answer any *specific* questions if you are attempting to do the same thing.

--
You received this message because you are subscribed to a topic in the Google Groups "drones-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/drones-discuss/D-NJ_sTBDM8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to drones-discus...@googlegroups.com.

[Pierre Kancir]

For common interess , his paper is freely accessible here : http://digitalcommons.georgiasouthern.edu/cgi/viewcontent.cgi?article=2324&context=etd

[jhua...@nd.edu]

Paul, is there a way we can get a copy of your thesis dissertation please?
Thank you.
Jane