Unexpected behaviour when changing flight mode and RTL (channel 8) at exactly the same time

[Scott Simpson]

This is related to receiver failsafe, not APM failsafe. 

I just had a deadfall crash from 70 meters.  This has been my first major hardware contribution to the arducopter project :)

I've been testing 3.2rc12 on APM with PPMSUM input.  I have my receiver failsafe set to enable channel 8 when out of range and channel 8 triggers RTL (in Extended Tuning). I left the failsafe flight mode set to stabilize and throttle to zero because I figured RTL would override those.  They don't seem to if they change at exactly the same time.  

I found it in the logs, but it's easy to reproduce this in Mission Planner connected to APM via USB.

If RTL is set to trigger on channel 8, and you also change the flight mode at exactly the same time (same PPMSUM packet), the RTL command is received (as seen in the log) but a few log messages later, the mode change is picked up.  This happens so fast, you don't see the RTL in mission planner.  You just see the mode change, even though channel 8 is enabled.  You should really see RTL shown in mission planner since RTL should override the flight mode.

Please let me know if anyone would like more information.  I still don't know enough about the code to recommend a patch, but I might at some point.  Right now, I was just hoping that we could at least track this for a future release.  It might save some hardware.

Thanks,

Scott

[Craig Elder]

This behavior is by design. You should set your receiver according to the instructions here http://copter.ardupilot.com/wiki/throttle-failsafe/#Throttle_Failsafe so that it drops CH3 below the FS_THR_VALUE

--
You received this message because you are subscribed to the Google Groups "drones-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to drones-discus...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Randy Mackay]

Scott,

 

     That’s  no fun at all.  Sorry for your loss.

 

     Ok, so after a quick review, I see the difference between initiating the RTL with ch8 vs ch5 is that ch8 is not “de-bounced” so that effect will take effect immediately while ch5 is debounced so the change won’t happen for 0.2 seconds.  This is more by accident than design, but it seems if both were changed at the same time, both would take effect but the ch7/ch8 initiate mode change would only be in effect for 0.2 seconds.

 

     Coincidentally Jonathan made a patch to master (but it won’t make AC3.2) which debounces the zero throttle so it must be at zero for 0.4 seconds before it plays into the failsafe decision to disarm the motors.  https://github.com/diydrones/ardupilot/commit/96f50b7cd72987c2598f782febcc5b559c315215

 

     I suspect this would have saved your copter although in this situation the way the failsafe was set-up sounds very dicey.  Any reason why you didn’t use AC’s failsafe features instead of the receiver’s?

 

-Randy

--

[Scott Simpson]

Thanks for the replies.

I assumed RTL which overrides modes when not in failsafe would also override the mode that is set during a failsafe.  i.e. When flying normally in any mode, I believe that if I switch channel 8 to RTL, I can set throttle to zero, play around with flight modes and the copter still keeps RTL'ing.  In failsafe, this is different because the commands are coming at the same time, and one is debounced, the "wrong one" - not RTL - wins. 

I'm not worried about this for my own use.  I can get the Arducopter failsafe working and I have that setup on my ezuhf/pixhawk configuration.  This configuration though I decided to do things differently.  Partly out of lazyness.

Doing failsafe in the receiver (and making assumptions like above) has worked for me up to now.  The stupid thing here is that I didn't test it at 1 meter.  A year ago when I tested Naza failsafe using the same sort of setup, it worked.  I'm not saying that Arducopter needs to do everything that Naza does, but Naza does set the bar high for newbie usability.

Regarding my crash, it was no big issue.  Also, I've been testing every 3.2 rc that's been out in wind, calm, pixhawk, apm, hex, quad.  This was my first major crash.  I was really lucky that I got away with only $30 in repair after a freefall 80 meters the ground. :)

 

[Jonathan Challinger]

Right, I think the issue that I fixed was made by Randy for you or someone in your exact situation.

--

[Scott Simpson]

 

Thanks, this sounds great... I'll keep testing other stuff, but like I said a few days ago, the current state of 3.2 is amazing!!!!! 

[Brian DeBusk]

Unfortunately all flight mode changes are edge-triggered.  So while you'd think flipping RTL would override the traditional flight modes, changing from something like Loiter to Stabilize while RTL is active puts you back into Stabilize mode.

Here's a common scenario:

- You're doing a maximum descent (throttle down) in loiter mode to bring your vehicle down.
- You think "hey, there's an easier way" and you flip the RTL switch
- As your craft gets close to home, you think: "I'll manually guide it in in Stabilize mode when it gets closer to the return point" -- so you prepare for the transition by changing the flight mode to Stabilize, thinking it will stay in RTL until you flip it off

Instead, it immediately switches to Stabilize mode, and your throttle is still down from your max descent in Loiter mode.  The bird falls out of the air and crashes.

I've thought about doing a patch in the control loop that checks for flight mode transitions and manually forces the craft to stay in RTL mode as long as it is chosen on Channels 7 or 8.

[Randy Mackay]

 

     It’s just a question of how you define or expect ch7/ch8 to work.  I’ve never personally thought of ch7/ch8 as an override of ch5 and I’m not sure it’s a great idea to force someone to put the ch7/ch8 switch back to its off position before letting them change flight modes with ch5.

 

     The code is partially consistent with thinking of it as an override though in that when the pilot puts ch7/ch8 back to the off position it reverts to the flight-mode specified by ch5.

 

-Randy

--

[Brian DeBusk]

I guess it comes down to how you see the CH7/CH8 switches.  If I consider RTL one of my CH5 flight modes, then edge-triggering makes total sense.

But CH7/CH8 really is a different issue.  If I've lost orientation, not sure how much battery I have left, or are a little worried about getting my vehicle home (i.e. it's a spec in the air), I'll fire the RTL from CH8 just to know I'm bringing it home safely.

I have a particularly tricky situation, because my landing area is about 4 x 4 meters with shrubbery and a pool on either side.  So as I get close to home, I like to guide it in using Stabilize.  Needless to say, this edge triggered issue did contribute to user error (because I do believe the vast majority of failures are user error), that caused a really nice 18" prop Octa to hit the drink hard.

[Robert Lefebvre]

I think there's a pretty good case to be made for making the Ch7/8 mode switches absolute.  It's a good panic thing, no matter what, this is the mode I want to be in.

[Scott Simpson]

Channel 7/8 currently overrides the mode and I originally found it the easiest way to get receiver RTL failsafe working.  I have my bottom right momentary programmed to RTL.  That's my panic switch.  My original problem report was that it overrides under normal flight but when the mode and channel 7/8 are switched at the exact same time (failsafe) it doesn't.  It sounds like you guys were already on this and it's patched for a future releases.

Since I opened the report, I reprogrammed my transmitter so that mode 6 is RTL and it's still overridden by my panic switch.  This just took a lot more ugly RX programming.  A side benefit of my new configuration is that if I don't have telemetry connected, I can quickly change my receiver failsafe to auto, land or any other mode that I have selected directly on the receiver.  This is a big +1 for receiver failsafe.

Thanks for all the discussion on this.  You guys are awesome!  

P.S. All 3 of my arducopters are now flying again with rc12 and my new RTL configuration...time for more testing :)

[Ben Dellar]

Hi Rob,

I would personally disagree with making the CH7/8 switches absolute as this could lead to a dangerous condition where a user loses all control of mode if they misunderstand their switch location. The scenario I imagine is them misplacing switches and setting CH7/8 to RTL then getting confused why their copter subsequently refuses to obey any mode change command on their primary mode switch or GCS mode control. Prior to getting my quads, I started off in the APM Plane world where all modes are through the mode switch (or GCS) and I believe this to be much safer and clearer approach.

[Jonathan Challinger]

I'd much rather have a "definitely in stabilize" switch than a "definitely in RTL" switch.

Knowing for absolute sure that a failsafe will not start an RTL or a LAND would be really nice for indoor flying or for situations where I'm landing the copter manually.

[john...@gmail.com]

In other words, ch7/8 should be a switch that will override any other modes settings with <insert mode>. A single trigger safety override option like the throttle cut we use on helicopters.

Exactly which mode is triggered could still be a user choice, since we have different needs. And if this is implemented I would like a full stop "mode" that enabled you to instantly kill the motors completely regardless of situation.

[David Pawlak]

The idea of a Ch7/8 override also implies "users' responsibility".

You put it there, it's up to you not to forget that you did. You put it there, and the autopilot thinks it should do some other failsafe but can't, you are taking that responsibility personally.

Is it infalably safe, potentially not. Does it have invaluable functionality, yes:

But I feel the user should have the option of selecting a programmable override.

Of course forced auto-mode selections should be limited by technical restrictions, (presence of GPS).

[Scott Simpson]

This is a great idea and intuitive.  Change it to "Force Mode" and allow possibly all modes.  As it's coded today, RTL IS a mode.  It doesn't make sense to not have 7/8 RTL override.  This idea removes confusion and enhances functionality.

I like all the idea of an all stop mode but also a "wind down" mode while we are on it.  Sort of like what the micro quads do when the battery dies.  The motors don't run full, but it comes down in a brainless way with motors spinning slower and slower.

[Robert Lefebvre]

Personally, I'm getting tired of passing up useful features because we have to cater to the lowest common denominator.  There seems to be a lot of demand for something like this.  It makes a lot of sense.  You implement it properly, you give clear instructions, and if some people screw it up, let the quads fall where they may.  No matter what we do, people will screw it up somehow.  

[Robert Lefebvre]

To be clear:  I screw up too.  I lost an helicopter in the lake at AVC because I flipped it to Acro instead of Alt Hold, and then rolled it over upside down.  I did not ask that we get rid of Acro mode because it caused me to crash.

Yes, we should try to make things as easy as possible for people.  Things should work properly.  They should be clearly documented.  They should be easy to use.  But we should not say we're just not going to do something, because somebody could be confused.  Having Overriding Modes makes an awful lot of sense for a lot of reasons.

[Brian DeBusk]

CH7 and CH8 already have drop-down selects in MP.

So the selects for each become:

<RTL Flight Mode>
<RTL Force Override>

And I remember seeing in the code where the switches are read.  With about eight lines of code, add an if statement that checks for RTL Force Override and just set the mode to RTL.

Everyone's happy and it's what?  20 mins of coding?  The worst part would probably be having to update Mission Planner so it was aware of the new CH7/CH8 options.

[Scott Simpson]

Sorry if I'm slow, but I don't understand how a ch7/8 RTL could ever not be considered a flight mode override.  I understand how simple, super simple, flip, etc don't override the mode because they are really mode modifiers, but shouldn't RTL, land, etc always override the set flight mode.  Maybe I'm misunderstanding something.  

I'm still hoping that none of what is being proposed breaks the patch to the original problem which was the inconsistency if the 7/8 switch is thrown during the mode debounce period.

[Jonathan Challinger]

We're talking about whether the ch8 mode select should be edge-triggered or level-triggered. Basically, can the mode change from RTL because of, for example, a failsafe, or a mode switch change?

[Robert Lefebvre]

A great example of why level-triggered would be useful, is for example a massive GPS glitch where you're blowing through fence lines.  You'd want to be able to go to Stabilize and have it stay there.  

[Scott Simpson]

Thanks Jonathan,

I'm definitely biased towards level triggering on anything that could change the mode.  Nice and simple and even appears consistent.  A fence breach or failsafe have elevated priorities (over anything the user sets), but still appear level triggered.

I'll bow out of the conversation now since I gave my 2 cents.  You guys have been living this for years, so I don't want to be annoying. :)