Acceptance of new plan while in error

[Kristoffer Gryte]

Hi,

I've been working on mechanisms for aborting, and I've made something similar to lost_comms that executes a predefined plan once an Abort is issued. I have encountered some weird behaviour; when my task triggers an Abort and changes mode to Error (e.g. when a weight sensor has exceeded some limit), the abort plan is started, and the vehicle immediately goes into maneuver. If I try to start a separate, normal plan, it immediately goes into Error and starts to loiter. 

To be more clear, here is a more complete list of the sequences of events:

1.  I start a normal plan, e.g. a goto
   
2.  I send an IMC message with a large weight, so that the my weight monitor triggers an abort. This also sets the Vehicle state to Error.
   
3.  The abort plan is started. This is set with the ignore_errors flag
   
4.  The UAV goes into error
   
5.  The UAV immediately goes into maneuvering, and starts flying towards the start of the abort plan
   
6.  From neptus, I start a normal plan (standard WP)
   
7.  The UAV goes into error and starts to loiter at its current location
   
8.  (The error is cleared, and it goes back to service once I send a weight that is below the threshold)
   

I find this behaviour undesired and weird; why am I (or another task) allowed to start a new plan, which will instantly fail? And I find it a bit weird that the UAV goes into Maneuver, and does not stay in error. Is it not possible to maneuver in error, even when the ignore_error is set for the plan? 

I would greatly appreciate if someone could shed some light on the design choices behind this behaviour, to help me understand the problem and possibly find a way to fix it.

Regards,

Kristoffer

[José Braga]

---------- Mensagem encaminhada ----------
De: "José Braga" <eejb...@gmail.com>
Data: 27/02/2018 10:45 da manhã
Assunto: Re: [lsts-toolchain] Acceptance of new plan while in error
Para: "LSTS Toolchain" <lsts-to...@googlegroups.com>
Cc:

Dear Kristoffer,

I might be able to help.

Can you upload all logs from that sequence?

Kind Regards,

José

--
www.github.com/LSTS/
www.lsts.pt
---
You received this message because you are subscribed to the Google Groups "LSTS Toolchain" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lsts-toolchain+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/lsts-toolchain.
To view this discussion on the web visit https://groups.google.com/d/msgid/lsts-toolchain/b7da78b5-3722-404b-86af-157071b58c6f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kristoffer Gryte]

Dear José,

Thanks for your quick reply. The log can be found here: https://drive.google.com/file/d/1JlXgjjVXuApCipzrYP5Bd7nd1KduNOvE/view?usp=sharing. Let me know if you need anything else. Since the sequence goes over multiple logs, I also paste the output below (with Supervisors.Vehicle with Debug level = spew)

Regards,

Kristoffer

****************************************************************************************************

[2018/02/26 11:27:18] - MSG [Plan.Engine] >> request -- Start Plan (go_TKO_far)

[2018/02/26 11:27:18] - MSG [Plan.Engine] >> plan initializing: go_TKO_far

[2018/02/26 11:27:18] - MSG [Plan.Engine] >> Goto1: executing maneuver

[2018/02/26 11:27:18] - MSG [Plan.Engine] >> reply -- Start Plan (go_TKO_far) -- Goto1: executing maneuver

[2018/02/26 11:27:18] - MSG [Transports.Logging] >> log stopped '20180226/112007'

[2018/02/26 11:27:18] - MSG [Transports.Logging] >> log started '20180226/112718_go_TKO_far'

[2018/02/26 11:27:18] - DBG [Supervisors.Vehicle] >> maneuver start request (11304/48/4)

[2018/02/26 11:27:18] - DBG [Supervisors.Vehicle] >> added stop

[2018/02/26 11:27:18] - DBG [Supervisors.Vehicle] >> dispatched stop request

[2018/02/26 11:27:18] - WRN [Supervisors.Vehicle] >> now in 'MANEUVERING' mode

[2018/02/26 11:27:18] - DBG [Supervisors.Vehicle] >> added start Goto

[2018/02/26 11:27:18] - DBG [Supervisors.Vehicle] >> Goto maneuver started

[2018/02/26 11:27:18] - DBG [Supervisors.Vehicle] >> (11304/48/4)

[2018/02/26 11:27:18] - MSG [Plan.Engine] >> request -- Get Plan ()

[2018/02/26 11:27:18] - MSG [Plan.Engine] >> reply -- Get Plan (go_TKO_far) -- OK

[2018/02/26 11:27:18] - MSG [Plan.Engine] >> request -- Get Plan ()

[2018/02/26 11:27:18] - MSG [Plan.Engine] >> reply -- Get Plan (go_TKO_far) -- OK

[2018/02/26 11:27:18] - DBG [Supervisors.Vehicle] >> maneuver stopped

[2018/02/26 11:27:18] - DBG [Supervisors.Vehicle] >> cleared stop request

[2018/02/26 11:27:18] - DBG [Supervisors.Vehicle] >> dispatched start request

[2018/02/26 11:27:18] - MSG [Control.Path.HeightGlideslope/AP-SIL] >> path (lat/lon): 63.63384 9.74185 to 63.63713 9.75424

[2018/02/26 11:27:18] - MSG [Control.Path.LOSnSMC/AP-SIL] >> path (lat/lon): 63.63384 9.74185 to 63.63713 9.75424

[2018/02/26 11:27:18] - MSG [Control.Path.LOStoTurn/AP-SIL] >> path (lat/lon): 63.63384 9.74185 to 63.63713 9.75424

[2018/02/26 11:27:18] - MSG [Control.Path.Longitudinal/AP-SIL] >> path (lat/lon): 63.63384 9.74185 to 63.63713 9.75424

[2018/02/26 11:27:18] - MSG [Control.UAV.Ardupilot/AP-SIL] >> ControlLoops FSM: Switched from INIT to INIT, due to T_DEACTIVATE_PITCH

[2018/02/26 11:27:18] - MSG [Control.UAV.Ardupilot/AP-SIL] >> ControlLoops FSM: Switched from INIT to INIT, due to T_DEACTIVATE_ROLL

[2018/02/26 11:27:18] - MSG [Control.UAV.Ardupilot/AP-SIL] >> speed control is NOT active

[2018/02/26 11:27:18] - MSG [Control.UAV.Ardupilot/AP-SIL] >> speed control is NOT active

[2018/02/26 11:27:18] - MSG [Control.UAV.Ardupilot/AP-SIL] >> speed control is NOT active

[2018/02/26 11:27:18] - MSG [Control.UAV.Ardupilot/AP-SIL] >> speed control is NOT active

[2018/02/26 11:27:19] - DBG [Supervisors.Vehicle] >> Goto maneuver started

[2018/02/26 11:27:19] - DBG [Supervisors.Vehicle] >> cleared start request

[2018/02/26 11:27:19] - DBG [Supervisors.Vehicle] >> some control loops are enabled now

[2018/02/26 11:27:41] - MSG [Monitors.AbortOnWeight2] >> Sent abort! severity 0, lim 0

[2018/02/26 11:27:41] - MSG [Monitors.UAV.FixedWingAbortPlan] >> Got abort, start checking conditions..

[2018/02/26 11:27:41] - MSG [Monitors.UAV.FixedWingAbortPlan] >> Executing abort plan. 

[2018/02/26 11:27:41] - WRN [Plan.DB] >> storing plan 'abort_plan' issued through a PlanControl request

[2018/02/26 11:27:41] - MSG [Plan.Engine] >> request -- Start Plan (abort_plan)

[2018/02/26 11:27:41] - MSG [Plan.Engine] >> plan initializing: abort_plan

[2018/02/26 11:27:41] - MSG [Monitors.Entities] >> Weight Abort 2 : Normal -> Error | Abort triggered

[2018/02/26 11:27:41] - MSG [Plan.Engine] >> Loiter1: executing maneuver

[2018/02/26 11:27:41] - MSG [Plan.Engine] >> reply -- Start Plan (abort_plan) -- Loiter1: executing maneuver

[2018/02/26 11:27:41] - MSG [Transports.Logging] >> log stopped '20180226/112718_go_TKO_far'

[2018/02/26 11:27:41] - MSG [Plan.DB] >> set plan (abort_plan) -- OK (updated)

[2018/02/26 11:27:41] - MSG [Transports.Logging] >> log started '20180226/112741_abort_plan'

[2018/02/26 11:27:41] - ERR [Supervisors.Vehicle] >> got abort request

[2018/02/26 11:27:41] - DBG [Supervisors.Vehicle] >> added stop

[2018/02/26 11:27:41] - DBG [Supervisors.Vehicle] >> dispatched stop request

[2018/02/26 11:27:41] - DBG [Supervisors.Vehicle] >> added start IdleManeuver

[2018/02/26 11:27:41] - WRN [Supervisors.Vehicle] >> now in 'ERROR' mode

[2018/02/26 11:27:41] - DBG [Supervisors.Vehicle] >> maneuver start request (11304/48/5)

[2018/02/26 11:27:41] - DBG [Supervisors.Vehicle] >> added stop

[2018/02/26 11:27:41] - WRN [Supervisors.Vehicle] >> now in 'MANEUVERING' mode

[2018/02/26 11:27:41] - DBG [Supervisors.Vehicle] >> added start Loiter

[2018/02/26 11:27:41] - DBG [Supervisors.Vehicle] >> Loiter maneuver started

[2018/02/26 11:27:41] - DBG [Supervisors.Vehicle] >> (11304/48/5)

[2018/02/26 11:27:41] - ERR [Plan.Engine] >> reply -- Start Plan (abort_plan) -- got abort request

[2018/02/26 11:27:41] - MSG [Plan.Engine] >> request -- Get Plan ()

[2018/02/26 11:27:41] - MSG [Plan.Engine] >> reply -- Get Plan (abort_plan) -- OK

[2018/02/26 11:27:41] - MSG [Plan.Engine] >> request -- Get Plan ()

[2018/02/26 11:27:41] - MSG [Plan.Engine] >> reply -- Get Plan (abort_plan) -- OK

[2018/02/26 11:27:41] - DBG [Supervisors.Vehicle] >> maneuver stopped

[2018/02/26 11:27:41] - DBG [Supervisors.Vehicle] >> cleared stop request

[2018/02/26 11:27:41] - DBG [Supervisors.Vehicle] >> a stop comes right after, ignoring this maneuver

[2018/02/26 11:27:41] - DBG [Supervisors.Vehicle] >> cleared start request

[2018/02/26 11:27:41] - DBG [Supervisors.Vehicle] >> cleared stop request

[2018/02/26 11:27:41] - DBG [Supervisors.Vehicle] >> dispatched start request

[2018/02/26 11:27:41] - MSG [Control.Path.HeightGlideslope/AP-SIL] >> path (lat/lon): 63.63554 9.74899 to 63.62860 9.72491

[2018/02/26 11:27:41] - MSG [Control.Path.LOSnSMC/AP-SIL] >> path (lat/lon): 63.63554 9.74899 to 63.62860 9.72491

[2018/02/26 11:27:41] - MSG [Control.Path.LOStoTurn/AP-SIL] >> path (lat/lon): 63.63554 9.74899 to 63.62860 9.72491

[2018/02/26 11:27:41] - MSG [Control.Path.Longitudinal/AP-SIL] >> path (lat/lon): 63.63554 9.74899 to 63.62860 9.72491

[2018/02/26 11:27:41] - MSG [Control.UAV.Ardupilot/AP-SIL] >> speed control is NOT active

[2018/02/26 11:27:41] - MSG [Control.UAV.Ardupilot/AP-SIL] >> speed control is NOT active

[2018/02/26 11:27:41] - MSG [Control.UAV.Ardupilot/AP-SIL] >> speed control is NOT active

[2018/02/26 11:27:41] - MSG [Control.UAV.Ardupilot/AP-SIL] >> speed control is NOT active

[2018/02/26 11:27:42] - DBG [Supervisors.Vehicle] >> Loiter maneuver started

[2018/02/26 11:27:42] - DBG [Supervisors.Vehicle] >> cleared start request

[2018/02/26 11:27:58] - MSG [Plan.Engine] >> request -- Start Plan (go_TKO_far)

[2018/02/26 11:27:58] - MSG [Plan.Engine] >> plan initializing: go_TKO_far

[2018/02/26 11:27:58] - MSG [Plan.Engine] >> Goto1: executing maneuver

[2018/02/26 11:27:58] - MSG [Plan.Engine] >> reply -- Start Plan (go_TKO_far) -- Goto1: executing maneuver

[2018/02/26 11:27:58] - MSG [Transports.Logging] >> log stopped '20180226/112741_abort_plan'

[2018/02/26 11:27:58] - MSG [Transports.Logging] >> log started '20180226/112758_go_TKO_far'

[2018/02/26 11:27:58] - DBG [Supervisors.Vehicle] >> maneuver start request (11304/48/6)

[2018/02/26 11:27:58] - DBG [Supervisors.Vehicle] >> added stop

[2018/02/26 11:27:58] - DBG [Supervisors.Vehicle] >> dispatched stop request

[2018/02/26 11:27:58] - DBG [Supervisors.Vehicle] >> added start Goto

[2018/02/26 11:27:58] - DBG [Supervisors.Vehicle] >> Goto maneuver started

[2018/02/26 11:27:58] - DBG [Supervisors.Vehicle] >> (11304/48/6)

[2018/02/26 11:27:58] - MSG [Plan.Engine] >> request -- Get Plan ()

[2018/02/26 11:27:58] - MSG [Plan.Engine] >> reply -- Get Plan (go_TKO_far) -- OK

[2018/02/26 11:27:58] - MSG [Plan.Engine] >> request -- Get Plan ()

[2018/02/26 11:27:58] - MSG [Plan.Engine] >> reply -- Get Plan (go_TKO_far) -- OK

[2018/02/26 11:27:58] - DBG [Supervisors.Vehicle] >> maneuver stopped

[2018/02/26 11:27:58] - DBG [Supervisors.Vehicle] >> cleared stop request

[2018/02/26 11:27:58] - DBG [Supervisors.Vehicle] >> dispatched start request

[2018/02/26 11:27:58] - MSG [Control.Path.LOSnSMC/AP-SIL] >> path (lat/lon): 63.63485 9.74519 to 63.63713 9.75424

[2018/02/26 11:27:58] - MSG [Control.Path.LOStoTurn/AP-SIL] >> path (lat/lon): 63.63485 9.74519 to 63.63713 9.75424

[2018/02/26 11:27:58] - MSG [Control.Path.Longitudinal/AP-SIL] >> path (lat/lon): 63.63485 9.74519 to 63.63713 9.75424

[2018/02/26 11:27:58] - MSG [Control.Path.HeightGlideslope/AP-SIL] >> path (lat/lon): 63.63485 9.74519 to 63.63713 9.75424

[2018/02/26 11:27:58] - MSG [Control.UAV.Ardupilot/AP-SIL] >> speed control is NOT active

[2018/02/26 11:27:58] - MSG [Control.UAV.Ardupilot/AP-SIL] >> speed control is NOT active

[2018/02/26 11:27:58] - MSG [Control.UAV.Ardupilot/AP-SIL] >> speed control is NOT active

[2018/02/26 11:27:58] - MSG [Control.UAV.Ardupilot/AP-SIL] >> speed control is NOT active

[2018/02/26 11:27:59] - DBG [Supervisors.Vehicle] >> Goto maneuver started

[2018/02/26 11:27:59] - DBG [Supervisors.Vehicle] >> cleared start request

[2018/02/26 11:27:59] - DBG [Supervisors.Vehicle] >> added stop

[2018/02/26 11:27:59] - DBG [Supervisors.Vehicle] >> dispatched stop request

[2018/02/26 11:27:59] - DBG [Supervisors.Vehicle] >> added start IdleManeuver

[2018/02/26 11:27:59] - WRN [Supervisors.Vehicle] >> now in 'ERROR' mode

[2018/02/26 11:27:59] - ERR [Plan.Engine] >> reply -- Get Plan (go_TKO_far) -- Weight Abort 2: Abort triggered

[2018/02/26 11:27:59] - MSG [Transports.Logging] >> log stopped '20180226/112758_go_TKO_far'

[2018/02/26 11:27:59] - MSG [Transports.Logging] >> log started '20180226/112759'

[2018/02/26 11:27:59] - DBG [Supervisors.Vehicle] >> maneuver stopped

[2018/02/26 11:27:59] - DBG [Supervisors.Vehicle] >> cleared stop request

[2018/02/26 11:27:59] - DBG [Supervisors.Vehicle] >> dispatched start request

[2018/02/26 11:27:59] - MSG [Control.UAV.Ardupilot/AP-SIL] >> ControlLoops FSM: Switched from INIT to INIT, due to T_DEACTIVATE_PITCH

[2018/02/26 11:27:59] - MSG [Control.UAV.Ardupilot/AP-SIL] >> ControlLoops FSM: Switched from INIT to INIT, due to T_DEACTIVATE_ROLL

[2018/02/26 11:28:00] - DBG [Supervisors.Vehicle] >> IdleManeuver maneuver started

[2018/02/26 11:28:00] - DBG [Supervisors.Vehicle] >> cleared start request

****************************************************************************************************

To unsubscribe from this group and stop receiving emails from it, send an email to lsts-toolchai...@googlegroups.com.

[José Braga]

Hi,

Just for reference, you can concatenate multiple logs with cat:

cat <log_folder>/Data.lsf.gz > <destination_folder>/Data.lsf.gz

Then copy one of the IMC.xml files and you are ready to go. 

Thanks for the files

Kind Regards,

José

To unsubscribe from this group and stop receiving emails from it, send an email to lsts-toolchain+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/lsts-toolchain.

To view this discussion on the web visit https://groups.google.com/d/msgid/lsts-toolchain/ff998c45-e351-42d6-9313-0e0952203088%40googlegroups.com.

[José Braga]

Hi again,

#1 why plan is started

* Plan scheduling and management is done in the plan engine task. When a plan Control start request is received, it checks if feasible, schedules payload activation (if any) and simply starts. At this point, it is completely agnostic of vehicle errors or state.

* Supervisors Vehicle is the upper layer managing of a plan proceeds by checking entities in error, and updating vehicle state. It is here that the flag IGNORE_ERRORS is used, if you notice

I am not sure of the exact reason why, I guess the idea is to have greater modularity where you can change the supervisor to add different strategies (instead of stopping, start a different plan, issue reports through several media..). I am guessing as this design choice far precedes my time in the lab.. 

There's a block diagram somewhere of the control architecture and it is pretty crazy. I'll send it to you once I find it. I am not sure how easy it is to change this behavior without opening Pandora's box, but it should be straightforward to test this in simulation.

Regarding vehicle staying in MANEUVER: the flag ignore errors is much recent than the underlying architecture, so this was a hack to force the vehicle running when an monitored entity error occurs. Specially if you have only an acoustic link and not wifi. For UAVs I guess a different design would fit best. Since the hack was not part of the development design, there are a few issues. But, the vehicle is maneuvering so I am not sure what the proper choice should be. I guess both, using an enumerated field instead

Kind Regards

José

[José Braga]

Here they are.

'PlanEngine_SM_PlanControl' is from 2012, 'Updated Vehicle Supervisor' is circa 2014

so there may be a few differences, but there should be the gist of it.

Kind Regards,

José Braga

____________________

jbr...@oceanscan-mst.com

www.oceanscan-mst.com

[Kristoffer Gryte]

Thank you very much for caring to explain this in such detail, José. I really appreciate it!

It will take some time for me to get an overview of the possibilities here, but I will let you know if I find a way this can be changed with reasonable ease.

Figures like those is something I've really missed when learning how DUNE works. Figuring out how one task is working is easy enough, but the higher level hierarchies take infinitely more time to learn. So if you have more diagrams like that, that can be posted, I think the whole community would appreciate it :)

Thanks again,

Kristoffer

[José Braga]

Sure, no problem. I'll see what I can find. The reason for such difficulty is because dune was not open sourced until almost 5y ago exactly. At that time, there were already 7y of developments and all architecture was laid out, with no documentation..

Kind Regards,

José

To unsubscribe from this group and stop receiving emails from it, send an email to lsts-toolchain+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/lsts-toolchain.

To view this discussion on the web visit https://groups.google.com/d/msgid/lsts-toolchain/47cb34e3-faaa-4c5d-ae75-e9cc61329776%40googlegroups.com.