Global variables

[Sergio Bossa]

There have been some discussions recently about global variables, and
I'm personally in favour of them: I believe they could be incredibly
useful to trigger flow actions based on another flow state, i.e. "if
at home do x, y, z", or "do x unless I'm at home" and so on.
Obviously, the same could be implemented by chaining flow calls, but
the flow diagrams you'd get would be quite awful and difficult to
maintain.

I understand global variables are generally a bad practice in
programming languages where you have better communication primitives,
but Automate is not a programming language nor it should be intended
for programmers.

Thoughts?

[Automate developer]

Global variables
Variables that can be accessed like regular variable but from any fiber may seem like a perfect solution. However in a "multi-threaded" environment like Automate, concurrency, i.e. where multiple fibers could read/write the same variable at the same time will cause problems. And the problem will be very difficult for users to understand and predict. For example, a check-and-set (if global variable "foo" equals null then set variable global variable foo to "2") may give an unexpected result since the variable may have been changed by another fiber inbetween check and the set.

The standard solution to the concurrency problem is to use locks, in Automate this would have been a "Variable lock" block with an enabled/disable checkbox and a variable name. The variable could then only be changed by the fiber holding the "lock". But this would of course require that the user uses the "lock" block around every check-and-set parts of a flow, which novices may forget. Flows would also grow larger with a lot of "Variable lock" blocks.

To implemement global variables i'd have to add an "variables" list dialog/screen to the flowchart, where the user could declare if a variable is local or global. It's not really an issue since i'll probably add this anyway, to allow users to rename variables. But another problem is how "global" should the variables be?

• They certainly can't be global across all flows, a user may download a lot of flows with the global variable "foo".
• They may not even be global within the same flow, the flow could be started multiple times  ("Allow parallel launch" enabled) and then the "global variables" become useless.
• One option would be to allow the user to choose which set of "global variables" a fiber has access to in the would-be "variables" list dialog/screen.
• The best solution is probably that the set of "global variables" are inherited, like regular variables, during a Fork.

There's a lot of issues, and the only use for "global variables" i see, would be to replace the File read/write blocks for storing persistent data.

I'll post about the alternatives, purpose built concurrency control blocks, as they come to mind.

[Automate developer]

Latch block
Useful for compound "when..." or "AND" condition triggers, instead creating a "tree" of fork blocks. It would probably include both a "Latch get" and a "Latch set" block. The "Latch set" block interface would allow the user to choose one of the "Latch get" blocks within the same flow, it would also take a boolean "open" value. The "Latch get" block have an "proceed" option to either get current number of "open" values (Immediate), or await until anything open/closes (When changed), it would also have an "open count" output variable. The "Latch set" block could even have two "in" dots, an "open" and "close" to make it easier.

So a flow could look like this:

      Flow beginning
          |
        Fork ----------------------------------------+
          |                                          |
        Fork --------------------+                   |
          |                      |                   |
          +----------+    +------+           +-------+
          |          |    |      |           |       |
       Latch get     |    | Location at --+  |  Time window --+
          |          |    |   (work)      |  |    (8:00 8h)   |
          |          |    |      |     +--+  |       |     +--+
  Expression true? --+    |      |     |     |       |     |
     openCount = 2        |     Latch set    |      Latch set
          |               |         |        |          |
(both work AND time)      +---------+        +----------+
          :

This is only an idea and may not work in practice, but something similar.

[Mark Newton]

That sounds good, but I have a question. From the example you provided, it looks as if you would have to use a seperate fiber for the latch get expression. If a flow required monitoring multiple latches/conditions then a separate fiber would be required for every different control/latch get correct? Wouldn't that still require spawning of multiples of forks?

[Automate developer]

A fiber can only await single "event", that's a must. I don't see how i could change that.

The "Latch", nor any other concurrency control block, will reduce the number of forks. But it could make a flow "wide" instead of "deep", i.e all forks would be made from the "main" fiber instead of nested in it's parents.
This may not be an improvement after all?

[Mark Newton]

It's an improvement on the current system none the less. Any system that allows conditional monitoring of multiple variables will work really.

[Sergio Bossa]

Agreed the Latch Block would be a good improvement, yet not a
substitute for global variables, but I can live without them :)

On Mon, Dec 8, 2014 at 4:13 AM, Mark Newton <markn...@gmail.com> wrote:
> It's an improvement on the current system none the less. Any system that
> allows conditional monitoring of multiple variables will work really.
>

> --
> You received this message because you are subscribed to the Google Groups
> "Automate" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to automate-use...@googlegroups.com.
> To post to this group, send email to automa...@googlegroups.com.
> Visit this group at http://groups.google.com/group/automate-user.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/automate-user/125118ad-4ab1-404d-bd12-b157a11858d5%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.



--
Sergio Bossa
http://www.linkedin.com/in/sergiob

[Mikael]

Some kind of message sending and receiving between fibers could be a substitute for both global variables and the latch block.

I am thinking of one block, send, which would send a message to a fiber,
and one block, receive, which would wait for messages from any fiber. It would get the URI of the sender as an variable (allows for sending a reply), and the message in another.

Maybe it is to complex and needs simplification.

/Mikma

On Monday, December 8, 2014 11:31:13 AM UTC+1, Sergio Bossa wrote:

Agreed the Latch Block would be a good improvement, yet not a
substitute for global variables, but I can live without them :)

[Automate developer]

Yes, a FIFO queue will probably also be included, since its a commonly used concurreny control structure.

But instead of sending messages with (dictionary) payloads i may try to make it pass variable updates/content.
You select which variables should be included, and when an "message" has been received the variables are update with their content from the sending fiber.
Then users wouldn't have to "package" the data into the dictionary data type before sending, and don't have to read the data from dictionary properties.

[Automate developer]

Two new blocks has been implemented for this; Variables give and Variables take
Try them out in the new alpha version:
https://groups.google.com/d/msg/automate-alpha/zjGiqwS-efI/YX9ZiWD-Ps0J

[gprs...@gmail.com]

-In addition to starting and stopping other flows from inside a different flow, it would be sensible to be able to check the state of other flows.
-how about those "when" blocks? Perhaps we could check state, force stop, and force go? E.G. I created a fiber using Mark Newton's Speak Caller ± VCommands which uses a "when ringing in call" block. I only want this fiber used when I'm not at the office but once the fiber is started and waiting for an incoming call, I have no way of stopping it. This might be resolved by breaking the fiber out to its own flow once we can control other flows but I suspect there is still value in controlling blocks waiting for state changes.
-could the current live blocks be shown along with the active fiber count? In multi fiber flows it can be easy to lose track of which fibers are where.
-maybe an option to search for blocks? I know I need to adjust block 132 based on the log but where was that block again?

Thanks again for such an outstanding app. I was happy to make the purchase in support of the effort.

[Automate developer]

You can check the if another flow is running with the Fiber running block, but only if you're using it from the flow that started it, since it can get the Fiber URI of the started flow. Same goes for stopping it with the Fiber stop block.
I got plans on adding a Flow URI field to the Flow stop block, so it also can be used to stop all fiber of another flow.

Every block, directly or indirectly connected to the started Flow beginning block of a running flow should be considered "live".

Block search, by id, sounds great, i'll add it.

[rocks...@gmail.com]

Ugh. Sorry, I see that I posted to the wrong thread...This should be in "Planned Features, to-do list"

[Automate developer]

The new alpha, version 1.1.2a1 now has "Atomic" blocks.
The values stored with them can be accessed from any fiber in the same flow, they're also persisted, until the flow has been altered.
Use them instead of saving data to external storage with the File write/read blocks.

[qwert...@gmail.com]

Atomic blocks are overkill, all you need is to implement an atomicity of every block, e.g.

1. Make visibility of vars for whole flowchart.
   
2. Every block should be locked (automatically without user interaction and even knowing about), locked until operation of output to vars or setting of vars finished.
   
3. Any block which uses the same variable should wait until first block finishes.
4. Any "deadlocks" and similar things are user responsibility.

The approach described above is better cause user needn't to store/get something...

[Henrik "The Developer" Lindqvist]

What you suggest doesn't work without "locking", what if a user wish to check-and-set a value, for example:
       :
 Expression true?
  global = null
       |
  Variable set
global = "foobar"
       :
The above example would cause unexpected behavior if another fiber changed the value of "global" in between the two blocks, or at any time.
Proper handling of concurrency isn't overkill, everything is done this way for a reason.

[qwert...@gmail.com]

Sorry, I meant "atomic load", "atomic store", ... and ... maybe, "variable give", "variable take" blocks saying "overkill". I wanted to say this approach is complicated still while changing variable visibility withing whole flowchart is better and simpler same time! I didn't mean the whole "atomic" section is useless!
I agree with you that proper handling of "get-and-set", "get-and-add", "add-and-get" problem requires atomic blocks.

[Henrik "The Developer" Lindqvist]

The Variables give/take blocks are used for inter-fiber communication, message passing, so they're not simply "variables".

The Atomic store block could possibly have been integrated into the Variable set block, and the Atomic load replaced with direct variable access in expressions, but that would cause the same check-and-set issue (requiring locking) as described earlier. As the Atomic blocks works now makes it easy to determine where a change of an "atomic" value will occur since the user explicitly load/save them.

This approach may seem "complicated", that's because it (concurrency) is, and should only be used where absolutely necessary with understanding of the implications.
The other automation apps may have implemented this differently, less complicated, but they likely did so without understanding, or ignoring, the consequences.