Issue 51 in fabricate: After group issue

[fabr...@googlecode.com]

Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 51 by wer...@beroux.com: After group issue
http://code.google.com/p/fabricate/issues/detail?id=51

What steps will reproduce the problem?
1. Specifying group='foo', after='foo' will never return (unless process is
killed)

What is the expected output? What do you see instead?
Should wait for other run(group='foo') but not self.
It is useful when for example updating a critical section like a database
with steps that should be done after other DB operations (like migration,
reindexing, testing).

What version of the product are you using? On what operating system?
fabricate v1.26 on Ubuntu x64 10.04

Example source code:

from fabricate import run, main

def build():
run('echo', 'aa', group='db')
run('echo', 'bb', group='db', after='db')

main(parallel_ok=True, jobs=3)

--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[fabr...@googlecode.com]

Comment #1 on issue 51 by ele...@gmail.com: After group issue
http://code.google.com/p/fabricate/issues/detail?id=51

What you are doing does not make sense, you are saying that 'echo bb'
should not run until all commands in the group db have completed. But the
group cannot complete until the 'echo db' command is complete since it is
part of the group, so the command never runs.

Put the 'echo bb' command in a different group, or just don't group it.

Grouping commands tell fabricate that those commands can run in parallel.
It is not for indicating logical groupings.

[fabr...@googlecode.com]

Comment #2 on issue 51 by wer...@beroux.com: After group issue
http://code.google.com/p/fabricate/issues/detail?id=51

Yes I noticed that's the way it works. The usage I made was not a
dependency graph buy a critical section. Meaning I don't expect it to wait
a command group='bb' is run but only to wait if there is one running in
that group, until it finishes.

Have a name group= is adding to the confusion as it implies more than one
item can be part of it.

Another part adding to that confusion, is that wait() will not wait if
nothing is run(), but wait= will until that group (or first of that group?)
is run().

I'd suggest to check if group=wait.

To it more clear and support more I'd also suggest to make wait= work like
a critical section (don't run while something of the group is running) and
rename the existing wait= to dependOn=.

Last, the Ctrl+C is not really working in multithreaded mode. The issue is
that it loses time in many occasions, which is the opposite of the goal of
multithreading.

[Lex Trotman]

On 13 September 2013 17:26, <fabr...@googlecode.com> wrote:

Comment #2 on issue 51 by wer...@beroux.com: After group issue
http://code.google.com/p/fabricate/issues/detail?id=51

Yes I noticed that's the way it works. The usage I made was not a
dependency graph buy a critical section. Meaning I don't expect it to wait
a command group='bb' is run but only to wait if there is one running in
that group, until it finishes.

The 'group' option specifies that this run command is part of a group of commands.  The 'after' option specifies an ordering dependency, that this command must not run until after *all* the commands in the named group have completed.  That is how those options are specified to work.

Have a name group= is adding to the confusion as it implies more than one
item can be part of it.

Indeed, any number of commands can be part of a group. A group is a name for that set of commands so they can be referred to.  Running after the group depends on them *all* completing, therefore no command can specify it needs to run after a group it is part of, nor can any other cycle exist in the ordering graph or there will be a deadlock as you experienced.

 

Another part adding to that confusion, is that wait() will not wait if
nothing is run(), but wait= will until that group (or first of that group?)
is run().

There is no wait() function or 'wait' option in the fabricate API?  Not sure what you mean.

 

I'd suggest to check if group=wait.

To it more clear and support more I'd also suggest to make wait= work like
a critical section (don't run while something of the group is running) and
rename the existing wait= to dependOn=.

Last, the Ctrl+C is not really working in multithreaded mode. The issue is
that it loses time in many occasions, which is the opposite of the goal of
multithreading.

Not sure what you mean here either.  The fabricate parallel processing is not multi-threaded, it is multi-process, it runs multiple commands in separate processes.  Aborting the main process with Ctrl+C may not cause child processes to abort unless they are part of the same process group.  Is that the issue?