Using 'create_after' and dependencies on subtask.

[Franken Stein]

Hi there.

I had found mby strange moment.

#!/usr/bin/doit -f

# -*- coding: utf-8 -*-

from doit import create_after

def PutNewEggs() :

print( 'Here you 10 new eggs' )

def SpamEgg( egg_num ) :

print( 'WHOA {}!'.format( egg_num ) )

def CatchEgg( catch_num ) :

print( 'GOTCHA {}!'.format( catch_num ) )

def task_PrepareEggs() :

return {

'actions': [ PutNewEggs ]

}

@create_after( executed = 'PrepareEggs' )

def task_SpamEggs() :

for egg_num in range( 10 ) :

yield {

'name': 'egg-{}'.format( egg_num ),

'actions': [ ( SpamEgg, [ egg_num ] ) ]

}

@create_after( executed = 'PrepareEggs' )

def task_CatchEggs() :

for catch_num in range( 10 ) :

yield {

'name': 'catch-{}'.format( catch_num ),

'actions': [ ( CatchEgg, [ catch_num ] ) ],

'task_dep': [ 'SpamEggs:egg-{}'.format( catch_num ) ]

}

DOIT_CONFIG = {

'default_tasks': [ 'CatchEggs' ]

}

So, now, if i run this script, i'll get next message:

.  PrepareEggs

Traceback (most recent call last):

  File "C:\Python34\lib\site-packages\doit\doit_cmd.py", line 168, in run

    return command.parse_execute(args)

  File "C:\Python34\lib\site-packages\doit\cmd_base.py", line 122, in parse_execute

    return self.execute(params, args)

  File "C:\Python34\lib\site-packages\doit\cmd_base.py", line 405, in execute

    return self._execute(**exec_params)

  File "C:\Python34\lib\site-packages\doit\cmd_run.py", line 253, in _execute

    return runner.run_all(self.control.task_dispatcher())

  File "C:\Python34\lib\site-packages\doit\runner.py", line 247, in run_all

    self.run_tasks(task_dispatcher)

  File "C:\Python34\lib\site-packages\doit\runner.py", line 206, in run_tasks

    node = task_dispatcher.generator.send(node)

  File "C:\Python34\lib\site-packages\doit\control.py", line 625, in _dispatcher_generator

    next_step = node.step()

  File "C:\Python34\lib\site-packages\doit\control.py", line 333, in step

    return next(self.generator)

  File "C:\Python34\lib\site-packages\doit\control.py", line 342, in _func

    for value in decorated(*args, **kwargs):

  File "C:\Python34\lib\site-packages\doit\control.py", line 451, in _add_task

    yield self._gen_node(node, task_dep)

  File "C:\Python34\lib\site-packages\doit\control.py", line 372, in _gen_node

    node = ExecNode(self.tasks[task_name], parent)

KeyError: 'SpamEggs:egg-0'

Crucial moment is at line : 

https://github.com/pydoit/doit/blob/master/doit/control.py#L372

`task_name` here equals to 'SpamEggs:egg-0', but there only 'SpamEggs' currently exists inside `self.tasks`.

I found it strange that doit searching only for subtask but don't predict that supertask may be delayed and its subtasks may not exist yet.

All looks like it's easy to find supertask delayed creator here, create subtasks and continue selecting task to be executed.

Mby some bug take place here?

[Eduardo Schettino]

I found it strange that doit searching only for subtask but don't predict that supertask may be delayed and its subtasks may not exist yet.

All looks like it's easy to find supertask delayed creator here, create subtasks and continue selecting task to be executed.

Mby some bug take place here?

 

Delayed tasks are created on demand. You 2 task-creator functions with create_after `PrepareEggs` but doit doesnt know which to create first... doit will not try to "find" or "predict" the correct order.

Just mark CatchEggs to be created after SpamEggs is executed like:

  @create_after( executed = 'SpamEggs' )

  def task_CatchEggs() :

 

Another option would be create the tasks for CatchEggs and SpamEggs inside a single function.
Because in a single function you have control which task will be created first and both tasks will created at the same time.

 cheers

[Franken Stein]

 

Delayed tasks are created on demand. You 2 task-creator functions with create_after `PrepareEggs` but doit doesnt know which to create first... doit will not try to "find" or "predict" the correct order.

Just mark CatchEggs to be created after SpamEggs is executed like:

  @create_after( executed = 'SpamEggs' )

  def task_CatchEggs() :

 

Another option would be create the tasks for CatchEggs and SpamEggs inside a single function.
Because in a single function you have control which task will be created first and both tasks will created at the same time.

 cheers

Hey. :)

Problem of joining `SpamEggs` and `CatchEggs` together is that in real life it impossible. Such configuration becomes overloaded, very complex in future support.

But in other hand, solution with setting @create_after( executed = 'SpamEggs' ) for `CatchEggs` looks like dependency duplication.

Ok, i see the reasons. My `SpamEggs` may not be able to construct when `CatchEggs` constructed and `CatchEggs:egg-0` selected to be executed.

And more than, `SpamEggs` may create tasks with totally different `basename`, and actual `SpamEggs:egg-0` may be created by different supertask.

It's clear for me now. :)

Thank for your response!