IdentityInterface as a gating node

[Fred Loney]

The IdentityInterface nodes are usually elided in the Nipype execution graph, e.g.:

ii = Node(IdentityInterface(fields=['one', 'two']), name='ii')
workflow.connect(upstream1, 'one', ii, 'one')
workflow.connect(upstream2, 'two', ii, 'two')
workflow.connect(ii, 'one', downstream1,'one')
workflow.connect(ii, 'two', downstream2,'two')

reduces in the execution graph to the ports:

upstream1--one-->downstream1
upstream2--two-->downstream2

Does the execution preserve the IdentityInterface node precedence constraint? I.e., if ii were any other interface, then ii could only start after both upstream1 and upstream2 completed. Is that constraint still the case when ii is an IdentityInterface, or is it possible that downstream1 could start before upstream2 completes? In other words, can an IdentityInterface be used as a gating node to constrain execution order of two unconnected nodes?

Fred

[Satrajit Ghosh]

hi fred,

in this case, downstream1 will start as soon as upstream1 finishes, independent of the status of upstream2. the identityinterface used to do the gating job in the past, but we found that it would make things wait unnecessarily when they could be running, so we started removing them during the creation of the execution graph. 

to do the gating, you could use a gating function node.

```

def gatingFunction(one, two):

    return one, two

gate = Node(Function(['one', 'two'], ['one', 'two'], gatingFunction), name='gatekeeper')

```

in fact you could use this paradigm to do manual intervention as well in the middle of a workflow.

```

def gatingFunction(one, two, checkdir):

    import os

    checkdir = compute_hash(one, two)

    if not os.path.exists(os.path.join(checkdir, 'checked.txt'):

         copyfiles(one, checkdir)

         copyfiles(two, checkdir)

         # send email

         raise IOError('Manual check file does not exist')

    return one, two

gate = Node(Function(['one', 'two'], ['one', 'two'], gatingFunction), name='gatekeeper')

```

cheers,

satra

[Fred Loney]

Thanks, Satra. A Function works just as well.

Fred