Hi,
Sorry that you had to wait so long for an answer. You can build
workflows that include manual steps. The manual interaction itself is
outside Luigi scope, however.
You are on the right track with markers. Your full pipeline will be a
combination of human and machine tasks. The challenge is the
integration between. When you have a human task before a machine task,
Luigi could poll for completion of the human task. In the general
case, you can use a custom ExternalTask, where you override the
complete() method. You can also poll for existence of files by
overriding output(), or run a database query, depending on how you
want the human to signal completion. If you use GSuite or Office 365,
a tick box or a row in an online spreadsheet might be a good signal
for handover.
Luigi will check for completion every time it runs. In order to make
the pipeline reliable, you schedule Luigi with a cron job or similar
service to poll regularly until the pipeline is completed.
This pattern is also applicable for data collection, where data is
pushed from source systems, rather than provided by a human.
Luigi will know nothing about the human part of the process. If you
would like a more integrated experience with UI visibility into the
pipeline, you will need a tool that has explicit support for human
workflows. Either manual process tools, such as Jira, or workflow
engine tools that support mixed workflows. If you already use a CI/CD
tool that has support for a mixed pipeline, e.g. Jenkins or GoCD, that
might be a good candidate.
Regards,
Lars Albertsson
Data engineering entrepreneur
www.scling.com,
www.mapflat.com
https://twitter.com/lalleal
+46 70 7687109
> --
> You received this message because you are subscribed to the Google Groups "Luigi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
luigi-user+...@googlegroups.com.
> To view this discussion on the web visit
https://groups.google.com/d/msgid/luigi-user/d87fe144-833e-4506-9fbc-5bd9b0bb64beo%40googlegroups.com.