DAG graph generation without running the pipeline

[Bong-Hyun Kim]

Hi all,

I am wondering if I can get the DAG without fully running the pipeline.
It looks like the DAG generation is associated to the NEXTFLOW RUN <pipeline> -with-dag flowchart.png.
Is there other way to get the DAG without running the pipeline?
My pipeline is quite slow, and I wish I can just draw the DAG without running the full pipeline.
If not, the separation of DAG to the pipeline running might be a good feature for analysis of complex pipelines and prototypings.

Otherwise, nextflow is awesome!

Best,

Bong-Hyun

[Brandon]

In sure, I want to do same with you in nextflow.

Alternatively, We can run the pipeline with small example data. it will be help for workflow development too.

2016년 7월 1일 금요일 오전 6시 26분 19초 UTC+9, Bong-Hyun Kim 님의 말:

[Paolo Di Tommaso]

The DAG is created dynamically at runtime. For this reason it can only rendered by launching a pipeline execution. 

As suggested by Brandon it's *always* a good practice include in your project a test dataset that allows you to run your pipeline. Other than to render the DAG, this is extremely useful for quick iterate your pipeline development-test-debug process. 

Alternatively, if you dataset if your dataset is composed by multiple input files, you can run your pipeline over just a small sample of them apply the take operator. 

Hope it helps.

Cheers,

Paolo

--
You received this message because you are subscribed to the Google Groups "Nextflow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+u...@googlegroups.com.
Visit this group at https://groups.google.com/group/nextflow.
For more options, visit https://groups.google.com/d/optout.

[Bong-Hyun Kim]

Thanks. You are absolutely right that it is always good to have small test set!

Bong-Hyun

You received this message because you are subscribed to a topic in the Google Groups "Nextflow" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nextflow/ggaxlRiSra8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nextflow+u...@googlegroups.com.

[Maxime Hebrard]

Hello, 

I come back on this old tread

It would be nice to be able to see the graph without running the pipeline.

All the information are in the .nf file. it should be possible to do the job by simply read the file.

I get your advice to run with small datasets, 

but some workflow needs a lots of dependencies, both software and resources (data)

it would be nice to be able to visualize what the workflow is doing without having to install and download everything.

[Paolo Di Tommaso]

The main problem here is that the full graph is only inferred at runtime. More in detail since the dynamic nature of the script interpretation a node can be executed while some other tasks still may not be interpreted.  

I think the full graph could be inferred analysing the abstract syntax tree during the script compilation phase. But it may be a quite challenging task. If someone want to try that it's welcome contribution. 

p