Workflow pipeline that generate 3D geometry files

[Nicholas Yue]

Hi,

  I would like to find out if there are any domain/use-case/pipeline of CWL where 3D geometry files are created as part of the process ?

Cheers

[Francis Charette Migneault]

I have used CWL to process LiDAR point clouds with different structures in the scene. Is this the kind of 3D geometries you are mentioning?

[Nicholas Yue]

Thank you Francis.

Yes, point clouds are one of them (including volumetric data like OpenVDB), others that which I am also interested are OBJ, STL i.e. those with surface area.

Are you able to point me to relevant higher view information on how your CWL pipeline fits into the bigger scheme of things ?

Thank you in advanced.

Cheers

[Peter Amstutz]

Hi Nicholas,

As it happens I'm pretty familiar with real-time 3D rendering pipelines but much less so with batch 3D rendering.  I think you've mentioned using Blender in the past?  If you could describe what you're trying to do a little more I might be able to give some high level suggestions.

Thanks,

Peter

--
You received this message because you are subscribed to the Google Groups "common-workflow-language" group.
To unsubscribe from this group and stop receiving emails from it, send an email to common-workflow-la...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/common-workflow-language/25fe4d86-107e-4c86-af08-9e29df7d896bn%40googlegroups.com.

[Nicholas Yue]

Thank you Peter.

At the moment, I am doing a general survey of the landscape of where CWL are in use (particularly outside of bioinformatics e.g. engineering) and if there are any work done associated with 3D geometries.

I will definitely write to this list as it is active with helpful people.

Cheers

[Francis Charette Migneault]

In my case, we are more using 3D geometries rather than generating. The context is more geospatial / earth-observation related.

I have a workflow that performs gridding of sampled data by LiDAR over massive regions of forested, rural and urban areas into tiles.

Each tile is then sent directly as point cloud to a dockerized Python ML application to predict classes.

An alternate branch of the workflow converts the tiled point clouds into 2D images using a C++ based tool wrapped in Docker, which are then sent to a different ML application to perform similar predictions in combination with digital elevation model data.

Predicted classes are then regrouped to form JSON FeatureCollection entries used in map services for tasks like urban planing and inventory management.

CWL is used to dispatch both for the encapsulation aspect of each sub-step application using different dependencies, and takes advantage of parallelization of the tiles and chaining of the steps because it is too big (memory and time wise) for processing in a single shot.

[Nicholas Yue]

On Mon, 8 Aug 2022 at 15:10, Francis Charette Migneault <francis.chare...@crim.ca> wrote:

CWL is used to dispatch both for the encapsulation aspect of each sub-step application using different dependencies, and takes advantage of parallelization of the tiles and chaining of the steps because it is too big (memory and time wise) for processing in a single shot.

Is the dispatch to an HPC cluster (what software are you using?) or to a big machine on the local network ?

Cheers

--

Nicholas Yue
https://www.linkedin.com/in/nicholasyue/

[Francis Charette Migneault]

It is dispatched on a CPU+GPU cluster maintained by my company, with OpenStack VMs used as worker nodes.

We use https://github.com/crim-ca/weaver to push HTTP requests that will run jobs based on processes defined with CWL.