Almost final draft 2 ready for review

[Peter Amstutz]

Hello everyone,

I am pleased to announce the almost final draft 2 specification for
the Common Workflow Language. I say "almost final" because it is now
closed to additional technical changes, and am requesting that
everyone who is interested please review it for copy editing or
clarification comments before it is tagged. Once draft 2 is tagged no
more changes will be accepted and we will begin working on the next
draft. Please direct your comments or fixes to the mailing list, or
with a pull request. Thanks!

The generated specification is here:

https://common-workflow-language.github.io/draft-2/

The specification is generated from this schema:

https://github.com/common-workflow-language/common-workflow-language/blob/master/schemas/draft-2/cwl-avro.yml

- Peter

[Michael Crusoe]

https://common-workflow-language.github.io/draft-2/#workflowstep

Under Subworkflows there is a formatting error as '(SubworkflowFeatureRequirement)(#SubworkflowFeatureRequirement) ' shows up in the rendered output.

https://common-workflow-language.github.io/draft-2/#commandlinetool

The word 'standalone' can be removed. For that matter 'command line' can also be removed as 'non-interactive' & 'POSIX' truly cover the bases.

Under 'Input binding': steps 3 & 4 talk about sorting. I think you mean the same thing in both instances (sort using UTF-8 encoding); perhaps that specification should be moved above the list?

I'm not seeing how to model a boolean so that both 'true' and 'false' result in differents strings being added to the command line.

The word 'metadata' never appears. We should document that CWL files can contain extra data about the tools and workflows. 

--
You received this message because you are subscribed to the Google Groups "common-workflow-language" group.
To unsubscribe from this group and stop receiving emails from it, send an email to common-workflow-la...@googlegroups.com.
To post to this group, send email to common-workf...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/common-workflow-language/CAEXjzRuDO3qFRZ81eCEUQqHhA4jBSMQRHrnKZVCBdd69QrgWFg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Michael R. Crusoe: Programmer & Bioinformatician cru...@ucdavis.edu 
The lab for Data Intensive Biology; University of California, Davis
https://impactstory.org/MichaelRCrusoe http://twitter.com/biocrusoe

[Peter Amstutz]

On Wed, Jun 24, 2015 at 6:13 PM, Michael Crusoe
<michael...@gmail.com> wrote:
> https://common-workflow-language.github.io/draft-2/#workflowstep
> Under Subworkflows there is a formatting error as
> '(SubworkflowFeatureRequirement)(#SubworkflowFeatureRequirement) ' shows up
> in the rendered output.

Fixed.

> https://common-workflow-language.github.io/draft-2/#commandlinetool
> The word 'standalone' can be removed. For that matter 'command line' can
> also be removed as 'non-interactive' & 'POSIX' truly cover the bases.

Done.

> Under 'Input binding': steps 3 & 4 talk about sorting. I think you mean the
> same thing in both instances (sort using UTF-8 encoding); perhaps that
> specification should be moved above the list?

Done.

> I'm not seeing how to model a boolean so that both 'true' and 'false' result
> in differents strings being added to the command line.

One way to do this is to define an "enum" where the symbols are the
two alternate command line flags. Another way is to use an expression
that overrides the default handling of boolean. A third way would be
to define two types with the command line flag in inputBinding.prefix
and use a union.

> The word 'metadata' never appears. We should document that CWL files can
> contain extra data about the tools and workflows.

Added a section "Extensions and Metadata"

Thanks,
Peter

[Michael Crusoe]

Looks good, thanks!

[porter...@gmail.com]

One thing I noticed—In the "2.2 Syntax" section there's a lot of references to avro-ld, but no precise description of exactly what avro-ld is or how to convert it to a standard avro schema or extract a json-ld context. This seems like an omission that could hamper alternative implementations, since implementers don't really have a description of how to validate CWL documents (except by reading the reference implementation).

Looks good, thanks!

>> email to common-workflow-language+unsub...@googlegroups.com.

>> To post to this group, send email to
>> common-workf...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/common-workflow-language/CAEXjzRuDO3qFRZ81eCEUQqHhA4jBSMQRHrnKZVCBdd69QrgWFg%40mail.gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> Michael R. Crusoe: Programmer & Bioinformatician cru...@ucdavis.edu
> The lab for Data Intensive Biology; University of California, Davis
> https://impactstory.org/MichaelRCrusoe http://twitter.com/biocrusoe

[Peter Amstutz]

Yes, that's fair.  The schema language ended up taking on a life of its own and I would like to split it out into its own project (with its own documentation, specification, etc) but have not had the time to do so yet.  If someone would like to volunteer to help I would be happy to chat about it.

I don't think this should block finalizing draft 2, validation is useful but not essential to write a working implementation, so this can be a goal for draft 3.  Would you like to file an issue?

Thanks,
Peter

[stefan.lo...@gmail.com]

Are there any notes or documentation on avro-ld? I'm interested in developing a Ruby implementation to contribute towards the project.

Thanks!

[stefan.lo...@gmail.com]

Hi,

I've been going through the reference implementation (specifically, how the Avro schema is "extended" for linked data) and I'm kind of confused how extending models are tracked (via extended_by). Let's say we have three records:

• Record A
• Record B (Extends A)
• Record C (Extends B)

What would the expected extends property be for each of those records? Does record C extend B? A? A and B?

What about the inherited_from property? I assume that is applied at the per-field level, so record C might have fields from record A -- those would be marked as inherited_from: A. Is that a fair assumption?

Also -- what's the purpose of the first_def function? When we generate the schema, I see that we're interested in filtering out abstract and doc nodes. Is that the only functional purpose of first_def?

Thanks and looking forward to contributing to the project!

[Peter Amstutz]

On Jul 17, 2015 8:04 PM, <stefan.lo...@gmail.com> wrote:
>
> Hi,

These are great questions, and just reminds me that I need to pull the schema language out into its own project.

> I've been going through the reference implementation (specifically, how the Avro schema is "extended" for linked data) and I'm kind of confused how extending models are tracked (via extended_by). Let's say we have three records:
> Record A
> Record B (Extends A)
> Record C (Extends B)
> What would the expected extends property be for each of those records? Does record C extend B? A? A and B?

B extends A, C extends B

"extends" mainly means that field definitive are copied from parent to child during processing.

> What about the inherited_from property? I assume that is applied at the per-field level, so record C might have fields from record A -- those would be marked as inherited_from: A. Is that a fair assumption?

That's correct.  That's only used for generating documentation currently.

> Also -- what's the purpose of the first_def function? When we generate the schema, I see that we're interested in filtering out abstract and doc nodes. Is that the only functional purpose of first_def?

This is to do support converting to "pure" Avro schema.  Avro supports recursive definitions but only if the recurrence of the definition is nested in the original definition.  This gets complex when the recurrence is indirect.  First_def expands type definitions the first place they are used and ensures the type symbol in each subsequent occurrence.

If I recall correctly, this is also involved in converting instances of abstract types into a union of concrete subtypes.

> Thanks and looking forward to contributing to the project!

Welcome!

>
> On Friday, July 17, 2015 at 1:00:07 PM UTC-7, stefan.lo...@gmail.com wrote:
>>
>> Are there any notes or documentation on avro-ld? I'm interested in developing a Ruby implementation to contribute towards the project.
>>
>> Thanks!
>>
>> On Wednesday, July 1, 2015 at 2:03:30 PM UTC-7, Peter Amstutz wrote:
>>>
>>> Yes, that's fair.  The schema language ended up taking on a life of its own and I would like to split it out into its own project (with its own documentation, specification, etc) but have not had the time to do so yet.  If someone would like to volunteer to help I would be happy to chat about it.
>>>
>>> I don't think this should block finalizing draft 2, validation is useful but not essential to write a working implementation, so this can be a goal for draft 3.  Would you like to file an issue?
>>>
>>> Thanks,
>>> Peter
>

> --
> You received this message because you are subscribed to the Google Groups "common-workflow-language" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to common-workflow-la...@googlegroups.com.

> To post to this group, send email to common-workf...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/common-workflow-language/6c82db9c-9a5d-4ce1-ae85-bb735da9e04e%40googlegroups.com.