Schema language

[Julien Le Dem]

I've opened an issue to pick the language to formalize the OpenLineage metadata.

I'm proposing an OpenAPI spec which is fairly portable and supports code generation in many languages:

https://github.com/OpenLineage/OpenLineage/issues/2

Please chime in if you have another preference or support this.

Here is an example of what an initial spec would look like:

https://github.com/OpenLineage/OpenLineage/pull/1/files

[Julien Le Dem]

In particular, avro schemas can be generated from that.

[Julien Le Dem]

I have added a simple generator to the PR so that we can get an idea of what the api looks like:

https://github.com/OpenLineage/OpenLineage/pull/1

[Julien Le Dem]

I have updated the initial spec to be more compact and have added a document to describe the contract.

You can focus on spec/OpenLineage.md and spec/OpenLineage.yml

https://github.com/OpenLineage/OpenLineage/pull/1/files#diff-343538dfec4460d0d703f9669b95f22b60d841e4e98d4a319a706a44f91b147e

--
You received this message because you are subscribed to a topic in the Google Groups "OpenLineage" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openlineage/2i7ogPl1IP4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openlineage...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openlineage/4396eeed-e94d-41ff-be39-9fbfb5168d08n%40googlegroups.com.

[Julien Le Dem]

I have updated the PR with a leaner spec:

https://github.com/OpenLineage/OpenLineage/pull/1/files

In particular it describes facets and how the model can be extended.

I find https://editor.swagger.io/ convenient to look at the yml file.

Julien