Excellent Pkl Overview...

[Jim Hargrave]

For those short on time watch this video - gives a great Pkl overview.

Pkl does have a maven artifact. There is heavy interest in Python support. I wouldn't be surprised if Python was added this year.

https://www.youtube.com/watch?v=lAxXWYAIt4k

Jim

[Jim Hargrave]

As I research this more Protobuffers isn't really considered a configuration management language. It can be extended to fill that role with third party packages, but IMHO you then loose the main selling point of PB as a well used standard.

These are additional configuration languages. I've never heard of any of them :-)

Cross-platform configuration management languages similar to Apple's Pkl include:

1. Jsonnet
2. HCL
3. Dhall
4. CUE

[Warnar Boekkooi]

Hey Jim,

Did you have a look at https://kdl.dev/ already? It looks like a type of format that may match most/all requirements.

[Jim Hargrave]

Interesting - this looks like a strong contender for a human config format. Very impressive cross-language support. It has some definite advantages over TOML.

Jim

--
You received this message because you are subscribed to the Google Groups "okapi-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to okapi-devel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/okapi-devel/c6c3beea-63eb-41a3-81b5-6acdedb10d7an%40googlegroups.com.

[Jim Hargrave]

I missed this:

https://github.com/kdl-org/kdl/blob/main/SCHEMA-SPEC.md

Looks like kdl could be a complete replacement for Pkl.

[Jim Hargrave]

kdl's validation is limited, but it does handle the most common use cases. What I like about Pkl is that it has a full validation language that gives you fine grained control of your validation logic across languages. I also like that with Pkl everything is in one file both template and validation logic.

Jim

[Jim Hargrave]

After ruminating on the available options for configuration with schema and validation I think tools like Kdl and Pkl, and even ProtoBuffer all have serious limitations. All require proprietary dependencies and learning custom formats. ProtoBuffer does not have validation (except with a third party package and then only for Java).

Of the three above ProtoBuffer has the most traction. But I think there is an easier way forward that will require fewer "custom" technologies. 

JSON Schema is a standard that has serious commercial backing and has support across many programming languages. This is about as standards based as we can get (unless we went pure XML).

TOML is a well used format for configuration. It's gained significant traction in the last few years. It can be easily converted (without loss) to JSON and back. TOML has ubiquitous IDE and editor support.

Because TOML is compatible with JSON it can be validated with JSON Schemas (by converting to JSON).

I think this will be my final proposal for the new Okapi configuration architecture. We only need to learn technologies that are widely used in the industry. Each filter or step will have a config template defined with JSON Schema. Humans and API's can use TOML (or JSON directly) to define their configurations files.

thoughts?

Jim

[Jim Hargrave]

Based on the discussion with Mihai here's a revised proposal.

1. Use ProtoBuffer to define template for data.
2. Optionally, If needed us JsonSchema for validation.
3. Accept two standard formats as Parameter config files: JSON and TOML (can be converted back and forth and validated with JsonSchema).
4. Use proto file to auto generate wiki (data driven). Mihai mentioned possibly having two proto files. One for data and another for doc and UI.
   

If we wait long enough and Apple's Pkl gets Python support then that would be a strong option. Pkl really does solve the complete problem.  Only weakness is that it is new and lacks Python and other language support. But those are being added quickly.

Jim