Document and fully support piqi-light syntax?

[Ignas Vyšniauskas]

Hi, Anton,

Toying around with piqi after awhile again, I only now discovered the
`Piqi-light` syntax.

I have one thing to say: it is *AWESOME*. Besides being more readable
in general, it fixes one thing I really disliked in the standard piqi
syntax. Namely, given

.$type [
.name $foo
<$stuff>
]

it wasn't clear that the type definition gets bound to the name $foo.
However in Piqi-light syntax this becomes

type $foo = $stuff

which is very straightforward and much closer to how one would define
data types in FP languages.

I wanted to encourage you to finalize Piqi-light syntax and to allow
it as input (i.e. write a parser for it), not merely output. This
would be an amazing usability improvement, because I am guessing that
many people are alienated by the custom Piq syntax.

I don't know if you have the full syntax worked-out, but I recommend
to leave space for appending arbitrary fields or even arbitrary piq
definitions to the types, i.e. something like

type bigNumber = int <[ deprecated; erlang-name = "big_number"; ]>

in the spirit of 'make simple things easy and complex things
possible'. Or maybe it could be done with "annotations", this would
maybe work more nicely with possible future-things such as docs:

@erlang-name = "big-number"
@deprecated
@doc "This type describes a pretty big number."
type bigNumber = int

Another suggestion is to include separators where they make sense
(e.g. between record field definitions) and avoid too many
magic-annotation-characters like `?`, `*, `-`, because they lead to
the syntax being very noisy (and thus harder to read).

Anyway, feel free to ignore these syntax suggestions, but I just
wanted to encourage you to push this out and mention it more visibly
in the documentation, since it's pretty damn cool.

--
Ignas

P.S. Any future plans for piqi?

[Anton Lavrik]

Hi Ignas,

Glad you liked the light version of the syntax. I surely drew inspiration from FP languages when designing it. There is also a little bit of conventional grammar syntax elements as you can probably tell. Speaking of which, would you rather use 'required', 'repeated', 'optional' keywords instead of '?' or '*' ?

The idea behind this syntax is to make piqi specs easier to read. Protocol definitions are usually written once, occasionally edited and read the rest of the time. Somewhat verbose syntax for composing specs shouldn't be a problem in practice unless all one does is write them, but it is rarely the case.

As in LISP, all the benefits and downsides of the Piq syntax come from the fact it's abstract. Piq is somewhat unique however. While being abstract, it is less verbose than other data representation languages like JSON (but surely more verbose than a special-purpose concrete syntax). But best of all, as long you don't mind some verbosity, you don't have to think about concrete syntax. Designing a good concrete syntax is always a lot more difficult than one may think.

Having the light syntax as output format allows me to keep it experimental and not worry about it too much. Implementing it as input notation is a more serious language design commitment.

That said, we may still do it. There is a value of things looking uniform when writing and reading them. And, by the way, I like your ideas about how to extend the schema syntax.

Speaking of future plans for Piqi, I need to stabilize command-line API, add support for Protobuf 2.5, 2.6. 3.0, finish the portable RPC spec discussed earlier on this list, and release everything as v1. There is also a list of things for piqi-erlang and piqi-rpc and I have plenty of ideas of what to do after that. The tricky part is finding time.

On a slightly separate topic, in addition to the schema syntax, you seem to suggest to use an alternative syntax for representing attributes and default values. For instance,

<[ deprecated; erlang-name = "big_number"; ]>

Why do you think it is better than the Piq syntax for that, i.e.

[ .deprecated .erlang-name "big_number" ]

I agree that your's is somewhat more conventional but Piq is also pretty natural for anyone who is familiar with command-line options. Don't really remember hearing any complains about the Piq syntax.

To evaluate which representation is better, just compare them side by side. Here's a quick example of Piq and two more conventional syntax approaches:

.person [

    .name "J. Random Hacker" .id 0

    .phone [ .number "(111) 123 45 67" ]

    .phone [ "(444) 123 45 67" .mobile ]

]

person: [

    name: "J. Random Hacker", id: 0,

    phone: [ number: "(111) 123 45 67" ],

    phone: [ "(444) 123 45 67", mobile ],

]

person = [

    name = "J. Random Hacker"; id = 0;

    phone = [ number = "(111) 123 45 67" ];

    phone = [ "(444) 123 45 67"; mobile ];

]

I personally like the second ("JSON-like") better than the third ("C-like"). One may argue JSON-like or C-like are more readable than Piq as they provide more visual cues as infix separators. As for me, such readability gain is marginal. On the other hand, you need to introduce at last one more lexical symbol for element separators. In theory, you could make separators optional but it can hurt readability when abused which defeats the reason for adding them in the first place. So to make it robust, you'd have to make such separators mandatory or maybe create some rules when they can be omitted, e.g. at the end of the line.

There's more to it. Unlike in Piq, you now don't have a good visual way of distinguishing between words in a quoted string, unquoted strings and symbols/atoms. So you use a different quoting style for atoms, e.g. 'work' and 'mobile'. You can probably get away with `work and `mobile as well, but it is way less conventional. And you obviously can't reuse any of ,.: for quotation, to avoid similarity with the separators.

Now, not only you've added more lexical elements, but you made (label : value) or (label = value) pairs look substantially different from just 'label' without a value. Yet, such labels are very close in nature regardless of whether they come with or without values. Not pretty. To overcome this, you may always require labels to come with values, e.g. (mobile: true) or (mobile = true). However, in this case, you'd actually have to write (phone-type: mobile) instead. Ha-ha.

I could go on, but let me just say that despite of Piq syntax looking unconventional and being purposefully minimalistic, it is amazingly practical. Here's an example of how I use Piq for SQL-like query DSL at work and how the same thing would look in the JSON-like notation. The second variant doesn't seem to add any clarity, it just has more visual noise.

.let [ c .count * ]

.select [

    location

    c

    .avg addressbook_size

    .histogram [ app_launch_t 1000 ]

]

.where [

    .eq [ platform android ]

    .range [ ts -1day ]

]

let: [ c, count: * ]

select: [

    location,

    c,

    avg: addressbook_size,

    histogram: [ app_launch_t, 1000 ],

]

where: [

    eq: [ platform, android ]

    range: [ ts, -1day ]

]

Anton

--
You received this message because you are subscribed to the Google Groups "piqi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to piqi+uns...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ignas Vyšniauskas]

Hi, Anton,

First of all, let me say that overall I like the Piq syntax and I'm
not saying it itself should somehow modified or replaced. I'm just
making the point that readability and familiarity are very important
when it comes to adoption of new languages.

On Mon, Mar 23, 2015 at 10:08 AM, Anton Lavrik <ala...@piqi.org> wrote:
> Speaking of
> which, would you rather use 'required', 'repeated', 'optional' keywords
> instead of '?' or '*' ?

Yes, as I mentioned, the symbolic representations seem noisy to me. I
would at the very worst apply them to the *type*:

type person = {
name :: string()
id :: int()
email :: string()? %% or even ?string() --
%% so that it's like a type constructor for
%% an 'option' type.
phone :: phone-number()* %% alt: * phone-number()
}

[In general, the repeated-field concept is, in my opinion, redundant
since it's isomorphic to a list type. I'd rather see lists as
primitive and use custom `.protobuf-*` flags to control the
serialization of fields with list type.]

Otherwise, just spell it out: (Optional string()), (Repeated phone-number()).

> As in LISP, all the benefits and downsides of the Piq syntax come from the
> fact it's abstract. Piq is somewhat unique however. While being abstract, it
> is less verbose than other data representation languages like JSON (but
> surely more verbose than a special-purpose concrete syntax). But best of
> all, as long you don't mind some verbosity, you don't have to think about
> concrete syntax. Designing a good concrete syntax is always a lot more
> difficult than one may think.

I completely agree and understand you here.

> Speaking of future plans for Piqi, I need to stabilize command-line API, add
> support for Protobuf 2.5, 2.6. 3.0, finish the portable RPC spec discussed
> earlier on this list, and release everything as v1. There is also a list of
> things for piqi-erlang and piqi-rpc and I have plenty of ideas of what to do
> after that. The tricky part is finding time.

If you wrote-up / categorised some of the plans (say, on the GitHub
wiki and/or issue tracker), in the future I might be able to
contribute and help you with some of it.

> On a slightly separate topic, in addition to the schema syntax, you seem to
> suggest to use an alternative syntax for representing attributes and default
> values. For instance,
>
> <[ deprecated; erlang-name = "big_number"; ]>
>
> Why do you think it is better than the Piq syntax for that, i.e.
>
> [ .deprecated .erlang-name "big_number" ]
>
> I agree that your's is somewhat more conventional but Piq is also pretty
> natural for anyone who is familiar with command-line options. Don't really
> remember hearing any complains about the Piq syntax.

The lack of an explicit separator means that `.` acts both as a prefix
to denote "name literals" and as a separator of sorts. This leads to
what you document in the docs in "2.13. Associativity control"[1].

For me, parenthesis are either for function application or "do
nothing". The distinction between `.foo .bar` and `.foo (.bar)` is
super confusing and unintuitive. At worst, I would expect the
behaviour of parenthesis to be "make `.foo` act as a constructor" and
writing `(.foo .bar)` to mean "Constructor .foo is _applied_ to the
_value_ .bar". However, such syntax is already reserved for repeated
name values, hence making the situation even more confusing.

So that's why I used (without really thinking about it) an alternative
syntax in the example, though it would probably make much more sense
to use just raw Piq syntax.

The allegory with command-line args/flags makes sense and it actually
provides a sane interpretation of `.foo (.bar)` (i.e. you read it as
-foo ".bar", so the value in parenthesis effectively becomes
"quoted"), but that's not what I'm thinking about when I write data
specifications! Perhaps the situation could also be improved by
mentioning very early on in the documentation that this is how you
should read the Piq syntax. The idea of 'quoting' does not really fit
in with the repeated name values syntax.

> To evaluate which representation is better, just compare them side by side.
> Here's a quick example of Piq and two more conventional syntax approaches:
>

> <..>

>
> As for me, such readability gain is
> marginal. On the other hand, you need to introduce at last one more lexical
> symbol for element separators. In theory, you could make separators optional
> but it can hurt readability when abused which defeats the reason for adding
> them in the first place. So to make it robust, you'd have to make such
> separators mandatory or maybe create some rules when they can be omitted,
> e.g. at the end of the line.

Once again, arguing about syntax is mostly futile, I too don't see
that much of a difference between the variants. I am mostly arguing
for improving familiarity among people who are newcomers to Piqi and
are used to the standard denotations of "assigning a value 'x' to a
field named 'a'".

That is, except for the separator part which I already mentioned as
problematic -- avoiding separators where they truly make sense is a
somewhat mysterious trend in computer languages I keep seeing.

> There's more to it. Unlike in Piq, you now don't have a good visual way of
> distinguishing between words in a quoted string, unquoted strings and
> symbols/atoms. So you use a different quoting style for atoms, e.g. 'work'
> and 'mobile'. You can probably get away with `work and `mobile as well, but
> it is way less conventional. And you obviously can't reuse any of ,.: for
> quotation, to avoid similarity with the separators.

Right, but -- if I understand this argument correctly -- on the
flipside you have the problem I described before with `.` acting
(implicitly) as a separator. I would actually see no problem with
explicitly denotating atoms -- I like it when the syntax makes the
data type explicit (instead of having it inferred by its usage).

> Now, not only you've added more lexical elements, but you made (label :
> value) or (label = value) pairs look substantially different from just
> 'label' without a value. Yet, such labels are very close in nature
> regardless of whether they come with or without values. Not pretty. To
> overcome this, you may always require labels to come with values, e.g.
> (mobile: true) or (mobile = true). However, in this case, you'd actually
> have to write (phone-type: mobile) instead. Ha-ha.

I don't really follow this argument. What is the use of a label
without a value? If it's a flag, it's implicitly always a boolean. If
it's an atom, it can be denoted as an atom?

Sorry for the wall-of-text that I'm producing here -- just trying to
synthesize my thoughts and give you some constructive feedback during
the process. I have some obscure plans for using Piqi as a backend for
a thing-I-might-build and so I'm re-evaluating all aspects of it. Piqi
is still by far the best data/interface-speification+serialization
infrastructure-thing out there (IMHO, hah) and I'm thinking on how to
make it even better :)

--
Ignas

[1]: http://piqi.org/doc/piq/#associativitycontrol

[Anton Lavrik]

On Mon, Mar 23, 2015 at 4:40 AM, Ignas Vyšniauskas <i.vysn...@gmail.com> wrote:

Hi, Anton,

First of all, let me say that overall I like the Piq syntax and I'm
not saying it itself should somehow modified or replaced. I'm just
making the point that readability and familiarity are very important
when it comes to adoption of new languages.

I agree. But coming up with new great stuff sort of contradicts sticking to what potential users may be already familiar with. If everything is good enough/familiar, why bother trying to come up with something new? So we have to find some balance here.

Piqi wouldn't exist without Piq. Try looking at piqi spec in JSON or XML. In no way this looks like a language somebody might use (I wouldn't). The trick was exactly in coming up with a way to self-describe Piqi AND represent such self-spec in an abstract but language-like notation.

On Mon, Mar 23, 2015 at 10:08 AM, Anton Lavrik <ala...@piqi.org> wrote:
> Speaking of
> which, would you rather use 'required', 'repeated', 'optional' keywords
> instead of '?' or '*' ?

Yes, as I mentioned, the symbolic representations seem noisy to me. I
would at the very worst apply them to the *type*:

type person = {
    name :: string()
    id :: int()
    email :: string()? %% or even ?string() --
                               %% so that it's like a type constructor for
                               %% an 'option' type.
    phone :: phone-number()* %% alt:  * phone-number()
}

[In general, the repeated-field concept is, in my opinion, redundant
since it's isomorphic to a list type. I'd rather see lists as
primitive and use custom `.protobuf-*` flags to control the
serialization of fields with list type.]

In Haskell, optional would turn into a Maybe (IIRC), in OCaml it is known as 'option'. So some languages do have formal representation for optional values at the type level.

Speaking of repeated vs lists. This is a great question. You'd probably be surprised to hear this, but Piqi wouldn't be created if I didn't discover the concept of 'repeated' fields in Protocol Buffers. Without repeated, .piqi would look like a mess of ever-nested stuff with twice as many brackets. So, in my view, repeated and lists are not exactly redundant. Moreover, the more I think about it the more I get convinced that between the two, the concept of repeated fields is more expressive than the concept of a list. In fact, I have an idea that high-level programming languages would benefit greatly in terms of expressiveness if repeated fields were introduced as a first-class concept.

Anyway, Piqi has both lists and repeated fields. The difference is that repeated field doesn't represent a first class value from the perspective of some mappings/target languages e.g. piqi -> protobuf -> C++/Java/Python, etc (their API's support getting element by index and iteration). The list, however, should always have a native mapping as a first-class container/object.

I agree that it makes things look more complicated and it is not as easy to improve as I would wish it were. Ideas are welcome!

Otherwise, just spell it out: (Optional string()), (Repeated phone-number()).

Sure, I'll play with this idea. This is easy to test.

> Speaking of future plans for Piqi, I need to stabilize command-line API, add
> support for Protobuf 2.5, 2.6. 3.0, finish the portable RPC spec discussed
> earlier on this list, and release everything as v1. There is also a list of
> things for piqi-erlang and piqi-rpc and I have plenty of ideas of what to do
> after that. The tricky part is finding time.

If you wrote-up / categorised some of the plans (say, on the GitHub
wiki and/or issue tracker), in the future I might be able to
contribute and help you with some of it.

 

Sure, help is always welcome. And if you wanted to work on the compact piqi syntax, I'd welcome that as well. In addition to things I already mentioned, here are some more lists:

https://github.com/alavrik/piqi/blob/master/TODO

https://github.com/alavrik/piqi-rpc/blob/master/TODO

https://github.com/alavrik/piqi-erlang/blob/master/TODO

http://piqi.org/roadmap/

> On a slightly separate topic, in addition to the schema syntax, you seem to
> suggest to use an alternative syntax for representing attributes and default
> values. For instance,
>
> <[ deprecated; erlang-name = "big_number"; ]>
>
> Why do you think it is better than the Piq syntax for that, i.e.
>
> [ .deprecated .erlang-name "big_number" ]
>
> I agree that your's is somewhat more conventional but Piq is also pretty
> natural for anyone who is familiar with command-line options. Don't really
> remember hearing any complains about the Piq syntax.

The lack of an explicit separator means that `.` acts both as a prefix
to denote "name literals" and as a separator of sorts. This leads to
what you document in the docs in "2.13. Associativity control"[1]. 

For me, parenthesis are either for function application or "do
nothing". The distinction between `.foo .bar` and `.foo (.bar)` is
super confusing and unintuitive.

Agree. I'd been hesitant for a very long time to allow '.foo bar', because there's no clear syntactic indication on whether bar is a symbol or a single-word unquoted string (or potentially even a variable identifier). But because Piq is a typed language, it is not that bad. Anyway, it is decided now and it should really help with other inconsistencies.

Keep in mind, that .foo (.bar) can be nicely written as .foo.bar and .foo (.bar 1) -- which hopefully looks ok to you -- can be written as .foo.bar 1

I am also thinking about supporting the same syntax for nested fields: .e.g. in addition to writing .foo [ .bar 1 .baz "aaa" ] we'll be allowed to write

.foo.bar 1

.foo.baz "aaa"

And then, knowing the type, piqi will place both of them inside their proper container. This could be a very nice syntax addition for config files and config options (kind of like sysctl).

At worst, I would expect the
behaviour of parenthesis to be "make `.foo` act as a constructor" and
writing `(.foo .bar)` to mean "Constructor .foo is _applied_ to the
_value_ .bar". However, such syntax is already reserved for repeated
name values, hence making the situation even more confusing.

(.foo .bar) expands exactly to .foo (.bar) and means exactly what you've written: constructor/label .foo is applied to the value .bar

This syntax is not reserved for repeated fields. In Piq, (...) is a _form_, similar to LISP. Its interpretation is determined by the very first element after the opening parenthesis. I could have defined (label .foo .bar) instead of (.foo .bar) but the latter is just shorter.

At the same time, the form (.foo 1 2 3 .bar) is, per definition of such forms, is expanded to .foo 1 .foo 2 .foo 3 .foo.bar

> There's more to it. Unlike in Piq, you now don't have a good visual way of
> distinguishing between words in a quoted string, unquoted strings and
> symbols/atoms. So you use a different quoting style for atoms, e.g. 'work'
> and 'mobile'. You can probably get away with `work and `mobile as well, but
> it is way less conventional. And you obviously can't reuse any of ,.: for
> quotation, to avoid similarity with the separators.

Right, but -- if I understand this argument correctly -- on the
flipside you have the problem I described before with `.` acting
(implicitly) as a separator. I would actually see no problem with
explicitly denotating atoms -- I like it when the syntax makes the
data type explicit (instead of having it inferred by its usage).

One of the goals for Piq was minimal syntax for data representation that wouldn't be overloaded with lexical markup.

> Now, not only you've added more lexical elements, but you made (label :
> value) or (label = value) pairs look substantially different from just
> 'label' without a value. Yet, such labels are very close in nature
> regardless of whether they come with or without values. Not pretty.

I don't really follow this argument. What is the use of a label
without a value? If it's a flag, it's implicitly always a boolean. If
it's an atom, it can be denoted as an atom?

I should have explained it better. First, a little bit of a background. In Piq, values of record {?foo, *bar :: int()} and a list of of variants [foo | bar :: int()] have the same syntax representation, e.g.:

[.foo .bar 1]

This is done on purpose, because I believe these two data types represent essentially the same data. Barring some optional|required|repeated  constraints.

As you can see, both fields/elements .foo and .bar 1 have consistent syntax representation.

In the alternative syntax, you'd have to write [ 'foo', bar : 1 ]. Now they look substantially different and this is what I mean by being not pretty.

Sorry for the wall-of-text that I'm producing here -- just trying to
synthesize my thoughts and give you some constructive feedback during
the process. I have some obscure plans for using Piqi as a backend for
a thing-I-might-build and so I'm re-evaluating all aspects of it. Piqi
is still by far the best data/interface-speification+serialization
infrastructure-thing out there (IMHO, hah) and I'm thinking on how to
make it even better :)

Thanks and I don't mind long comments at all. This is a great feedback and you've been making some really good points.

Anton

[Ignas Vyšniauskas]

On Fri, Mar 27, 2015 at 7:55 AM, Anton Lavrik <ala...@piqi.org> wrote:
> Speaking of repeated vs lists. This is a great question. You'd probably be
> surprised to hear this, but Piqi wouldn't be created if I didn't discover
> the concept of 'repeated' fields in Protocol Buffers. Without repeated,
> .piqi would look like a mess of ever-nested stuff with twice as many
> brackets. So, in my view, repeated and lists are not exactly redundant.
> Moreover, the more I think about it the more I get convinced that between
> the two, the concept of repeated fields is more expressive than the concept
> of a list. In fact, I have an idea that high-level programming languages
> would benefit greatly in terms of expressiveness if repeated fields were
> introduced as a first-class concept.

You mean that e.g.

.field [ $FOO ] .field [ $BAR ]

is so much better than (in non-exsistant syntax):

.fields [ [$FOO], [$BAR] ]

?

I might agree / don't-care syntax-wise, but structurally I disagree
that this should be the case.

> https://github.com/alavrik/piqi/blob/master/TODO
> https://github.com/alavrik/piqi-rpc/blob/master/TODO
> https://github.com/alavrik/piqi-erlang/blob/master/TODO
> http://piqi.org/roadmap/

Thanks, I'll try to take a look. I'll make a GH issue if I start
working on anything.

> I am also thinking about supporting the same syntax for nested fields: .e.g.
> in addition to writing .foo [ .bar 1 .baz "aaa" ] we'll be allowed to write
>
> .foo.bar 1
> .foo.baz "aaa"
>
> And then, knowing the type, piqi will place both of them inside their proper
> container. This could be a very nice syntax addition for config files and
> config options (kind of like sysctl).

I'd avoid more syntactic sugar. It would imply you always have to know
the container type in order to mentally parse the declarations (i.e.
"is this a variant or a record?"),
and again, I always prefer the syntax to explicitly denotate the data
type. The situation
can be become very bad in the case of nested data types.

As you mention later, currently the syntax is ambigious only for
variant lists and records,
but this would make it worse, if I understand correctly.

> (.foo .bar) expands exactly to .foo (.bar) and means exactly what you've
> written: constructor/label .foo is applied to the value .bar

In the docs (on "2.13. Associativity control") you don't mention that
(.foo .bar) === .foo (.bar).
You definitely should.

> This syntax is not reserved for repeated fields. In Piq, (...) is a _form_,
> similar to LISP. Its interpretation is determined by the very first element
> after the opening parenthesis. I could have defined (label .foo .bar)
> instead of (.foo .bar) but the latter is just shorter.

Ah, this makes perfect sense. You might want to mention it in the docs
as it will definitely help at least those familiar with LISP.

> I should have explained it better. First, a little bit of a background. In
> Piq, values of record {?foo, *bar :: int()} and a list of of variants [foo |
> bar :: int()] have the same syntax representation, e.g.:
>
> [.foo .bar 1]
>
> This is done on purpose, because I believe these two data types represent
> essentially the same data. Barring some optional|required|repeated
> constraints.
>
> As you can see, both fields/elements .foo and .bar 1 have consistent syntax
> representation.
>
> In the alternative syntax, you'd have to write [ 'foo', bar : 1 ]. Now they
> look substantially different and this is what I mean by being not pretty.

Why couldn't I write [foo, bar: 1] or ['foo', 'bar': 1] or [`foo, `bar: 1]?

--
Ignas

[Anton Lavrik]

On Mon, Mar 30, 2015 at 2:13 AM, Ignas Vyšniauskas <i.vysn...@gmail.com> wrote:

On Fri, Mar 27, 2015 at 7:55 AM, Anton Lavrik <ala...@piqi.org> wrote:
> Speaking of repeated vs lists. This is a great question. You'd probably be
> surprised to hear this, but Piqi wouldn't be created if I didn't discover
> the concept of 'repeated' fields in Protocol Buffers. Without repeated,
> .piqi would look like a mess of ever-nested stuff with twice as many
> brackets. So, in my view, repeated and lists are not exactly redundant.
> Moreover, the more I think about it the more I get convinced that between
> the two, the concept of repeated fields is more expressive than the concept
> of a list. In fact, I have an idea that high-level programming languages
> would benefit greatly in terms of expressiveness if repeated fields were
> introduced as a first-class concept.

You mean that e.g.

    .field [ $FOO ] .field [ $BAR ]

is so much better than (in non-exsistant syntax):

    .fields [ [$FOO], [$BAR] ]

?

I might agree / don't-care syntax-wise, but structurally I disagree
that this should be the case.

Such nesting would apply to all syntax elements. For example, you'd have typedefS, includeS, recordS, variantS, and so on.

This is why vast majority of concrete notations tend to flatten things. For example, you write typedef a =;  typedef b = instead of typedefs [ [ a =  ] [ b = ] ]. Similarly, include a; include b instead of includes [ a, b]

This is also pretty easy to test in case of Piqi. Take any practical .piqi definition file and run

piqi convert -t json -o - x.piqi | sed -e 's/"//g'

By the way, sometimes this *can* be useful and there is a way to express it in Piq as:

(.field [$FOO] [$BAR]) which expands to .field [$FOO] .field [$BAR]

> I am also thinking about supporting the same syntax for nested fields: .e.g.
> in addition to writing .foo [ .bar 1 .baz "aaa" ] we'll be allowed to write
>
> .foo.bar 1
> .foo.baz "aaa"
>
> And then, knowing the type, piqi will place both of them inside their proper
> container. This could be a very nice syntax addition for config files and
> config options (kind of like sysctl).

I'd avoid more syntactic sugar. It would imply you always have to know
the container type in order to mentally parse the declarations (i.e.
"is this a variant or a record?"),
and again, I always prefer the syntax to explicitly denotate the data
type. The situation
can be become very bad in the case of nested data types.

Well, this is just an idea. It can be applied selectively, like a special mode for config files / config options or something like that. A slight syntax distinction may help to disambiguate it.

> (.foo .bar) expands exactly to .foo (.bar) and means exactly what you've
> written: constructor/label .foo is applied to the value .bar

In the docs (on "2.13. Associativity control") you don't mention that
(.foo .bar) === .foo (.bar).
You definitely should.

It is kind of documented. Could be improved for sure.

http://piqi.org/doc/piq/#builtinsyntaxabbreviation

http://piqi.org/doc/piq/#repeatednamedvalueconstructionabbreviation

 

> I should have explained it better. First, a little bit of a background. In
> Piq, values of record {?foo, *bar :: int()} and a list of of variants [foo |
> bar :: int()] have the same syntax representation, e.g.:
>
> [.foo .bar 1]
>
> This is done on purpose, because I believe these two data types represent
> essentially the same data. Barring some optional|required|repeated
> constraints.
>
> As you can see, both fields/elements .foo and .bar 1 have consistent syntax
> representation.
>
> In the alternative syntax, you'd have to write [ 'foo', bar : 1 ]. Now they
> look substantially different and this is what I mean by being not pretty.

Why couldn't I write [foo, bar: 1]

You could. If you also support quotation for foo.

 

or ['foo', 'bar': 1]

Unnecessary quotation for bar. Most human-readable notations try to avoid that. Besides ': creates visual clutter after bar.

 

or [`foo, `bar: 1]

This is better. Minimal and works well with the syntax sugar. The downside is that this is not conventional :)

By the way with sugar removed, you get [`foo `bar 1] which is close to the original notation of [.foo .bar 1]

I tested this notation on various objects with and without sugar and compared it to the Piq notations.

In most fonts, backtick is much thinner than dot. This makes it less prominent. This also means you need a separator in front of a backtick, e.g. [`foo, `bar 1] On the other hand, dot is an excellent separator on its own, there's no need for a comma or any other separator in front of a dot.

With backticks, colons don't look that bad. But with dot-prefixed labels, placing a colon after them doesn't make thing look better at all. You've got enough visual distinction already.

Overall, Piq is much better. I tested all kind of stuff, including '-', ':' and so on before settling on the dot. It is pretty hard to beat.

But you know what's cool? While playing with it I found a combination that looks really well with commas. Basically, remove the dot in front of the labels, add comma separators but don't use infix colons between labels and values -- this is the key. This notation has a downside that your labels/keywords look no different from unquoted strings / identifiers, but this is pretty much true for all programming languages. And it looks really good. This can be further improved by making commas at the end of the line optional. With this, notation requires even less symbols than Piq. Something to think about.

Anton