reusing defined types

[Motiejus Jakštys]

Hi,

I have these two fields:

.variant [
.name filter-parm
.option [ .type float ]
.option [ .type int ]
.option [ .type string ]
.option [ .type order ]
]

.variant [
.name primitive-value
.option [ .type float ]
.option [ .type int ]
.option [ .type string ]
]


In practice they are related, filter-parm is just extension of
primitive-value. Any way to express one with another?

I tried .include, but seems like it works only as a top-level
declaration only:

Warning: types.piqi:113:5: unknown field: include

Thank you!
Motiejus

[Anton Lavrik]

Hi Motiejus,

Piqi doesn't support structured inheritance per se. However, there's a
mechanism called "Extensions" that may be useful case similar to
yours. You can read more about it here:
http://piqi.org/doc/piqi/#extensions

For example, using extensions, your example can be rewritten as:

.variant [
.name filter-parm
.option [ .type order ]
]

.variant [
.name primitive-value

% this is ugly and may not be acceptable, but there's no work around;
% obviously, records don't have such limitation, because they may
% legitimately have no fields
.option [ .name variant-definition-must-have-at-least-one-option ]
]

.extend [
(.typedef filter-parm primitive-value)

% options shared between "filter-param" and "primitive-value" variants
.with.option [ .type float ]
.with.option [ .type int ]
.with.option [ .type string ]
]

As pointed out in the comment, this mechanism works better for records
rather than for variants.

Anton


I deliberately chose no to deal with inheritance in Piqi, however

[Motiejus Jakštys]

On Wed, Jan 23, 2013 at 6:54 AM, Anton Lavrik <ala...@piqi.org> wrote:
>
> yours. You can read more about it here:
> http://piqi.org/doc/piqi/#extensions

Thanks. I looked at them, but didn't realize that I can extend
multiple models at once. This completely covers my use case.

> I deliberately chose no to deal with inheritance in Piqi, however

Out of curiosity, why? (I don't suggest it's good, rather just want to
know why).

--
Motiejus Jakštys

[Anton Lavrik]

On Wed, Jan 23, 2013 at 2:18 AM, Motiejus Jakštys <desir...@gmail.com> wrote:
> On Wed, Jan 23, 2013 at 6:54 AM, Anton Lavrik <ala...@piqi.org> wrote:
>>
>> yours. You can read more about it here:
>> http://piqi.org/doc/piqi/#extensions
>
> Thanks. I looked at them, but didn't realize that I can extend
> multiple models at once. This completely covers my use case.

Great!

I don't know what I was thinking yesterday... Your original question
has nothing to do with structured inheritance of any sort. Moreover,
there is a native support for your use case without even using
extensions. It is as simple as this:

piqic-erlang already turns

.variant [
.name primitive-value
.option [ .type float ]
.option [ .type int ]
.option [ .type string ]
]

.variant [
.name filter-parm
.option [ .type primitive-value ]
.option [ .type order ]
]

into

-type(t_primitive_value() ::
{float, number()}
| {int, integer()}
| {string, string() | binary()}
).
-type(t_filter_parm() ::
t_primitive_value()
| {order, t_order()}
).

>> I deliberately chose no to deal with inheritance in Piqi, however
> Out of curiosity, why? (I don't suggest it's good, rather just want to
> know why).

There are several main reasons.

First of all, modeling data using structured inheritance is very hard.
It is hard to get hierarchies right -- this topic comes in various
flavors: 1) whether something should inherit another thing or include
it; 2) often times people would use inheritance in order to take a
shortcut and use inheritance even though the logical relationship
between the two objects is unclear and/or can't be established; 3) new
facts (e.g. new data structures) can change your perspective on
previous inheritance decisions; 4) overall, there's no single unified
view on inheritance among people who use it -- this makes everything
highly subjective and this is a really bad thing for protocol design.

Second of all, all those difficulties and ambiguities cause people to
constantly reconsider and refactor their specifications. Needless to
say, this is a bad idea for protocols that need to be stable, portable
and backward compatible with previous versions.

These problems don't have clear solutions other than banning
inheritance use altogether and modeling data in a very direct and
obvious way.

Anton

[Motiejus Jakštys]

I didn't know you can kind-of nest variants that way. This is fantastic.

>>> I deliberately chose no to deal with inheritance in Piqi, however
>> Out of curiosity, why? (I don't suggest it's good, rather just want to
>> know why).
>
> There are several main reasons.
>
> First of all, modeling data using structured inheritance is very hard.
> It is hard to get hierarchies right -- this topic comes in various
> flavors: 1) whether something should inherit another thing or include
> it; 2) often times people would use inheritance in order to take a
> shortcut and use inheritance even though the logical relationship
> between the two objects is unclear and/or can't be established; 3) new
> facts (e.g. new data structures) can change your perspective on
> previous inheritance decisions; 4) overall, there's no single unified
> view on inheritance among people who use it -- this makes everything
> highly subjective and this is a really bad thing for protocol design.
>
> Second of all, all those difficulties and ambiguities cause people to
> constantly reconsider and refactor their specifications. Needless to
> say, this is a bad idea for protocols that need to be stable, portable
> and backward compatible with previous versions.
>
> These problems don't have clear solutions other than banning
> inheritance use altogether and modeling data in a very direct and
> obvious way.

Once you learn some hidden corners, piqi way of data modeling becomes
indeed simple. I suggest adding the above example somewhere to
documentation of variants.

Thanks for your reply.

Regards,
Motiejus Jakštys

[Motiejus Jakštys]

On Thu, Jan 24, 2013 at 09:12:06AM -0600, Anton Lavrik wrote:
> On Wed, Jan 23, 2013 at 2:18 AM, Motiejus Jakštys <desir...@gmail.com> wrote:
> > On Wed, Jan 23, 2013 at 6:54 AM, Anton Lavrik <ala...@piqi.org> wrote:
> >>
> >> yours. You can read more about it here:
> >> http://piqi.org/doc/piqi/#extensions
> >
> > Thanks. I looked at them, but didn't realize that I can extend
> > multiple models at once. This completely covers my use case.
>
> Great!
>
> I don't know what I was thinking yesterday... Your original question
> has nothing to do with structured inheritance of any sort. Moreover,
> there is a native support for your use case without even using
> extensions. It is as simple as this:
>
> piqic-erlang already turns
>
> .variant [
> .name primitive-value
> .option [ .type float ]
> .option [ .type int ]
> .option [ .type string ]
> ]
> .variant [
> .name filter-parm
> .option [ .type primitive-value ]
> .option [ .type order ]
> ]

Hi again,

That worked wonderfully. However, is there a way to do the same for
enums?

.enum [
.name error-common
.option [ .name unknown .code 1 ]
.option [ .name backend .code 2 ]
...
]

.enum [
.name getApplicationToken-error-enum
.option [ .name sql .code 99 ]
.option [ .type error-common ]
]

Now it complains:

activity-errors.piqi:21:33: enum options must specify name

Piqi 0.5.7

Thanks,
Motiejus

[Anton Lavrik]

This type of variant "flattening" is not supported for enums.

When you use this trick with variants, they are still represented as
nested records in all three external formats. They are only
"flattened" at Erlang (and OCaml) language level for convenience.
However, enums values can not be nested like variants in external
formats, which means they would need to be "unrolled" or "flattened"
at the Piqi language level. This, in turn, would complicate the
language semantics in many ways. For instance, this will bring up a
lot of questions such as how to enforce enum code uniqueness and how
to do that across modules and in the presence of backward and
forward-compatible protocol changes.

With that said, there is a Piqi language feature that provides
syntax-level extensions for type definitions, field, options and
functions. It can be viewed as a very primitive macros mechanism. At
the same time, it is much more explicit than "inheritance". If you
want to reuse similar enum options in several definitions you could do
this:

$ cat t.piqi

.enum [
.name getApplicationToken-error-enum
.option [ .name sql .code 99 ]

]

.enum [
.name some-other-enum-type
.option [ .name o .code 100]
]

.extend [
(.typedef getApplicationToken-error-enum some-other-enum-type)

% some common options
.with.option [ .name unknown .code 1 ]
.with.option [ .name backend .code 2 ]
]

the above is equivalent to

$ piqi expand t.piqi

.enum [
.name getApplicationToken-error-enum
.option [
.name sql
.code 99
]
.option [

.name unknown
.code 1
]
.option [
.name backend
.code 2
]

]

.enum [
.name some-other-enum-type
.option [
.name o
.code 100
]

.option [
.name unknown
.code 1
]
.option [
.name backend
.code 2
]

]


Anton