Erlang function type namespacing

[Ignas Vyšniauskas]

Hi,

Would it not make sense to *always* namespace Erlang function
input|output|error type names using the module name, even in the case of
a custom `.erlang-type-prefix "foo_`?

I see them as an internal generated type, which should not clash with
anything explicitly declared.

So e.g. if I have a function `user` and an explicitly defined type
`user_error()` it will clash with the type `user_error()` generated by
the `.error` field of a function.

I would propose to always namespace the function types by prefixing
with the module name (since the module is their "parent")[*].
Moreover, perhaps the function types should not be even added to the
_piqi.hrl files, only _piqi_impl.hrl.

If I wanted to expose the function input / output type, I could just
explicitly define a record for that, otherwise there is little need to
"access" it.

Alternatively / additionally, is there any way to work
around this? I would like to have unqualified type names (via
.erlang-type-prefix ""), however I want the generated function I/O type
names to be qualified by the module name.

(The reason why I need that is because if you try include piqi header
files of erlang module into another and both have unqualified
types with matching function names, then it causes the function type
names to clash even though none of the *declared* types actually clash.)

Thanks,
Ignas

[*]: I realise this doesn't resolve the issue as it is still possible to
craft clashes, but it's at least an improvement. A better solution would
be to just store the generated types in separate module altogether. The
annoying problem, as usual, is record definitions, which *need* to exist
in a header file.

[Anton Lavrik]

Hi Ignas,

Your question doesn't seem to specific to Erlang. While trying to figure out how to reproduce this clash, I realized that this should be invalid in Piqi and should have been caught by "piqi compile" in the first place. So, there was a bug that I have just fixed.

Here's an example of what's valid and what's not with respect to implicit types derived from function parameters.

.function [

    .name user

    .input int

    .error user-error

    .output user-output

]

% this is valid, because it matches the implicit user-input type

.alias [

    .name user-input

    .type int

]

% this would be invalid, because it clashes with the implicit user-input type

%.alias [

%    .name user-input

%    .type string

%]

% this is valid, because introducing an implicit

% .alias [ .name user-error .type user-error ] would be silly

.alias [

    .name user-error

    .type int

]

% this is naturally valid as well

.record [

    .name user-output

    .field [

        .name f

        .optional

    ]

]

Implicitly generated types for function parameters are very useful when it comes to integrating with piqi. That includes piqi compiler backends. For instance, piqic-erlang doesn't know anything about functions, yet it is able to generate correct type definitions for its function parameters and correct stubs for multi-format serializers. In piqic-erlang-rpc, it helps to simplify stubs generation as well. For example, knowing that the function accepts input, it can easily derive the name of the input type from the function name.

Not to mention, you can reference implicit types from anywhere else as if they were real types. Here's an made-up and rather simplistic example:

t.piqi:

.function [

    .name user

    .error int

    .output user-error

]

And this is possible, because the above is essentially equivalent to this:

$ piqi expand --functions t.piqi

.alias [

    .name user-output

    .type user-error

]

.alias [

    .name user-error

    .type int

]

.function [

    .name user

    .output user-error

    .error int

]

Another reason implicitly generated types share the same namespace with the explicitly defined ones is that it makes it really easy to take an implicitly defined type and turn it into an explicit one and the other way around without breaking backward compatibility.

Sometimes this may result in clashes like in your case. Not sure how to avoid them without sacrificing clarity. However, it is possible to minimize them by following some naming conventions such as using verbs for function names and nouns for type names.

Anton

--
You received this message because you are subscribed to the Google Groups "piqi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to piqi+uns...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Ignas Vyšniauskas]

On 02/02/2014 12:40 AM, Anton Lavrik wrote:
> Your question doesn't seem to specific to Erlang. While trying to
> figure out how to reproduce this clash, I realized that this should
> be invalid in Piqi and should have been caught by "piqi compile" in
> the first place. So, there was a bug that I have just fixed.

Ah, indeed, nice fix!

> Implicitly generated types for function parameters are very useful
> when it comes to integrating with piqi. That includes piqi compiler
> backends. For instance, piqic-erlang doesn't know anything about
> functions, yet it is able to generate correct type definitions for
> its function parameters and correct stubs for multi-format
> serializers. In piqic-erlang-rpc, it helps to simplify stubs
> generation as well. For example, knowing that the function accepts
> input, it can easily derive the name of the input type from the
> function name.

Sure, I am not saying they should not exist and completely understand
the need for a uniform treatment of all types. However, the possible
ways to fix this I proposed would not at all break this -- you would
simply only reference generated types in the generated code (such as in
piqi-erlang-rpc ), and separately expose the explicitly defined types to
the user, which would themselves reference the implicit generated types.

> Sometimes this may result in clashes like in your case. Not sure how
> to avoid them without sacrificing clarity. However, it is possible
> to minimize them by following some naming conventions such as using
> verbs for function names and nouns for type names.

I see. Well, in my case of one unqualified module importing types from
another unqualified module, no verb/noun convention can work, because
the clash is due to function I/O typenames between the two modules (e.g.
if both module1 and module2 define an `insert` function), not just due
to function and data typename clashes.

Anyway, thanks for the answer, I guess what you're saying makes sense,
it's probably easier to just treat everything simply and equally.

To work around this I now do a horrible hack, where I explicitly
introduce module-qualified type aliases for function input/output/errors
and then perform a nasty regex to remove the unqualified versions from
the generated Erlang code. Another possible option is to split up the
modules into those defining only data types and others also defining
functions.

--
Ignas

[Anton Lavrik]

Somehow I missed this part form your original message...

If I understand correctly, you want most type names to stay unqualified for convenience and brevity and use qualified names in cases in which they (may) cause clashes between types of different modules.

Here's an example of how you can do it manually for one function parameter at a time (i guess now I have to fix piqic-erlang-rpc to handle this properly...):

t.piqi:

.erlang-type-prefix ""

.function [

    .name user

    % this will generate qualified -type t_user_output () :: foo()

    .output.alias [

        .erlang-name "t_user_output"

        .type foo

    ]

    % this will generated qualified -type t_user_error() :: #t_user_error{}.

    .error [

        .erlang-name "t_user_error"

        .field [

            .name f

            .optional

        ]

    ]

]

.record [

    .name foo

    .field [

        .name f

        .optional

    ]

]

(.custom-field erlang-name erlang-type-prefix)

For more sophisticated use cases, you could write something like piqc-erlang-rpc that generates things the way you need them. For dealing with records and their redefinitions you can use something like https://github.com/alavrik/erl_aliases

Anton

[Ignas Vyšniauskas]

Dear Anton,

> Somehow I missed this part form your original message...
>
> If I understand correctly, you want most type names to stay
> unqualified for convenience and brevity and use qualified names in
> cases in which they (may) cause clashes between types of different
> modules.
>
> Here's an example of how you can do it manually for one function
> parameter at a time

Yes, this is *exactly* what I wanted, thank you so much! And I don't
care about the fact that it needs to be done for each
method individually because I'm generating these definitions from
something else anyway.

> (i guess now I have to fix piqic-erlang-rpc to handle this
> properly...)

What is improper about the handling now? The generated piqic-erlang
artefacts seem to be correct to me.

Thanks again. I keep being impressed with the flexibility piqi provides
and discovering neat features I have missed.

--
Ignas

[Ignas Vyšniauskas]

On 02/02/2014 11:52 PM, Ignas Vyšniauskas wrote:
>> (i guess now I have to fix piqic-erlang-rpc to handle this
>> properly...)
>
> What is improper about the handling now? The generated piqic-erlang
> artefacts seem to be correct to me.

I see what you mean now, piqic-erlang-rpc assumes that the method
input/output types in Erlang are always of the form
"<ErlPrefix>_<FuncName>_<IO-Type>" and there's no information in the
`#func` record about the type names of the I/O records at all.

I would be happy to provide a patch if you could point me in the
preferred direction of fixing it.

I assume one would just need to extend the #func record created in
`piqic.erl` with information about the IO type names?

--
Ignas

[Ignas Vyšniauskas]

On 02/03/2014 01:15 AM, Ignas Vyšniauskas wrote:
> I assume one would just need to extend the #func record created in
> `piqic.erl` with information about the IO type names?

Ah, I missed the fact that #func actually contains the piqi type names
of input/output/error.

So alternatively I can lookup the record typedef form the context and
get the erlang typename from there.

Would this be the preferred way? Or would you rather populate the #func
record with that data explicitly?

--
Ignas

[Anton Lavrik]

On Sun, Feb 2, 2014 at 4:45 PM, Anton Lavrik <ala...@piqi.org> wrote:

Fixing this right now. The fix is to obtain Erlang type name by calling piqic:resolve_type_name(<function-name>-input|output|error) and taking typedef_erlname of the result.

This is fixed now.

Anton

[Anton Lavrik]

Accidentally took this discussion thread off list. Returning it back.

On Mon, Feb 3, 2014 at 8:57 AM, Ignas Vyšniauskas <bali...@gmail.com> wrote:

On 02/03/2014 01:45 AM, Anton Lavrik wrote:
> Note that there's a small usability issue with using .erlang-name.
> For instance, piqic-erlang generates serialization functions from the
> above t.piqi as t_piqi:parse_t_user_output(..) and
> t_piqi:get_t_user_output(...)
>
> To prefix only the type names and not the function names, we'd have
> to allow .erlang-type-prefix property at the type definition level.
> This is fairly straightforward to implement in case you need it. I'll
> be happy to merge it.

Hmm, I'm now bumping into this and indeed the naming becomes is a bit
awkward.

I'm not sure if also adding `.erlang-type-prefix` to all typedefs is
such a great idea, since it will make it even more confusing on how to
figure out the exact Erlang typenames.

Agree. It is not all clear whether a typedef-level .erlang-type-prefix should override a module-level setting or stick another prefix.

 

I was thinking of instead separately generating the `parse_*` and
`gen_*` functions of input/output/errors in the *_piqi_rpc.erl modules
-- those are needed anyway for the
input decoding functions and would provide a nice abstraction, so that
essentially the _piqi_rpc.erl module is sufficient for all your
input/output encoding / decoding (if, for example, you are using some
other transport) and you don't care about about the intput / output
types are defined.

As an example, given foobar.piqi with:

.function [
    .name destroy
    .input.alias [
        .erlang-name foobar
        .type great-input
    ]
]

In foobar_piqi_rpc.erl you would generate

    parse_destroy_input(X) ->
        foobar_piqi:parse_foobar(X).

and similarly for `gen_*` and output, error things. This is already
similar to what it is done in the *_default_impl.erl modules.

The point is that the function names and the input|output|error fields
are always fixed, so the user can trust these not to change and should
not care about how exactly the types are defined.

Would you accept such a change? I think I'm going to implement it in any
case as it is greatly needed in our case, but it would be nice to be
able to contribute it back to you and stay in sync.

Sure. I think this is a great idea.

Anton

 

[Ignas Vyšniauskas]

On 02/04/2014 07:05 AM, Anton Lavrik wrote:
> Sure. I think this is a great idea.

Awesome, here you go then:
https://github.com/alavrik/piqi-rpc/pull/3

Please see the NOTE and share your opinion on this either here or in the PR.

--
Ignas