Generating mocks / sample data from piqi specs

[Ignas Vyšniauskas]

Hi,

I'm contemplating on how to leverage piqi and piqi defaults for
generating mock data from piqi-specs. In particular I am interested in
testing and mocking piqi-rpc modules.

Currently in *Erlang* there are the `*_default_impl.erl` modules and
`default_*()` methods for getting default data samples in the RPC
modules, both generated by piqic_erlang_rpc.erl

Unfortunately the current implementation of default generation is
somewhat of a hack, with primitive type defaults hard-coded in the
`piqic_erlang_rpc.erl` file and etc.

What I would love to have is default and fuzzy data generation in any
format from a piqi spec, e.g. being able to do:

piqi example --default --type foo/bar -t json

and also

piqi example --random --type piqi/int -t json


This could be extremely useful. Moreover, then it would become
language agnostic and e.g. the Erlang default generation would be simply
a call to piqi tools.

The example generation could also be partial, i.e. all specifications do
not need to be immediately supported.

Is there any development in this direction? Perhaps there are already
some things in place that allow you to do things like that in piqi that
I am not aware of? If yes, what is the best / most flexible way to
generate sample/default data (preferably without invoking Erlang)?

Regards,
Ignas

[Anton Lavrik]

On Tue, Jan 7, 2014 at 3:56 AM, Ignas Vyšniauskas <i.vysn...@gmail.com> wrote:

Hi,

I'm contemplating on how to leverage piqi and piqi defaults for
generating mock data from piqi-specs. In particular I am interested in
testing and mocking piqi-rpc modules.

Currently in *Erlang* there are the `*_default_impl.erl` modules and
`default_*()` methods for getting default data samples in the RPC
modules, both generated by piqic_erlang_rpc.erl

Unfortunately the current implementation of default generation is
somewhat of a hack, with primitive type defaults hard-coded in the
`piqic_erlang_rpc.erl` file and etc.

What I would love to have is default and fuzzy data generation in any
format from a piqi spec, e.g. being able to do:

        piqi example --default --type foo/bar -t json

and also

        piqi example --random --type piqi/int -t json


This could be extremely useful. Moreover, then it would become
language agnostic and e.g. the Erlang default generation would be simply
a call to piqi tools.

Agree -- that would be really cool! There could also be a mixed mode, when some values are set to known defaults and others are assigned randomly.

Is there any development in this direction? Perhaps there are already
some things in place that allow you to do things like that in piqi that
I am not aware of?

I've always wanted to play with this idea. Nothing on the implementation side yet.

 

If yes, what is the best / most flexible way to
generate sample/default data (preferably without invoking Erlang)?

In fact, if I wanted something quick and practical, I'd look at generating it in Erlang. There's code in piqic-erlang that can generate real defaults in Erlang syntax. In fact, it is used for generating field defaults for Erlang records. I would also look at Proper -- it has a randomized generation mode that works based on Erlang types. This way it should be possible to use it for generating randomized values based on _piqi.hrl types. If you need JSON output, converting generated Erlang values to JSON is trivial.

On the other hand, if you are up to some OCaml hacking, this would be a really cool piqi feature. There are a couple of QuickCheck implementations for OCaml to use as inspiration. Also, hacking on Piqi has never been easier. I've recently made some changes in master that greatly simplified the code layout and the build process. I'll be happy to help if you have questions about Piqi internals.

Anton

[Ignas Vyšniauskas]

On 01/08/2014 11:11 AM, Anton Lavrik wrote:
> Agree -- that would be really cool! There could also be a mixed mode,
> when some values are set to known defaults and others are assigned
> randomly.

I realised while discussing with Motiejus that what is needed is more
like a "canonical example" flag `--canonical` which would pick either
defaults or explicitly provided values (via some extension like
`.example`). Otherwise this mashes up the meanings of "standard example
of a type" and "default value of a field".

> In fact, if I wanted something quick and practical, I'd look at
> generating it in Erlang. There's code in piqic-erlang that can
> generate real defaults in Erlang syntax. In fact, it is used for
> generating field defaults for Erlang records. I would also look at
> Proper -- it has a randomized generation mode that works based on
> Erlang types. This way it should be possible to use it for
> generating randomized values based on _piqi.hrl types. If you need
> JSON output, converting generated Erlang values to JSON is trivial.

Indeed if the intention would only be internal testing of Erlang code,
then piqic_erlang and PropEr would already do most of the work.

However, my goal is mocking, testing and benchmarking abstract
interfaces, so this will possibly be done using other languages /
tools, not only Erlang. Moreover, it would definitely be way hackier to
implement it in Erlang. I also want to take advantage of this when
generating documentation, so Erlang makes little sense again.

> On the other hand, if you are up to some OCaml hacking, this would
> be a really cool piqi feature. There are a couple of QuickCheck
> implementations for OCaml to use as inspiration. Also, hacking on
> Piqi has never been easier. I've recently made some changes in
> master that greatly simplified the code layout and the build process.
> I'll be happy to help if you have questions about Piqi internals.

QuickCheck is way too much for this! Though it could be re-used for the
primitive type generation.

I won't try to hack this into Erlang and have a go at OCaml... at
some point when my backlog is smaller. I'll let you know whether/when I
get anything done :-)

Thanks again,
Ignas

[Anton Lavrik]

On Wed, Jan 8, 2014 at 7:30 AM, Ignas Vyšniauskas <i.vysn...@gmail.com> wrote:

On 01/08/2014 11:11 AM, Anton Lavrik wrote:
> Agree -- that would be really cool! There could also be a mixed mode,
> when some values are set to known defaults and others are assigned
> randomly.

I realised while discussing with Motiejus that what is needed is more
like a "canonical example" flag `--canonical` which would pick either
defaults or explicitly provided values (via some extension like
`.example`). Otherwise this mashes up the meanings of "standard example
of a type" and "default value of a field".

Agree. I would also make .example a repeated field so that one could specify several examples for a field or type.

To further avoid confusion, we may either fully ignore field defaults for the purpose of example generation or provide an option for treating field defaults as examples. I'm thinking that field defaults may not serve as good examples and since these values are going to be assigned automatically during parsing anyway, what's the point of having them as examples?

Anton