Quarkus and Contract First Development

[clement escoffier]

Hello,

A couple of Quarkus contributors discussed offline contract-first API development and how it can be achieved in Quarkus.

In this mail, I will try to capture the result of these discussions; it's not an easy task, they got pretty chatty about it. But let's try. Bear with me; I'm not an expert on the topic; I'm just the messenger. 

To simplify and scope the conversation, we will only focus on REST. The principles can be applied to the other type of interactions (event-driven, GraphQL, and so on)

What's contract-first API development?

The idea behind contract-first API development is to have a set of API descriptors that drive your development. They are generally OpenAPI descriptors in the REST world, but that's not the only format (we will talk about that later). 

Immediately, you can imagine that contract first API development for Quarkus would be to generate a project skeleton based on an OpenAPI descriptor. The generation produces the project structure, the resource skeleton, and the data structures that are part of the API contract (body, entity...).

That is nice, but, well, Quarkus deserves better ;-)

Client-Side generation

What can be done for the server can be done for the client. In most distributed systems, you need to call remote endpoint. Here, the idea would be to generate the (MicroProfile) Rest Client interface from an OpenAPI descriptor. So, you don't have to write the interface yourself. As for the server-side, it would generate the Rest Client interface (but this time, no need to implement anything) and the data structures. It can even introduce the proper configuration attribute in your "application.properties" to configure the URL. 

How can this be done?

Generating code from the OpenAPI descriptor is not something new. In addition, the Apicurio project (https://www.apicur.io/) started, as part of the Apicurio Studio, to support the server-side generation. Granted, it may be a bit outdated, but the base is there (Eric, if you are around, maybe you can give us an update). 

Trigging the generation can be done either from the dev console or via the "qs" cli. 

Ok, but can't we go further?

Mocking Services

From here, we are more talking about ideas than concrete implementations (it was the case before, but it was easy to see how it could be implemented). 

When implementing an application requiring a service, you end up having to call that service. But what if it's not yet implemented or available? What could be done is to generate a mock version from the OpenAPI descriptor. Maybe, some fixtures can be required, but if the OpenAPI descriptor is complete enough (with examples and so on), we have everything to run a mock version of the service. So, if you use dev services, it would just start that fake version and configure your application to call it. So, you are not stuck and can start implementing your business logic. Of course, as soon as the service is ready, you would not use the mock version. 

Conformance Testing

Back to the server-side. Even though you generated the skeleton, things can quickly drift. Again, if the OpenAPI descriptor is complete enough, Quarkus could run conformance tests against your API. 

The purpose of the Pact (https://docs.pact.io/), among other tools, is precisely this. However, Pact does not use OpenAPI as input, which requires a second type of descriptor. Not the end of the world, but the idea would be to avoid that. The Pact documentation explains very well the different purpose (https://docs.pact.io/faq/convinceme#but-i-use-swaggeropenapi) 

Thoughts? Ideas? 

Clement

[Mattia Mascia]

Hi Clement,

Nice discussion.

Regarding the code generation I would extend  the "official" community one : https://openapi-generator.tech/docs/generators

It's pretty nice and works well.

Best regards 

Mattia 

--
You received this message because you are subscribed to the Google Groups "Quarkus Development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to quarkus-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CAKW6ficvQpe15McuaYC2j_3ROMRtfjuZa-dtPz3i4GArkCHs%2BA%40mail.gmail.com.

--

MATTIA MASCIA

He / Him / His

PRINCIPAL CONSULTANT

 Switzerland  

mma...@redhat.com    M: +41 79 41 14 377 

[William Burke]

I don't like the idea of promoting this style of development.  This is CORBA/WS-* all over again.  The contract is already defined by the HTTP specification.  Standard HTTP methods, response codes, and headers are all well defined and their semantics should be honored or gracefully ignored..  All the app-specific complexity should be in the representation. 

Code generation tools encourage bloating of libraries and thus an explosion in RSS consumed.  Let's learn from the lessons of the past guys.

To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CAGzaM-41_aGoMQZrQ8orRwV%2B_bnRwEoM_VzkUPc6op7mnehphA%40mail.gmail.com.

--

Bill Burke

Red Hat

[Paul Carter-Brown]

I tend to agree with Bill when it comes to the resources and methods, but when it comes to the JSON models I hate crafting them by hand.

What I wouldn't mind is a standard way of generating the rest model classes with all the Json properties, bean validation annotations etc from a single file. It's one thing I've carried over from the SOAP days is to use XSD to define the model and jaxb to generate the model classes. But for some reason XSD is out of favour and nothing has evolved to take its place (that I know of)?

We have a hack with a custom jaxb processor that turns something like this:

    <!-- @Schema(description="A generic way for identifying a card. Any one of these fields can uniquely identify a card.
    Only one should be provided as conflicting identifiers may cause issues") -->
    <xs:complexType name="CardIdentifier">
        <xs:sequence>
            <!-- @Positive -->
            <!-- @Schema(description="Unique system generated identifier for a card")-->
            <xs:element name="cardId" type="xs:long" minOccurs="0"/>
            <!-- @Size(min=5, max=20) -->
            <!-- @Schema(description="Pack Id of a card")-->
            <xs:element name="packId" type="xs:string" minOccurs="0"/>
            <!-- @Size(min=16, max=16) -->
            <!-- @Schema(description="Primary account number of the card")-->
            <xs:element name="pan" type="xs:string" minOccurs="0"/>
            <!-- @Size(min=10, max=20) -->
            <!-- @Schema(description="QRCode on the card which uniquely identifies it")-->
            <xs:element name="qrCode" type="xs:string" minOccurs="0"/>
        </xs:sequence>
    </xs:complexType>

Into this:

@Schema(description = "A generic way for identifying a card. Any one of these fields can uniquely identify a card. Only one should be provided as conflicting identifiers may cause issues")
public class CardIdentifier implements Serializable
{
    private final static long serialVersionUID = 1L;
    @Positive(message = "Value '${validatedValue}' for CardIdentifier's cardId is invalid: It must be positive")
    @Schema(description = "Unique system generated identifier for a card")
    protected Long cardId;
    @Size(message = "Value '${validatedValue}' for CardIdentifier's packId is invalid: It must be a minimum of {min} and maximum of {max}", min = 5, max = 20)
    @Schema(description = "Pack Id of a card")
    protected String packId;
    @Size(message = "Value '${validatedValue}' for CardIdentifier's pan is invalid: It must be a minimum of {min} and maximum of {max}", min = 16, max = 16)
    @Schema(description = "Primary account number of the card")
    protected String pan;
    @Size(message = "Value '${validatedValue}' for CardIdentifier's qrCode is invalid: It must be a minimum of {min} and maximum of {max}", min = 10, max = 20)
    @Schema(description = "QRCode on the card which uniquely identifies it")
    protected String qrCode;

    public Long getCardId() {
        return cardId;
    }

...

...

Its pretty opinionated though but works for us. One file defining the outward facing model which is processed to java at build time (a single 1000 lines XSD generates about 8000 lines of java across maybe 100 classes) As per jaxb it supports complex types, enums, lists etc etc. I could picture something like this being done with a Quarkus extension in a nice reusable way where you can define your model in a file and Quarkus does the rest.

To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CAL%3DE%3DjQtevm05EkMUvzfNvj%3DyFiwX4B%2BZpjmU8GhhzeM1ww%3DzA%40mail.gmail.com.

[Eric Wittmann]

Right - Apicurio Studio has a feature where a Quarkus (JAX-RS) project can be generated from an OpenAPI document.  As mentioned, it generates an old project format right now, but that part is easily updated.  The idea we talked about recently was to move the generation out of tools like Studio and instead support a Quarkus extension that generates the JAX-RS layer on the fly from the OpenAPI file (integrated into Quarkus dev mode).  We (apicurio team) will be putting together a little POC based on the generator used in Studio today, as a way to start playing around with this idea and fostering discussion/collaboration.

Note that our generator only creates JAX-RS interfaces and POJOs.  The idea is that any time the contract changes, we can just regenerate those assets, overwriting what was there before.  No implementation classes are created, so we don't risk accidentally deleting code.  It's not perfect, as there are plenty of edge cases and idiosyncrasies...and our impl is pretty naive and opinionated, but it should hopefully be a start to see how well this might work.

We've been using the generator based workflow internally and it works pretty well.  I find it much easier to make changes to my REST API by updating the contract in Studio first, then using the codegen to update the JAX-RS interfaces.  After that it's a matter of implementing any missing/changed methods and classes.  The end result is much the same as if I did it code-first, but I get a slightly better outcome (IMO) because my OpenAPI file is very good (e.g. has nice descriptions and examples) without cluttering up my Java code with endless annotations.  From my perspective, integrating the workflow better into Quarkus would be sweet.

[Antonio Goncalves]

Hi all,

I just joined a project where I had to implement a resource exposing a few SCIM endpoints[1] (not all of them). We had the official OpenAPI SCIM contract, therefore, we generated pure JAX-RS code with SwaggerCodegen[2]. To be honest, it saved us a lot of time because the SCIM model is complex (40/50 classes). So having a generator that would generate JAX-RS code and mocks working out of the box with Quarkus would have been nice

Antonio

[1] https://tools.ietf.org/html/rfc7644

[2] https://swagger.io/tools/swagger-codegen/

To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CAL%3DE%3DjQtevm05EkMUvzfNvj%3DyFiwX4B%2BZpjmU8GhhzeM1ww%3DzA%40mail.gmail.com.

--

Antonio Goncalves 
Contractor, Java Champion and Pluralsight author

Blog | Twitter | LinkedIn | Author | Pluralsight | Devoxx France | Voxxed Microservices

[Georgios Andrianakis]

On Thu, Apr 29, 2021, 18:50 clement escoffier <clement....@gmail.com> wrote:

Hello,

A couple of Quarkus contributors discussed offline contract-first API development and how it can be achieved in Quarkus.

In this mail, I will try to capture the result of these discussions; it's not an easy task, they got pretty chatty about it. But let's try. Bear with me; I'm not an expert on the topic; I'm just the messenger. 

To simplify and scope the conversation, we will only focus on REST. The principles can be applied to the other type of interactions (event-driven, GraphQL, and so on)

What's contract-first API development?

The idea behind contract-first API development is to have a set of API descriptors that drive your development. They are generally OpenAPI descriptors in the REST world, but that's not the only format (we will talk about that later). 

Immediately, you can imagine that contract first API development for Quarkus would be to generate a project skeleton based on an OpenAPI descriptor. The generation produces the project structure, the resource skeleton, and the data structures that are part of the API contract (body, entity...).

That is nice, but, well, Quarkus deserves better ;-)

Client-Side generation

What can be done for the server can be done for the client. In most distributed systems, you need to call remote endpoint. Here, the idea would be to generate the (MicroProfile) Rest Client interface from an OpenAPI descriptor. So, you don't have to write the interface yourself. As for the server-side, it would generate the Rest Client interface (but this time, no need to implement anything) and the data structures. It can even introduce the proper configuration attribute in your "application.properties" to configure the URL. 

How can this be done?

Generating code from the OpenAPI descriptor is not something new. In addition, the Apicurio project (https://www.apicur.io/) started, as part of the Apicurio Studio, to support the server-side generation. Granted, it may be a bit outdated, but the base is there (Eric, if you are around, maybe you can give us an update). 

Trigging the generation can be done either from the dev console or via the "qs" cli. 

Ok, but can't we go further?

Mocking Services

From here, we are more talking about ideas than concrete implementations (it was the case before, but it was easy to see how it could be implemented). 

When implementing an application requiring a service, you end up having to call that service. But what if it's not yet implemented or available? What could be done is to generate a mock version from the OpenAPI descriptor. Maybe, some fixtures can be required, but if the OpenAPI descriptor is complete enough (with examples and so on), we have everything to run a mock version of the service. So, if you use dev services, it would just start that fake version and configure your application to call it. So, you are not stuck and can start implementing your business logic. Of course, as soon as the service is ready, you would not use the mock version. 

This is the part I am most excited about. I think that if we can nail the experience on this part, it will go a long way

Conformance Testing

Back to the server-side. Even though you generated the skeleton, things can quickly drift. Again, if the OpenAPI descriptor is complete enough, Quarkus could run conformance tests against your API. 

The purpose of the Pact (https://docs.pact.io/), among other tools, is precisely this. However, Pact does not use OpenAPI as input, which requires a second type of descriptor. Not the end of the world, but the idea would be to avoid that. The Pact documentation explains very well the different purpose (https://docs.pact.io/faq/convinceme#but-i-use-swaggeropenapi) 

Thoughts? Ideas? 

Clement

[Kevin Viet]

Mocking Services

From here, we are more talking about ideas than concrete implementations (it was the case before, but it was easy to see how it could be implemented). 

When implementing an application requiring a service, you end up having to call that service. But what if it's not yet implemented or available? What could be done is to generate a mock version from the OpenAPI descriptor. Maybe, some fixtures can be required, but if the OpenAPI descriptor is complete enough (with examples and so on), we have everything to run a mock version of the service. So, if you use dev services, it would just start that fake version and configure your application to call it. So, you are not stuck and can start implementing your business logic. Of course, as soon as the service is ready, you would not use the mock version. 

If I may, I agree with Georgios, having a full blown solution integrated with Quarkus would be neat:

By experience, in my company and even if no solution is perfect, there's no common ground technology for mocking/ghosting remote services.

To me the following list of function would be expected : 

Basic query/reply 

Matching replies from queries emitted by the service

Creation of matching expectation

The possibility to create expectation of incoming queries in the mocking system

Record/replay

Possibility to record traffic to ease the creation of mock to be replayed later.

Performance/load testing

Last but not least : Internally we created a ghosting solution based on Vertx because we needed a mocking solution that literally scales during the load testing of our applications.  

Regards

To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CALeTM-nX1hKddGpwktD6Ly87iS3woEtWpQUEKpMdyXSN9-7M0w%40mail.gmail.com.

[Ken Yee]

Recently finished a Spring Boot microservice that was autogen'd off an OpenAPI/Swagger spec and I had similar misgivings before but it seems to work well enough and needs less boilerplate.  There's a maven plugin that autogens/updates the skeleton if you change/update the OpenAPI spec that is also in the project (I wrote this spec using Swaggerhub).

Main negative is the generator is for Java and I used Kotlin for the core business/delegates logic section of it so the skeleton generate delegation methods that are incorrectly spelled as nullable instead of following the OpenAPI doc.  One nice thing that it handled was an endpoint that had multipart files and a JSON object.  Doing that by hand is really hairy in Spring :-(

If Quarkus can handle this as well as it does what it can do now, it should be a nice seamless experience.  The hard part will be trying to generate a skeleton that can handle JWT, auth, multipart, validation, etc. :-)

To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CAL%3DE%3DjQtevm05EkMUvzfNvj%3DyFiwX4B%2BZpjmU8GhhzeM1ww%3DzA%40mail.gmail.com.

[Wojciech Trocki]

The beauty of contract-first development is that once Quarkus provides Contract like in quickstart: https://github.com/quarkusio/quarkus-quickstarts/tree/main/openapi-swaggerui-quickstart
any community library can be used to base on that contract for generating frontend source code.

In AeroGear we have done client-side, mock and test generation for OpenAPI and GraphQL. 

For OpenAPI specifically, I would recommend - OpenAPI generator community.
This does OpenAPI based generation for many clients and server languages: https://openapi-generator.tech/docs/generators/

Offlicial generators are basic but there are dozens of libraries that do more framework-specific generations for React and others.

For example:
https://api-platform.com/docs/client-generator/react/

There are also more high-level assembly tools like https://www.jhipster.tech that help to drive things by API/Model and even produce basic UI.

[Stephane Epardaud]

We actually have an issue open for generating MP REST interfaces from an OpenAPI definition (written in another language, not Java): https://github.com/quarkusio/quarkus/issues/16401 so big +1 for supporting this one one or another (upstream, in Quarkus, whatever, as long as it's documented and trivial to use from our tools).

This actually differs from traditionnal JAX-RS endpoint generated, because we want to be able to support RESTEasy Reactive extensions as well as Mutiny variants for async calls.

To Bill's point about generated clients being worse than pure HTTP calls, let's illustrate two calls:

  // MP-REST

  List<Order> orders = mpRestClient.getOrders(customerId, OrderType.Open);

  // PURE JAX-RS CLIENT

  WebTarget target = ClientBuilder.newClient().target("http://example.com");
  target.queryParam("type", OrderType.Open).path("/orders/"+customerId);
  List<Order> orders = target.request().accept(MediaType.APPLICATION_JSON).get(new GenericType<List<Order>>(){});

Now, granted, some HTTP APIs are better than others, and the JAX-RS one is particularly nasty and verbose and confusing and error-prone, and HELL YEAH we should do a better one for people who don't like proxies (like me, BTW), because ideally I want this to work:

  // RESTEASY-REACTIVE FICTIONAL HTTP CLIENT

  List<Order> orders = myService.query("type", OrderType.Open).path("/orders/"+customerId).get();

But even then, my point is that even though we're targeting "pure" HTTP and no WDSL-like generation and impure non-REST paperwork: we're still hard-coding the call handshake and underlying information exchange protocol in our code. So we're creating a tight coupling with the protocol. So Bill, I share your distrust of interfaces for doing clients, but the fact is that when done well, it's just not worse than writing "pure" HTTP calls in my experience. And we're not even providing a nice HTTP client for those who want to go "pure". And even then, I feel only HATEOAS has a real edge over client interface generation, because there we're reducing the coupling further.

To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CANhb1xYJxtDx8r6gLgyYDLYEwMFYwJPCUryGJWGmMKdxcYuj1w%40mail.gmail.com.

--

Stéphane Épardaud

[Stephane Epardaud]

As for mocking services, that's a real issue with micro-services, and a story we must support.

I have the impression that mocking an MP-REST client interface should be trivial and already supported, no?

For mocking the HTTP layers, I did ask if Vert.x had an equivalent to okhttp's mocking like we use in the Kubernetes extension: https://quarkus.io/guides/kubernetes-client#testing where there are two modes of mocking the K8s server:

- Adding mock HTTP calls by specification, such as: httpMockRecorder.whenGet("/orders/*").answerWith(() -> Arrays.of(new Order().setId(12)))

- Using CRUD to set up mock data, which is then recorded by a special CRUD/mock endpoint and will lead to the correct answers being returned: crudRecorder.post("/orders/23").entity(Arrays.of(new Order().setId(12)));

The CRUD kind I feel is probably too specific to satisfy every kind of remote service. The HTTP mocking is pretty nice. Sure this could also be done using fixtures for more complex operations, but an API is a nice start. We then "just" have to make sure that our HTTP clients will delegate to the mocks if they exist and match.

--

Stéphane Épardaud

[Stephane Epardaud]

On Fri, 30 Apr 2021 at 00:47, Kevin Viet <kevin...@gmail.com> wrote:

Basic query/reply 

Matching replies from queries emitted by the service

Yup, this can be done with with MP-REST interface mocking with Mockito, or for HTTP with something like I mentioned the K8s client does via okhttp mocking

 

Creation of matching expectation

The possibility to create expectation of incoming queries in the mocking system

Again, can be done in both cases. Mockito supports those verifications, and our version of okhttp mocking shoud support that too for the HTTP mocking.

 

Record/replay

Possibility to record traffic to ease the creation of mock to be replayed later.

This is very interesting, and something I thought about too, but you have to have real data to record first, so where would that happen? The user using curl? Using the SwaggerUI? Recording production exchanges? That's what I haven't been able to figure out. But yeah, something that looks at exchanges "somewhere" and spits out HTTP fixtures would be awesome.

 

Performance/load testing

Last but not least : Internally we created a ghosting solution based on Vertx because we needed a mocking solution that literally scales during the load testing of our applications.  

 You want to test the load, or you want the mocking to scale under load?

[Max Rydahl Andersen]

> I don't like the idea of promoting this style of development. This is
> CORBA/WS-* all over again. The contract is already defined by the
> HTTP
> specification. Standard HTTP methods, response codes, and headers are
> all
> well defined and their semantics should be honored or gracefully
> ignored..
> All the app-specific complexity should be in the representation.
>
> Code generation tools encourage bloating of libraries and thus an
> explosion
> in RSS consumed. Let's learn from the lessons of the past guys.

I tend to agree with you here Bill - thus my main interest around
"automation" is
the server side generation where you would need to implement the full
api thus
it isn't a choice.

For client interactions I would prefer if there is a way the generation
could be selective instead of "everything" in the client.

but then again - I tend to prefer using jackson and have it ignore
properties by default
so I just get what I need anyway.

/max

>>> *What's contract-first API development?*

>>>
>>> The idea behind contract-first API development is to have a set of
>>> API
>>> descriptors that drive your development. They are generally OpenAPI
>>> descriptors in the REST world, but that's not the only format (we
>>> will talk
>>> about that later).
>>>
>>> Immediately, you can imagine that contract first API development for
>>> Quarkus would be to generate a project skeleton based on an OpenAPI
>>> descriptor. The generation produces the project structure, the
>>> resource
>>> skeleton, and the data structures that are part of the API contract
>>> (body,
>>> entity...).
>>>
>>> That is nice, but, well, Quarkus deserves better ;-)
>>>
>>>

>>> *Client-Side generation*

>>>
>>> What can be done for the server can be done for the client. In most
>>> distributed systems, you need to call remote endpoint. Here, the
>>> idea would
>>> be to generate the (MicroProfile) Rest Client interface from an
>>> OpenAPI
>>> descriptor. So, you don't have to write the interface yourself. As
>>> for the
>>> server-side, it would generate the Rest Client interface (but this
>>> time, no
>>> need to implement anything) and the data structures. It can even
>>> introduce
>>> the proper configuration attribute in your "application.properties"
>>> to
>>> configure the URL.
>>>
>>>

>>> *How can this be done?*

>>>
>>> Generating code from the OpenAPI descriptor is not something new. In
>>> addition, the Apicurio project (https://www.apicur.io/) started, as
>>> part
>>> of the Apicurio Studio, to support the server-side generation.
>>> Granted, it
>>> may be a bit outdated, but the base is there (Eric, if you are
>>> around,
>>> maybe you can give us an update).
>>>
>>>
>>> Trigging the generation can be done either from the dev console or
>>> via
>>> the "qs" cli.
>>>
>>>
>>> Ok, but can't we go further?
>>>
>>>

>>> *Mocking Services*

>>>
>>> From here, we are more talking about ideas than concrete
>>> implementations
>>> (it was the case before, but it was easy to see how it could be
>>> implemented).
>>>
>>>
>>> When implementing an application requiring a service, you end up
>>> having
>>> to call that service. But what if it's not yet implemented or
>>> available?
>>> What could be done is to generate a mock version from the OpenAPI
>>> descriptor. Maybe, some fixtures can be required, but if the OpenAPI
>>> descriptor is complete enough (with examples and so on), we have
>>> everything
>>> to run a mock version of the service. So, if you use dev services,
>>> it would
>>> just start that fake version and configure your application to call
>>> it. So,
>>> you are not stuck and can start implementing your business logic. Of
>>> course, as soon as the service is ready, you would not use the mock
>>> version.
>>>
>>>

>>> *Conformance Testing*

>>> <https://groups.google.com/d/msgid/quarkus-dev/CAKW6ficvQpe15McuaYC2j_3ROMRtfjuZa-dtPz3i4GArkCHs%2BA%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .

>>>
>> --
>>
>> MATTIA MASCIA
>>
>> He / Him / His
>>
>> PRINCIPAL CONSULTANT
>>

>> Switzerland
>>
>> mma...@redhat.com M: +41 79 41 14 377 <+41794114377>

>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups
>> "Quarkus Development mailing list" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an
>> email to quarkus-dev...@googlegroups.com.
>> To view this discussion on the web visit

>> https://groups.google.com/d/msgid/quarkus-dev/CAGzaM-41_aGoMQZrQ8orRwV%2B_bnRwEoM_VzkUPc6op7mnehphA%40mail.gmail.com
>> <https://groups.google.com/d/msgid/quarkus-dev/CAGzaM-41_aGoMQZrQ8orRwV%2B_bnRwEoM_VzkUPc6op7mnehphA%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .

>>
>
>
> --
> Bill Burke
> Red Hat
>

> --
> You received this message because you are subscribed to the Google
> Groups "Quarkus Development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to quarkus-dev...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/quarkus-dev/CAL%3DE%3DjQtevm05EkMUvzfNvj%3DyFiwX4B%2BZpjmU8GhhzeM1ww%3DzA%40mail.gmail.com.


/max
https://xam.dk/about

[Max Rydahl Andersen]

>
> Trigging the generation can be done either from the dev console or via
> the
> "qs" cli.

dev console or cli is for me not that interesting here.
sure - we can have it but I would much rather see if we can just fit it
into the extension lifecycle
like we've done with grpc and kogito done similar.

There shouldn't need to be a dev console or cli step here.

At least when doing service side contract first development.

/max

>
>
> Ok, but can't we go further?
>
>

> *Mocking Services*

>
> From here, we are more talking about ideas than concrete
> implementations
> (it was the case before, but it was easy to see how it could be
> implemented).
>
>
> When implementing an application requiring a service, you end up
> having to
> call that service. But what if it's not yet implemented or available?
> What
> could be done is to generate a mock version from the OpenAPI
> descriptor.
> Maybe, some fixtures can be required, but if the OpenAPI descriptor is
> complete enough (with examples and so on), we have everything to run a
> mock
> version of the service. So, if you use dev services, it would just
> start
> that fake version and configure your application to call it. So, you
> are
> not stuck and can start implementing your business logic. Of course,
> as
> soon as the service is ready, you would not use the mock version.
>
>

> *Conformance Testing*

>
> Back to the server-side. Even though you generated the skeleton,
> things can
> quickly drift. Again, if the OpenAPI descriptor is complete enough,
> Quarkus
> could run conformance tests against your API.
>
>
> The purpose of the Pact (https://docs.pact.io/), among other tools, is
> precisely this. However, Pact does not use OpenAPI as input, which
> requires
> a second type of descriptor. Not the end of the world, but the idea
> would
> be to avoid that. The Pact documentation explains very well the
> different
> purpose (https://docs.pact.io/faq/convinceme#but-i-use-swaggeropenapi)
>
>
> Thoughts? Ideas?
>
>
> Clement
>

> --
> You received this message because you are subscribed to the Google
> Groups "Quarkus Development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to quarkus-dev...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/quarkus-dev/CAKW6ficvQpe15McuaYC2j_3ROMRtfjuZa-dtPz3i4GArkCHs%2BA%40mail.gmail.com.


/max
https://xam.dk/about

[Max Rydahl Andersen]

On 29 Apr 2021, at 17:58, Mattia Mascia wrote:

> Hi Clement,
>
> Nice discussion.
>
> Regarding the code generation I would extend the "official" community
> one
> : https://openapi-generator.tech/docs/generators

separate from this discussion I would love to see someone contribute a
decent backend for such generator that would
work with MP client; for both classic and non-reactive.


on this one - then feedback I generally seem to get about these
generators are that their
api is not very fluent, bulky and just quite incomplete/hard to
maintain. Some discussions at
https://github.com/quarkusio/quarkus/issues/3905 in
the last few weeks.

main Challenge though seem to be that latest openapi gen does not work
with latest openapi features because they haven't been able to migrate
all the various implementations.

I haven't tried it so it might be a non-issue but it is not the first
time I heard it over the years.

But it worries me if the generator already now is being held back :/

/max

>
> It's pretty nice and works well.
>
> Best regards
>
> Mattia
>
>
> On Thu, 29 Apr 2021 at 16:50, clement escoffier
> <clement....@gmail.com>
> wrote:
>
>> Hello,
>>
>>
>> A couple of Quarkus contributors discussed offline contract-first API
>> development and how it can be achieved in Quarkus.
>>
>>
>> In this mail, I will try to capture the result of these discussions;
>> it's
>> not an easy task, they got pretty chatty about it. But let's try.
>> Bear with
>> me; I'm not an expert on the topic; I'm just the messenger.
>>
>>
>> To simplify and scope the conversation, we will only focus on REST.
>> The
>> principles can be applied to the other type of interactions
>> (event-driven,
>> GraphQL, and so on)
>>
>>

>> *What's contract-first API development?*
>>

>> The idea behind contract-first API development is to have a set of
>> API
>> descriptors that drive your development. They are generally OpenAPI
>> descriptors in the REST world, but that's not the only format (we
>> will talk
>> about that later).
>>
>> Immediately, you can imagine that contract first API development for
>> Quarkus would be to generate a project skeleton based on an OpenAPI
>> descriptor. The generation produces the project structure, the
>> resource
>> skeleton, and the data structures that are part of the API contract
>> (body,
>> entity...).
>>
>> That is nice, but, well, Quarkus deserves better ;-)
>>
>>

>> *Client-Side generation*

>>
>> What can be done for the server can be done for the client. In most
>> distributed systems, you need to call remote endpoint. Here, the idea
>> would
>> be to generate the (MicroProfile) Rest Client interface from an
>> OpenAPI
>> descriptor. So, you don't have to write the interface yourself. As
>> for the
>> server-side, it would generate the Rest Client interface (but this
>> time, no
>> need to implement anything) and the data structures. It can even
>> introduce
>> the proper configuration attribute in your "application.properties"
>> to
>> configure the URL.
>>
>>

>> *How can this be done?*
>>

>> Generating code from the OpenAPI descriptor is not something new. In
>> addition, the Apicurio project (https://www.apicur.io/) started, as
>> part
>> of the Apicurio Studio, to support the server-side generation.
>> Granted, it
>> may be a bit outdated, but the base is there (Eric, if you are
>> around,
>> maybe you can give us an update).
>>
>>
>> Trigging the generation can be done either from the dev console or
>> via the
>> "qs" cli.
>>
>>
>> Ok, but can't we go further?
>>
>>

>> *Mocking Services*

>>
>> From here, we are more talking about ideas than concrete
>> implementations
>> (it was the case before, but it was easy to see how it could be
>> implemented).
>>
>>
>> When implementing an application requiring a service, you end up
>> having to
>> call that service. But what if it's not yet implemented or available?
>> What
>> could be done is to generate a mock version from the OpenAPI
>> descriptor.
>> Maybe, some fixtures can be required, but if the OpenAPI descriptor
>> is
>> complete enough (with examples and so on), we have everything to run
>> a mock
>> version of the service. So, if you use dev services, it would just
>> start
>> that fake version and configure your application to call it. So, you
>> are
>> not stuck and can start implementing your business logic. Of course,
>> as
>> soon as the service is ready, you would not use the mock version.
>>
>>

>> *Conformance Testing*

>> <https://groups.google.com/d/msgid/quarkus-dev/CAKW6ficvQpe15McuaYC2j_3ROMRtfjuZa-dtPz3i4GArkCHs%2BA%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .

>>
> --
>
> MATTIA MASCIA
>
> He / Him / His
>
> PRINCIPAL CONSULTANT
>

> Switzerland
>
> mma...@redhat.com M: +41 79 41 14 377 <+41794114377>

>
> --
> You received this message because you are subscribed to the Google
> Groups "Quarkus Development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to quarkus-dev...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/quarkus-dev/CAGzaM-41_aGoMQZrQ8orRwV%2B_bnRwEoM_VzkUPc6op7mnehphA%40mail.gmail.com.


/max
https://xam.dk/about

[Stephane Epardaud]

On Fri, 30 Apr 2021 at 11:19, Max Rydahl Andersen <mand...@redhat.com> wrote:

There shouldn't need to be a dev console or cli step here.

At least when doing service side contract first development.

I'm not sure what you mean here. I look at it from the perspective of dependencies:

As a user, I want to consume a web service. Let's pretend it's K8s, so I import `quarkus-kubernetes` or whatever gets me the K8s HTTP client.

Now, let's suppose it's not, and all I got is an OpenAPI file? The equivalent step would be to import `io.quarkus:gimme-client?http://service/openapi.yml` but that's not going to fly, is it? So what?

$ q make-http-service http://service/openapi.yml

Would be a decent way to get the service interfaces generated and configured for usage, even though its source files would appear in my git tree. I guess I can live with that.

[Laurent Broudoux]

Hi Clement and all,

Just wanted to share what we're doing on the API/microservices mocking and conformance testing sides with Microcks (https://microcks.io).

It's a Kubernetes native tooling tackling especially this 2 challenges using contract / specification (OpenAPI descriptors) + embedded examples :

1- For generating simulations of designed service with request/response matching rules, EL for message templating, etc...

2- For contract testing implementations once developed ; validating schemas and protocols semantics.

I recently had a talk at BarcelonaJUG with an introduction to the tool : https://youtu.be/p5gdmrPFTw8?t=2208

One nice thing about Microcks is that is handles both synchronous REST APIs and new AsyncAPI specification. We also had a blog post recently on how this can be used to simulate CloudEvents : https://microcks.io/blog/simulating-cloudevents-with-asyncapi/ - I thing this technique can be used to ease developer life when it comes to deal with Knative events, Kogito or event Debezium events : one do not have to replicate complex setup to have CloudEvents at hand for developing.

Let me know if you see any interest looking at Microcks ; it is written in Java (partly SpringBoot and now the new part in Quarkus) so some stuffs may be reused ...

[Kevin Viet]

Record/replay
Possibility to record traffic to ease the creation of mock to be replayed later.

This is very interesting, and something I thought about too, but you have to have real data to record first, so where would that happen? The user using curl? Using the SwaggerUI? Recording production exchanges? That's what I haven't been able to figure out. But yeah, something that looks at exchanges "somewhere" and spits out HTTP fixtures would be awesome.

This is more for non regression scenarios :

___________________             ___________         ___________________________          _____________
| Non Regression Tool |.  -----> | Microservice | ----> |Ghosting solution in record mode|  ---> | Real REST Api |
--------------------------------            ------------------          ----------------------------------------------          ----------------------

The non regression tool might be any tool of the industry namely : 

• - Gatling
• - SOAP UI
• - Postman 

Performance/load testing
Last but not least : Internally we created a ghosting solution based on Vertx because we needed a mocking solution that literally scales during the load testing of our applications.  

 

 You want to test the load, or you want the mocking to scale under load?

It might be a little specific on how we do things internally but our mocking was not able to scale under the load of some of our applications:  mainly IO bound  and generate a lot of API calls. 

So when doing performance campaigns on these apps we did not want the mocking to be the bottleneck so we did a mocking solution based on vertx that scales very well.

--

You received this message because you are subscribed to the Google Groups "Quarkus Development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to quarkus-dev...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CAKU9E9uZOrap5SGtMp-eXU%3Dv9BftSbK0JPtweUe_qXODGCQHTg%40mail.gmail.com.

[clement escoffier]

Hello,

Thanks for all these ideas! Let's try to organize them a little bit and see where we can go from there. 

From these discussions, there are two facets to this work:

1. server/client generation based on OpenAPI description
2. mock / record-replay services

These are pretty different, so let's cover them separately. 

Server/Client generation based on OpenAPI

For the server-side, the problem is simple, at least well-known. You provide an OpenAPI descriptor, and it bootstraps the server implementation (Jax-RS resources and data objects). There are already plenty of generators like this, but I will come to that a bit later.

The client-side generation is less mature, or at least not as common. The idea is to generate the Microprofile REST Client interface based on an OpenAPI descriptor. It is similar to server-side generation. Granted, if you want to create the REST Client interface for an extremely extensive API (Github or Kubernetes), it may not be practical. However, if the service you target had a dozen of endpoints, that sounds possible. Solutions such as filtering by prefix could also help. The advantage here is to ease the writing of such REST clients, which while being marvelous, are still new. 

As I said before, there are plenty of generators, primarily targetting the server-side. I discussed this quickly with Eric from the Apicurio project, and we will work on an extension that handles the server and client generation, as there is a lot of code shared between these two tasks. 

Faking services

The second side of the work is quite different, but it also can be a significant differentiator. The idea is to provide developers a feature to invoke services that do not yet exist or are not accessible. You can see it as a mock of the remote service or a fake service.

There are multiple ways to achieve this:

1. describing the API in the code with expectations,
2. running the fake service based on its OpenAPI description,
3. recording service interactions (in production, for example) and replaying them.

All these approaches are not incompatible, and they can all complement each other.

The first approach is slightly similar to what Wiremock or Prism provides. You write a spec (or code it) that describes the stub, and potentially, some verification rules and dev services can run the service for you. In dev or test modes, you can run your application and call that service. 

The second approach is somewhat similar. However, instead of writing the descriptor, it would take an OpenAPI descriptor as input. Microcks does this, for example. If your OpenAPI descriptor contains enough details about the service, it can produce fake responses. Yes, you may have a bit less control over the HTTP responses, but it's a smooth setup (especially combined with the client-side generation mentioned above). 

The third approach is in the same vein. However, this time, the behavior of the fake service is recorded. It means that the service runs somewhere, and you were able to record the request/response. Multiple tools are doing this, such as Pact, Wiremock (yes, it seems to be a new feature); it's also on the roadmap for Microcks. 

Another aspect that Microcks and Wiremocks provide is the ability to control the latency between your application and the fake service. I'm a big fan of that feature, as it let you verify your resilience strategy (timeout, retry, circuit-breaker, and so on)

I don't know the landscape enough, but it seems that multiple tools already exist, so we should select and integrate.

Conformance

That's also an interesting topic. Conformance would allow verifying that you implement a service correctly or that the service you are calling behaves as expected. Microcks supports this. It takes the OpenAPI descriptor as input and verifies that the service implements it correctly (it also supports versions, as it is very likely to have multiple versions of a service). 

Last words

This email focuses solely on HTTP/REST. But as highlighted by Laurent, the same principles would work for event-driven architectures. 

For the server/client-side generation, there is a plan. The second part is more challenging but looks very promising. It would fit very well with the developer experience Quarkus provides. 

Clement

[Stuart Douglas]

On Fri, 7 May 2021 at 01:49, clement escoffier <clement....@gmail.com> wrote:

Hello,

Thanks for all these ideas! Let's try to organize them a little bit and see where we can go from there. 

From these discussions, there are two facets to this work:

1. server/client generation based on OpenAPI description
2. mock / record-replay services

These are pretty different, so let's cover them separately. 

Server/Client generation based on OpenAPI

For the server-side, the problem is simple, at least well-known. You provide an OpenAPI descriptor, and it bootstraps the server implementation (Jax-RS resources and data objects). There are already plenty of generators like this, but I will come to that a bit later.

The client-side generation is less mature, or at least not as common. The idea is to generate the Microprofile REST Client interface based on an OpenAPI descriptor. It is similar to server-side generation. Granted, if you want to create the REST Client interface for an extremely extensive API (Github or Kubernetes), it may not be practical. However, if the service you target had a dozen of endpoints, that sounds possible. Solutions such as filtering by prefix could also help. The advantage here is to ease the writing of such REST clients, which while being marvelous, are still new. 

As I said before, there are plenty of generators, primarily targetting the server-side. I discussed this quickly with Eric from the Apicurio project, and we will work on an extension that handles the server and client generation, as there is a lot of code shared between these two tasks. 

It would be really cool if we could do this from the Dev UI, just point it at a URL, hit a button and you have a fully generated client.

 

Faking services

The second side of the work is quite different, but it also can be a significant differentiator. The idea is to provide developers a feature to invoke services that do not yet exist or are not accessible. You can see it as a mock of the remote service or a fake service.

There are multiple ways to achieve this:

1. describing the API in the code with expectations,
2. running the fake service based on its OpenAPI description,
3. recording service interactions (in production, for example) and replaying them.

All these approaches are not incompatible, and they can all complement each other.

The first approach is slightly similar to what Wiremock or Prism provides. You write a spec (or code it) that describes the stub, and potentially, some verification rules and dev services can run the service for you. In dev or test modes, you can run your application and call that service. 

The second approach is somewhat similar. However, instead of writing the descriptor, it would take an OpenAPI descriptor as input. Microcks does this, for example. If your OpenAPI descriptor contains enough details about the service, it can produce fake responses. Yes, you may have a bit less control over the HTTP responses, but it's a smooth setup (especially combined with the client-side generation mentioned above). 

The third approach is in the same vein. However, this time, the behavior of the fake service is recorded. It means that the service runs somewhere, and you were able to record the request/response. Multiple tools are doing this, such as Pact, Wiremock (yes, it seems to be a new feature); it's also on the roadmap for Microcks. 

Another aspect that Microcks and Wiremocks provide is the ability to control the latency between your application and the fake service. I'm a big fan of that feature, as it let you verify your resilience strategy (timeout, retry, circuit-breaker, and so on)

I don't know the landscape enough, but it seems that multiple tools already exist, so we should select and integrate.

We could record all data from testsuite runs, use it to generate a 'test image' that can repeat the same responses back, or use this to generate pact rules.

 

Conformance

That's also an interesting topic. Conformance would allow verifying that you implement a service correctly or that the service you are calling behaves as expected. Microcks supports this. It takes the OpenAPI descriptor as input and verifies that the service implements it correctly (it also supports versions, as it is very likely to have multiple versions of a service). 

We could have a verify filter in dev/test mode that will fail the response if it does not confirm to the API definition. We could probably just make this enabled by default so you get this verification out of the box.

Stuart

 

To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CAKW6ficaBuRqao2o8-fh3AMV-sm6Ah67QuvAScgsQETsjNMmsQ%40mail.gmail.com.

[Maciej Swiderski]

pt., 7 maj 2021 o 03:31 Stuart Douglas <sdou...@redhat.com> napisał(a):

On Fri, 7 May 2021 at 01:49, clement escoffier <clement....@gmail.com> wrote:

Hello,

Thanks for all these ideas! Let's try to organize them a little bit and see where we can go from there. 

From these discussions, there are two facets to this work:

1. server/client generation based on OpenAPI description
2. mock / record-replay services

These are pretty different, so let's cover them separately. 

Server/Client generation based on OpenAPI

For the server-side, the problem is simple, at least well-known. You provide an OpenAPI descriptor, and it bootstraps the server implementation (Jax-RS resources and data objects). There are already plenty of generators like this, but I will come to that a bit later.

The client-side generation is less mature, or at least not as common. The idea is to generate the Microprofile REST Client interface based on an OpenAPI descriptor. It is similar to server-side generation. Granted, if you want to create the REST Client interface for an extremely extensive API (Github or Kubernetes), it may not be practical. However, if the service you target had a dozen of endpoints, that sounds possible. Solutions such as filtering by prefix could also help. The advantage here is to ease the writing of such REST clients, which while being marvelous, are still new. 

As I said before, there are plenty of generators, primarily targetting the server-side. I discussed this quickly with Eric from the Apicurio project, and we will work on an extension that handles the server and client generation, as there is a lot of code shared between these two tasks. 

It would be really cool if we could do this from the Dev UI, just point it at a URL, hit a button and you have a fully generated client.

When it comes to client generation from OpenAPI I'd like to add a few points here that could be taken into consideration when implementing this feature. I have been doing something like that in my project based on an open api tools generator and I think that the most important thing is that it should only generate methods on operations that are to be used. So at least there should be an option to define what operation ids or paths+http methods you are interested in. This significantly helps when working with big OpenAPI definitions.

In addition to that here are some points about the generated methods for clients 

- generate at least two methods - one with all parameters and second one with just required parameters - in many cases the later would be most frequently used and makes it way simpler to be used. There might be lots of query params that are optional so always requiring them can be not the best.

- take into account security schemes especially for those endpoints that use query param based auth like api key etc. That should be part of the generated method so it can be easily given when being invoked. 

- providing some pluggable security schemes support for rest calls based on generated clients can be properly authenticated like adding http headers. Sometimes this can be simple using basic auth but might be more complex when using various OAuth flows

- optional generation of the data model as data model might be already part of the application itself or its dependencies - here is a bit tricky part as it somehow needs to provide model to the generated client interface so maybe relying on jandex to check if given model is already there or similar

These are the main pain points I ran into when generating MP Rest Client interfaces from OpenAPI definition. Hope this can be a useful input for the upcoming implementation of this very useful feature.

Maciej

To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CAD%2BL2cx_Vk3iNm0G5cT3cvEQF8p%3Dvrmti9EYoy3E7CEBdFGDBg%40mail.gmail.com.

[clement escoffier]

Le ven. 7 mai 2021 à 07:13, Maciej Swiderski <swidersk...@gmail.com> a écrit :

pt., 7 maj 2021 o 03:31 Stuart Douglas <sdou...@redhat.com> napisał(a):

On Fri, 7 May 2021 at 01:49, clement escoffier <clement....@gmail.com> wrote:

Hello,

Thanks for all these ideas! Let's try to organize them a little bit and see where we can go from there. 

From these discussions, there are two facets to this work:

1. server/client generation based on OpenAPI description
2. mock / record-replay services

These are pretty different, so let's cover them separately. 

Server/Client generation based on OpenAPI

For the server-side, the problem is simple, at least well-known. You provide an OpenAPI descriptor, and it bootstraps the server implementation (Jax-RS resources and data objects). There are already plenty of generators like this, but I will come to that a bit later.

The client-side generation is less mature, or at least not as common. The idea is to generate the Microprofile REST Client interface based on an OpenAPI descriptor. It is similar to server-side generation. Granted, if you want to create the REST Client interface for an extremely extensive API (Github or Kubernetes), it may not be practical. However, if the service you target had a dozen of endpoints, that sounds possible. Solutions such as filtering by prefix could also help. The advantage here is to ease the writing of such REST clients, which while being marvelous, are still new. 

As I said before, there are plenty of generators, primarily targetting the server-side. I discussed this quickly with Eric from the Apicurio project, and we will work on an extension that handles the server and client generation, as there is a lot of code shared between these two tasks. 

It would be really cool if we could do this from the Dev UI, just point it at a URL, hit a button and you have a fully generated client.

+1 for the Dev UI. It got lost in my second email (but was mentioned in my first ;-))

 

When it comes to client generation from OpenAPI I'd like to add a few points here that could be taken into consideration when implementing this feature. I have been doing something like that in my project based on an open api tools generator and I think that the most important thing is that it should only generate methods on operations that are to be used. So at least there should be an option to define what operation ids or paths+http methods you are interested in. This significantly helps when working with big OpenAPI definitions.

In addition to that here are some points about the generated methods for clients 

- generate at least two methods - one with all parameters and second one with just required parameters - in many cases the later would be most frequently used and makes it way simpler to be used. There might be lots of query params that are optional so always requiring them can be not the best.

- take into account security schemes especially for those endpoints that use query param based auth like api key etc. That should be part of the generated method so it can be easily given when being invoked. 

- providing some pluggable security schemes support for rest calls based on generated clients can be properly authenticated like adding http headers. Sometimes this can be simple using basic auth but might be more complex when using various OAuth flows

- optional generation of the data model as data model might be already part of the application itself or its dependencies - here is a bit tricky part as it somehow needs to provide model to the generated client interface so maybe relying on jandex to check if given model is already there or similar

These are the main pain points I ran into when generating MP Rest Client interfaces from OpenAPI definition. Hope this can be a useful input for the upcoming implementation of this very useful feature.

That's very good points! We should definitely provide options for these.

I don't believe the generator will be a one-off task and done. We will need several iterations to understand these kinds of issues and improve the customizability and usability. 

By any chance, can you share the openapi descriptor? At the moment is it's all handwavy, but soon we would need to have more structured ideas. 

Clement

[Maciej Swiderski]

Clement, what do you mean by openapi descriptor? 

Maciej

[clement escoffier]

You mentioned you worked on a project using generation. I was wondering if you had a set of OpenAPI descriptors we could reuse and look at what should be generated for these.

I know there are plenty on the Internet, but if you have a set with clear expectations, it would be great. 

Clement

[swidersk...@gmail.com]

Usual pet store is quite good to start with. In addition this one might be good too https://app.swaggerhub.com/apis/IdRatherBeWriting/open-weather_map_api/2.5.2 

I also used quarkus apps open api of the rest service but as far as I remember it does not include the security info based on what is configured in the app like Basic Auth with properties or OAuth/OID so that part is not there.

Maciej

[clement escoffier]

Awesome, thanks! 

Clement

[Max Rydahl Andersen]

On 7 May 2021, at 3:31, Stuart Douglas wrote:

It would be really cool if we could do this from the Dev UI, just point it at a URL, hit a button and you have a fully generated client.

am I the only one that think this is the least interesting option :)

But if this dev ui is just using the info from application.properties or annotations defining which endpoints you are after then it would be nice
to have the button to (re)generate.

/max
https://xam.dk/about

[Benjamin Raimondi]

Le jeudi 6 mai 2021 à 17:49:31 UTC+2, clement....@gmail.com a écrit :

Hello,

Thanks for all these ideas! Let's try to organize them a little bit and see where we can go from there. 

From these discussions, there are two facets to this work:

1. server/client generation based on OpenAPI description
2. mock / record-replay services

These are pretty different, so let's cover them separately. 

Server/Client generation based on OpenAPI

For the server-side, the problem is simple, at least well-known. You provide an OpenAPI descriptor, and it bootstraps the server implementation (Jax-RS resources and data objects). There are already plenty of generators like this, but I will come to that a bit later.

One of the firsts difficulties when we  began with Quarkus (2 year earlier) was the code generation. We used OpenApiGenerator with jaxrs-resteasy generator and we add to override some mustache template to add @RegisterForReflexion.

Some works on this librairy could make it quarkus ready, and It would be great :)

The client-side generation is less mature, or at least not as common. The idea is to generate the Microprofile REST Client interface based on an OpenAPI descriptor. It is similar to server-side generation. Granted, if you want to create the REST Client interface for an extremely extensive API (Github or Kubernetes), it may not be practical. However, if the service you target had a dozen of endpoints, that sounds possible. Solutions such as filtering by prefix could also help. The advantage here is to ease the writing of such REST clients, which while being marvelous, are still new. 

As I said before, there are plenty of generators, primarily targetting the server-side. I discussed this quickly with Eric from the Apicurio project, and we will work on an extension that handles the server and client generation, as there is a lot of code shared between these two tasks. 

Faking services

The second side of the work is quite different, but it also can be a significant differentiator. The idea is to provide developers a feature to invoke services that do not yet exist or are not accessible. You can see it as a mock of the remote service or a fake service.

There are multiple ways to achieve this:

1. describing the API in the code with expectations,
2. running the fake service based on its OpenAPI description,
3. recording service interactions (in production, for example) and replaying them.

All these approaches are not incompatible, and they can all complement each other.

The first approach is slightly similar to what Wiremock or Prism provides. You write a spec (or code it) that describes the stub, and potentially, some verification rules and dev services can run the service for you. In dev or test modes, you can run your application and call that service. 

The second approach is somewhat similar. However, instead of writing the descriptor, it would take an OpenAPI descriptor as input. Microcks does this, for example. If your OpenAPI descriptor contains enough details about the service, it can produce fake responses. Yes, you may have a bit less control over the HTTP responses, but it's a smooth setup (especially combined with the client-side generation mentioned above). 

The third approach is in the same vein. However, this time, the behavior of the fake service is recorded. It means that the service runs somewhere, and you were able to record the request/response. Multiple tools are doing this, such as Pact, Wiremock (yes, it seems to be a new feature); it's also on the roadmap for Microcks. 

Another aspect that Microcks and Wiremocks provide is the ability to control the latency between your application and the fake service. I'm a big fan of that feature, as it let you verify your resilience strategy (timeout, retry, circuit-breaker, and so on)

I don't know the landscape enough, but it seems that multiple tools already exist, so we should select and integrate.

Conformance

That's also an interesting topic. Conformance would allow verifying that you implement a service correctly or that the service you are calling behaves as expected. Microcks supports this. It takes the OpenAPI descriptor as input and verifies that the service implements it correctly (it also supports versions, as it is very likely to have multiple versions of a service). 

Last words

This email focuses solely on HTTP/REST. But as highlighted by Laurent, the same principles would work for event-driven architectures.

 You talk earlier of the resilient strategy for http service, maybe using toxiproxy, that is a TCP proxy and not an HTTP proxy,  should have a more generic use

[clement escoffier]

Yes, Toxiproxy is a TCP proxy (so can be used for HTTP, even if the set of toxics are rather low level (you can return a 500 for example)). Last week, I struggled a lot with it.... kind of broke my heart :-(. But no worries, I have the beginning of a plan.

Clement

 

To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/f9c640b9-e812-4a54-b925-0a59c9492c4bn%40googlegroups.com.