Can I generate pact contracts using swagger

[Gerry Mc]

Hi Guys

Our development team is going to start using swagger and we also wish to implement CDC testing using swagger. Is it possible to define or generate the pact contracts using Swagger or do we do that separately?

[Smith, David]

Hi Gerry,

This questions comes up a lot, you are not alone in wondering whether this is a suitable approach.

Pact works around two actors, the consumer and the provider. The consumer calls the provider and in doing so, defines the Pact contracts (Consumer Driven Contract Testing). A provider may produce swagger docs to describe their API, but if it also produced it's own Pact descriptions they would have no value as they wouldn't have been tested by the consumer. It's a bit like marking your own exam results.

So in answer to your question: Yes, you will need to generate them separately using a CDC/Pact test suite because Swagger and Pact descriptions are created by philosophically opposite processes.

Hope that helps,

Dave

-- 

David Smith | Principal Software Engineer | Online Technology | ITV plc

200 Gray's Inn Road, London, WC1X 8HF  |  Tel: 07754 417715

On 2 May 2018 at 22:10, Gerry Mc <gerrym...@gmail.com> wrote:

Hi Guys

Our development team is going to start using swagger and we also wish to implement CDC testing using swagger. Is it possible to define or generate the pact contracts using Swagger or do we do that separately?

--
Pact-Support email group is being deprecated, please use StackOverflow instead to help make questions and answers more visible: http://stackoverflow.com/questions/tagged/pact
---
You received this message because you are subscribed to the Google Groups "Pact Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pact-support+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

ITV plc (Registration No. 4967001) (ITV) is incorporated in England and Wales with its registered office at The London Television Centre, Upper Ground, London SE1 9LT. Please visit www.itv.com for further information.

Please note that all bookings sent to an ITV broadcaster are made subject to either (i) in respect of all broadcasters except the STV broadcasters, the broadcaster's standard airtime sales terms and conditions located at http://www.itvmedia.co.uk/legal/broadcasterstermsandconditions2018, or (ii) in respect of the STV broadcasters only the STV broadcasters standard airtime sales terms and conditions located at http://www.itvmedia.co.uk/legal/stvbroadcasterstermsandconditions2018 and are deemed to be made in accordance with any deal agreement in place between the Buyer and ITV Commercial, a division of ITV Broadcasting Limited ("Deal Agreement").

Bookings will only incorporate the booking parameters set out in the relevant Deal Agreement and any conflicting requests shall be deemed excluded from the relevant booking. If the Buyer wishes to include any campaign specific terms in the booking agreement, these must be negotiated separately with ITV Commercial (acting on behalf of the relevant ITV broadcaster) and agreed in writing. ITV Commercial reserves the right to re-coup over-delivery on an agreed booking on future booking agreements.

[Beth]

Nicely explained Dave! I'm filing away "marking your own exam results" (TM Dave Smith) for future reference.

[Bart Van Bos]

Hi Dave,

I do not agree with your statement on "It's a bit like marking your own exam results". Allow me to explain why...

It is true in a certain sense that PACT and Swagger/OpenAPI are philosophically opposite processes, as in who takes the lead in the expected API. I would not state it so hard however, since both are different but yet complementary.

• PACT is in essence (and I see most of the community completely missing this point) a behavioral contract. It is not by accident that people with Cucumber BDD testing will be very familiar with a PACT contract.
  
• Swagger is in essence an interface/form contract.
  

In my opinion, you need both to fully and accurately describe an API, and it does not matter who does this task (this is where I disagree with your statement on correcting your own exam):

• PACT's weakness lies within not clearly describing the full form (shape) of an API. It is easy to miss one behavioral scenario and it does not scale well to cover all possible input/output combo's (what I mean be that is that your PACT file can become very very huge, with lot's of repetition and redundancy (an example: a scenario describing all failure cases, including 401/403 responses). 
• Swagger weakness lies within the fact you cannot describe behavior in a formal matter (only with some free text or comments within the swagger comments).

That's why I always advise my client to use both, because you need both in real projects most of the time.

PS: I find the word contract a very bad name for PACT Behavior Driven Test Cases. In fact, PACT scenario's are test cases and the fact the PACT (but also Spring Cloud Contract) community, uses the word contract is the source of much confusion with the community and it's users (maybe this is a mere marketing decision, to compete with Swagger/OpenAPI, but this is not needed if you follow my logic described here).

PPS: In order to have some development flow that suites both, a following scenario is what I have seen working before:

1. Top down Swagger/OpenAPI contract creation by non-developers.
2. Generate a skeleton of PACT scenario's from this (as in endpoints/verbs/return codes), but leave the actual content empty and to filled in. The fact that you generate this from 1, leaves no room for missing combo's or mismatches between Swagger & PACT.
3. Complete the content (defining the behavior).
4. Start the rest of the usual PACT DEV flow.

>>> Who does this is only of second grade importance to me. In cases where consumer driven is workable, certainly do so. In cases where producers own the API, you have enforced testable behavior.

Best regards,

Bart

On Wednesday, May 2, 2018 at 11:19:25 PM UTC+2, Dave Smith wrote:

Hi Gerry,

This questions comes up a lot, you are not alone in wondering whether this is a suitable approach.

Pact works around two actors, the consumer and the provider. The consumer calls the provider and in doing so, defines the Pact contracts (Consumer Driven Contract Testing). A provider may produce swagger docs to describe their API, but if it also produced it's own Pact descriptions they would have no value as they wouldn't have been tested by the consumer. It's a bit like marking your own exam results.

So in answer to your question: Yes, you will need to generate them separately using a CDC/Pact test suite because Swagger and Pact descriptions are created by philosophically opposite processes.

Hope that helps,

Dave

-- 

David Smith | Principal Software Engineer | Online Technology | ITV plc

200 Gray's Inn Road, London, WC1X 8HF  |  Tel: 07754 417715

On 2 May 2018 at 22:10, Gerry Mc <gerrym...@gmail.com> wrote:

Hi Guys

Our development team is going to start using swagger and we also wish to implement CDC testing using swagger. Is it possible to define or generate the pact contracts using Swagger or do we do that separately?

--
Pact-Support email group is being deprecated, please use StackOverflow instead to help make questions and answers more visible: http://stackoverflow.com/questions/tagged/pact
---
You received this message because you are subscribed to the Google Groups "Pact Support" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pact-support...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Dave Smith]

Hi Bart,

Thank you for your thoughts. I disagree in a number of areas, and while I'm not inclined to argue I would offer a few observations for your consideration.

I do agree that you do need both Swagger and Pact contracts.

Pact testing is not a form of BDD testing, it is a form of contract testing taken from the perspective of the consumer ....hence the name ...Consumer Driven Contract Testing.

The fact that you only have to describe the parts of a contract that you will actually consume/use and not the whole contract, is in fact a strength, not a weakness as you suggest:

1. The consumer is stating and informing the provider team of the areas of the API they derive value from.
2. It is not the consumers job to fully document the providers API.
   
3. The consumer team need not concern themselves with every aspect of a providers API unnecessarily, only the bits they care about.

The thing that I love most about the Pact process though, is that the contract files are generated by the consumer as an output of an integration test that exercises real code. This means that if the implementation/code changes, the test changes, and the contract changes (or the test fails). If you generate Pact contract files from Swagger docs then you skip this frankly magical and super valuable step, effectively making the whole process ...a bit pointless.

Kind regards,

Dave

[Bart Van Bos]

Hi Dave,

BDD is an approach where one describes the behavior in a machine understandable format and generates tests from that. BDD is at it's core an attempt of contract behavior specification (what you like to call a "contract") and the RoR community in the past went to great length to extend this all the way to user story break down using Gherkin/Cucumber in agile development environments that where aiming for continuous integration. 

This is exactly what PACT is. It is not a contract, but a mere enumeration of test cases that describe expected behavior. I realize this discussion is fishy as it boils down on a common understanding and hard definition of what a contract is exactly. However, looking at the number of concerns and question on various fora, I do not seem the only one experiencing this misconception (between a form-contract and a behavior-contract).

Also with your remark on the consumer only describing area's of the API he/she uses... I partially agree. This is a valid statement in happy case scenario's. In failure cases (the HTTP 4xx and 5xx cases) in real life this is unrealistic, as that consumer's behavior is in reality determined by the input of another system (whether another micro service or an end-user using an UI interface, doesn't matter). 

Don't miss understand me, I completely acknowledge and understand the added value of consumer driven development (I will avoid the word contract) in a micro-service context where consumer and producer can cooperate directly. I just think that both PACT and Spring Cloud Contract mis-use terminology and create unnecessary confusion. The fact that so many people are asking for PACT to Swagger/OpenAPI tooling or vice versa is not because they do not understand it (as way too often is the answers of the enlightened extremist in this area), but because both approaches aren't perfect and a double approach is needed.

Kindly,

Bart Van Bos

SW & ICT Engineering - AllBits BVBA

Mobile: +32 485 630 628
E-mail: bartv...@gmail.com
BTW: BE.0678.829.457

IBAN: BE23 9731 7830 1491
Address: Lobroeken 25, 3191 Hever

[Beth]

Hi all,

I have to say, I read this thread with a massive smile on my face because I loved the level of understanding and deep consideration of the concepts that was shown! I have a few points to add, though I'm not really going to wade in to the debate.

1. The term "contract" (most likely) came from this article by Ian Robertson https://martinfowler.com/articles/consumerDrivenContracts.html (happy to be corrected on this, but it's the earliest known reference I know of) and was in existence in 2006, well before Pact was first created. His definition is not technically specific, but does seem to have an element of "behaviour" to it.

2. Bart, you make an excellent point that Pact is in fact a series of test cases, rather than a "schema" type contract, and I agree that this is confusing as the term "contract" is overloaded. There used to be a fairly clear explanation of this "contract by example" in the Pact docs near the start, but it seems to have gone missing in the latest rewrite. I'll see where I can put it back in.

3. I think generating test scenarios (that is, actual CODE) from a swagger document is actually a great idea. We have discussed this before, but it's never come up as the number 1 priority, so has not been done (yet). What people often ask about, however, is generating the pact itself from a Swagger document. This is the thing that the docs claim is pointless (though I guess generating a pact from a Swagger document and verifying it against the provider could be an easy way of verifying that the Swagger doc was in sync with the provider code - it just wouldn't be a consumer contract)

4. Yes, I agree that Pact and Swagger work well together. I'm currently working on improving our Swagger support in the Pact Broker (allowing you to publish swagger documents for use with Atlassian's Pact Swagger request validator).

Bart, I thought you expressed how Pact and Swagger work together and complement each other's weaknesses quite eloquently. Would you be interested in writing a few paragraphs on it for including in docs.pact.io?

Cheers,
Beth

[Beth]

PS. I've updated this section (added second paragraph), please let me know what you think Bart/Dave https://github.com/pact-foundation/pact.io/blob/documentation/readme.md#consumer-driven-contracts