How to validate API in integration tests with Swagger?

[Diff]

I'm trying to figure out the best way to have my API documentation be the source of truth and use it to validate the actual Java REST code. We're using the contract first or consumer contract type of approach, so we don't want the documentation to be generated from annotated code necessarily and updating every time a developer makes a change.

One thought has been to use Swagger, but I'm not sure how best to make it be used for validating the API. Ideally, it'd be good to have the validation occur in the build or integration testing process to see if the response (and request if possible) match what's expected. The responses don't have to come through a real server setup but as long as it goes through the existing code and returns a response.  I know there are a lot of uses and tools for Swagger and just trying to wrap my head around it. Or if there is a better alternative to work with Java code.

[Ron Ratovsky]

Before diving into a possibly longer answer, have you seen our swagger-inflector project?

--
You received this message because you are subscribed to the Google Groups "Swagger" group.
To unsubscribe from this group and stop receiving emails from it, send an email to swagger-swaggers...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Diff]

No, I haven't heard of it.

To unsubscribe from this group and stop receiving emails from it, send an email to swagger-swaggersocket+unsub...@googlegroups.com.

[Ron Ratovsky]

Well, I’d suggest taking a look at the project. There’s a very simple quick start there that shows its capabilities.

It’s a tad different approach to writing your API. It still uses Jersey, but not quite the JAX-RS annotations.

However, it provides validations and other perks. It can even break your build if the code doesn’t match your spec, and provide you debugging details as to what’s missing.

To unsubscribe from this group and stop receiving emails from it, send an email to swagger-swaggers...@googlegroups.com.

[Adam Kliment]

I would say this sounds more like a job for API Blueprint [1] + Dredd HTTP testing framework [2]. There are some convertors around for API Blueprint -> Swagger and back.

[1] http://apiblueprint.org
[2] https://github.com/apiaryio/dredd http://dredd.readthedocs.org/en/latest/

I was totally thinking about adding support for Swagger to Dredd, but I miss some key features for this task in Swagger. I'll put them together and let you know. Mainly it's lack of request examples - they can be only generated from given schemas at this moment.

Is anyone else interested in testing Swaggers against backend implementation and putting them into continuous integration?

Adam

Disclaimer: I'm a Dredd maintainer.

[Ron Ratovsky]

Definitely an interesting option. So Diff, you have more than one thing to go with.

The swagger-inflector will serve as the base for your application, so it’s different. It is based on the design-first approach, building your code around the API and not the spec from the API.

From: <swagger-sw...@googlegroups.com> on behalf of Adam Kliment <ad...@apiary.io>
Reply-To: "swagger-sw...@googlegroups.com" <swagger-sw...@googlegroups.com>
Date: Tuesday, 19 January 2016 at 10:30
To: Swagger <swagger-sw...@googlegroups.com>
Subject: Re: How to validate API in integration tests with Swagger?

--

[Diff]

Ron, definitely sounds intriguing.  I'm especially interested in your comment of "It can even break your build if the code doesn’t match your spec".  Are there any code examples with that setup?

I'm not a fan of heavily annotated code, so this could be useful.

[Ron Ratovsky]

Zero annotations. One of the great things about it. In fact, no JAX-RS annotations as well.

Just follow the README for the project, and there are a few samples in the repo. If you have any specific questions, we’d love to cover those.

It’s actually one of my favorite projects now – love the concept of it.

To unsubscribe from this group and stop receiving emails from it, send an email to swagger-swaggers...@googlegroups.com.

[Diff]

Adam, do you know of any existing Swagger (or other related API formats) that have the ability to test them against the backend implementation?  I'd really like to have something as a part of continuous integration to validate that the code hasn't changed and broken the contract with the Swagger spec.  And I'd prefer to not use annotations in the code.  I've seen several tools that do validation but all they are comparing is the Swagger generated from the code against the expected Swagger - whereas I want to validate the actual response from the code against what's expected.  I'm trying to decide if Swagger is the right option still or not.

[tony tam]

The swagger-inflector java project does this:

https://github.com/swagger-api/swagger-inflector#payload-validation

[Diff]

Cool.  I was hoping to find some examples of this in action but haven't found any yet.  I'm also curious how to get this to work in a continuous integration type concept.  Any suggestions?

On Monday, January 25, 2016 at 11:18:55 AM UTC-6, tony tam wrote:

The swagger-inflector java project does this:

https://github.com/swagger-api/swagger-inflector#payload-validation

To unsubscribe from this group and stop receiving emails from it, send an email to swagger-swaggersocket+unsub...@googlegroups.com.

[tony tam]

Yes it’s ready for CI—take a look at some of the configurations.

https://github.com/swagger-api/swagger-inflector#development-lifecycle

Note you can configure the library to _fail_ to run if there are unimplemented operations, etc.

To unsubscribe from this group and stop receiving emails from it, send an email to swagger-swaggers...@googlegroups.com.

[Diff]

Is the ResponseContext required for the payload validation to work?

On Monday, January 25, 2016 at 11:18:55 AM UTC-6, tony tam wrote:

The swagger-inflector java project does this:

https://github.com/swagger-api/swagger-inflector#payload-validation

To unsubscribe from this group and stop receiving emails from it, send an email to swagger-swaggersocket+unsub...@googlegroups.com.

[tony tam]

Yes it is.  The entity inside it will be examined against the schema

To unsubscribe from this group and stop receiving emails from it, send an email to swagger-swaggers...@googlegroups.com.

[Diff]

A couple more questions:

- For the payload validation to work, based on my initial work with it, it appears you have to call the APIs to get them to do the validation.  Is that the way it works?  I assumed it automatically figured it out somehow.

- Are there any examples of swagger-inflector being used in a CI environment?  I'm interested to see the setup and code that would be running against swagger-inflector.  I see how to do the inflector configuration and have played with those settings successfully, but I want to see how the integration tests, or jenkins job (for instance) or whatever, and the related project would be setup as well.

On Tuesday, January 26, 2016 at 11:53:30 AM UTC-6, tony tam wrote:

Yes it’s ready for CI—take a look at some of the configurations.

https://github.com/swagger-api/swagger-inflector#development-lifecycle

Note you can configure the library to _fail_ to run if there are unimplemented operations, etc.

[tony tam]

That’s correct, it will validate the actual data being sent to/from the API.  This is a runtime check and is really intended for development purposes.  You can, for example, fail tests if the data is not matching the schema.  You shouldn’t get into production with payload schemas enabled, and there is some (low but non-zero) overhead.

For the examples for a CI environment, not yet but there will be one shortly.  I have a number of workflow examples to share, just trying to wrap them up.  I’ve opened this ticket so you can follow it and see when it’s complete:

https://github.com/swagger-api/swagger-inflector/issues/77

To unsubscribe from this group and stop receiving emails from it, send an email to swagger-swaggers...@googlegroups.com.