How to test/validate API

[Felipe Sere]

Hi guys,

In the following weeks a team and I are going to build a new API for internal use.

One question that has come up is how do specify/validate/test our API.

Just to clarify, here is what I am thinking about when I mean specify/validate/test:

I want to make sure that the implementation of my endpoints behave the way I intend them to.

So I'll be looking at things like Can I call GET on a specific URL? Does it return the right status code? Is the structure of response OK (i.e. are certain elements there?).

An important thing for me is that I don't want any upfront code generation or spec-generation.

The development of the API and this 'spec' goes hand in hand, but they should not depend upon each other.

Have you guys done this? What has been your experience? What tools have you chosen/built?

Ideally, I could take the spec, turn it into HTML and with some examples and talk it through with client developers.

--

Cheers,
Felipe

[Jack Repenning]

On Apr 1, 2015, at 12:42 PM, Felipe Sere <felip...@gmail.com> wrote:

I want to make sure that the implementation of my endpoints behave the way I intend them to.

So I'll be looking at things like Can I call GET on a specific URL? Does it return the right status code? Is the structure of response OK (i.e. are certain elements there?).

Swagger/JSON, Mulesoft/RAML, and Apiary/Blueprint all provide varying degrees and styles of help, here. Each has its own strengths and weaknesses  (google for comparisons:  you'll be amazed at how many options you have you never though about!).

I am most familiar with Apiary/Blueprint. Here's how this works there:

1. You write Blueprint (a combination of end-user documentation and executable, verifying specification).

2. You write your implementation however you like (no code generation in this system).

3. You can test either against the spec (a "Mock server") or against your real code (a "Proxy server").

4. You can test through a browser, your own client program, or a companion test automation tool.

5. In any of that testing, Apiary watches the request/reply stream and provides a success/fail log with diffs.

-- 
Jack Repenning
Repenni...@gmail.com

[Felipe Seré]

Thanks Jack!

I’ll have to see if Apiary/Blueprint is free or a product I’d have to buy into.

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/d/optout.

---

[Magnus Lassi]

I really like ATDD especially Cucumber for business applications where it often makes a lot of sense. I recently ran across an interesting blog post by Steve Klabnik who frequents this mailing list and may be able to chime in if he still likes the approach he wrote about using Cucumber at: http://blog.balancedpayments.com/tdd-your-api/ 

I like how it describes it at a high level and how easily you can describe the possible pathways through the API. They have the code on github: https://github.com/balanced/balanced-api . 

[Andrew Braae]

If you use RAML, there is a tool for measuring test coverage described here:

http://forums.raml.org/t/test-tool-that-verifies-request-response-against-a-raml/471/2

To quote: "It does the right thing in the sense that, instead of trying to auto-test your API (which can only work for the most naive things), raml-tester invites you to fully cover your API with integration tests and, as you're doing so, validates that the requests and responses conform to the RAML spec and its related schemas."

[Owen Rubel]

Functional tests can be done with Curl and Spock.

[Felipe Seré]

I have been looking at abao (https://github.com/cybertk/abao/) which does pretty much the same.
Does anyone have experience with it?

What I like about it is that the setup is smaller, as no Java/Spring-MVC is going to be needed.
The goal of these tests is to make sure some flows work. Proper TDD will be done underneath in whatever language we use to implement the service.

On a similar note: What do you guys test on an API level? I have a tendency to cover a lot underneath with unit tests…

--

[André Tavares]

Django, for example, has a test client that can be used to build unit tests against the API endpoints. I use it to write tests for my APIs built with Django.

Do you use any framework that might have a similar thing?

[Felipe Seré]

Not really. I am relatively free in my choice of language.
One thing I wanted to avoid is use my production classes/code to validate the correctness of my API…