Suggestion: use cucumber + rest-assured for contract testing

[pf...@thoughtworks.com]

Hi all,

I have created a contract test repo: https://github.com/OpenLMIS/openlmis-contract-tests

I wrote a sanity test using cucumber + rest-assured.

Cucumber allows us to write tests in a more readable manner, behind it, rest-assured would sending http requests and verifying response.

Please take a look and let me know if you have any questions.

Thanks

[Jake Watson]

Thanks Pengfei -- I really like the readability of the tests.

SolDevelo guys -- want to take some of the Postman tests and try them out in Cucumber/gherkin?

[Paweł Gesek]

Hello,

When in Seattle we decided on Postman as opposed to the RestAssured tests we were starting as it would be easier for less technical users to write tests. From what I can see Cucumber is a BDD framework, so the tests (at least the spec part) would be Java code, then we would write these test cases as specs. Am I getting that right?

How would you see the flow of us using it? If the product owner/QAs would write the specs - who do you see writing the steps? Do you think this would ultimately fall more on the shoulders of the development team or the QA?

Regards,
Paweł

--
You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev...@googlegroups.com.
To post to this group, send email to openlm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/6ffbdc93-91e7-48f5-a9d8-15b08557b619%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[pf...@thoughtworks.com]

Hi Paweł,

Cucumber specs can be written in gherkin, which is its DSL. 

But the step definition will be written in Java code, which I think will be more of developer's workload.

Postman would be ok too if it could be incorporated into Jenkins build.

Is there a Postman Collection already? If so, I can try to run that as contract test and see how it goes.

And Cucumber has a JS version as well, we can still use it to write specs and behind the specs there are step definitions filled with postman collections.

The major benefit I see in Cucumber is that it makes it easier for less technical people to see what is being covered by tests.

[Josh Zamor]

Interesting, in considering Cucumber, there are some tests from v1 used with Selenium:

https://github.com/OpenLMIS/open-lmis/tree/master/test-modules/functional-tests/src/test/resources/org/openlmis/functional

I’d be curious to see how well Cucumber makes clear the JSON needed at the endpoints, and what if any overlap there is with example JSON that could/should be in RAML?

Thanks,

Josh

To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/ead88b05-d4f1-4e61-abc4-fde99fd355f1%40googlegroups.com.

[pf...@thoughtworks.com]

I just tried to integrate postman with cucumber.js, it did not go well.

A postman collection is basically a json file that describes the end point under test, the expected response.

{
 "id": "04e3f4e4-1b93-7f3f-0fd7-9af00f57161a",
 "name": "cuke",
 "description": "try to use this in cucumber",
 "order": [
 "d12a2691-e5d4-b83d-a6e2-7fa576c72313"
 ],
 "folders": [],
 "timestamp": 1472192231695,
 "owner": 0,
 "public": false,
 "published": false,
 "requests": [
 {
 "id": "d12a2691-e5d4-b83d-a6e2-7fa576c72313",
 "headers": "Content-Type: application/json\n",
 "url": "http://jsonplaceholder.typicode.com/posts",
 "preRequestScript": "",
 "pathVariables": {},
 "method": "POST",
 "data": [],
 "dataMode": "raw",
 "version": 2,
 "tests": "var jsonData = JSON.parse(responseBody);\ntests[\"has title\"] = jsonData.title === 'foo';",
 "currentHelper": "normal",
 "helperAttributes": {},
 "time": 1472192692688,
 "name": "post to json place holder",
 "description": "send post to json placeholder",
 "collectionId": "04e3f4e4-1b93-7f3f-0fd7-9af00f57161a",
 "responses": [],
 "rawModeData": "{\"title\":\"foo\"}"
 }
 ]
}

It could be ran with newman, the postman cli tool.

Or it could be ran with newman's API, like:

var newman = require('newman');

newman.run({
    collection: require('./some_test_collection.json'),//it could be url here as well
    reporters: 'cli'
}, function (err) {
    if (err) { throw err; }
    console.log('collection run complete!');
});

As we can see here, the smallest granularity of control that newman gives us is a collection, which contains all steps of a test already.

So defining cucumber steps(given, when, then, smaller steps) with postman collection would not work out.

Due to the granularity of control that newman exposes, any BDD framework that requires us to define the setup, the action, the assertion separately will not work with postman.

[Paweł Gesek]

Yes, I agree that Postman and Cucumber are two separate technologies that are not made for each other and I doubt anyone is mixing them currently, so we will have to decide on either one.

FYI there was a sample Postman collection posted in the comments of https://openlmis.atlassian.net/secure/RapidBoard.jspa?rapidView=46&view=detail&selectedIssue=OLMIS-965&sprint=70

Regards,
Paweł

To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/2deb0054-1f1b-412c-a705-594c7213e004%40googlegroups.com.