Best way to use predicates to match request SOAP Bodies to responses?
1,675 views
Skip to first unread message
Janne Mattila
Feb 9, 2018, 11:04:45 AM2/9/18
to mountebank-discuss
I use mountebank to mock a SOAP web service. I want to match SOAP request bodies to responses. SOAP headers contain noise that breaks matching, so I want to disregard those. For example request RQ1
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:myheader="http://myheader">
<env:Header>
<myheader:time>1517241715</myheader:time>
</env:Header>
<env:Body>
<FooRequest xmlns="http:/foo">
<foo>123</foo>
</FooRequest>
</env:Body>
</env:Envelope>
should result in response RES1
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
<env:Body>
<FooResponse xmlns="http://foo">
<a>Computer 1</a>
</FooResponse>
</env:Body>
</env:Envelope>
and request RQ2
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:myheader="http://myheader">
<env:Header>
<myheader:time>1517241790</myheader:time>
</env:Header>
<env:Body>
<BarRequest xmlns="http:/bar">
<bar>asdasfsa</bar>
</BarRequest>
</env:Body>
</env:Envelope>
in response RES2
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
<env:Body>
<BarResponse xmlns="http://bar">
<b>Tsugu</b>
</BarResponse>
</env:Body>
</env:Envelope>
Q1: seems that I can't use predicateGenerators & xpath to simply conveniently match to the full Body element, since xpathing only works with scalar values, is this still the case? (https://groups.google.com/forum/#!topic/mountebank-discuss/sEMVYeBZ9Xg)
Q2: PredicateGenerators & Except
I used PredicateGenerators with Except regexp to leave SOAP Header out of the matching:
"predicateGenerators": [
{
"matches": {
"body": true
},
"except" : "<env:Header>([\\s\\S]*?)<\/env:Header>"
}
],
this works and returns correct responses for RQ1 and RQ2. However, if RQ1 is repeated multiple times during the recording, and these result in different responses (RES1 = Computer 1, RES 2 = Computer2, ...), only RES1 gets returned. Each request-response pair gets its own stub, and only the first stub is returned when playing back.
Am I using PredicateGenerators & Except correctly, and why aren't all the requests that match the same predicate with "except" stored in same stub?
Q3: PredicateGenerators & XPath
Alternatively I created PredicateGenerator with xpath that identifies enough of the soap body contents to make matching work. For example this matches to all the text content inside the soap body:
"predicateGenerators": [
{
"matches": {
"body": true
},
"xpath": {
"selector": "//*[(normalize-space()) and (text()) and ancestor::soap:Body]",
"ns": {
"soap": "http://schemas.xmlsoap.org/soap/envelope/"
}
}
}
This solution did not have the problem of Q2: for example repeating RQ1 twice resulted in
"predicates": [
{
"xpath": {
"selector": "//*[(normalize-space()) and (text()) and ancestor::soap:Body]",
"ns": {
"soap": "http://schemas.xmlsoap.org/soap/envelope/"
}
},
"deepEquals": {
"body": [
"\n ",
"123"
]
}
}
],
"responses": [
{
"is": { // 2 different responses here...
Adding multiple different xpath selectors I was able to match most request to correct responses while disregarding soap header noise. End result is overly complex though and has some problems e.g. when xpath results in empty body.
Brandon Byars
Feb 9, 2018, 3:23:31 PM2/9/18
to Janne Mattila, mountebank-discuss
Hi Janne,
Thanks for the detailed information in your request. Responses below in red:
Q1: seems that I can't use predicateGenerators & xpath to simply conveniently match to the full Body element, since xpathing only works with scalar values, is this still the case? (https://groups.google.com/forum/#!topic/mountebank-discuss/sEMVYeBZ9Xg)
Yes, that is still the case.
Q2: PredicateGenerators & ExceptI used PredicateGenerators with Except regexp to leave SOAP Header out of the matching:
{
"matches": {
"body": true
},
"except" : "<env:Header>([\\s\\S]*?)<\/env:Header>"
}
],this works and returns correct responses for RQ1 and RQ2. However, if RQ1 is repeated multiple times during the recording, and these result in different responses (RES1 = Computer 1, RES 2 = Computer2, ...), only RES1 gets returned. Each request-response pair gets its own stub, and only the first stub is returned when playing back.Am I using PredicateGenerators & Except correctly, and why aren't all the requests that match the same predicate with "except" stored in same stub?
Can you show the proxy configuration surrounding the predicateGenerators? You'd need to set the mode to proxyAlways to record multiple responses in the same stub.
My attempt at doing this worked. For reference, here's the imposter config file I used (res1.xml and res2.xml corresponded to your response XMLs above):
{
"imposters": [
{
"protocol": "http",
"port": 3001,
"stubs": [{
"responses": [
{ "is": { "body": "<%- stringify(filename, 'res1.xml') %>"} },
{ "is": { "body": "<%- stringify(filename, 'res2.xml') %>"} }
]
}]
},
{
"protocol": "http",
"port": 3000,
"stubs": [{
"responses": [{
"proxy": {
"mode": "proxyAlways",
"to": "http://localhost:3001",
"predicateGenerators": [{
"matches": { "body": true },
"except": "<env:Header>([\\s\\S]*?)<\/env:Header>"
}]
}
}]Then I sent the following set of shell commands (req1.xml and req2.xml correspond to your requests above, the -d@file curl parameter POSTs that file to the url):
}]
}
]
}
MacBook-Pro-5:mountebank:% curl http://localhost:3000/ -d@troubleshooting/req1.xml
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
<env:Body>
<FooResponse xmlns="http://foo">
<a>Computer 1</a>
</FooResponse>
</env:Body>
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
<env:Body>
<BarResponse xmlns="http://bar">
<b>Tsugu</b>
</BarResponse>
</env:Body>
</env:Envelope>%
MacBook-Pro-5:mountebank:% mb replay
MacBook-Pro-5:mountebank:% curl http://localhost:3000/ -d@troubleshooting/req1.xml
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
<env:Body>
<FooResponse xmlns="http://foo">
<a>Computer 1</a>
</FooResponse>
</env:Body>
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
<env:Body>
<BarResponse xmlns="http://bar">
<b>Tsugu</b>
</BarResponse>
</env:Body>
</env:Envelope>%
Agreed this is complex.
Janne Mattila
Feb 12, 2018, 10:31:09 AM2/12/18
to mountebank-discuss
Hi,
this is the configuration I used to setup recording:
{
"port": 8800,
"protocol": "http",
"name": "proxyToProxy",
"stubs": [
{
"responses": [
{
"proxy": {
"to": "http://localhost:8900/",
"predicateGenerators": [
{
"matches": {
"body": true
},
"except" : "<env:Header>([\\s\\S]*?)<\/env:Header>"
}
],
"mode": "proxyAlways"
}
}
]
}
]
}I then had a SoapUI rest mock service running at localhost:8900.
This is the imposters.ejs that resulted from recording:
{
"imposters": [
{
"protocol": "http",
"port": 8800,
"name": "proxyToProxy",
"stubs": [
{
"predicates": [
{
"except": "<env:Header>([\\s\\S]*?)</env:Header>",
"deepEquals": {
"body": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<env:Envelope xmlns:env=\"http://schemas.xmlsoap.org/soap/envelope/\"\n xmlns:myheader=\"http://myheader\">\n <env:Header>\n <myheader:time>1517241715</myheader:time>\n </env:Header>\n <env:Body>\n <FooRequest xmlns=\"http:/foo\">\n <foo>123</foo>\n </FooRequest>\n </env:Body>\n</env:Envelope>\n"
}
}
],
"responses": [
{
"is": {
"statusCode": 200,
"headers": {
"Content-Type": "application/xml",
"Content-Length": 222,
"Server": "Jetty(6.1.26)",
"Connection": "close"
},
"body": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<env:Envelope xmlns:env=\"http://schemas.xmlsoap.org/soap/envelope/\">\n <env:Body>\n <FooResponse xmlns=\"http://foo\"> \n <a>Computer 1</a>\n </FooResponse>\n </env:Body>\n</env:Envelope>",
"_mode": "text"
}
}
]
},
{
"predicates": [
{
"except": "<env:Header>([\\s\\S]*?)</env:Header>",
"deepEquals": {
"body": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<env:Envelope xmlns:env=\"http://schemas.xmlsoap.org/soap/envelope/\"\n xmlns:myheader=\"http://myheader\">\n <env:Header>\n <myheader:time>1517241716</myheader:time>\n </env:Header>\n <env:Body>\n <FooRequest xmlns=\"http:/foo\">\n <foo>123</foo>\n </FooRequest>\n </env:Body>\n</env:Envelope>\n"
}
}
],
"responses": [
{
"is": {
"statusCode": 200,
"headers": {
"Content-Type": "application/xml",
"Content-Length": 225,
"Server": "Jetty(6.1.26)",
"Connection": "close"
},
"body": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<env:Envelope xmlns:env=\"http://schemas.xmlsoap.org/soap/envelope/\">\n <env:Body>\n <FooResponse xmlns=\"http://foo\"> \n <a>Computer 2000</a>\n </FooResponse>\n </env:Body>\n</env:Envelope>",
"_mode": "text"
}
}
]
},
{
"predicates": [
{
"except": "<env:Header>([\\s\\S]*?)</env:Header>",
"deepEquals": {
"body": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<env:Envelope xmlns:env=\"http://schemas.xmlsoap.org/soap/envelope/\"\n xmlns:myheader=\"http://myheader\">\n <env:Header>\n <myheader:time>1517241790</myheader:time>\n </env:Header>\n <env:Body>\n <BarRequest xmlns=\"http:/bar\">\n <bar>asdasfsa</bar>\n </BarRequest>\n </env:Body>\n</env:Envelope>\n"
}
}
],
"responses": [
{
"is": {
"statusCode": 200,
"headers": {
"Content-Type": "application/xml",
"Content-Length": 217,
"Server": "Jetty(6.1.26)",
"Connection": "close"
},
"body": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<env:Envelope xmlns:env=\"http://schemas.xmlsoap.org/soap/envelope/\">\n <env:Body>\n <BarResponse xmlns=\"http://bar\"> \n <b>Tsugu</b>\n </BarResponse>\n </env:Body>\n</env:Envelope>",
"_mode": "text"
}
}
]
}
]
}
]
}
Noteworthy here is that those two FooRequests are stored in separate stubs, even though I would like them to be one FooRequest and two responses to that.
Janne Mattila
Feb 14, 2018, 3:52:15 AM2/14/18
to mountebank-discuss
Hi,
finally had time to go through your example. Yes it works when matching req1 to resp1 and req2 to resp2, but it still fails to group multiple req1's into same stub. I might have described the use case(s) so that its a bit hard to follow.
Let's say that we have the req1.xml with the original timestamp 1517241715 (which is "excepted" from predicateMatching). And in addition to that we have another req1 ("req1b") with a different timestamp, which produces a different response (mocked system has state which has been modified meanwhile).
req1b.xml:
<myheader:time>1517241722</myheader:time>
</env:Header>
<env:Body>
<FooRequest xmlns="http:/foo">
<foo>123</foo>
</FooRequest>
</env:Body>
</env:Envelope>
Now when we record with req1.xml and req1b.xml AND those return different responses, we would like to receive corresponding responses in the playback. But only the resp1.xml gets returned for both requests, since those request are stored in separate stubs even though they represent the same stub when it comes to matching.
You can use your existing test setup with curl commands by creating req1b.xml and imagining that res1.xml and res2.xml should be responses to req1.xml and req1b.xml this time.
/ # curl http://localhost:3000 -d @req1.xml
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
<env:Body>
<FooResponse xmlns="http://foo">
<a>Computer 1</a>
</FooResponse>
</env:Body>
</env:Envelope>
/ # curl http://localhost:3000 -d @req1b.xml
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
<env:Body>
<BarResponse xmlns="http://bar">
<b>Tsugu</b>
</BarResponse>
</env:Body>
</env:Envelope>
/ # mb replay
/ # curl http://localhost:3000 -d @req1.xml
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
<env:Body>
<FooResponse xmlns="http://foo">
<a>Computer 1</a>
</FooResponse>
</env:Body>
</env:Envelope>
/ # curl http://localhost:3000 -d @req1b.xml
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
<env:Body>
<FooResponse xmlns="http://foo">
<a>Computer 1</a>
</FooResponse>
</env:Body>
</env:Envelope>
and the recorded imposters have req1 and req1b in different stubs (and req1 only ever gets to be used)
/ # curl localhost:2525/imposters?replayable=true
{
"imposters": [
{
"protocol": "http",
"port": 3000,
"stubs": [
{
"predicates": [
{
"except": "<env:Header>([\\s\\S]*?)</env:Header>",
"deepEquals": {
"body": "<env:Envelope xmlns:env=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:myheader=\"http://myheader\"> <env:Header> <myheader:time>1517241715</myheader:time> </env:Header> <env:Body> <FooRequest xmlns=\"http:/foo\"> <foo>123</foo> </FooRequest> </env:Body></env:Envelope>"
}
}
],
"responses": [
{
"is": {
"statusCode": 200,
"headers": {
"Connection": "close",
"Date": "Wed, 14 Feb 2018 08:33:16 GMT",
"Transfer-Encoding": "chunked"
},
"body": "<env:Envelope xmlns:env=\"http://schemas.xmlsoap.org/soap/envelope/\">\n <env:Body>\n <FooResponse xmlns=\"http://foo\">\n <a>Computer 1</a>\n </FooResponse>\n </env:Body>\n</env:Envelope>",
"_mode": "text"
}
}
]
},
{
"predicates": [
{
"except": "<env:Header>([\\s\\S]*?)</env:Header>",
"deepEquals": {
"body": "<env:Envelope xmlns:env=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:myheader=\"http://myheader\"> <env:Header> <myheader:time>1517241722</myheader:time> </env:Header> <env:Body> <FooRequest xmlns=\"http:/foo\"> <foo>123</foo> </FooRequest> </env:Body></env:Envelope>"
}
}
],
"responses": [
{
"is": {
"statusCode": 200,
"headers": {
"Connection": "close",
"Date": "Wed, 14 Feb 2018 08:33:18 GMT",
"Transfer-Encoding": "chunked"
},
"body": "<env:Envelope xmlns:env=\"http://schemas.xmlsoap.org/soap/envelope/\">\n <env:Body>\n <BarResponse xmlns=\"http://bar\">\n <b>Tsugu</b>\n </BarResponse>\n </env:Body>\n</env:Envelope>",
"_mode": "text"
}
}
]
}
]
},...
perjantai 9. helmikuuta 2018 22.23.31 UTC+2 Brandon Byars kirjoitti:
Brandon Byars
Feb 17, 2018, 11:46:23 AM2/17/18
to Janne Mattila, mountebank-discuss
OK, thanks for the clarification. I did reproduce your issue.
The problem seems to be that the "except" parameter doesn't use multiline regex for its replacement, so the pattern doesn't actually exclude the header. This does mean that you are correct in your original assessment that the only currently supported way that solves your problem is to make more complex predicateGenerators.
I'm actually quite thankful for you bringing this problem up. I had been contemplating deprecating the "except" parameter (as in removing the docs, not the code) since the matches predicate does the same job. Your use case shows that there still is a need for using it for proxy recording since there's no obvious way to incorporate the matches predicate there. I'm reluctant to simply change the regex to multiline as that's likely a breaking change for someone, but will play with other options, maybe adding a new parameter (exceptMultiline).
-Brandon
--
You received this message because you are subscribed to the Google Groups "mountebank-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mountebank-discuss+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Janne Mattila
Feb 19, 2018, 2:05:53 AM2/19/18
to mountebank-discuss
Thanks Brandon! I hope you can find a suitable solution (be it exceptMultiline, XPath matching for non-scalar values, or whatever...) since I do think there's a real use case for this feature.
Reply all
Reply to author
Forward
0 new messages