Custom REST API with content type application/x-www-form-urlencoded

[mgmort mgmort]

I am trying to get data from an API using the REST connection manager. the api returns data as a urlstring - ""key1=value1&key2=value2&key3=value3..."
I wrote a JS function called  urlStringToJson() using Object.fromEntries that converts the string into a JSON object.
Then, in the read endpoint result extraction, i used "{{=[urlStringToJson(response)]}}".

in the test http dialog, i can see that the function is correctly parsing the string into json, but only when i choose content type application/x-www-form-urlencoded.

The log viewer also shows  a Template evaluation result which appears to be correct json, and matches the json sample of the defined fields.

but when i try to get data from the source, i only get nulls. How do i make sure the right content type is being used to extract the response? 

[Ivan Peev]

Hi,

Please inspect with the Fiddler tracing app the response returned by the service. Then post a copy of the response here for review.

[mgmort mgmort]

Here is the raw response from the api (i have randomized some of the data for security):
HTTP/1.1 200 OK
Content-Type: text/html
Connection: close
Date: Sat, 19 Jul 2025 18:35:08 GMT
Strict-Transport-Security: max-age=31536000; includeSubDomains
ICOM: web3b

Id=3104111112&CCode=0&Amount=414&ACode=9993004&TransType=טלפונית&Bank=2&Payments=1&TashType=רגיל&Brand=2&Issuer=2&Tmonth=05&Tyear=2028&L4digit=2468&Coin=1&CardName=Visa&UID=25012345678428258859999&UserId=038111234&ClientLName=אבגדה&ClientName=מאיר&street=רחוב&city=עיר שלי&zip=1234527&phone=&cell=&email=notm...@gmail.com&voucherNumber=12345019

Here is the function I added in Configuration > Script Modules:
function urlStringToJson(urlString) {
    return Object.fromEntries(
        urlString.toString().split('&').map(function(pair) {
            var parts = pair.split('=');
            var key = parts[0];
            var value = parts[1] || '';
            return [decodeURIComponent(key), decodeURIComponent(value)];
        })
    );
}

here is the expression i used for Configuration > Resources > MyEndpoint > Read > Result Extraction:
{{=[urlStringToJson(response)]}}

and here is the XML for the fields I defined in Configuration > Resources > MyEndpoint > Fields:
<Fields>
  <Field Name="Id" Template="ShortText" />
  <Field Name="CCode" Template="ShortText" />
  <Field Name="Amount" Template="ShortText" />
  <Field Name="ACode" Template="ShortText" />
  <Field Name="TransType" Template="ShortText" />
  <Field Name="Bank" Template="ShortText" />
  <Field Name="Payments" Template="ShortText" />
  <Field Name="TashType" Template="ShortText" />
  <Field Name="Brand" Template="ShortText" />
  <Field Name="Issuer" Template="ShortText" />
  <Field Name="Tmonth" Template="ShortText" />
  <Field Name="Tyear" Template="ShortText" />
  <Field Name="L4digit" Template="ShortText" />
  <Field Name="Coin" Template="ShortText" />
  <Field Name="CardName" Template="ShortText" />
  <Field Name="UID" Template="ShortText" />
  <Field Name="UserId" Template="ShortText" />
  <Field Name="ClientLName" Template="ShortText" />
  <Field Name="ClientName" Template="ShortText" />
  <Field Name="street" Template="ShortText" />
  <Field Name="city" Template="ShortText" />
  <Field Name="email" Template="ShortText" />
  <Field Name="voucherNumber" Template="ShortText" />
  <Field Name="firstPayment" Template="ShortText" />
</Fields>

The raw response is from actually running the SSIS package, so I know I am reaching the api and getting the data.
What am I missing to actually load the fields and create data rows in the data flow?

thanks

[Ivan Peev]

In the  urlStringToJson function include on top alert(urlString) . Then execute the package in VS. This should give you a clue what is being received for processing as an input.

You can use alert in your functions in different places to confirm the input data is processed correctly.

[mgmort mgmort]

Playing with alert() and some different functions, i can see that the response is an arraybuffer object, but i cant figure out how to get to its contents.

in fiddler, the response headers show:

HTTP/1.1 200 OK
Content-Type: text/html

Transfer-Encoding: chunked
Connection: close
Date: Mon, 21 Jul 2025 09:57:10 GMT
Strict-Transport-Security: max-age=31536000; includeSubDomains
ICOM: web2b
Content-Encoding: gzip

and there is a yellow button: "Response body is encoded, Click to decode", and only after clicking is the response readable under 'TextView'.

What do i have to add to my result extraction script to perform this decoding on the response?

[mgmort mgmort]

Looking at the connection manager verbose log, I see that maybe the response is returning from the api but not being picked up by the connection manager:
21/07/2025 13:06:36| Template evaluation result: c4f66d2f919b826021f5dd4efbfab7983053431a1f11568e4ac21f96e42b2a42
21/07/2025 13:06:36| Send HTTP request: GET /GetTransDetails
--- Parameter 'Masof': 123456789
--- Parameter 'id': abcdefghi
--- Parameter 'partner': XXXXX
--- Parameter 'type': uuuuuu
--- Parameter 'signature': a-09jaksea-9enav0anvr-0a93eivn-a09vn-0e9vna-09ivna-0fa0-9finva-0s

21/07/2025 13:06:36| Unexpected HTTP response: 397 bytes
21/07/2025 13:06:36| HTTP response data:
21/07/2025 13:06:36| Evaluate template: {{=[urlStringToJson(response)]}}
--- Parameter 'resource': {
  "name": "GetTransDetails"
}

even though fiddler shows a successful response from the same call.

[Ivan Peev]

Okay. The engine is returning the raw buffer. Include the following code in your function:

var Encoding = getClrType("System.Text.Encoding");

urlString = Encoding.UTF8.GetString(urlString);

inspect what urlString looks like now.

[mgmort mgmort]

Great, using Encoding.UTF8.GetString(urlString); before converting the string to JSON solves the problem.

Interestingly, alert(Encoding.UTF8.GetString(urlString)) still shows an empty string, but the data is being processed correctly, so i'm not too worried about it.

Thanks for your help.