POST http request issue.

[Denis Putnam]

I am getting the error: [renew-jwt-token.js] makeTokenRenewalRequest(): responseData={"error_description":"Invalid client or client credentials.","error":"invalid_client"}

Can someone tell me what I am doing wrong to get this error?  Does the body_str have to go through encoding that I am unaware of?

The body_string is in bold and I write it to the outputStream with:

 outputStream.writeBytes(getBytes(body_str));

var HttpURLConnection = Java.type("java.net.HttpURLConnection");
var BufferedReader = Java.type("java.io.BufferedReader");
var InputStreamReader = Java.type("java.io.InputStreamReader");
var URL = Java.type("java.net.URL");

var body_str = "grant_type="+grantType+"&username="+userName+"&password="+pwd;
  var auth = client_id + ":" + secret;

  var encodedAuth = base64EncodeBytes(getBytes(auth));
 encodedAuth="+JSON.stringify(encodedAuth));

  //var encodedBody = base64EncodeBytes(getBytes(body_str));
  var encodedBody = encodeURIComponent(body_str);
encodedBody="+JSON.stringify(encodedBody));


  var url = new URL(ping_url);
  logger("makeTokenRenewalRequest(): url="+url);
  var connection = url.openConnection();

  connection.setRequestMethod("POST")
  connection.setRequestProperty('Content-Type', 'application/x-www-form-urlencoded');
  connection.setRequestProperty('cache-control', 'no-cache');
  connection.setRequestProperty('Authorization', 'Basic '+ encodedAuth);
  connection.setDoOutput(true);
 

  var outputStream = connection.getOutputStream();
 body_str="+encodeURIComponent(body_str));
  outputStream.writeBytes(getBytes(body_str));

[Simon Bennetts]

Some context would be very useful here :)

What ae you trying to achieve?

Where are you running this script?

Is it a ZAP script, and if so which type?

Can you compare the request you generate with a known good request?

Cheers,

Simon

[Denis Putnam]

Hi Simon,

I am trying to execute an httpsender javascript that will renew a jwt token after it expires so that the ascan will be able to run the penetration scan for a restful api.

So the Basic Authorization generates the correct encodedAuth value that I have compared to our get_jwt() library function in python.   I set up the 'body_str' with the grant_type, username, and password, but I get the  responseData={"error_description":"Invalid client or client credentials.","error":"invalid_client"} when I print the responseData.

The ping_url is going out to our internal url to get the jwt token just like it does in python.

Do you have any clue as to why the http post request fails with the invalid client or client credentials?  Am I setting up the 'body_str" correctly?  Am I calling the outputStream.write properly?

Sincerely,

Denis

[Simon Bennetts]

Hi Denis,

Ah, in that case you shouldnt be using those methods - they are too low level.

This script does something a bit similar: https://github.com/zaproxy/community-scripts/blob/main/targeted/ElasticSearchExploit.js

Those ZAP classes will do a lot more for you than the ones you are using.

Does that help?

Cheers,

Simon

[Simon Bennetts]

Note that you dont need to create an HttpSender in your case - you can use helper.getHttpSender()

[Denis Putnam]

Hi Simon,

Is there a 

var uri=httpRequestHeader.setURI()

Can you point me to the document(s) the describe what is in the 'msg' argument?

I see these, but I don't think it's everything I need.  https://javadoc.io/doc/org.zaproxy/zap/latest/org/parosproxy/paros/network/HttpSender.html

Sincerely,

Denis

[thc...@gmail.com]

https://javadoc.io/doc/org.zaproxy/zap/latest/org/parosproxy/paros/network/HttpRequestHeader.html

>>>> 'body_str' with the grant_type, username, and password, but I get the *responseData={"error_description":"Invalid
>>>> client or client credentials.","error":"invalid_client"} *when I print

>>>> the responseData.
>>>>
>>>> The ping_url is going out to our internal url to get the jwt token just
>>>> like it does in python.
>>>>
>>>> Do you have any clue as to why the http post request fails with the
>>>> invalid client or client credentials? Am I setting up the 'body_str"
>>>> correctly? Am I calling the outputStream.write properly?
>>>>
>>>> Sincerely,
>>>> Denis
>>>>
>>>> On Thursday, June 1, 2023 at 4:04:42 AM UTC-4 psi...@gmail.com wrote:
>>>>
>>>>> Some context would be very useful here :)
>>>>> What ae you trying to achieve?
>>>>> Where are you running this script?
>>>>> Is it a ZAP script, and if so which type?
>>>>> Can you compare the request you generate with a known good request?
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Simon
>>>>>
>>>>> On Wednesday, 31 May 2023 at 21:01:26 UTC+1 denis...@gmail.com wrote:
>>>>>

>>>>>> I am getting the error: *[renew-jwt-token.js]
>>>>>> makeTokenRenewalRequest(): responseData={"error_description":"Invalid
>>>>>> client or client credentials.","error":"invalid_client"}*

>>>>>>
>>>>>> Can someone tell me what I am doing wrong to get this error? Does the
>>>>>> body_str have to go through encoding that I am unaware of?
>>>>>>
>>>>>> The body_string is in bold and I write it to the outputStream with:

>>>>>> * outputStream.writeBytes(getBytes(body_str));*

>>>>>>
>>>>>> var HttpURLConnection = Java.type("java.net.HttpURLConnection");
>>>>>> var BufferedReader = Java.type("java.io.BufferedReader");
>>>>>> var InputStreamReader = Java.type("java.io.InputStreamReader");
>>>>>> var URL = Java.type("java.net.URL");

>>>>>> *var body_str =
>>>>>> "grant_type="+grantType+"&username="+userName+"&password="+pwd;*

>>>>>> var auth = client_id + ":" + secret;
>>>>>>
>>>>>> var encodedAuth = base64EncodeBytes(getBytes(auth));
>>>>>> encodedAuth="+JSON.stringify(encodedAuth));
>>>>>>
>>>>>> //var encodedBody = base64EncodeBytes(getBytes(body_str));
>>>>>> var encodedBody = encodeURIComponent(body_str);
>>>>>> encodedBody="+JSON.stringify(encodedBody));
>>>>>>
>>>>>>
>>>>>> var url = new URL(ping_url);
>>>>>> logger("makeTokenRenewalRequest(): url="+url);
>>>>>> var connection = url.openConnection();
>>>>>>
>>>>>> connection.setRequestMethod("POST")
>>>>>> connection.setRequestProperty('Content-Type',
>>>>>> 'application/x-www-form-urlencoded');
>>>>>> connection.setRequestProperty('cache-control', 'no-cache');
>>>>>> connection.setRequestProperty('Authorization', 'Basic '+
>>>>>> encodedAuth);
>>>>>> connection.setDoOutput(true);
>>>>>>
>>>>>>
>>>>>> var outputStream = connection.getOutputStream();
>>>>>> body_str="+encodeURIComponent(body_str));

>>>>>> * outputStream.writeBytes(getBytes(body_str));*
>>>>>>
>>>>>
>

[Denis Putnam]

Thank you.

I found it via https://www.javadoc.io/doc/org.zaproxy/zap/latest/org/parosproxy/paros/network/HttpMessage.html

[Denis Putnam]

Hi,

I have pasted the entirety of my code for the httpsender javascript.

Why is the Authorization and body data showing in the zap gui in the request tab.

The message.getRequestBody() returns {"error_description":"grant_type is required","error":"invalid_request"}.  Why is this if I have set it?

Does the body format look correct?  Do I need to convert it to bytes?  Do I need to use this format?  var body_str = "grant_type=" + grantType + "&username=" + userName + "&password=" + pwd;

Please set me on the right path. :)

Sincerely,

Denis

// Check if the token is expired (you'll need to implement your own logic here) const isExpired = isTokenExpired();
var HttpSender = Java.type("org.parosproxy.paros.network.HttpSender")
var ScriptVars = Java.type("org.zaproxy.zap.extension.script.ScriptVars");

var URI = Java.type("org.apache.commons.httpclient.URI")

var message = null;
var request_initiator;

PARAMETER_VARIABLE = "jwt_data";
jwt_data = null;
username = null;
password = null;
client_id = null;
secret = null;
ping_url = null;
var myScriptVars = {
  token: "NOT_SET"
};


// Logging with the script name is super helpful!
function logger() {
print('[' + this['zap.script.name'] + '] ' + arguments[0]);
}

// Parse and store headers where we can get at them quickly
function initializeJwtData(variableName) {
  logger("Initializing the jwt data..");
  jwt_data = JSON.parse(ScriptVars.getGlobalVar(variableName));
logger("InitializeJwtData(): jwt_data="+JSON.stringify(jwt_data));
  username = jwt_data.username;
  logger("initilizeJwtData(): username="+username);
  password = jwt_data.password;
  client_id = jwt_data.client_id;
  logger("initilizeJwtData(): client_id="+client_id);
  secret = jwt_data.secret;
  logger("initilizeJwtData(): secret="+secret);
  ping_url = jwt_data.ping_url;
  //ping_url = jwt_data.ping_url.substring(0, jwt_data.ping_url.length - 1);
  logger("initilizeJwtData(): ping_url="+ping_url);
  var token = renewToken()
  myScriptVars = {
    token: token
  };
}

/*
 * Processes messages by adding user-specified headers (overwriting original
 * values if header already exists). This may be pointless for some initiators
 * (CHECK_FOR_UPDATES) and redundant for others (FUZZER).
 *
 * Called before forwarding the message to the server.
 *
 * @param {HttpMessage} msg - The message that will be forwarded to the server.
 * @param {int} initiator - The initiator that generated the message.
 * @param {HttpSenderScriptHelper} helper - A utility object with helper functions.
 */
function sendingRequest(msg, initiator, helper) {
  logger("sendingRequest() called...");
  logger("sendingRequest() msg="+msg);
  message = msg;
  request_initiator = initiator
 
  if (jwt_data === null) {
    initializeJwtData(PARAMETER_VARIABLE);
  }
 
  message.getRequestHeader().setHeader('Content-type', 'application/json');
  message.getRequestHeader().setHeader('x-abc-jws-token', getScriptVar('token'));
  message.getRequestHeader().setHeader('x-abc-channel', 'api');
  message.getRequestHeader().setHeader('x-abc-api-type', 'private');
}

/* Called after receiving the response from the server.
 *
 * @param {HttpMessage} msg - The message that was forwarded to the server.
 * @param {int} initiator - The initiator that generated the message.
 * @param {HttpSenderScriptHelper} helper - A utility object with helper functions.
 */
function responseReceived(msg, initiator, helper) {
// Nothing to do here
}

function makeTokenRenewalRequest() {
  logger("makeTokenRenewalRequest(): Called...");
  //var body_str = "{ \"grant_type\": \"password\",\"username\":\""+username+"\",\"password\":\""+password+"\" }";
  var grantType = "password";
  var userName = username;
  var pwd = password;
  //var body_str = "grant_type=" + grantType + "&username=" + userName + "&password=" + encodeURIComponent(pwd);
  //var body_str = "grant_type=" + grantType + "&username=" + userName + "&password=" + pwd;

  var auth = client_id + ":" + secret;

  logger("makeTokenRenewalRequest(): DEBUG0 auth bytes="+getBytes(auth));
  // Encode the auth string.
  var encodedAuth = base64EncodeBytes(getBytes(auth));
  logger("maketTokenRenewalRequest(): DEBUG1 encodedAuth="+JSON.stringify(encodedAuth));
  var authorization = 'Basic '+encodedAuth;

  // Set the headers for the jwt request.
  var httpRequestHeader = message.getRequestHeader()
  httpRequestHeader.setHeader('Content-type', 'application/x-www-form-urlencoded');
  httpRequestHeader.setHeader('cache-control', 'no-cache');
  httpRequestHeader.setHeader('Authorization', authorization);

  // Set the payload/body of the request.
  var body_str = '{"grant_type":' + grantType + '"username":' + userName + '"password":' + pwd + '}';
  message.getRequestBody().setBody(body_str);
  httpRequestHeader.setContentLength(message.getRequestBody().length());

  // Set the request method.
  httpRequestHeader.setMethod("POST");

  // Create the uri with the ping url for the jwt service.
  var uri = new URI(ping_url);
  logger("maketTokenRenewalRequest(): DEBUG1.5 encodedAuth="+JSON.stringify(encodedAuth));
  httpRequestHeader.setURI(uri);

  logger("makeTokeRenewalRequest(): request_initiator="+request_initiator)
  logger("makeTokeRenewalRequest(): MANUAL_REQUEST_INITIATOR="+HttpSender.MANUAL_REQUEST_INITIATOR)
  var sender = new HttpSender(request_initiator);
  //var sender = new HttpSender(HttpSender.MANUAL_REQUEST_INITIATOR);

  sender.sendAndReceive(message)

  requestBody = message.getResponseBody();

  logger("makeTokenRenewalRequest(): DEBUG2 requestBody="+requestBody);


  var responseData = JSON.parse(requestBody);
  logger("makeTokenRenewalRequest(): responseData="+JSON.stringify(responseData));

  // None of this logic works at this time.
  var renewedToken
  renewedToken = responseData.access_token;
  logger("makeTokenRenewalRequest(): responseData.access_token="+responseData.access_token);
  logger("makeTokenRenewalRequest(): renewedToken="+renewedToken);
  if ( renewedToken == null) {
    renewedToken = "Unable to set the JWT token. "+JSON.stringify(responseData)
  }

  myScriptVars = {
    token: renewedToken
  }
  logger("makeTokenRenewalRequest(): returning="+renewedToken);
  return renewedToken;
}

// Function to check if the token is expired
function isTokenExpired() {
  // Retrieve the token from myScriptVars
  var currentToken = getScriptVar('token');
  if (currentToken === "") {
    return true;
  }

  // Implement your own logic to determine token expiration
  // Return true if the token is expired, false otherwise
  // For example:
  var decodedToken = decodeJWT(currentToken);
  logger("isTokenExpired(): decodedToken="+decodedToken);
  var currentTimestamp = Math.floor(Date.now() / 1000);
  return decodedToken.exp < currentTimestamp;
}

// Function to renew the token
function renewToken() {
  // Implement your own logic to renew the token
  // This can involve making an API request to a token renewal endpoint
  // For example:
  logger("renewToken(): Called...");
  var renewedToken = makeTokenRenewalRequest();


  return renewedToken;
}

// Function to decode a JWT token (you may need to import a JWT library for this)
function decodeJWT(token) {
  // Implement your own logic to decode the JWT token
  // You can use a JWT library or decode it manually
  // For example:
  //const decodedToken = decode(token);
  logger("decodeJWT(): Called...");
  var decodedToken = decodeURIComponent(token);
  logger("decodeJWT(): returning decodedToken="+decodedToken);
  return decodedToken;
}

function getBytes(str) {
  var bytes = [];

  for (var i = 0; i < str.length; i++) {
    var charCode = str.charCodeAt(i);

    if (charCode < 0x80) {
      bytes.push(charCode);
    } else if (charCode < 0x800) {
      bytes.push(0xc0 | (charCode >> 6));
      bytes.push(0x80 | (charCode & 0x3f));
    } else if (charCode < 0x10000) {
      bytes.push(0xe0 | (charCode >> 12));
      bytes.push(0x80 | ((charCode >> 6) & 0x3f));
      bytes.push(0x80 | (charCode & 0x3f));
    } else {
      bytes.push(0xf0 | (charCode >> 18));
      bytes.push(0x80 | ((charCode >> 12) & 0x3f));
      bytes.push(0x80 | ((charCode >> 6) & 0x3f));
      bytes.push(0x80 | (charCode & 0x3f));
    }
  }

  return bytes;
}

function base64EncodeBytes(bytes) {
  var base64Chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=";
  var base64String = "";

  for (var i = 0; i < bytes.length; i += 3) {
    var byte1 = bytes[i];
    var byte2 = bytes[i + 1];
    var byte3 = bytes[i + 2];

    var charIndex1 = byte1 >> 2;
    var charIndex2 = ((byte1 & 3) << 4) | (byte2 >> 4);
    var charIndex3 = ((byte2 & 15) << 2) | (byte3 >> 6);
    var charIndex4 = byte3 & 63;

    base64String += base64Chars.charAt(charIndex1);
    base64String += base64Chars.charAt(charIndex2);
    base64String += base64Chars.charAt(charIndex3);
    base64String += base64Chars.charAt(charIndex4);
  }

  // Handle padding if the number of bytes is not divisible by 3
  var padding = bytes.length % 3;
  if (padding === 1) {
    var lastByte = bytes[bytes.length - 1];
    var charIndex1 = lastByte >> 2;
    var charIndex2 = (lastByte & 3) << 4;

    base64String += base64Chars.charAt(charIndex1);
    base64String += base64Chars.charAt(charIndex2);
    base64String += "==";
  } else if (padding === 2) {
    var secondLastByte = bytes[bytes.length - 2];
    var lastByte = bytes[bytes.length - 1];
    var charIndex1 = secondLastByte >> 2;
    var charIndex2 = ((secondLastByte & 3) << 4) | (lastByte >> 4);
    var charIndex3 = (lastByte & 15) << 2;

    base64String += base64Chars.charAt(charIndex1);
    base64String += base64Chars.charAt(charIndex2);
    base64String += base64Chars.charAt(charIndex3);
    base64String += "=";
  }

  return base64String;
}


// Function to get a ScriptVar value
function getScriptVar(varName) {
  logger("getScriptVar(): Called...");
  // Retrieve the value from myScriptVars
  var ltoken;
  ltoken = renewToken()
  myScriptVars = {
    token: ltoken
  }
  logger("getScriptVar(): ltoken="+ltoken);
  logger("getScriptVar(): returning="+myScriptVars[varName]);
  return myScriptVars[varName];
}

The Request tab contents:

POST https://fmsso-devl-api-int.fanniemae.com/as/token.oauth2 HTTP/1.1
Host: fmsso-devl-api-int.fanniemae.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0
Pragma: no-cache
cache-control: no-cache
Accept: application/json
Content-type: application/x-www-form-urlencoded
x-fnma-api-type: private
x-fnma-jws-token: Unable to set the JWT token. {"error_description":"grant_type is required","error":"invalid_request"}
x-fnma-channel: api
Content-Length: 77
Authorization: Basic c3NvLXAyeC1GMTMtZ...
{"grant_type":password"username":f13extd"password":secretpwd}

[Denis Putnam]

So I changed the body_str to   var body_str = "grant_type=" + grantType + "&username=" + userName + "&password=" + pwd;

Now I am getting {"error_description":"Invalid client or client credentials.","error":"invalid_client"}

This is actually an improvement. :)

So I looked at our python code for get_jwt() and this is the lines of code that sets up the authorization.

            encoded_auth_bytes = base64.b64encode(auth.encode("utf-8"))
            encoded_auth_str = str(encoded_auth_bytes, "utf-8")
            authorization = "Basic " + encoded_auth_str

The encoded_auth_str is 3 characters longer than what I get in the javascript code.

Is there equivalent Java.type() libraries that I can use in my javascript to make sure they are the same?  I think in need the encode('utf-8") function.

Sincerely,

Denis

[Denis Putnam]

Ok, I decided to pass in the encodedAuth string from my python code and now it works!

So the only question I have left is why do I see:

Authorization: Basic c3NvLXAyeC1GMTMt...

grant_type=password&username=f13extd&password=secret_pwd

in the Request window of the ZAP Gui?

Sincerely,

Denis.