OpenAPI schema specifies a JSON array of objects, but ZAP sends a string

[Jernej Rus]

Hello. Some of our website's OpenAPI endpoints specify that the request body be a JSON array of objects. Abridged example:

"/subscribers/api/v2/": {
  "post": {
    "tags": [
      "Subscribers API"
    ],
    "summary": "Creates list of subscribers",
    "description": "Creates list of subscribers",
    "operationId": "Create subscribers",
    "requestBody": {
      "content": {
        "application/json": {
          "schema": {
            "type": "array",
            "items": {
              "$ref": "#/components/schemas/Subscriber"
            }
          }
        }
      },
      "required": true
    },
    "responses": {
      "200": {
        "description": "OK",
        "content": {
          "application/json": {
            "schema": {
              "$ref": "#/components/schemas/Report"
            }
          }
        }
      },
      "500": {
        "description": "Internal Server Error",
        "content": {
          "application/json": {
            "schema": {
              "$ref": "#/components/schemas/ErrorMessage"
            }
          }
        }
      }
    },
    "security": [
      {
        "subscriber-api": []
      }
    ]
  },
}

However, ZAP sends a string:

This causes the endpoints to respond with 400 Bad Request:

ZAP also logs a warning message. Example (with sensitive information masked):

1810763 [ZAP-ActiveScanner-16] WARN  org.parosproxy.paros.core.scanner.VariantJSONQuery - Failed to parse the request body for url [MASKED]/subscriber-api/subscribers/api/v2/ : Input is invalid JSON; does not start with '{' or '[', c=34
java.lang.IllegalArgumentException: Input is invalid JSON; does not start with '{' or '[', c=34
at org.parosproxy.paros.core.scanner.VariantJSONQuery.parseObject(VariantJSONQuery.java:139) ~[zap-2.16.1.jar:2.16.1]
at org.parosproxy.paros.core.scanner.VariantJSONQuery.parseContent(VariantJSONQuery.java:89) ~[zap-2.16.1.jar:2.16.1]
at org.parosproxy.paros.core.scanner.VariantAbstractRPCQuery.setRequestContent(VariantAbstractRPCQuery.java:148) ~[zap-2.16.1.jar:2.16.1]
at org.parosproxy.paros.core.scanner.VariantAbstractRPCQuery.setMessage(VariantAbstractRPCQuery.java:60) [zap-2.16.1.jar:2.16.1]
at org.parosproxy.paros.core.scanner.AbstractAppParamPlugin.scan(AbstractAppParamPlugin.java:91) [zap-2.16.1.jar:2.16.1]
at org.parosproxy.paros.core.scanner.AbstractPlugin.run(AbstractPlugin.java:391) [zap-2.16.1.jar:2.16.1]
at java.base/java.lang.Thread.run(Thread.java:840) [?:?]

I found no errors in our OpenAPI schemas. Does ZAP know how to send an array in the body, or is this a bug in ZAP?

[Simon Bennetts]

Unfortunately that example doesnt work because it is too cut down, even if I provide the correct wrapper.

I've just asked ChatGPT to generate a test case for me, and ZAP seemed to handle it fine.

Can you try with the latest version of the OpenAPI add-on - we released that yesterday and it has support for a load of 3.1 features.

If it still doesnt work for you can you create a simple but complete example which fails?

LLMs seem to be good at doing that, so just try different prompts :)

Cheers,

Simon

[Jernej Rus]

Hello. Sorry for the late reply. The offending requests are sent by ZAP when spidering our API during the openapi jobs. Then the activeScan job tries to use/parse the requests and logs WARN messages.

If I use the latest version of the add-on, ZAP still sends the malformed requests, but activeScan logs different kinds of error messages:

1179050 [ZAP-ActiveScanner-5] ERROR org.zaproxy.zap.extension.jwt.attacks.PayloadAttack - Payload is not a valid JSON Object
org.json.JSONException: A JSONObject text must begin with '{' at 1 [character 2 line 1]
at org.json.JSONTokener.syntaxError(JSONTokener.java:507)
at org.json.JSONObject.<init>(JSONObject.java:222)
at org.json.JSONObject.<init>(JSONObject.java:406)
at org.zaproxy.zap.extension.jwt.attacks.PayloadAttack.executeNullByteAttack(PayloadAttack.java:81)
at org.zaproxy.zap.extension.jwt.attacks.PayloadAttack.executeAttack(PayloadAttack.java:107)
at org.zaproxy.zap.extension.jwt.attacks.ServerSideAttack.execute(ServerSideAttack.java:86)
at org.zaproxy.zap.extension.jwt.JWTActiveScanRule.performAttackServerSideConfigurations(JWTActiveScanRule.java:161)
at org.zaproxy.zap.extension.jwt.JWTActiveScanRule.scan(JWTActiveScanRule.java:125)
at org.parosproxy.paros.core.scanner.AbstractAppParamPlugin.scan(AbstractAppParamPlugin.java:233)
at org.parosproxy.paros.core.scanner.AbstractAppParamPlugin.scan(AbstractAppParamPlugin.java:136)
at org.zaproxy.zap.extension.jwt.JWTActiveScanRule.scan(JWTActiveScanRule.java:93)
at org.parosproxy.paros.core.scanner.AbstractAppParamPlugin.scan(AbstractAppParamPlugin.java:96)
at org.parosproxy.paros.core.scanner.AbstractPlugin.run(AbstractPlugin.java:402)
at java.base/java.lang.Thread.run(Thread.java:840)

Here is a less-abridged example:

"post": {
  "tags": [
    "Subscribers API"
  ],
  "summary": "Creates list of subscribers",
  "description": "Creates list of subscribers",

  "operationId": "CreateSubscribers",

...

"Subscriber": {
  "type": "object",
  "properties": {
    "supiType": {
      "type": "string"
    },
    "supi": {
      "type": "string",
      "example": "311480143456796"
    },
    "plmnId": {
      "type": "string",
      "example": "31148"
    },
    "id": {
      "type": "integer",
      "format": "int32",
      "readOnly": true
    },
    "opKey": {
      "type": "string",
      "example": "01 12 14 88 35 05 29 36 25 36 71 79 58 32 13 63"
    },
    "opcKey": {
      "type": "string",
      "example": "32 67 17 53 41 28 42 08 11 74 80 32 74 94 64 71"
    },
    "authKey": {
      "type": "string",
      "example": "58 66 42 27 97 60 53 23 71 31 96 11 13 90 56 16"
    },
    "amfVal": {
      "type": "string",
      "example": "54 25"
    },
    "sqnVal": {
      "type": "string",
      "example": "32 42 44 01 48 10"
    },
    "ciphering": {
      "type": "string",
      "example": "NEA0"
    },
    "integrity": {
      "type": "string",
      "example": "NIA0"
    },
    "deviceName": {
      "type": "string"
    },
    "firstName": {
      "type": "string"
    },
    "lastName": {
      "type": "string"
    },
    "homeAddress": {
      "type": "string"
    },
    "city": {
      "type": "string"
    },
    "created": {
      "type": "integer",
      "format": "int64",
      "readOnly": true
    },
    "modified": {
      "type": "integer",
      "format": "int64",
      "readOnly": true
    },
    "gpsi": {
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "profile": {
      "type": "string"
    },
    "active": {
      "type": "boolean"
    },
    "status": {
      "type": "string"
    },
    "templateType": {
      "$ref": "#/components/schemas/TemplateType"
    },
    "encPermanentKey": {
      "type": "string"
    },
    "encOpcKey": {
      "type": "string"
    },
    "dnnConfiguration": {
      "type": "object",
      "additionalProperties": {
        "$ref": "#/components/schemas/DnnConfiguration"
      }
    },
    "traceData": {
      "$ref": "#/components/schemas/SubscriberInfoTraceData"
    }
  }
}

I'm guessing ZAP code provides the value "John Doe" in https://github.com/zaproxy/zap-extensions/blob/239af836e33e2fbcff1b6347a5d8fbde48553ebe/addOns/openapi/src/main/java/org/zaproxy/zap/extension/openapi/generators/DataGenerator.java#L58 Does the code contain an error?

[Simon Bennetts]

Sorry, fragments are of no use.

Please supply a cut down but full example, e.g. one which is reported as valid in something like https://apinotes.io/openapi-validator

In theory I should be able to work out a full example from what you have supplied, but I tried a couple of quick attempts and they failed, so its dropped right down my list.

Cheers,

Simon

[Jernej Rus]

Hello. File example_01.txt contains a correct request that I got from our Swagger UI. I'm sending a file to preserve indentation. As mentioned previously, ZAP should send such a request, but sends "John Doe" instead.

The website that you linked confirms our schema is valid:

[Jernej Rus]

More examples from other endpoints.

* Example 02

Schema:
"post": {

  "tags": [
    "Subscribers API"
  ],
  "summary": "Get list of subscribers",
  "description": "Return subscribers with same supi as passed",
  "operationId": "getSubscribers_1",

  "requestBody": {
    "content": {
      "application/json": {
        "schema": {
          "type": "array",
          "items": {

            "type": "string"
          }
        }
      }
    },
    "required": true
  }

Swagger example:

[ "string" ]

ZAP sends:

"John Doe"

* Example 03
Schema:

"post": {
  "tags": [
    "Dumps"
  ],
  "summary": "Generate and store dumps for multiple datasources",
  "operationId": "generateAndStoreMultipleDumps",
  "requestBody": {
    "content": {
      "application/json": {
        "schema": {
          "$ref": "#/components/schemas/MultiDumpRequestDto"
        }
      }
    },
 ...

"MultiDumpRequestDto": {
  "type": "object",
  "properties": {
    "datasourceIds": {

      "type": "array",
      "items": {
        "type": "string"
      }
    }
  }
}

Swagger example:

{ "datasourceIds": [ "string" ] }

ZAP sends:

{"datasourceIds":"John Doe"}

[Simon Bennetts]

Thats weird, I'm getting errors when trying to import the first file.

It also fails https://apinotes.io/openapi-validator with "Unable to determine OpenAPI/Swagger version. Must include "openapi" or "swagger" field."

[Jernej Rus]

Hello. I validated a schema. File example_01.txt contains an example request. I cannot post the entire schema, but here is a minimal compliant version.

[Simon Bennetts]

Great - thats exactly what we need.

I've tried it out and can see the incorrect requests.

Many thanks,

Simon

[Jernej Rus]

Great, thanks for letting me know. I was going crazy trying to find a bug in our APIs 🤪.