ZAP Automation Framework for SOAP and REST Scanning

[Sai Theja Pamarty]

Hi,

I'm using the default SOAP plan available in the automation framework in ZAP GUI. For Rest I couldn't find any dedicated framework...

FYI, the below automation framework is taken from another system GUI and I'm running that framework using the command line on Linux (which doesn't have any GUI and is a remote server).

The plan I'm using is,

---

env:

  contexts:

  - name: "Default Context"

    urls:

    - "http://www.emaple.com/ws/"

    includePaths: []

    excludePaths: []

    authentication:

      parameters: {}

      verification:

        method: "response"

        pollFrequency: 60

        pollUnits: "requests"

    sessionManagement:

      method: "cookie"

      parameters: {}

    technology:

      exclude: []

  parameters:

    failOnError: true

    failOnWarning: false

    progressToStdout: true

  vars: {}

jobs:

- parameters:

    scanOnlyInScope: true

    enableTags: false

  rules: []

  name: "passiveScan-config"

  type: "passiveScan-config"

- parameters:

    wsdlFile: ""

    wsdlUrl: " http://www.emaple.com/ws/vs?WSDL"

  name: "soap"

  type: "soap"

- parameters:

    context: "Default Context"

    user: ""

    url: " http://www.emaple.com/ws/vs?WSDL"

    maxDuration: 0

    maxDepth: 0

    maxChildren: 0

  name: "spider"

  type: "spider"

  tests:

  - onFail: "INFO"

    statistic: "automation.spider.urls.added"

    site: ""

    operator: ">="

    value: 100

    name: "At least 100 URLs found"

    type: "stats"

- parameters: {}

  name: "passiveScan-wait"

  type: "passiveScan-wait"

- parameters:

    context: "Default Context"

    user: ""

    policy: ""

    maxRuleDurationInMins: 0

    maxScanDurationInMins: 0

  policyDefinition:

    defaultStrength: "medium"

    defaultThreshold: "medium"

    rules: []

  name: "activeScan"

  type: "activeScan"

- parameters:

    template: "risk-confidence-html"

    theme: "original"

    reportDir: "C:\\Users\\p\\Desktop\\zap"

    reportFile: ""

    reportTitle: "ZAP Scanning Report"

    reportDescription: ""

    displayReport: true

  risks:

  - "info"

  - "low"

  - "medium"

  - "high"

  confidences:

  - "falsepositive"

  - "low"

  - "medium"

  - "high"

  - "confirmed"

  sections:

  - "siteRiskCounts"

  - "responseBody"

  - "appendix"

  - "alertTypes"

  - "responseHeader"

  - "alertTypeCounts"

  - "riskConfidenceCounts"

  - "alerts"

  - "aboutThisReport"

  - "contents"

  - "requestBody"

  - "reportDescription"

  - "reportParameters"

  - "requestHeader"

  - "summaries"

  name: "report"

  type: "report"

I'm getting the following errors, 

 [main ] ERROR WSDLCustomParser - Unable to communicate with SOAP server. Server may be not available.
javax.net.ssl.SSLHandshakeException: The server selected protocol version TLS10 is not accepted by client preferences [TLS12]

for TLS, is there any way to allow this through the framework? other than going to java.security file and changing those into legacy algorithms.

[ZAP-telemetry-start] ERROR ExtensionCallHome - Connect to https://tel.zaproxy.org:443 [tel.zaproxy.org/172.67.129.53, tel.zaproxy.org/104.21.1.121] failed: connect timed out

is there any way to define proxy settings in the zap framework? can you aso provide example like how to give it? I generally use export http_proxy=.......... and https_proxy=......... this method to use proxy in a session. 

for the default full-scan framework, I'm getting the following error,

[ZAP-ActiveScanner-0] WARN  DomXssScanRule - Skipping scanner, failed to start browser: Cannot find firefox binary in PATH. Make sure firefox is installed. OS appears to be: LINUX
Build info: version: 'unknown', revision: 'unknown', time: 'unknown'
System info: host: '', ip: '', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-1160.81.1.0.1.el7.x86_64', java.version: '11.0.16.1'
Driver info: driver.version: FirefoxDriver
[ZAP-Scanner-0] INFO  HostProcess - skipped plugin [failed to start or connect to the browser] http://10.196.150.192 | DomXssScanRule in 0.046s with 0 message(s) sent and 0 alert(s) raised.

I'm using Linux through the cmd line and there's no graphical interface available. what should I do about this?

Thank you in advance.

Regards

Sai Theja

[Sai Theja Pamarty]

Hi,

I was able to resolve the proxy issue. the firefox not installed (I don't have a GUI) and REST Scan. Is there support for REST scan.means can I scan any REST application using the framework?

[Sai Theja Pamarty]

Hi,

getting new errors for soap scanning,

JobUtils - Automation Framework failed to find method setUSer on org.parosproxy.paros.core.scanner.ScannerParam

and

WARN SitesTreeHelper - Malformed SOAP Message

Thanks & Regards

Sai Theja

[Sai Theja Pamarty]

I've installed firefox but getting error like

couldn't load XPCOM

and soap errors

JobUtils - Automation Framework failed to find method setUSer on org.parosproxy.paros.core.scanner.ScannerParam

and

WARN SitesTreeHelper - Malformed SOAP Message

and REST scan, is there any automaton framework available or can we scan REST in ZAP which is not from OpenAPI?

[kingthorin+owaspzap]

> WARN SitesTreeHelper - Malformed SOAP Message

Is a warning and seems pretty straight forward. Whatever the server sent, was not well formed.

> is there any automaton framework available

There is an automation framework, you seem to already be using it.

> can we scan REST in ZAP which is not from OpenAPI?

You can scan anything with ZAP if you can somehow import or explore it.

[Sai Theja Pamarty]

Hi,

JobUtils - Automation Framework failed to find method setUser on org.parosproxy.paros.core.scanner.ScannerParam

what does this error mean?

> can we scan REST in ZAP which is not from OpenAPI?

You can scan anything with ZAP if you can somehow import or explore it.

So, we can scan REST using ZAP Automation Framework. Can we do that using the full scan or any other way available in automation framework? In OpenAPI its asking for OpenAPI Definition or something.

Thanks & Regards

Sai Theja

[Sai Theja Pamarty]

Hi,

regarding the error,

JobUtils - Automation Framework failed to find method setUser on org.parosproxy.paros.core.scanner.ScannerParam

what does this error mean?

So, we can scan REST using ZAP Automation Framework. Can we do that using the full scan or any other way available in the automation framework? Ror OpenAPI it's asking for OpenAPI Definition or something. But I don't have the flexibility to use the ZAP GUI app and will only use the cmd line to invoke the automation framework. 

The rest application which I want to scan is just a URL..

Thanks & Regards

Sai Theja

[Simon Bennetts]

Answered below:

JobUtils - Automation Framework failed to find method setUser on org.parosproxy.paros.core.scanner.ScannerParam

what does this error mean?

You can ignore that, its not a real error. I thought we had fixed it but I'll have another look.

So, we can scan REST using ZAP Automation Framework. Can we do that using the full scan or any other way available in the automation framework? Ror OpenAPI it's asking for OpenAPI Definition or something. But I don't have the flexibility to use the ZAP GUI app and will only use the cmd line to invoke the automation framework. 

The rest application which I want to scan is just a URL..

Do you have an API definition, if so in what format?

Cheers,

Simon

[Sai Theja Pamarty]

Hi Simon,

Regarding the API Definition. We use SoftwareAG (webMethods) to create REST API. We use "REST V2 Resource" as REST API Descriptor.

Thanks & Regards

Sai Theja

[Sai Theja Pamarty]

Hi,

I'm also getting the following warnings or error

[main ] INFO  CommandLine - Job activeScan finished
2023-03-29 12:49:02,791 [main ] INFO  CommandLine - Job report started
2023-03-29 12:49:03,549 [main ] INFO  CommandLine - Job report generated report /opt/SP/devops/DevOpsShare/ZaProxy/zap/Test/reports/zapsoap.html
2023-03-29 12:49:03,549 [main ] INFO  CommandLine - Job report finished
2023-03-29 12:49:03,550 [main ] INFO  CommandLine - Automation plan succeeded!
2023-03-29 12:49:05,642 [ZAP-DomXssReaper] INFO  DomXssScanRule - Reaper thread exiting 0
2023-03-29 12:49:20,690 [ZAP-IO-EventExecutor-3-3] WARN  MainServerHandler - Failed to write/forward the HTTP response to the client: java.util.concurrent.RejectedExecutionException: event executor terminated
2023-03-29 12:49:20,691 [ZAP-IO-EventExecutor-3-3] WARN  AbstractChannelHandlerContext - Failed to submit an exceptionCaught() event.
java.util.concurrent.RejectedExecutionException: event executor terminated
2023-03-29 12:50:40,633 [ZAP-IO-EventExecutor-3-2] WARN  AbstractEventExecutor - A task raised an exception. Task: io.netty.channel.DefaultChannelPipeline$4@66b065b1
java.util.concurrent.RejectedExecutionException: event executor terminated

2023-03-29 12:50:42,771 [main ] INFO  CommandLineBootstrap - OWASP ZAP 2.12.0 terminated.

I only pasted some of the errors. As most of them were repeating for a long time. This occurred while I was using the full-scan automation framework. 

Thanks & Regards

Sai Theja

[Sai Theja Pamarty]

Hi,

Regarding the REST API Definition. 

We use SoftwareAG (webMethods) to create REST API. We use "REST V2 Resource" available in SoftwareAG as REST API Descriptor.

Can we import The 'rest v2 resource' into zap automation framework?

Thanks & Regards
Sai Theja Pamarty

[Simon Bennetts]

Hiya,

I must admit that "REST V2 Resource" is a format I have not heard about before.

Do you know where it is defined?

You will not currently be able to import that directly into ZAP, unless it is also compatible with the existing formats ZAP supports.

Do you know any tools which support this format?

You may be able to proxy those tools through ZAP.

Cheers,

Simon

[Sai Theja Pamarty]

Hi,

Regarding "REST V2 Resource", it's available in SoftwareAG tool.

I've tried to scan REST using cmdline and Automation Framework.

For ZAP Automation Framework I'm getting the following error

The descriptor is Swagger 2.0

the cmd used:

/usr/lib/jvm/java-11-openjdk-11.0.18.0.10-1.el7_9.x86_64/bin/java -Xmx7998m -XX:+UseG1GC -jar /opt/SP/devops/DevOpsShare/ZaProxy/ZAP_2.12.0/zap-2.12.0.jar -autorun /opt/SP/devops/DevOpsShare/ZaProxy/zap/Test/YamlConfig/restconfig.yaml -cmd -silent

The Error:

2023-04-05 08:26:45,789 [main ] ERROR ExtensionAutomation - java.nio.charset.MalformedInputException: Input length = 1
org.yaml.snakeyaml.error.YAMLException: java.nio.charset.MalformedInputException: Input length = 1
    at org.yaml.snakeyaml.reader.StreamReader.update(StreamReader.java:218) ~[?:?]
    at org.yaml.snakeyaml.reader.StreamReader.ensureEnoughData(StreamReader.java:176) ~[?:?]
    at org.yaml.snakeyaml.reader.StreamReader.ensureEnoughData(StreamReader.java:171) ~[?:?]
    at org.yaml.snakeyaml.reader.StreamReader.peek(StreamReader.java:126) ~[?:?]
    at org.yaml.snakeyaml.scanner.ScannerImpl.scanToNextToken(ScannerImpl.java:1198) ~[?:?]
    at org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:308) ~[?:?]
    at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:248) ~[?:?]
    at org.yaml.snakeyaml.parser.ParserImpl$ParseImplicitDocumentStart.produce(ParserImpl.java:213) ~[?:?]
    at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:165) ~[?:?]
    at org.yaml.snakeyaml.parser.ParserImpl.checkEvent(ParserImpl.java:155) ~[?:?]
    at org.yaml.snakeyaml.composer.Composer.getSingleNode(Composer.java:140) ~[?:?]
    at org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:151) ~[?:?]
    at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:490) ~[?:?]
    at org.yaml.snakeyaml.Yaml.load(Yaml.java:429) ~[?:?]
    at org.zaproxy.addon.automation.AutomationPlan.<init>(AutomationPlan.java:71) ~[?:?]
    at org.zaproxy.addon.automation.ExtensionAutomation.runAutomationFile(ExtensionAutomation.java:433) ~[?:?]
    at org.zaproxy.addon.automation.ExtensionAutomation.execute(ExtensionAutomation.java:549) ~[?:?]
    at org.parosproxy.paros.extension.ExtensionLoader.runCommandLine(ExtensionLoader.java:535) ~[zap-2.12.0.jar:2.12.0]
    at org.parosproxy.paros.control.Control.runCommandLine(Control.java:442) ~[zap-2.12.0.jar:2.12.0]
    at org.zaproxy.zap.CommandLineBootstrap.start(CommandLineBootstrap.java:91) ~[zap-2.12.0.jar:2.12.0]
    at org.zaproxy.zap.ZAP.main(ZAP.java:94) ~[zap-2.12.0.jar:2.12.0]
Caused by: java.nio.charset.MalformedInputException: Input length = 1
    at java.nio.charset.CoderResult.throwException(CoderResult.java:274) ~[?:?]
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339) ~[?:?]
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) ~[?:?]
    at java.io.InputStreamReader.read(InputStreamReader.java:181) ~[?:?]
    at org.yaml.snakeyaml.reader.UnicodeReader.read(UnicodeReader.java:125) ~[?:?]
    at org.yaml.snakeyaml.reader.StreamReader.update(StreamReader.java:183) ~[?:?]
    ... 20 more
2023-04-05 08:26:45,794 [main ] ERROR CommandLine - Unexpected error accessing file /opt/SP/devops/DevOpsShare/ZaProxy/zap/Test/YamlConfig/restconfig.yaml : java.nio.charset.MalformedInputException: Input length = 1 - see log for details
2023-04-05 08:26:45,795 [main ] ERROR CommandLineBootstrap - null
java.lang.NullPointerException: null
    at org.zaproxy.addon.automation.ExtensionAutomation.execute(ExtensionAutomation.java:551) ~[?:?]
    at org.parosproxy.paros.extension.ExtensionLoader.runCommandLine(ExtensionLoader.java:535) ~[zap-2.12.0.jar:2.12.0]
    at org.parosproxy.paros.control.Control.runCommandLine(Control.java:442) ~[zap-2.12.0.jar:2.12.0]
    at org.zaproxy.zap.CommandLineBootstrap.start(CommandLineBootstrap.java:91) ~[zap-2.12.0.jar:2.12.0]
    at org.zaproxy.zap.ZAP.main(ZAP.java:94) ~[zap-2.12.0.jar:2.12.0]

env:
  contexts:
  - name: "Default Context"

    URLs:

- name: "www.ndajad.com/service/1.0"     includePaths: []

    excludePaths: []
    authentication:
      parameters: {}
      verification:
        method: "response"
        pollFrequency: 60
        pollUnits: "requests"
    sessionManagement:
      method: "cookie"
      parameters: {}

  parameters:
    failOnError: true
    failOnWarning: false
    progressToStdout: true
  vars: {}
jobs:
- parameters:
    scanOnlyInScope: true
    enableTags: false
  rules: []
  name: "passiveScan-config"
  type: "passiveScan-config"
- parameters:

    apiFile: "/devops/DevOpsShare/ZaProxy/zap/Test/restdescriptor/Descriptor.json"
    apiUrl: ""
    targetUrl: "www.ndajad.com/service/1.0/sreiveadrees"
  name: "openapi"
  type: "openapi"
- parameters:
    maxDuration: 5000

  name: "passiveScan-wait"
  type: "passiveScan-wait"
- parameters:
    context: "Default Context"
    user: ""
    policy: ""
    maxRuleDurationInMins: 0
    maxScanDurationInMins: 0
  policyDefinition:
    defaultStrength: "medium"
    defaultThreshold: "medium"
    rules: []
  name: "activeScan"
  type: "activeScan"
- parameters:

    template: "traditional-html-plus"
    theme: "light"
    reportDir: "/devops/DevOpsShare/ZaProxy/zap/Test/reports"
    reportFile: "zaprest"

    reportTitle: "ZAP Scanning Report"
    reportDescription: ""

    displayReport: false

  risks:
  - "info"
  - "low"
  - "medium"
  - "high"
  confidences:
  - "falsepositive"
  - "low"
  - "medium"
  - "high"
  - "confirmed"
  sections:

  - "passingrules"
  - "instancecount"
  - "alertdetails"
  - "alertcount"
  - "params"
  - "chart"
  - "statistics"
  name: "report"
  type: "report"

For cmdline: /devops/DevOpsShare/ZaProxy/zap/Test/restdescriptor/RS_VfEdgeAddressManagementServicesV200_SIT3.json -openapitargeturl www.ndajad.com/service/1.0/sreiveadrees -silent -quickout /devops/DevOpsShare/ZaProxy/zap/Test/reports/zaprest.html

The Error:

2023-04-06 08:08:12,164 [main ] INFO  DbMigrate - Current version of schema "PUBLIC": 1
2023-04-06 08:08:12,165 [main ] INFO  DbMigrate - Schema "PUBLIC" is up to date. No migration necessary.
2023-04-06 08:08:12,172 [main ] INFO  CallbackService - Started callback service on 0.0.0.0:42182
2023-04-06 08:08:12,197 [main ] INFO  ExtensionCallHome - Shh! Silent mode or telemetry turned off
2023-04-06 08:08:18,585 [main ] INFO  CommandLineBootstrap - OWASP ZAP 2.12.0 terminated.

It's not producing any reports.

I also tried adding quickurl to the above command but only an empty report was produced.

If possible, I would like to understand why both are not working. 

Thanks & Regards

Sai Theja

[Sai Theja Pamarty]

Hi,

Regarding the above errors..... 

Thanks & Regards 

Sai Theja

[Sai Theja Pamarty]

Hi,

Regarding the rest error?

Thanks & regards 

Sai Theja