ERROR ExtensionAutomation - null

55 views
Skip to first unread message

mir

unread,
Apr 16, 2023, 12:24:06 AM4/16/23
to OWASP ZAP Developer Group
Hello,
I am trying to create an automation framework file
I run the command ./zap.sh -cmd -port 8090 -autogenmax zap.yaml on a remote Linux machine,
The file was indeed created but in the terminal it writes me the error
Failed to get template for job type: import
My zap.sh version is 2.12.0
I started working on the file and writing data in it, then I run the command "./zap.sh -cmd -port 8090 -autorun zap.yaml" in the terminal
I get:
Found Java version 11.0.18
Available memory: 64310 MB
Using JVM args: -Xmx16077m
Job authentication set parameters = {port=8090, hostname=localhost}
Job sessionManagement set parameters = {script=null, scriptEngine=null}
Job users set credentials = {password=null, username=null}
Job alertFilter set ruleId = 1
Job alertFilter set newRisk = High
Job passiveScan-config set maxAlertsPerRule = 10
Job passiveScan-config set scanOnlyInScope = true
Job passiveScan-config set enableTags = false
Job passiveScan-config set disableAllRules = false
Job passiveScan-config ignoring rule with no id
Job script set action = run
Job: script Script name is required, but not specified
Job: script Script type is required, but not specified
Job requestor set url = https://www.example.com/path
Job spider/stats set name = At least X URLs found
Job spider/stats set type = stats
Job spider/stats set statistic = automation.spider.urls.added
Job spider/stats set operator = >=
Job spider/stats set value = 100
Job spider/stats set onFail = Info
Job spider adding test of type stats : At least X URLs found
Job spiderAjax/stats set name = At least X URLs found
Job spiderAjax/stats set type = stats
Job spiderAjax/stats set statistic = spiderAjax.urls.added
Job spiderAjax/stats set operator = >=
Job spiderAjax/stats set value = 100
Job spiderAjax/stats set onFail = Info
Job spiderAjax adding test of type stats : At least X URLs found
Job passiveScan-wait set maxDuration = 5
Job passiveScan-wait/alert set name = test one
Job passiveScan-wait/alert set type = alert
Job passiveScan-wait/alert set action = passIfAbsent
Job passiveScan-wait/alert set scanRuleId = 1
Job passiveScan-wait/alert set url = http://www.example.com/path
Job passiveScan-wait/alert set onFail = Info
Job passiveScan-wait adding test of type alert : test one
Job report set reportDir = /home/report
Job report set reportFile = report.txt
Job report set reportTitle = test
Job report set reportDescription = test
Job report set displayReport = true
Unexpected error accessing file /home/zap.yaml : null - see log for details
null

Usage:
        zap.sh [Options]
Core options:
        -version                 Reports the ZAP version
        -cmd                     Run inline (exits when command line options complete)
        -daemon                  Starts ZAP in daemon mode, i.e. without a UI
        -config <kvpair>         Overrides the specified key=value pair in the configuration file
        -configfile <path>       Overrides the key=value pairs with those in the specified properties file
        -dir <dir>               Uses the specified directory instead of the default one
        -installdir <dir>        Overrides the code that detects where ZAP has been installed with the specified directory
        -h                       Shows all of the command line options available, including those added by add-ons
        -help                    The same as -h
        -newsession <path>       Creates a new session at the given location
        -session <path>          Opens the given session after starting ZAP
        -lowmem                  Use the database instead of memory as much as possible - this is still experimental
        -experimentaldb          Use the experimental generic database code, which is not surprisingly also still experimental
        -nostdout                Disables the default logging through standard output
        -silent                  Ensures ZAP does not make any unsolicited requests, including check for updates
Add-on options:
        -graphqlfile <path>       Imports a GraphQL Schema from a File
        -graphqlurl <url>         Imports a GraphQL Schema from a URL
        -graphqlendurl <url>      Sets the Endpoint URL
        -certload <path>         Loads the Root CA certificate from the specified file name
        -certpubdump <path>      Dumps the Root CA public certificate into the specified file name, this is suitable for importing into browsers
        -certfulldump <path>     Dumps the Root CA full certificate (including the private key) into the specified file name, this is suitable for importing into ZAP
        -host <host>             Overrides the host of the main proxy, specified in the configuration file
        -port <port>             Overrides the port of the main proxy, specified in the configuration file
        -hud                     Launches a browser configured to proxy through ZAP with the HUD enabled, for use in daemon mode
        -hudurl <url>            Launches a browser as per the -hud option with the specified URL
        -hudbrowser <browser>    Launches a browser as per the -hud option with the specified browser, supported options: Chrome, Firefox by default Firefox
        -notel                   Turns off telemetry calls
        -autorun <filename>      Run the automation jobs specified in the file
        -autogenmin <filename>   Generate template automation file with the key parameters
        -autogenmax <filename>   Generate template automation file with all parameters
        -autogenconf <filename>  Generate template automation file using the current configuration
        -script <script>         Run the specified script from commandline or load in GUI
        -addoninstall <addOnId>   Installs the add-on with specified ID from the ZAP Marketplace
        -addoninstallall          Install all available add-ons from the ZAP Marketplace
        -addonuninstall <addOnId> Uninstalls the Add-on with specified ID
        -addonupdate              Update all changed add-ons from the ZAP Marketplace
        -addonlist                List all of the installed add-ons
        -quickurl <target url>   The URL to attack, e.g. http://www.example.com
        -quickout <filename>     The file to write the HTML/JSON/MD/XML results to (based on the file extension)
        -quickprogress:          Display progress bars while scanning
        -openapifile <path>      Imports an OpenAPI definition from the specified file name
        -openapiurl <url>        Imports an OpenAPI definition from the specified URL
        -openapitargeturl <url>  The Target URL, to override the server URL present in the OpenAPI definition. Refer to the help for supported format.

in the log file

2023-04-13 12:24:13,579 [main ] ERROR ExtensionAutomation - null
java.lang.NullPointerException: null
at org.zaproxy.addon.reports.automation.OutputSummaryJob.verifyParameters(OutputSummaryJob.java:104) ~[?:?]
at org.zaproxy.addon.automation.AutomationPlan.<init>(AutomationPlan.java:119) ~[?:?]
at org.zaproxy.addon.automation.ExtensionAutomation.runAutomationFile(ExtensionAutomation.java:491) ~[?:?]
at org.zaproxy.addon.automation.ExtensionAutomation.execute(ExtensionAutomation.java:607) ~[?:?]
at org.parosproxy.paros.extension.ExtensionLoader.runCommandLine(ExtensionLoader.java:535) ~[zap-2.12.0.jar:2.12.0]
at org.parosproxy.paros.control.Control.runCommandLine(Control.java:442) ~[zap-2.12.0.jar:2.12.0]
at org.zaproxy.zap.CommandLineBootstrap.start(CommandLineBootstrap.java:91) ~[zap-2.12.0.jar:2.12.0]
at org.zaproxy.zap.ZAP.main(ZAP.java:94) ~[zap-2.12.0.jar:2.12.0]
2023-04-13 12:24:13,595 [main ] ERROR CommandLine - Unexpected error accessing file /home/zap.yaml : null - see log for details
2023-04-13 12:24:13,595 [main ] ERROR CommandLineBootstrap - null
java.lang.NullPointerException: null
at org.zaproxy.addon.automation.ExtensionAutomation.execute(ExtensionAutomation.java:609) ~[?:?]
at org.parosproxy.paros.extension.ExtensionLoader.runCommandLine(ExtensionLoader.java:535) ~[zap-2.12.0.jar:2.12.0]
at org.parosproxy.paros.control.Control.runCommandLine(Control.java:442) ~[zap-2.12.0.jar:2.12.0]
at org.zaproxy.zap.CommandLineBootstrap.start(CommandLineBootstrap.java:91) ~[zap-2.12.0.jar:2.12.0]
at org.zaproxy.zap.ZAP.main(ZAP.java:94) ~[zap-2.12.0.jar:2.12.0]
2023-04-13 12:24:18,617 [main ] INFO  CommandLineBootstrap - OWASP ZAP 2.12.0 terminated.

Please help me fix the problem, it's really urgent for me
Thank you!!!

thc...@gmail.com

unread,
Apr 16, 2023, 2:01:33 AM4/16/23
to zaproxy...@googlegroups.com
Hi.

In the `outputSummary` job configuration add a valid path in
`summaryFile`, or remove the job.


Best regards.
Message has been deleted

mir

unread,
Apr 16, 2023, 3:45:24 AM4/16/23
to OWASP ZAP Developer Group
Thanks!!
Now it picks up running and then falls
The output in the terminal is:


Found Java version 11.0.18
Available memory: 64310 MB
Using JVM args: -Xmx16077m
Job authentication set parameters = {port=8090, hostname=localhost}
Job users set credentials = {password=null, username=null}
Job alertFilter set ruleId = 1
Job alertFilter set newRisk = High
Job passiveScan-config set maxAlertsPerRule = 10
Job passiveScan-config set scanOnlyInScope = true
Job passiveScan-config set enableTags = false
Job passiveScan-config set disableAllRules = false
Job passiveScan-config ignoring rule with no id
Job script set action = run
Job: script Script name is required, but not specified
Job: script Script type is required, but not specified
Job requestor set url = https://www.example

Job spider/stats set name = At least X URLs found
Job spider/stats set type = stats
Job spider/stats set statistic = automation.spider.urls.added
Job spider/stats set operator = >=
Job spider/stats set value = 100
Job spider/stats set onFail = Info
Job spider adding test of type stats: At least X URLs found
Job spiderAjax/stats set name = At least X URLs found
Job spiderAjax/stats set type = stats
Job spiderAjax/stats set statistic = spiderAjax.urls.added
Job spiderAjax/stats set operator = >=
Job spiderAjax/stats set value = 100
Job spiderAjax/stats set onFail = Info
Job spiderAjax adding test of type stats: At least X URLs found
Job passiveScan-wait set maxDuration = 5
Job passiveScan-wait/alert set name = test one
Job passiveScan-wait/alert set type = alert
Job passiveScan-wait/alert set action = passIfAbsent
Job passiveScan-wait/alert set scanRuleId = 1
Job passiveScan-wait/alert set onFail = Info
Job passiveScan-wait adding test of type alert: test one
Job report set reportDir = /home/report

Job report set reportFile = report.txt
Job report set reportTitle = test

Job report set reportDescription = test
Job report set displayReport = true
Automation plan failures:


         Job: script Script name is required, but not specified
         Job: script Script type is required, but not specified

My file is like this:
env:                                  
  contexts :                          
    - name: context 1                  
      urls: ['https://www.example']                          
      includePaths:                    
      excludePaths:                    
      authentication:
        method:                        
        parameters:                    
          hostname: localhost                  
          port: 8090                      
          technology:
        exclude:                      
      users:                          
      - name:                          
        credentials:                  
          username:                    
          password:                    
  vars:                                
  parameters:
    failOnError: true                          
    failOnWarning: false              
    progressToStdout: true            

jobs:

  - type: alertFilter                  
    parameters:
      deleteGlobalAlerts: true        
    alertFilters:                      
      - ruleId: 1                      
        newRisk: 'High'                
        context:                      
        url:                          
        urlRegex:                      
        parameter:                    
        parameterRegex:                
        attack:                        
        attackRegex:                  
        evidence:                      
        evidenceRegex:                
  - type: passiveScan-config          
    parameters:
      maxAlertsPerRule: 10            
      scanOnlyInScope: true            
      maxBodySizeInBytesToScan:        
      enableTags: false                
      disableAllRules: false          
    rules:                            
    - id:                              
      name:                            
      threshold:                      
  - type: script
    parameters:
      action: 'run'                  
      type:                      
      engine:                    
      name:                      
      file:                      
      target:                    
  - type: requestor                    
    parameters:
      user:                            
    requests:                          
      - url: 'https://www.example'                          
        name:                          
        method:                        
        httpVersion:                  
        headers:                      
        data:                          
        responseCode:                  
  - type: spider                       # The traditional spider - fast but doesnt handle modern apps so well
    parameters:
      context:                         # String: Name of the context to spider, default: first context
      user:                            # String: An optional user to use for authentication, must be defined in the env
      url:                             # String: Url to start spidering from, default: first context URL
      maxDuration:                     # Int: The max time in minutes the spider will be allowed to run for, default: 0 unlimited
      maxDepth:                        # Int: The maximum tree depth to explore, default 5
      maxChildren:                     # Int: The maximum number of children to add to each node in the tree
      acceptCookies:                   # Bool: Whether the spider will accept cookies, default: true
      handleODataParametersVisited:    # Bool: Whether the spider will handle OData responses, default: false
      handleParameters:                # Enum [ignore_completely, ignore_value, use_all]: How query string parameters are used when checking if a URI has already been visited, default: use_all
      maxParseSizeBytes:               # Int: The max size of a response that will be parsed, default: 2621440 - 2.5 Mb
      parseComments:                   # Bool: Whether the spider will parse HTML comments in order to find URLs, default: true
      parseGit:                        # Bool: Whether the spider will parse Git metadata in order to find URLs, default: false
      parseDsStore:                    # Bool: Whether the spider will parse .DS_Store files in order to find URLs, default: false
      parseRobotsTxt:                  # Bool: Whether the spider will parse 'robots.txt' files in order to find URLs, default: true
      parseSitemapXml:                 # Bool: Whether the spider will parse 'sitemap.xml' files in order to find URLs, default: true
      parseSVNEntries:                 # Bool: Whether the spider will parse SVN metadata in order to find URLs, default: false
      postForm:                        # Bool: Whether the spider will submit POST forms, default: true
      processForm:                     # Bool: Whether the spider will process forms, default: true
      requestWaitTime:                 # Int: The time between the requests sent to a server in milliseconds, default: 200
      sendRefererHeader:               # Bool: Whether the spider will send the referer header, default: true
      threadCount:                     # Int: The number of spider threads, default: 2
      userAgent:                       # String: The user agent to use in requests, default: '' - use the default ZAP one
    tests:
      - name: 'At least X URLs found'                   # String: Name of the test, default: statistic + operator + value
        type: 'stats'                                   # String: Type of test, only 'stats' is supported for now
        statistic: 'automation.spider.urls.added'       # String: Name of an integer / long statistic, currently supported: 'automation.spider.urls.added'
        operator: '>='                                  # String ['==', '!=', '>=', '>', '<', '<=']: Operator used for testing
        value: 100                                      # Int: Change this to the number of URLs you expect to find
        onFail: 'info'                                  # String: One of 'warn', 'error', 'info', mandatory
  - type: spiderAjax                   # The ajax spider - slower than the spider but handles modern apps well
    parameters:
      context:                         # String: Name of the context to spider, default: first context
      url:                             # String: Url to start spidering from, default: first context URL
      maxDuration:                     # Int: The max time in minutes the ajax spider will be allowed to run for, default: 0 unlimited
      maxCrawlDepth:                   # Int: The max depth that the crawler can reach, default: 10, 0 is unlimited
      numberOfBrowsers:                # Int: The number of browsers the spider will use, more will be faster but will use up more memory, default: 1
      runOnlyIfModern:                 # Boolean: If true then the spider will only run if a "modern app" alert is raised, default: false
      inScopeOnly:                     # Boolean: If true then any URLs requested which are out of scope will be ignored, default: true
      browserId:                       # String: Browser Id to use, default: firefox-headless
      clickDefaultElems:               # Bool: When enabled only click the default element: 'a', 'button' and input, default: true
      clickElemsOnce:                  # Bool: When enabled only click each element once, default: true
      eventWait:                       # Int: The time in milliseconds to wait after a client side event is fired, default: 1000
      maxCrawlStates:                  # Int: The maximum number of crawl states the crawler should crawl, default: 0 unlimited
      randomInputs:                    # Bool: When enabled random values will be entered into input element, default: true
      reloadWait:                      # Int: The time in milliseconds to wait after the URL is loaded, default: 1000
      elements:                        # A list of HTML elements to click - will be ignored unless clickDefaultElems is false
     
    tests:
      - name: 'At least X URLs found'        # String: Name of the test, default: statistic + operator + value
        type: 'stats'                        # String: Type of test, only 'stats' is supported for now
        statistic: 'spiderAjax.urls.added'   # String: Name of an integer / long statistic, currently supported: 'spiderAjax.urls.added'
        operator: '>='                       # String ['==', '!=', '>=', '>', '<', '<=']: Operator used for testing
        value: 100                           # Int: Change this to the number of URLs you expect to find
        onFail: 'info'                       # String [warn, error, info]: Change this to 'warn' or 'error' for the test to take effect
  - type: delay                        # Pause the plan for a set period of time or event (file created, programmatic method called, API endpoint called)
    parameters:
      time:                            # String: The time to wait, format any of ['hh:mm:ss', 'mm:ss', 'ss'], default: 0
      fileName:                        # String: Name of a file which will cause the job to end early if created, default: empty
  - type: passiveScan-wait             # Passive scan wait for the passive scanner to finish
    parameters:
      maxDuration: 5                   # Int: The max time to wait for the passive scanner, default: 0 unlimited
    tests:
      - name: 'test one'                       # Name of the test, optional
        type: alert                            # Specifies that the test is of type 'alert'
        action: passIfAbsent     # String: The condition (presence/absence) of the alert, default: passIfAbsent  
        scanRuleId: 1                           # Integer: The id of the scanRule which generates the alert, mandatory  
        alertName:                             # String: The name of the alert generated, optional
        url:        # String: The url of the request corresponding to the alert generated, optional
        method:                                # String: The method of the request corresponding to the alert generated, optional
        attack:                                # String: The actual attack which generated the alert, optional
        param:                                 # String: The parameter which was modified to generate the alert, optional
        evidence:                              # String: The evidence corresponding to the alert generated, optional
        confidence:                            # String: The confidence of the alert, one of 'False Positive', 'Low', 'Medium', 'High', 'Confirmed', optional
        risk:                                  # String: The risk of the alert, one of 'Informational', 'Low', 'Medium', 'High', optional
        otherInfo:                             # String: Addional information corresponding to the alert, optional
        onFail: 'info'                         # String: One of 'warn', 'error', 'info', mandatory
  - type: report                       # Report generation
    parameters:
      template:                        # String: The template id, default : modern
      theme:                           # String: The template theme, default: the first theme defined for the template (if any)
      reportDir: '/home/report'                      # String: The directory into which the report will be written
      reportFile: 'report.txt'                     # String: The report file name pattern, default: {{yyyy-MM-dd}}-ZAP-Report-[[site]]
      reportTitle: 'test'                    # String: The report title
      reportDescription: 'test'              # String: The report description
      displayReport: True                  # Boolean: Display the report when generated, default: false
    risks:                             # List: The risks to include in this report, default all
      - high
      - medium
      - low
      - info
    confidences:                       # List: The confidences to include in this report, default all
      - high
      - medium
      - low
      - falsepositive
    sections:                          # List: The template sections to include in this report - see the relevant template, default all
 
please help me

ב-יום ראשון, 16 באפריל 2023 בשעה 09:01:33 UTC+3, thc202 כתב/ה:

thc...@gmail.com

unread,
Apr 16, 2023, 4:21:14 AM4/16/23
to zaproxy...@googlegroups.com
If you don't plan to add/run any script you should remove the `script` job.

Best regards.

mir

unread,
Apr 16, 2023, 4:39:36 AM4/16/23
to OWASP ZAP Developer Group
I fixed everything necessary,
Now it falls on the first error

"Found Java version 11.0.18
Available memory: 64310 MB
Using JVM args: -Xmx16077m
Writing template to /home/zap.yaml

Failed to get template for job type: import"

in the log file:
2023-04-16 11:35:55,770 [main ] ERROR CommandLine - Failed to get template for job type: import
java.lang.NullPointerException: null
    at java.io.Reader.<init>(Reader.java:167) ~[?:?]
    at java.io.InputStreamReader.<init>(InputStreamReader.java:109) ~[?:?]
    at org.apache.commons.io.IOUtils.copy(IOUtils.java:1021) ~[commons-io-2.11.0.jar:2.11.0]
    at org.apache.commons.io.IOUtils.toString(IOUtils.java:2839) ~[commons-io-2.11.0.jar:2.11.0]
    at org.zaproxy.addon.exim.automation.ImportJob.getResourceAsString(ImportJob.java:154) ~[?:?]
    at org.zaproxy.addon.exim.automation.ImportJob.getTemplateDataMax(ImportJob.java:149) ~[?:?]
    at org.zaproxy.addon.automation.ExtensionAutomation.lambda$generateTemplateFile$1(ExtensionAutomation.java:287) ~[?:?]
    at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[?:?]
    at java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357) ~[?:?]
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485) ~[?:?]
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
    at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:?]
    at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:?]
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
    at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) ~[?:?]
    at org.zaproxy.addon.automation.ExtensionAutomation.generateTemplateFile(ExtensionAutomation.java:283) ~[?:?]
    at org.zaproxy.addon.automation.ExtensionAutomation.execute(ExtensionAutomation.java:627) ~[?:?]
    at org.parosproxy.paros.extension.ExtensionLoader.runCommandLine(ExtensionLoader.java:535) ~[zap-2.12.0.jar:2.12.0]
    at org.parosproxy.paros.control.Control.runCommandLine(Control.java:442) ~[zap-2.12.0.jar:2.12.0]
    at org.zaproxy.zap.CommandLineBootstrap.start(CommandLineBootstrap.java:91) ~[zap-2.12.0.jar:2.12.0]
    at org.zaproxy.zap.ZAP.main(ZAP.java:94) ~[zap-2.12.0.jar:2.12.0]
2023-04-16 11:36:01,775 [main ] INFO  CommandLineBootstrap - OWASP ZAP 2.12.0 terminated.
ב-יום ראשון, 16 באפריל 2023 בשעה 11:21:14 UTC+3, thc202 כתב/ה:

thc...@gmail.com

unread,
Apr 16, 2023, 4:46:22 AM4/16/23
to zaproxy...@googlegroups.com
If you already have the plan you should remove `-autogenmax`, that's
only needed to create a base plan for one to customise.

Best regards.

mir

unread,
Apr 19, 2023, 2:57:20 AM4/19/23
to OWASP ZAP Developer Group
Thank you very much!!

ב-יום ראשון, 16 באפריל 2023 בשעה 11:46:22 UTC+3, thc202 כתב/ה:
Reply all
Reply to author
Forward
0 new messages