Report is generated locally (MacOS) but not on CI server

[Marvie Rosal]

I got it working on my local MacOS machine:
Job report started
Job report generated report /zap/wrk/report.html

Job report finished
Automation plan succeeded!

However, when I run the same on our CI Server (Buildkite - Linux), it doesn't generate the report:

Job report started
Job report failed to generate report: /zap/wrk/report.html
Job report finished
Automation plan failures:
Job report failed to generate report: /zap/wrk/report.html

Do I need to include $(pwd) on the reportDir parameter too as it currently it looks like this:

reportDir: "/zap/wrk/"

[kingthorin+owaspzap]

Does the output directory exist on the CI server? Are you using docker, did you map it properly? (with pwd)

[Marvie Rosal]

it creates a zap/wrk folder based on the following path:
/var/lib/buildkite-agent/builds/buildkite-agents-test-i-008h34zx61194pl6f-1/agyletime/zap-test:/zap/wrk/:rw

Also, runs the automation framework successfully based on this command

docker run -v $(pwd):/zap/wrk/:rw --rm -t owasp/zap2docker-stable bash -c "zap.sh -cmd -addonupdate; zap.sh -cmd -autorun /zap/wrk/zap.yaml"

I actually tried to include reportDir: "${pwd)/zap/wrk/" but still didn't work.

[Simon Bennetts]

"$(pwd)" will only work on the Linux command line, not in the report yaml.

You should use an absolute path in the yaml, in your case starting "/zap/wrk/".

If you still have problems check the zap.log file:

https://www.pingdom.com/blog/webpages-are-getting-larger-every-year-and-heres-why-it-matters/

Cheers,

Simon

[Simon Bennetts]

Ugh, c-n-p problem - link should have been to https://www.zaproxy.org/faq/somethings-not-working-what-should-i-do/#check-the-log-file

[Marvie Rosal]

seems like the difference between local and CI is this error:
1659425713518   Marionette      INFO    Stopped listening on port 34983
Missing chrome or resource URL: resource://gre/modules/UpdateListener.jsm
Missing chrome or resource URL: resource://gre/modules/UpdateListener.sys.mjs
console.error: "Error during quit-application-granted: [Exception... \"File error: Not found\"  nsresult: \"0x80520012 (NS_ERROR_FILE_NOT_FOUND)\"  location: \"JS frame :: resource:///modules/BrowserGlue.jsm :: _onQuitApplicationGranted/tasks< :: line 2006\"  data: no]"
1659425714170   Marionette      INFO    Stopped listening on port 43267
Missing chrome or resource URL: resource://gre/modules/UpdateListener.jsm
Missing chrome or resource URL: resource://gre/modules/UpdateListener.sconsole.error: "Error during quit-application-granted: [Exception... \"File error: Not found\"  nsresult: \"0x80520012 (NS_ERROR_FILE_NOT_FOUND)\"  location: \"JS yframes .:m:j sr
esource:///modules/BrowserGlue.jsm :: _onQuitApplicationGranted/tasks< :: line 2006\"  data: no]"

I googled it up and seems to be firefox related when I already set this on my yaml file:

- parameters:

browserId: chrome-headless

name: "spiderAjax"

type: "spiderAjax"

tests:

- onFail: "INFO"

statistic: "spiderAjax.urls.added"

site: ""

operator: ">="

value: 100

name: "At least 100 URLs found"

type: "stats"

[Simon Bennetts]

Yes, they look like browser errors but I dont think they will stop the AF plan from running.

Are there any other errors in the log file?

Cheers,

Simon

[Marvie Rosal]

Log:
Starting ChromeDriver 106.0.5249.61 (511755355844955cd3e264779baf0dd38212a4d0-refs/branch-heads/5249@{#569}) on port 8386

Only local connections are allowed.

Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.

ChromeDriver was started successfully.

[1665529538.662][SEVERE]: bind() failed: Cannot assign requested address (99)

Job spiderAjax found 0 URLs

Job spiderAjax test of type stats failed: At least 100 URLs found [0 < 100]

Job spiderAjax finished

Job passiveScan-wait started

Job passiveScan-wait finished

Job activeScan started

1665529554506    geckodriver    INFO    Listening on 127.0.0.1:31517

1665529554491    geckodriver    INFO    Listening on 127.0.0.1:2480

1665529554536    mozrunner::runner    INFO    Running command: "/usr/lib/firefox/firefox" "--marionette" "-headless" "-no-remote" "-profile" "/tmp/rust_mozprofilewAimxm"

1665529554549    mozrunner::runner    INFO    Running command: "/usr/lib/firefox/firefox" "--marionette" "-headless" "-no-remote" "-profile" "/tmp/rust_mozprofile2eNT1H"

*** You are running in headless mode.

*** You are running in headless mode.

[GFX1-]: glxtest: libpci missing

[GFX1-]: glxtest: Unable to open a connection to the X server

[GFX1-]: glxtest: libEGL missing

[GFX1-]: No GPUs detected via PCI

[GFX1-]: glxtest: libpci missing

[GFX1-]: glxtest: Unable to open a connection to the X server

[GFX1-]: glxtest: libEGL missing

[GFX1-]: No GPUs detected via PCI

1665529555683    Marionette    INFO    Marionette enabled

1665529555690    Marionette    INFO    Listening on port 37237

Read port: 37237

1665529555738    Marionette    INFO    Marionette enabled

1665529555746    Marionette    INFO    Listening on port 36163

Read port: 36163

1665529555898    RemoteAgent    WARN    TLS certificate errors will be ignored for this session

1665529555957    RemoteAgent    WARN    TLS certificate errors will be ignored for this session

[GFX1-]: RenderCompositorSWGL failed mapping default framebuffer, no dt

console.warn: SearchSettings: "get: No settings file exists, new profile?" (new NotFoundError("Could not open the file at /tmp/rust_mozprofilewAimxm/search.json.mozlz4", (void 0)))

[GFX1-]: RenderCompositorSWGL failed mapping default framebuffer, no dt

console.warn: SearchSettings: "get: No settings file exists, new profile?" (new NotFoundError("Could not open the file at /tmp/rust_mozprofile2eNT1H/search.json.mozlz4", (void 0)))

Missing chrome or resource URL: resource://gre/modules/UpdateListener.sys.mjs

Missing chrome or resource URL: resource://gre/modules/UpdateListener.sys.mjs

1665529592989    Marionette    WARN    Ignoring event 'DOMContentLoaded' because document has an invalid readyState of 'complete'.

1665529593074    Marionette    WARN    Ignoring event 'DOMContentLoaded' because document has an invalid readyState of 'complete'.

1665529603312    Marionette    WARN    Ignoring event 'DOMContentLoaded' because document has an invalid readyState of 'complete'.

1665529603477    Marionette    WARN    Ignoring event 'DOMContentLoaded' because document has an invalid readyState of 'complete'.

1665529618996    Marionette    WARN    Ignoring event 'DOMContentLoaded' because document has an invalid readyState of 'complete'.

1665529629349    Marionette    WARN    Ignoring event 'DOMContentLoaded' because document has an invalid readyState of 'complete'.

1665529629454    Marionette    INFO    Stopped listening on port 37237

Missing chrome or resource URL: resource://gre/modules/UpdateListener.sys.mjs

console.error: "Error during quit-application-granted: [Exception... \"File error: Not found\" nsresult: \"0x80520012 (NS_ERROR_FILE_NOT_FOUND)\" location: \"JS frame :: resource:///modules/BrowserGlue.jsm :: _onQuitApplicationGranted/tasks< :: line 2009\" data: no]"

1665529630277    Marionette    INFO    Stopped listening on port 36163

Missing chrome or resource URL: resource://gre/modules/UpdateListener.sys.mjs

console.error: "Error during quit-application-granted: [Exception... \"File error: Not found\" nsresult: \"0x80520012 (NS_ERROR_FILE_NOT_FOUND)\" location: \"JS frame :: resource:///modules/BrowserGlue.jsm :: _onQuitApplicationGranted/tasks< :: line 2009\" data: no]"

Job activeScan finished

Job report started

Job report failed to generate report: /zap/wrk/report.html

Job report finished

Automation plan failures:

   Job report failed to generate report: /zap/wrk/report.html

make: *** [dast] Error 1

🚨 Error: The command exited with status 2

[Simon Bennetts]

That looks like stdout.

We need to know if there are any errors in the zap.log file: https://www.zaproxy.org/faq/somethings-not-working-what-should-i-do/#check-the-log-file

Also, does the CI/CD job have write access to /zap/wrk ?

Cheers,

Simon

[Marvie Rosal]

Sorry can't access the log on the CI (Its not generating artefact). However, this command worked:

docker run -v $(shell pwd):/zap/wrk/:rw --rm -t owasp/zap2docker-stable bash -c "zap.sh -cmd -addonupdate; zap.sh -cmd -autorun /zap/wrk/zap.yaml; cat zap-report.html"

This is based on the following snippet in the `zap.yaml` file: 

- type: report

  parameters:

    template: high-level-report 

    reportDir: /zap 

    reportFile: zap-report 

    reportTitle: zap-report 

[Simon Bennetts]

Does that mean its all working for you now?

[Marvie Rosal]

Yes, one main difference is changing the reportDir to /zap instead of /zap/wrk

[Marvie Rosal]

Now, my problem is I try to run it locally and doesn't work anymore. It gets stuck here:
 ~ % docker run -v $(pwd):/zap/wrk/:rw --rm -t owasp/zap2docker-stable bash -c "zap.sh -cmd -addonupdate; zap.sh -cmd -autorun /zap/wrk/zap.yaml"
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Found Java version 11.0.16
Available memory: 7851 MB
Using JVM args: -Xmx1962m
4523 [main] INFO  org.parosproxy.paros.Constant - Copying default configuration to /home/zap/.ZAP/config.xml
5193 [main] INFO  org.parosproxy.paros.Constant - Creating directory /home/zap/.ZAP/session
5195 [main] INFO  org.parosproxy.paros.Constant - Creating directory /home/zap/.ZAP/dirbuster
5196 [main] INFO  org.parosproxy.paros.Constant - Creating directory /home/zap/.ZAP/fuzzers
5197 [main] INFO  org.parosproxy.paros.Constant - Creating directory /home/zap/.ZAP/plugin
Oct 14, 2022 7:11:58 AM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.

I also cant find the log files as the ZAP folder is missing in 

• Mac OS: ~/Library/Application Support/ZAP

I am running MacOS on M1 chip.

[AppSec LN]

Hi Simon,

I am also getting the same issue. Unable to get the reports in CI.

Please see the logs below:

Job report generated report /home/zap/ZAP-Report.html

Job report finished, time taken: 00:00:05
Job report started
Job report generated report /home/zap/Risk-Confidence.html
Job report finished, time taken: 00:00:00
Automation plan succeeded!

Copy Reports
Error response from daemon: Could not find the file /home/zap/ZAP-Report.html in container owasp
Error response from daemon: Could not find the file /home/zap/Risk-Confidence.html in container owasp

Could you please help.

[Simon Bennetts]

How are you running ZAP?

What command are you using?

Feel free to obfuscate sensitive data.

These details matter :)

Cheers,

Simon

[AppSec LN]

Running docker zap using automation plan.

 

Please find the command:

docker run -v "$(Build.SourcesDirectory)/ZAP/":/zap/wrk/:rw -t owasp/zap2docker-stable zap.sh -cmd -autorun /zap/wrk/zap.yaml

[thc...@gmail.com]

Either save the reports to /zap/wrk/ or add volume for /home/zap/ as well.

Best regards.

On 06/02/2024 13:29, AppSec LN wrote:
> Running docker zap using automation plan.
>
> Please find the command:
>

> *docker run -v "$(Build.SourcesDirectory)/ZAP/":/zap/wrk/:rw -t
> owasp/zap2docker-stable zap.sh -cmd -autorun /zap/wrk/zap.yaml*

>>>> - Mac OS: ~/Library/Application Support/ZAP

>>>>
>>>>
>>>> I am running MacOS on M1 chip.
>>>>
>>>> On Friday, October 14, 2022 at 6:13:43 PM UTC+11 Marvie Rosal wrote:
>>>>
>>>>> Yes, one main difference is changing the reportDir to /zap instead of
>>>>> /zap/wrk
>>>>>
>>>>> On Thursday, October 13, 2022 at 6:22:05 PM UTC+11 psi...@gmail.com
>>>>> wrote:
>>>>>
>>>>>> Does that mean its all working for you now?
>>>>>>
>>>>>> On Thursday, 13 October 2022 at 00:58:43 UTC+2 Marvie Rosal wrote:
>>>>>>
>>>>>>> Sorry can't access the log on the CI (Its not generating artefact).
>>>>>>> However, this command worked:
>>>>>>> docker run -v $(shell pwd):/zap/wrk/:rw --rm -t
>>>>>>> owasp/zap2docker-stable bash -c "zap.sh -cmd -addonupdate; zap.sh -cmd

>>>>>>> -autorun /zap/wrk/zap.yaml*; cat zap-report.html*"

[AppSec LN]

I've tried the below 4 options in azure devops pipeline but none of them working. could you please check and let me know the workable option

Option 1: If I defined the report directory as  /zap/wrk/ and report file name is ZAP-Report.html

Error Message: Job report failed to generate report: Cannot create directory '/zap/wrk/ZAP-Report' 

Option 2: blank directory name "",   file name "ZAP-Report.html", copy command from  container name

Error response from daemon: Could not find the file /home/zap/ZAP-Report.html in container owasp

Option 3: blank directory name "",   file name "ZAP-Report.html" copy command from  container id

Error response from daemon: Could not find the file /home/zap/ZAP-Report.html in container abcda1321

Option 4: add volume for /home/zap
docker run -v "$(Build.SourcesDirectory)/ZAP/reports":/home/zap/:rw owasp/zap2docker-stable

docker run -v "$(Build.SourcesDirectory)/ZAP/":/zap/wrk/:rw -t owasp/zap2docker-stable zap.sh -cmd -autorun /zap/wrk/zap.yaml

Error: reports not found in $(Build.SourcesDirectory)/ZAP/reports

[Simon Bennetts]

Option 1 is likely to be the best one to focus on.

The problem you are seeing is a permissions one, and so something that ZAP cannot control.

Try creating a ZAP-Report.html file in the mapped directory and giving write access to everyone.

Cheers,

Simon

[AppSec LN]

Yes it is related to permissions. I tried the below command and reports got generated. ActiveScan completed quickly without any results.

docker run -v $(pwd):/zap/wrk/:rw -t --user root owasp/zap2docker-stable zap.sh -cmd -autorun /zap/wrk/zap.yaml

if I use the below command (without user permissions), no reports but full scan happening (no reports in pipeline scan - detailed reports in desktop)  

docker run -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable zap.sh -cmd -autorun /zap/wrk/zap.yaml

[Simon Bennetts]

Thats good to know, but we dont recommend running ZAP with the root user - we know the AJAX spider will fail as Firefox will refuse to run.

Have you tried pre-creating the file and giving full access to it?

Cheers,

Simon

[Matthew Rosenberg]

I was also running into a similar issue when developing locally (MacOS) versus running on a CI server (ubuntu-latest).

TLDR; My workaround right now is to temporarily chmod 777 the directory that I'm attaching to the zap container.

Details

I’m running spider and spiderAjax with a report job:

- type: report

  parameters:

    template: traditional-pdf

    reportDir: /zap/wrk

    reportFile: "***"

My reports were failing with similar output (even though my plan was configured to produce pdf):

... 

Job report started 

Job report failed to generate report: /zap/wrk/***.html

Job report finished, time taken: 00:00:00 

  Automation plan failures: Job report failed to generate report: /zap/wrk/***.html 

My script now looks like:

chmod 777 $(pwd)/wrk # Temporarily more permissive 

docker run \ 

 --rm \

 -v $(pwd)/wrk:/zap/wrk/:rw \

 --user zap \

 zaproxy/zap-stable zap.sh -cmd -autorun "/zap/wrk/my-plan.yml"

# Reset permissions after the scan completes 

chmod 775 $(pwd)/wrk

This version works on both MacOS and in CI.

But, I do see a large output of “[Fatal Error] :1:1: Content is not allowed in prolog.” logs in the CI environment that I don't get locally (I understand from other threads that those can be ignored).