How to convert context with ZEST authentication script to docker image .py scripts commands?

Pavel Hrachou

unread,

Dec 22, 2020, 10:04:20 AM12/22/20

to OWASP ZAP User Group

In following message and files sensitive and possibly sensitive data replaced with "APPNAME" and "DATA", so owasp ZAP expected to have troubles during import of them.

It works in WASP Zed Attack Proxy on windows, so now I need to make it work with docker scripts.
So I can have something like this
zap-full-scan.py -t https:// APPNAME.com -d -r zap-full-scan.py-$(date +%Y-%m-%d_%H:%M).html -n APPNAME .context -U DATA

Current issue:
108500 [Thread-11] ERROR org.zaproxy.zap.authentication.ScriptBasedAuthenticationMethodType - Unable to find script while loading Script Based Authentication Method for name: Azure auth v2.zst

Pavel Hrachou

unread,

Dec 22, 2020, 10:04:56 AM12/22/20

to OWASP ZAP User Group

sanitized.7z

Pavel Hrachou

unread,

Dec 22, 2020, 10:06:17 AM12/22/20

to OWASP ZAP User Group

Somehow can't add context and script directly so used archive. If someone can explain why - I will be grateful.

вторник, 22 декабря 2020 г. в 18:04:56 UTC+3, Pavel Hrachou:

eri...@augment1security.com

unread,

Dec 22, 2020, 10:29:01 AM12/22/20

to OWASP ZAP User Group

Hi,

If I understand correctly, you have something working on the desktop zap but now you want it to work in docker. Reason why you are getting that error is because your script needs to be in the docker container. You can do it by the zap api or you can copy it using docker commands.

You can check out https://augment1security.com/api-scanning/owasp-zap-api-scanning-with-authentication-from-desktop-to-docker-part-1/ for some guidance. Please let me know how you go.

Best Regards,
Eric W.
Blog: https://augment1security.com/blog/
Twitter: @aug1sec
Facebook: https://www.facebook.com/aug1sec

Pavel Hrachou

unread,

Dec 23, 2020, 6:09:28 AM12/23/20

to OWASP ZAP User Group

" If I understand correctly, you have something working on the desktop zap but now you want it to work in docker. Reason why you are getting that error is because your script needs to be in the docker container. "

Exactly.

" You can check out https://augment1security.com/api-scanning/owasp-zap-api-scanning-with-authentication-from-desktop-to-docker-part-1/ for some guidance. Please let me know how you go. "

1) Guide tells that I should add scripts to config.

owasp-zap-api-scanning-with-authentication-from-desktop-to-docker-part-3-2.png

I use ZEST engine script. In guide it's python script. So I assume for me it's something like this.

<name>response_json</name>

<resHeadRegex>type\s*:\s*application\/json</resHeadRegex>

</scanner>

</autoTagScanners>

</pscans>

<name>AzureAuth.zst</name>

<engine>zest</engine>

<type>authentication</type>

<enabled>false</enabled>

<file>/zap/wrk/scripts/authentication/AzureAuth.zst</file>

</scripts>

</script>

</config>

But what should I use for engine and enabled parameter?

2) I don't need to modify zap-api-scan.py cause I don't have augment1security_accesstoken_setter.py?

ZAP on windows holds this session without this modification, so maybe I don't need it?

3) Is it normal that I don't have in /zap/xml file log4j.properties?
I see this string in zap-script.sh
docker cp ${ROOTPATH}/log4j.properties $(docker inspect --format="{{.Id}}" zap_container):/zap/xml
But in container based on zap stable 2.10.0 docker Image on this address I don't see such file.

$ ls

common-user-agents.txt config.xml drivers.dtd drivers.xml report.html.xsl report.md.xsl reportCompare.xsl

$ pwd

/zap/xml

4) Should I modify zap_common.py?
I see it's significantly changed after guide was made.

вторник, 22 декабря 2020 г. в 18:29:01 UTC+3, eri...@augment1security.com:

Pavel Hrachou

unread,

Dec 23, 2020, 6:25:14 AM12/23/20

to OWASP ZAP User Group

So I came to conclusion that

I create in \zap\wrk\auth_files\files_to_copy_in_zap_folder\scripts\authentication\AzureAuth.zst

{

"about": "This is a Zest script. For more details about Zest visit https://developer.mozilla.org/en-US/docs/Zest",

"zestVersion": "0.8",

"title": "AzureAuth.zst",

"description": "This is script for azure based authentication.",

"prefix": "",

"type": "StandAlone",

"parameters": {

"tokenStart": "{{",

"tokenEnd": "}}",

"tokens": {

"Username": "",

"enteredAppClassFlag": "project",

"LoginURL": "",

"Password": ""

},

"elementType": "ZestVariables"

},

"statements": [

{

"message": "Authentification script started on {{LoginURL}}",

"index": 1,

"enabled": true,

"elementType": "ZestActionPrint"

},

{

"windowHandle": "Site",

"browserType": "firefox",

"url": "{{LoginURL}}",

"capabilities": "",

"headless": true,

"index": 3,

"enabled": true,

"elementType": "ZestClientLaunch"

},

{

"rootExpression": {

"windowHandle": "Site",

"type": "classname",

"element": "{{enteredAppClassFlag}}",

"not": false,

"elementType": "ZestExpressionClientElementExists"

},

"ifStatements": [

{

"message": "Already logged in",

"index": 26,

"enabled": true,

"elementType": "ZestActionPrint"

}

],

"elseStatements": [

{

"set": {

"start": 0,

"end": 50,

"step": 1,

"elementType": "ZestLoopTokenIntegerSet"

},

"statements": [

{

"rootExpression": {

"windowHandle": "Site",

"type": "cssselector",

"element": "input[type\u003d\"email\"] ",

"not": false,

"elementType": "ZestExpressionClientElementExists"

},

"ifStatements": [

{

"message": "Login form rendered",

"index": 5,

"enabled": true,

"elementType": "ZestActionPrint"

},

{

"index": 0,

"enabled": true,

"elementType": "ZestControlLoopBreak"

}

],

"elseStatements": [

{

"milliseconds": 200,

"index": 7,

"enabled": true,

"elementType": "ZestActionSleep"

},

{

"message": "Waiting login form to render. Attempt {{TempWaitCycle}} from 50",

"index": 8,

"enabled": true,

"elementType": "ZestActionPrint"

}

],

"index": 1,

"enabled": true,

"elementType": "ZestConditional"

},

{

"rootExpression": {

"value": "49",

"variableName": "LoginURL",

"caseExact": false,

"not": false,

"elementType": "ZestExpressionEquals"

},

"ifStatements": [

{

"message": "Expired login form waiting",

"priority": "HIGH",

"index": 9,

"enabled": true,

"elementType": "ZestActionFail"

}

],

"elseStatements": [],

"index": 2,

"enabled": true,

"elementType": "ZestConditional"

}

],

"variableName": "TempWaitCycle",

"index": 31,

"enabled": true,

"elementType": "ZestLoopInteger"

},

{

"value": "{{Username}}",

"windowHandle": "Site",

"type": "name",

"element": "loginfmt",

"index": 3,

"enabled": true,

"elementType": "ZestClientElementSendKeys"

},

{

"windowHandle": "Site",

"type": "classname",

"element": "ext-primary",

"index": 4,

"enabled": true,

"elementType": "ZestClientElementClick"

},

{

"message": "Entered login",

"index": 5,

"enabled": true,

"elementType": "ZestActionPrint"

},

{

"set": {

"start": 0,

"end": 50,

"step": 1,

"elementType": "ZestLoopTokenIntegerSet"

},

"statements": [

{

"rootExpression": {

"windowHandle": "Site",

"type": "cssselector",

"element": "input[type\u003d\"password\"] ",

"not": false,

"elementType": "ZestExpressionClientElementExists"

},

"ifStatements": [

{

"message": "Password form rendered",

"index": 10,

"enabled": true,

"elementType": "ZestActionPrint"

},

{

"index": 0,

"enabled": true,

"elementType": "ZestControlLoopBreak"

}

],

"elseStatements": [

{

"milliseconds": 200,

"index": 12,

"enabled": true,

"elementType": "ZestActionSleep"

},

{

"message": "Waiting password form to render. Attempt {{TempWaitCycle}} from 50",

"index": 13,

"enabled": true,

"elementType": "ZestActionPrint"

}

],

"index": 1,

"enabled": true,

"elementType": "ZestConditional"

},

{

"rootExpression": {

"value": "49",

"variableName": "LoginURL",

"caseExact": false,

"not": false,

"elementType": "ZestExpressionEquals"

},

"ifStatements": [

{

"message": "Expired password form waiting",

"priority": "HIGH",

"index": 14,

"enabled": true,

"elementType": "ZestActionFail"

}

],

"elseStatements": [],

"index": 2,

"enabled": true,

"elementType": "ZestConditional"

}

],

"variableName": "TempWaitCycle",

"index": 6,

"enabled": true,

"elementType": "ZestLoopInteger"

},

{

"value": "{{Password}}",

"windowHandle": "Site",

"type": "name",

"element": "passwd",

"index": 3,

"enabled": true,

"elementType": "ZestClientElementSendKeys"

},

{

"milliseconds": 5000,

"index": 4,

"enabled": true,

"elementType": "ZestActionSleep"

},

{

"windowHandle": "Site",

"type": "cssselector",

"element": "input[type\u003d\"submit\"] ",

"index": 5,

"enabled": false,

"elementType": "ZestClientElementClick"

},

{

"windowHandle": "Site",

"type": "classname",

"element": "ext-primary",

"index": 6,

"enabled": true,

"elementType": "ZestClientElementClick"

},

{

"message": "Entered password",

"index": 7,

"enabled": true,

"elementType": "ZestActionPrint"

},

{

"set": {

"start": 0,

"end": 50,

"step": 1,

"elementType": "ZestLoopTokenIntegerSet"

},

"statements": [

{

"rootExpression": {

"windowHandle": "Site",

"type": "id",

"element": "KmsiDescription",

"not": false,

"elementType": "ZestExpressionClientElementExists"

},

"ifStatements": [

{

"message": "Stay signed in form rendered",

"index": 8,

"enabled": true,

"elementType": "ZestActionPrint"

},

{

"index": 9,

"enabled": true,

"elementType": "ZestControlLoopBreak"

}

],

"elseStatements": [

{

"milliseconds": 200,

"index": 10,

"enabled": true,

"elementType": "ZestActionSleep"

},

{

"message": "Waiting stay signed in form to render. Attempt {{TempWaitCycle}} from 50",

"index": 11,

"enabled": true,

"elementType": "ZestActionPrint"

}

],

"index": 1,

"enabled": true,

"elementType": "ZestConditional"

},

{

"rootExpression": {

"value": "49",

"variableName": "LoginURL",

"caseExact": false,

"not": false,

"elementType": "ZestExpressionEquals"

},

"ifStatements": [

{

"message": "Expired stay signed in form waiting",

"priority": "HIGH",

"index": 12,

"enabled": true,

"elementType": "ZestActionFail"

}

],

"elseStatements": [],

"index": 2,

"enabled": true,

"elementType": "ZestConditional"

}

],

"variableName": "TempWaitCycle",

"index": 7,

"enabled": true,

"elementType": "ZestLoopInteger"

},

{

"windowHandle": "Site",

"type": "classname",

"element": "ext-secondary",

"index": 13,

"enabled": true,

"elementType": "ZestClientElementClick"

},

{

"message": "Entered \"Do not exit the system?\"",

"index": 14,

"enabled": true,

"elementType": "ZestActionPrint"

},

{

"message": "Auth finished",

"index": 15,

"enabled": true,

"elementType": "ZestActionPrint"

},

{

"set": {

"start": 0,

"end": 50,

"step": 1,

"elementType": "ZestLoopTokenIntegerSet"

},

"statements": [

{

"rootExpression": {

"windowHandle": "Site",

"type": "classname",

"element": "{{enteredAppClassFlag}}",

"not": false,

"elementType": "ZestExpressionClientElementExists"

},

"ifStatements": [

{

"message": "Entered app",

"index": 16,

"enabled": true,

"elementType": "ZestActionPrint"

},

{

"milliseconds": 200,

"index": 17,

"enabled": true,

"elementType": "ZestActionSleep"

},

{

"index": 18,

"enabled": true,

"elementType": "ZestControlLoopBreak"

}

],

"elseStatements": [

{

"milliseconds": 200,

"index": 19,

"enabled": true,

"elementType": "ZestActionSleep"

},

{

"message": "Waiting entering app. Attempt {{TempWaitCycle}} from 50",

"index": 20,

"enabled": true,

"elementType": "ZestActionPrint"

}

],

"index": 1,

"enabled": true,

"elementType": "ZestConditional"

},

{

"rootExpression": {

"value": "49",

"variableName": "LoginURL",

"caseExact": false,

"not": false,

"elementType": "ZestExpressionEquals"

},

"ifStatements": [

{

"message": "Expired enter app waiting",

"priority": "HIGH",

"index": 21,

"enabled": true,

"elementType": "ZestActionFail"

}

],

"elseStatements": [],

"index": 2,

"enabled": true,

"elementType": "ZestConditional"

}

],

"variableName": "TempWaitCycle",

"index": 6,

"enabled": true,

"elementType": "ZestLoopInteger"

}

],

"index": 4,

"enabled": true,

"elementType": "ZestConditional"

},

{

"windowHandle": "Site",

"sleepInSeconds": 0,

"index": 22,

"enabled": true,

"elementType": "ZestClientWindowClose"

}

],

"authentication": [],

"index": 0,

"enabled": true,

"elementType": "ZestScript"

}

I create in \zap\wrk\auth_files\files_to_copy_in_zap_folder\xml\config.xml such file

<?xml version="1.0" encoding="UTF-8"?>

</scanner>

<name>html_tag_password</name>

<config>Password</config>

</scanner>

<name>html_type_password</name>

<config>Password</config>

<resBodyRegex>type\s*=\s*['"]?password['"]?</resBodyRegex>

</scanner>

<name>html_type_hidden</name>

<config>Hidden</config>

<resBodyRegex>type\s*=\s*['"]?hidden['"]?</resBodyRegex>

</scanner>

<name>html_type_upload</name>

<config>Upload</config>

</scanner>

<name>html_tag_object</name>

<config>Object</config>

</scanner>

<name>html_tag_script</name>

<config>Script</config>

</scanner>

<name>html_mailto</name>

<config>MailTo</config>

</scanner>

<name>html_setcookie</name>

<config>SetCookie</config>

<resHeadRegex>set-cookie</resHeadRegex>

</scanner>

<name>html_comment1</name>

<config>Comment</config>

</scanner>

<name>html_comment2</name>

<config>Comment</config>

</scanner>

<name>response_json</name>

<resHeadRegex>type\s*:\s*application\/json</resHeadRegex>

</scanner>

</autoTagScanners>

</pscans>

<name>AzureAuth.zst</name>

<type>authentication</type>

<enabled>false</enabled>

<file>/zap/wrk/scripts/authentication/AzureAuth.zst</file>

</scripts>

</script>

</config>

I copy in \zap\wrk\auth_files\APPNAME.ctxt.context my context.

Than I run such script

cp -rv /zap/wrk/auth_files/files_to_copy_in_zap_folder/* /zap/

I run scan using

zap-full-scan.py -t https://APPNAME.com/ -d -r zap-full-$(date +%Y-%m-%d_%H:%M).html -n /zap/wrk/auth_files/APPNAME.context -U %User Name from context, not his id%

That's how it should work for my case with Zest based auth?

среда, 23 декабря 2020 г. в 14:09:28 UTC+3, Pavel Hrachou:

eri...@augment1security.com

unread,

Dec 23, 2020, 10:35:42 AM12/23/20

to OWASP ZAP User Group

Ok, let's approach this one thing at a time.

1) But what should I use for engine and enabled parameter?
My ans: You might want to take a look at the config.xml when you had zap open in desktop mode. You should see that the
engine is "Mozilla Zest"
enabled is true

2) I don't need to modify zap-api-scan.py cause I don't have augment1security_accesstoken_setter.py?

My ans : what is the purpose of augment1security_accesstoken_setter.py in the context of the scenario in the blog? is that the same purpose in your use case?
We need to understand what is needed and what is not in your use case

3) Is it normal that I don't have in /zap/xml file log4j.properties?

My ans: When the blog post was created, it was create when the version of zap is 2.9
In the latest docker stable image, you can find that log4j.properties has been renamed to log4j2.properties in the /home/zap/.ZAP folder. If I recall correctly, /home/zap/.ZAP_D is for weekly images.

4) Should I modify zap_common.py?

My ans : The modification does not seem to be needed anymore. Haven't had a chance to have a closer look at the new scripts. You will have to give it a go and see what the zap.log says.

Miscellaneous comments

"I copy in \zap\wrk\auth_files\APPNAME.ctxt.context my context."

My ans: not sure what you meant by the last part "my context". Do you mean to say "I copy in \zap\wrk\auth_files\APPNAME.ctxt.context which is my context file." ?
Might be a good idea to attach long files instead of copy pasting it into the thread.

"zap-full-scan.py -t https://APPNAME.com/ -d -r zap-full-$(date +%Y-%m-%d_%H:%M).html -n /zap/wrk/auth_files/APPNAME.context -U %User Name from context, not his id% "

My ans: would appreciate if you could put in some punctuation so that it makes it easier to understand what you are trying to convey. I am referring to the last bit "Name from context, not his id% ". You are putting them all in the same line as the command you are trying to execute. The command "zap-full-scan.py -t https://APPNAME.com/ -d -r zap-full-$(date +%Y-%m-%d_%H:%M).html -n /zap/wrk/auth_files/APPNAME.context -U %User" seems to be ok. Give it a go and see what happens.

Best Regards,
Eric W.
Blog: https://augment1security.com/blog/
Twitter: @aug1sec
Facebook: https://www.facebook.com/aug1sec

Pavel Hrachou

unread,

Dec 24, 2020, 5:56:17 AM12/24/20

to OWASP ZAP User Group

"My ans: You might want to take a look at the config.xml when you had zap open in desktop mode. You should see that the"

Oh, I see. In zap installation folder default version of file.
But in C:\Users\USERNAME\OWASP ZAP\ on windows there is actual config.xml file.

</autoTagScanners>

</pscans>

<name>AzureAuth.zst</name>

<description>This is script for azure ZEST based authentication.</description>

<engine>Mozilla Zest</engine>

<type>authentication</type>

<file>/zap/wrk/scripts/authentication/AzureAuth.zst</file>

</scripts>

</script>

Updated to this. Tbh, interested to know, what enabled means. I have no idea where to find info about it, in reasonable amount of time.

"My ans : what is the purpose of augment1security_accesstoken_setter.py in the context of the scenario in the blog? is that the same purpose in your use case?

We need to understand what is needed and what is not in your use case"

I have cookie based session. It's defined in mine context. This script used to add accessToken from global variable set by other script used in this guide. So I assume that it's useless for me. Unless I need similar workaround for auth working in desktop mode, when I convert it into docker command.

" My ans: When the blog post was created, it was create when the version of zap is 2.9
In the latest docker stable image, you can find that log4j.properties has been renamed to log4j2.properties in the /home/zap/.ZAP folder. If I recall correctly, /home/zap/.ZAP_D is for weekly images. "

Oh it's in C:\Users\USERNAME\OWASP ZAP\ on windows. I was looking in zap folder.
That's it's content:

"name = ZAP Home Config

rootLogger.level = info

rootLogger.appenderRef.stdout.ref = stdout

rootLogger.appenderRef.rolling.ref = RollingFile

appender.console.type = Console

appender.console.name = stdout

appender.console.layout.type = PatternLayout

appender.console.layout.pattern = %r [%t] %-5level %logger{36} - %msg%n

property.filename = ${sys:zap.user.log}/zap.log

appender.rolling.type = RollingFile

appender.rolling.name = RollingFile

appender.rolling.fileName = ${filename}

appender.rolling.filePattern = ${filename}.%i

appender.rolling.immediateFlush = true

appender.rolling.layout.type = PatternLayout

appender.rolling.layout.pattern = %d [%-5t] %-5p %c{1} - %m%n

appender.rolling.policies.type = Policies

appender.rolling.policies.size.type = SizeBasedTriggeringPolicy

appender.rolling.policies.size.size=4MB

appender.rolling.strategy.type = DefaultRolloverStrategy

appender.rolling.strategy.max = 3

logger.paros.name = org.parosproxy.paros

logger.paros.level = info

logger.zap.name = org.zaproxy.zap

logger.zap.level = info

logger.commonshttpclient.name = org.apache.commons.httpclient

logger.commonshttpclient.level = error

# Disable Jericho log, it logs HTML parsing issues as errors.

logger.jericho.name = net.htmlparser.jericho

logger.jericho.level = off

# Prevent Crawljax from logging too many, not so useful, INFO messages.

# For example:

# INFO Crawler - New DOM is a new state! crawl depth is now 10

# INFO Crawler - Crawl depth is now 1

# INFO Crawler - Crawl depth is now 2

logger.crawljaxCrawler.name = com.crawljax.core.Crawler

logger.crawljaxCrawler.level = warn

# INFO UnfiredCandidateActions - There are 64 states with unfired actions

logger.crawljaxStateMachine.name = com.crawljax.core.state.StateMachine

logger.crawljaxStateMachine.level = warn

# INFO StateMachine - State state106 added to the StateMachine.

logger.crawljaxUnfired.name = com.crawljax.core.UnfiredCandidateActions

logger.crawljaxUnfired.level = warn"
Okay I change those strings to this values
rootLogger.level = debug

logger.paros.level = debug

logger.zap.level = debug

"My ans: would appreciate if you could put in some punctuation so that it makes it easier to understand what you are trying to convey. I am referring to the last bit "Name from context, not his id% ". You are putting them all in the same line as the command you are trying to execute. The command "zap-full-scan.py -t https://APPNAME.com/ -d -r zap-full-$(date +%Y-%m-%d_%H:%M).html -n /zap/wrk/auth_files/APPNAME.context -U %User" seems to be ok. Give it a go and see what happens."

Oh, %User Name from context, not his id% and many others was words I used as sanitizing replacement for sensitive info. Let me try to use this color to highlight such replacements. Sounds like a bad option, but enclosing in $, %, {} and other colors not sounds better. Hope you have better option in mind.
So now I have:
📦auth_files
┣ 📂files_to_copy_in_zap_folder
┃ ┣ 📂scripts
┃ ┃ ┗ 📂authentication
┃ ┃ ┃ ┣ 📜AzureAuth.zst
┃ ┣ 📂xml
┃ ┃ ┗ 📜config.xml
┃ ┗ 📜log4j2.properties
┣ 📜copy_files.sh
┣ 📜 APPNAME.context

┃ ┃ ┃ ┣ 📜AzureAuth.zst
Not changed since I given it previous time.

┃ ┣ 📂xml
┃ ┃ ┗ 📜config.xml
┃ ┗ 📜log4j2.properties
Described bofore in this message.

┣ 📜 APPNAME.context
<?xml version="1.0" encoding="UTF-8" standalone="no"?>

<name>http://APPNAME.com/</name>

<desc/>

<incregexes>http://APPNAME.com.*</incregexes>

<incregexes>https://APPNAME.com.*</incregexes>

<excregexes>h*SignOut*</excregexes>

<tech>

<include>Db.MySQL</include>

<include>Language.ASP</include>

<include>Language.C</include>

<include>Language.JavaScript</include>

<include>OS.Linux</include>

<include>OS.MacOS</include>

<include>OS.Windows</include>

<include>WS.Apache</include>

<include>WS.Tomcat</include>

<exclude>Db.CouchDB</exclude>

<exclude>Db.Firebird</exclude>

<exclude>Db.HypersonicSQL</exclude>

<exclude>Db.Microsoft Access</exclude>

<exclude>Db.Microsoft SQL Server</exclude>

<exclude>Db.MongoDB</exclude>

<exclude>Db.Oracle</exclude>

<exclude>Db.PostgreSQL</exclude>

<exclude>Db.SAP MaxDB</exclude>

<exclude>Db.SQLite</exclude>

<exclude>Db.Sybase</exclude>

<exclude>Language</exclude>

<exclude>Language.JSP/Servlet</exclude>

<exclude>Language.Java</exclude>

<exclude>Language.PHP</exclude>

<exclude>Language.Python</exclude>

<exclude>Language.Ruby</exclude>

<exclude>Language.XML</exclude>

</tech>

<class>org.zaproxy.zap.model.StandardParameterParser</class>

<config>{"kvps":"&","kvs":"=","struct":[]}</config>

</urlparser>

<class>org.zaproxy.zap.model.StandardParameterParser</class>

<config>{"kvps":"&","kvs":"=","struct":[]}</config>

</postparser>

<pollurl>http://APPNAME.com/api/Authorization</pollurl>

<pollunits>REQUESTS</pollunits>

<loggedin>\Q"fullName"\E</loggedin>

<loggedout>\QUnauthorizedE</loggedout>

<name>AzureAuth.zst</name>

<params>EXCLUDED SANITIZED DATA</params>

</script>

</authentication>

<users>

<user>1882;true;EXCLUDED SANITIZED DATA</user>

</users>

</session>

<basic>

<body/>

</basic>

</authorization>

</context>

</configuration>

┣ 📜copy_files.sh
consist string:
cp -rv /zap/wrk/auth_files/files_to_copy_in_zap_folder/* /zap/

So now I do:

I run copy_files.sh

I run scan using

zap-full-scan.py -t https://APPNAME.com/ -d -r zap-full-scan-$(date +%Y-%m-%d_%H-%M).html -n /zap/wrk/auth_files/APPNAME.context -U USERNAME
I use replacement to "USERNAME" instead of "%User" due to fact that I guide was used userid in one place.

$ zap-full-scan.py -t https://APPNAME.com/ -d -r zap-full-scan-$(date +%Y-%m-%d_%H-%M).html -n /zap/wrk/auth_files/APPNAME.context -U USERNAME

2020-12-24 10:46:19,943 Could not find custom hooks file at /home/zap/.zap_hooks.py

2020-12-24 10:46:19,943 Trigger hook: cli_opts, args: 1

2020-12-24 10:46:19,944 Using port: 53141

2020-12-24 10:46:19,944 Trigger hook: start_zap, args: 2

2020-12-24 10:46:19,944 Starting ZAP

2020-12-24 10:46:19,944 Params: ['zap-x.sh', '-daemon', '-port', '53141', '-host', '0.0.0.0', '-config', 'database.recoverylog=false', '-config', 'api.disablekey=true', '-config', 'api.addrs.addr.name=.*', '-config', 'api.addrs.addr.regex=true', '-config', 'spider.maxDuration=0', '-addonupdate', '-addoninstall', 'pscanrulesBeta', '-addoninstall', 'ascanrulesBeta']

2020-12-24 10:46:19,952 Starting new HTTP connection (1): localhost:53141

2020-12-24 10:46:20,954 Starting new HTTP connection (1): localhost:53141

2020-12-24 10:46:21,956 Starting new HTTP connection (1): localhost:53141

2020-12-24 10:46:22,958 Starting new HTTP connection (1): localhost:53141

2020-12-24 10:46:23,960 Starting new HTTP connection (1): localhost:53141

2020-12-24 10:46:24,962 Starting new HTTP connection (1): localhost:53141

2020-12-24 10:46:25,966 Starting new HTTP connection (1): localhost:53141

2020-12-24 10:46:26,969 Starting new HTTP connection (1): localhost:53141

2020-12-24 10:46:27,017 http://localhost:53141 "GET http://zap/JSON/core/view/version/ HTTP/1.1" 200 20

2020-12-24 10:46:27,018 ZAP Version 2.10.0

2020-12-24 10:46:27,019 Took 7 seconds

2020-12-24 10:46:27,020 Trigger hook: zap_started, args: 2

2020-12-24 10:46:27,020 Tune

2020-12-24 10:46:27,020 Disable all tags

2020-12-24 10:46:27,023 Starting new HTTP connection (1): localhost:53141

2020-12-24 10:46:27,029 http://localhost:53141 "GET http://zap/JSON/pscan/action/disableAllTags/?apikey= HTTP/1.1" 200 15

2020-12-24 10:46:27,030 Set max pscan alerts

2020-12-24 10:46:27,031 Starting new HTTP connection (1): localhost:53141

2020-12-24 10:46:27,037 http://localhost:53141 "GET http://zap/JSON/pscan/action/setMaxAlertsPerRule/?maxAlerts=10&apikey= HTTP/1.1" 200 15

2020-12-24 10:46:27,038 Trigger hook: zap_tuned, args: 1

2020-12-24 10:46:27,038 Trigger hook: zap_import_context, args: 2

2020-12-24 10:46:27,039 Starting new HTTP connection (1): localhost:53141

2020-12-24 10:46:27,057 http://localhost:53141 "GET http://zap/JSON/context/action/importContext/?contextFile=%2Fzap%2Fwrk%2Fvdm.talrace.context&apikey= HTTP/1.1" 400 52

2020-12-24 10:46:27,057 Failed to load context file /zap/wrk/vdm.talrace.context : does_not_exist

2020-12-24 10:46:27,058 Trigger hook: zap_import_context_wrap, args: 1

2020-12-24 10:46:27,058 Trigger hook: zap_set_scan_user, args: 2

ERROR <class 'TypeError'>

2020-12-24 10:46:27,059 Unexpected error: <class 'TypeError'>

Traceback (most recent call last):

File "/zap//zap-full-scan.py", line 337, in main

zap_set_scan_user(zap, user)

File "/zap/zap_common.py", line 104, in _wrap

return_data = func(*args_list, **kwargs)

File "/zap/zap_common.py", line 577, in zap_set_scan_user

for usr in context_users:

TypeError: 'NoneType' object is not iterable

Found Java version 11.0.9.1

Available memory: 2994 MB

Using JVM args: -Xmx748m

1389 [main] INFO org.zaproxy.zap.DaemonBootstrap - OWASP ZAP 2.10.0 started 24/12/2020, 10:42:09 with home /home/zap/.ZAP/

1448 [main] INFO org.parosproxy.paros.common.AbstractParam - Setting config database.recoverylog = false was false

1448 [main] INFO org.parosproxy.paros.common.AbstractParam - Setting config api.disablekey = true was true

1449 [main] INFO org.parosproxy.paros.common.AbstractParam - Setting config api.addrs.addr.name = .* was .*

1450 [main] INFO org.parosproxy.paros.common.AbstractParam - Setting config api.addrs.addr.regex = true was true

1450 [main] INFO org.parosproxy.paros.common.AbstractParam - Setting config spider.maxDuration = 0 was 0

1459 [main] INFO org.parosproxy.paros.network.SSLConnector - Reading supported SSL/TLS protocols...

1459 [main] INFO org.parosproxy.paros.network.SSLConnector - Using a SSLEngine...

1560 [main] INFO org.parosproxy.paros.network.SSLConnector - Done reading supported SSL/TLS protocols: [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2, TLSv1.3]

1561 [main] INFO org.parosproxy.paros.extension.option.OptionsParamCertificate - Unsafe SSL renegotiation disabled.

1977 [main] INFO hsqldb.db.HSQLDB379AF3DEBD.ENGINE - dataFileCache open start

1995 [main] INFO hsqldb.db.HSQLDB379AF3DEBD.ENGINE - dataFileCache commit start

2009 [main] INFO hsqldb.db.HSQLDB379AF3DEBD.ENGINE - dataFileCache commit end

2009 [main] INFO hsqldb.db.HSQLDB379AF3DEBD.ENGINE - dataFileCache open end

2066 [ZAP-daemon] INFO org.zaproxy.zap.control.ExtensionFactory - Loading extensions

3232 [ZAP-daemon] INFO org.zaproxy.zap.control.ExtensionFactory - Installed add-ons: [[id=alertFilters, version=10.0.0], [id=ascanrules, version=38.0.0], [id=ascanrulesBeta, version=33.0.0], [id=bruteforce, version=10.0.0], [id=commonlib, version=1.2.0], [id=diff, version=10.0.0], [id=directorylistv1, version=4.0.0], [id=domxss, version=10.0.0], [id=encoder, version=0.4.0], [id=formhandler, version=3.0.0], [id=fuzz, version=13.1.0], [id=gettingStarted, version=12.0.0], [id=graaljs, version=0.1.0], [id=graphql, version=0.2.0], [id=help, version=11.0.0], [id=hud, version=0.12.0], [id=importurls, version=7.0.0], [id=invoke, version=10.0.0], [id=onlineMenu, version=8.0.0], [id=openapi, version=17.0.0], [id=pscanrules, version=31.0.0], [id=pscanrulesBeta, version=24.0.0], [id=quickstart, version=29.0.0], [id=replacer, version=8.0.0], [id=retire, version=0.6.0], [id=reveal, version=3.0.0], [id=saverawmessage, version=5.0.0], [id=savexmlmessage, version=0.1.0], [id=scripts, version=28.0.0], [id=selenium, version=15.3.0], [id=soap, version=4.0.0], [id=spiderAjax, version=23.2.0], [id=tips, version=7.0.0], [id=webdriverlinux, version=23.0.0], [id=websocket, version=23.0.0], [id=zest, version=33.0.0]]

3292 [ZAP-daemon] ERROR org.zaproxy.zap.ZAP.UncaughtExceptionLogger - Exception in thread "ZAP-daemon"

java.awt.AWTError: Can't connect to X11 window server using ':1.0' as the value of the DISPLAY variable.

at sun.awt.X11GraphicsEnvironment.initDisplay(Native Method) ~[?:?]

at sun.awt.X11GraphicsEnvironment$1.run(X11GraphicsEnvironment.java:102) ~[?:?]

at java.security.AccessController.doPrivileged(Native Method) ~[?:?]

at sun.awt.X11GraphicsEnvironment.<clinit>(X11GraphicsEnvironment.java:61) ~[?:?]

at java.lang.Class.forName0(Native Method) ~[?:?]

at java.lang.Class.forName(Class.java:315) ~[?:?]

at java.awt.GraphicsEnvironment$LocalGE.createGE(GraphicsEnvironment.java:101) ~[?:?]

at java.awt.GraphicsEnvironment$LocalGE.<clinit>(GraphicsEnvironment.java:83) ~[?:?]

at java.awt.GraphicsEnvironment.getLocalGraphicsEnvironment(GraphicsEnvironment.java:129) ~[?:?]

at sun.awt.X11.XToolkit.<clinit>(XToolkit.java:231) ~[?:?]

at java.lang.Class.forName0(Native Method) ~[?:?]

at java.lang.Class.forName(Class.java:315) ~[?:?]

at java.awt.Toolkit$2.run(Toolkit.java:588) ~[?:?]

at java.awt.Toolkit$2.run(Toolkit.java:583) ~[?:?]

at java.security.AccessController.doPrivileged(Native Method) ~[?:?]

at java.awt.Toolkit.getDefaultToolkit(Toolkit.java:582) ~[?:?]

at javax.swing.ImageIcon.<init>(ImageIcon.java:198) ~[?:?]

at javax.swing.ImageIcon.<init>(ImageIcon.java:217) ~[?:?]

at org.zaproxy.zap.extension.hud.ExtensionHUD.<clinit>(ExtensionHUD.java:87) ~[?:?]

at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]

at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]

at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]

at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?]

at org.zaproxy.zap.control.AddOnLoaderUtils.loadAndInstantiateClassImpl(AddOnLoaderUtils.java:118) ~[zap-2.10.0.jar:2.10.0]

at org.zaproxy.zap.control.AddOnLoaderUtils.loadAndInstantiateClass(AddOnLoaderUtils.java:64) ~[zap-2.10.0.jar:2.10.0]

at org.zaproxy.zap.control.AddOnLoader.loadAddOnExtension(AddOnLoader.java:897) ~[zap-2.10.0.jar:2.10.0]

at org.zaproxy.zap.control.AddOnLoader.loadAddOnExtensions(AddOnLoader.java:886) ~[zap-2.10.0.jar:2.10.0]

at org.zaproxy.zap.control.AddOnLoader.getExtensions(AddOnLoader.java:842) ~[zap-2.10.0.jar:2.10.0]

at org.zaproxy.zap.control.AddOnLoader.getExtensions(AddOnLoader.java:813) ~[zap-2.10.0.jar:2.10.0]

at org.zaproxy.zap.control.ExtensionFactory.loadAllExtension(ExtensionFactory.java:109) ~[zap-2.10.0.jar:2.10.0]

at org.parosproxy.paros.control.Control.addExtension(Control.java:185) ~[zap-2.10.0.jar:2.10.0]

at org.parosproxy.paros.control.AbstractControl.loadExtension(AbstractControl.java:54) ~[zap-2.10.0.jar:2.10.0]

at org.parosproxy.paros.control.Control.init(Control.java:141) ~[zap-2.10.0.jar:2.10.0]

at org.parosproxy.paros.control.Control.initSingletonWithoutViewAndProxy(Control.java:371) ~[zap-2.10.0.jar:2.10.0]

at org.zaproxy.zap.HeadlessBootstrap.initControl(HeadlessBootstrap.java:58) ~[zap-2.10.0.jar:2.10.0]

at org.zaproxy.zap.DaemonBootstrap$1.run(DaemonBootstrap.java:75) ~[zap-2.10.0.jar:2.10.0]

at java.lang.Thread.run(Thread.java:834) [?:?]

2020-12-24 10:46:27,074 Trigger hook: pre_exit, args: 3

среда, 23 декабря 2020 г. в 18:35:42 UTC+3, eri...@augment1security.com:

eri...@augment1security.com

unread,

Dec 24, 2020, 8:18:07 AM12/24/20

to OWASP ZAP User Group

1) " Updated to this. Tbh, interested to know, what enabled means. I have no idea where to find info about it, in reasonable amount of time. "
For some scripts, you need to enable them for them to be used. You can find out more about scripting here - https://augment1security.com/general/a-gentle-introduction-to-zap-scripts-part-1/

2) There is some discrepancy here. You are getting

" Failed to load context file /zap/wrk/vdm.talrace.context : does_not_exist "

but you have specified " -n /zap/wrk/auth_files/APPNAME.context " on the commandline
where is " /zap/wrk/vdm.talrace.context " specified?

3) can i check with you if you are running your scan as described in the blog post that I gave earlier which is to have a script that kicks off everything like here - https://github.com/augmentonesecurity/api_scanning_with_auth_from_desktop_to_docker/blob/master/zap-script.sh?

Best Regards,
Eric W.
Blog: https://augment1security.com/blog/
Twitter: @aug1sec
Facebook: https://www.facebook.com/aug1sec

Pavel Hrachou

unread,

Dec 24, 2020, 8:48:27 AM12/24/20

to OWASP ZAP User Group

1) So it's the same as load on start in settings of script, just called confusingly other way?

2) I just not replaced sensitive info. That's all discrepancy. Can we hide it?

3) I don't use this script because I'm using CLI inside of container and I do zap-full-scan.py. This script run docker container, copy files in places, run scan, copy report, kill container. With my current approach most of this steps irrelevant. I will create script like this, or something else, later. Now I need to make it work. Polishing and automation later.
That's how I copy all the files I need to change in zap folder.

$ sh copy_files.sh

'/zap/wrk/auth_files/files_to_copy_in_zap_folder/log4j2.properties' -> '/zap/log4j2.properties'

'/zap/wrk/auth_files/files_to_copy_in_zap_folder/scripts/authentication/AzureAuth.zst' -> '/zap/scripts/authentication/AzureAuth.zst'

'/zap/wrk/auth_files/files_to_copy_in_zap_folder/xml/config.xml' -> '/zap/xml/config.xml'

"can i check with you if you are running your scan as described in the blog post"

Do you mean skype, discord, microsoft teams etc like variant? Because I'm ready for something like this.

четверг, 24 декабря 2020 г. в 16:18:07 UTC+3, eri...@augment1security.com:

eri...@augment1security.com

unread,

Dec 24, 2020, 9:38:33 AM12/24/20

to OWASP ZAP User Group

2) If its discrepancy, it's quite confusing and its impeding the ability to troubleshoot what the real issue is

3) The approach described in the blog works so might be a good idea to follow as closely as possible that approach, get something to work first then see how we can customize it later.

Best Regards,
Eric W.
Blog: https://augment1security.com/blog/
Twitter: @aug1sec
Facebook: https://www.facebook.com/aug1sec

Pavel Hrachou

unread,

Dec 24, 2020, 10:47:32 AM12/24/20

to OWASP ZAP User Group

2) Agree. But no sanitizing is bad too. Can you offer some better solution?

3) Right now I think that yeah, it makes sense. Just case in guide really different to mine. So I think right now more effective will be to try to resolve this issue from log.

java.awt.AWTError: Can't connect to X11 window server using ':1.0' as the value of the DISPLAY variable.

It will take some time.

четверг, 24 декабря 2020 г. в 17:38:33 UTC+3, eri...@augment1security.com:

Reply all

Reply to author

Forward