| I've been developing a pipeline that uses serverless to deploy an AWS website backed with lambda functions. I started noticing that some expected log statements were absent from the console log. Upon looking at the physical log in /var/lib/jenkins/jobs/OurProjectPortal/branches/SomeBranchName/builds/155/log, I saw that part of the log had been overwritten with NUL bytes (0x00), and this corresponded with the missing log output in the console log. Sample:
vi /var/lib/jenkins/jobs/OurProjectPortal/branches/SomeBranchName/builds/155/log
========
[2019-08-12T19:35:23.994Z] + echo Installing 'serverless' and plugins:
[2019-08-12T19:35:23.994Z] Installing 'serverless' and plugins:
[2019-08-12T19:35:23.994Z] + [ -e node_modules.zip ]
[2019-08-12T19:35:23.994Z] + unzip node_modules.zip
[2019-08-12T19:35:45.805Z] ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@....
@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@+ echo This decrypts secrets.blah-blah.yml.encrypted, containing db credentials and cognito user pool
[2019-08-12T19:35:45.805Z] This decrypts secrets.blah-blah.yml.encrypted, containing db credentials and cognito user pool
[2019-08-12T19:35:45.805Z] + echo
In previous situations where I've seen this, it's been caused by two processes writing to the same file using buffered I/O and no locking. Here's the affected part of the Jenkinsfile:
sedhex=2f5b2e5d5b5e205c2f302d392e5d2a202f207b2020732f2e2a5c285b2e5d5b5e205c2f302d392e5d2a5c29202e2a2f5c312f3b2070207d
sedprog=\$(echo \$sedhex | xxd -p -r)
echo "Installing 'serverless' and plugins:"
if [ -e node_modules.zip ]; then
unzip node_modules.zip > /dev/null 2>&1
else
npm install serverless serverless-python-requirements serverless-secrets-plugin serverless-plugin-git-variables
fi
echo ""
echo "Creating dummy secrets file:" > /dev/null
(echo "a:"; echo " n: x") > secrets.blah-blah.yml
echo "Zipping up python functions and requirements:"
echo "Count|Filename Extension"
zip -r9 function.zip -x '*/requirements.txt' \
\$(echo */requirements.txt | sed s,/requirements.txt,,g) \
| sed -n "\$sedprog" \
| sort | uniq -c | sort -nr
echo ""
echo "Waiting for CloudFormation stack to be available:"
while aws cloudformation list-stacks | jq -r '.StackSummaries[] | select(.StackName == "blah-blah") | .StackStatus' | tee /dev/stderr | grep _IN_PROGRESS; do
date +"%F %T"
sleep 30
done
echo "This decrypts secrets.blah-blah.yml.encrypted, containing db credentials and cognito user pool"
echo ""
serverless decrypt --stage blah-blah --password 'blah-blah'
echo ""
Is it possible that the
is what's causing this? Thanks! |
