[JIRA] (JENKINS-55998) recordIssues cannot see /workspace within docker.image.inside

6 views
Skip to first unread message

kerrhome (JIRA)

unread,
Feb 6, 2019, 2:06:02 PM2/6/19
to jenkinsc...@googlegroups.com
Shannon Kerr created an issue
 
Jenkins / Bug JENKINS-55998
recordIssues cannot see /workspace within docker.image.inside
Issue Type: Bug Bug
Assignee: Ulli Hafner
Components: warnings-ng-plugin
Created: 2019-02-06 19:05
Environment: Jenkins 2.138.1 and NG Warnings plugin version 2.1.2
Priority: Major Major
Reporter: Shannon Kerr

I'm running multiple parallel builds.  Some of these builds are run within docker containers.  The code would look something like this:

 

 

docker.image("$docker_server/$docker_image-dev-$arch:$build_tag").inside("--privileged -v $docker_image-$arch-ccache:/root/.ccache") {
    sh '''#!/bin/bash -e
        pushd /workspace
        <build here>
    '''
    recordIssues(tools: [gcc4(id: "gcc4-$os$arch", name: "GNU C Compiler $target")])
}

It does appear to capture all/only [?] the issues for that build within the docker container, but on the build page we are unable to see any of the issues in source context.  The build happens at /workspace within the docker container instead of $WORKSPACE, so the parallel build containers may share the one $WORKSPACE.  I don't think this is our issue though.

It appears the console output being parsed has the actual absolute path to the source files and this location is not visible to recordIssues. Since I can't tell recordIssues to ignore /workspace and instead look for the source in $WORKSPACE, it seems I'm stuck.

Example output from info screen:

Error Messages

Can't resolve absolute paths for some files:
- /workspace/live555/groupsock/../../live555/transport/include/TransportMediaSession.h
- /workspace/TinyXml/pixelproDevice.cpp
- /workspace/live555/BasicUsageEnvironment/../shared/ipplugin.h
- /workspace/TinyXml/../../live555/transport/include/StreamsockTransport.h
- /opt/gtest/1.8.0/x64/include/gtest/gtest.h
- /workspace/TinyXml/Services/soapC_003.cpp
- /workspace/TinyXml/./source/TransportMultiFramedRTPSource.cpp
- /workspace/TinyXml/tests.cpp
- /workspace/TinyXml/AudioOut.h
  ... skipped logging of 453 additional errors ...
Can't create fingerprints for some files:
- '/workspace/TinyXml/../AES/aeskey.c', IO exception has been thrown: java.nio.file.NoSuchFileException: /workspace/TinyXml/../AES/aeskey.c
- '/workspace/TinyXml/../AES/aeskey.c', IO exception has been thrown: java.nio.file.NoSuchFileException: /workspace/TinyXml/../AES/aeskey.c
.......

 

Information Messages

Parsing console log (workspace: '/var/lib/docker/builder/repo/workspace/_Branch_build_updates-SDSYGOEWO53Z6ASKV6W4GSRWQU4DXCNNDGKGTWMQJ4O7LTMGYQVQ')
Post processing issues on 'DockerAgent0' with encoding 'UTF-8'
Resolving absolute file names for all issues
-> 0 resolved, 473 unresolved, 0 already resolved
Copying affected files to Jenkins' build folder /var/jenkins_home/jobs/Server Branch/branches/build_updates/builds/169
-> 0 copied, 0 not in workspace, 473 not-found, 0 with I/O error
Resolving module names from module definitions (build.xml, pom.xml, or Manifest.mf files)
-> resolved module names for 3061 issues
Resolving package names (or namespaces) by parsing the affected files
-> resolved package names of 473 affected files
No filter has been set, publishing all 3061 issues
Creating fingerprints for all affected code blocks to track issues over different builds
-> created fingerprints for 0 issues
Skipping blaming as requested in the job configuration
Using reference build 'Server Branch/build_updates #166' to compute new, fixed, and outstanding issues
Issues delta (vs. reference build): outstanding: 2683, new: 378, fixed: 189
No quality gates have been set - skipping
Health report is disabled - skipping

 

We do not have this issue with the Legacy Warnings plugin.  Of course I have moved the previous warnings() call from a post-stage success{} block call to a recordIssues() call inside the docker.image.inside{} block, but I also tried the recordIssues() call in the same post-satge success{} block and had the same (no source) result.  So, it appears this just no longer works the way it did with the Legacy Warnings plugin.

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

kerrhome (JIRA)

unread,
Feb 6, 2019, 2:19:02 PM2/6/19
to jenkinsc...@googlegroups.com
Shannon Kerr commented on Bug JENKINS-55998
 
Re: recordIssues cannot see /workspace within docker.image.inside

It's kind of odd.  Here is a snippet from the console output:

include/TransportMediaSession.h:114:32: warning: 'transport::TransportMediaSubsession::m_parent' will be initialized after [-Wreorder]
 TransportMediaSession& m_parent;
 ^~~~~~~~

As you can see, there is no absolute path.  I checked a different build, pre NG Warnings plugin upgrade, and I can click on the file and see the warning in context, so it must be something with the new next gen plugin.  I'm glad to provide whatever info I can to help.  Thanks!

kerrhome (JIRA)

unread,
Feb 6, 2019, 2:39:02 PM2/6/19
to jenkinsc...@googlegroups.com
Shannon Kerr updated an issue
 
Change By: Shannon Kerr
(related to JENKINS-56001)

I'm running multiple parallel builds.  Some of these builds are run within docker containers.  The code would look something like this:

 

 
{code:java}

docker.image("$docker_server/$docker_image-dev-$arch:$build_tag").inside("--privileged -v $docker_image-$arch-ccache:/root/.ccache") {
    sh '''#!/bin/bash -e
        pushd /workspace
        <build here>
    '''
    recordIssues(tools: [gcc4(id: "gcc4-$os$arch", name: "GNU C Compiler $target")])
}
{code}

It does appear to capture all/only [?] the issues for that build within the docker container, but on the build page we are unable to see any of the issues in source context.  The build happens at /workspace within the docker container instead of $WORKSPACE, so the parallel build containers may share the one $WORKSPACE.  I don't think this is our issue though.

It appears the console output being parsed has the actual absolute path to the source files and this location is not visible to recordIssues. Since I can't tell recordIssues to ignore /workspace and instead look for the source in $WORKSPACE, it seems I'm stuck.

Example output from info screen:
{code:java}

Error Messages

Can't resolve absolute paths for some files:
- /workspace/live555/groupsock/../../live555/transport/include/TransportMediaSession.h
- /workspace/TinyXml/pixelproDevice.cpp
- /workspace/live555/BasicUsageEnvironment/../shared/ipplugin.h
- /workspace/TinyXml/../../live555/transport/include/StreamsockTransport.h
- /opt/gtest/1.8.0/x64/include/gtest/gtest.h
- /workspace/TinyXml/Services/soapC_003.cpp
- /workspace/TinyXml/./source/TransportMultiFramedRTPSource.cpp
- /workspace/TinyXml/tests.cpp
- /workspace/TinyXml/AudioOut.h
  ... skipped logging of 453 additional errors ...
Can't create fingerprints for some files:
- '/workspace/TinyXml/../AES/aeskey.c', IO exception has been thrown: java.nio.file.NoSuchFileException: /workspace/TinyXml/../AES/aeskey.c
- '/workspace/TinyXml/../AES/aeskey.c', IO exception has been thrown: java.nio.file.NoSuchFileException: /workspace/TinyXml/../AES/aeskey.c
.......{code}
 
{code:java}

Information Messages

Parsing console log (workspace: '/var/lib/docker/builder/repo/workspace/_Branch_build_updates-SDSYGOEWO53Z6ASKV6W4GSRWQU4DXCNNDGKGTWMQJ4O7LTMGYQVQ')
Post processing issues on 'DockerAgent0' with encoding 'UTF-8'
Resolving absolute file names for all issues
-> 0 resolved, 473 unresolved, 0 already resolved
Copying affected files to Jenkins' build folder /var/jenkins_home/jobs/Server Branch/branches/build_updates/builds/169
-> 0 copied, 0 not in workspace, 473 not-found, 0 with I/O error
Resolving module names from module definitions (build.xml, pom.xml, or Manifest.mf files)
-> resolved module names for 3061 issues
Resolving package names (or namespaces) by parsing the affected files
-> resolved package names of 473 affected files
No filter has been set, publishing all 3061 issues
Creating fingerprints for all affected code blocks to track issues over different builds
-> created fingerprints for 0 issues
Skipping blaming as requested in the job configuration
Using reference build 'Server Branch/build_updates #166' to compute new, fixed, and outstanding issues
Issues delta (vs. reference build): outstanding: 2683, new: 378, fixed: 189
No quality gates have been set - skipping
Health report is disabled - skipping
{code}

 

We do not have this issue with the Legacy Warnings plugin.  Of course I have moved the previous warnings() call from a post-stage success{} block call to a recordIssues() call inside the docker.image.inside{} block, but I also tried the recordIssues() call in the same post-satge success{} block and had the same (no source) result.  So, it appears this just no longer works the way it did with the Legacy Warnings plugin.

ullrich.hafner@gmail.com (JIRA)

unread,
Feb 6, 2019, 4:26:02 PM2/6/19
to jenkinsc...@googlegroups.com
Ulli Hafner commented on Bug JENKINS-55998
 
Re: recordIssues cannot see /workspace within docker.image.inside

The absolute path is automatically resolved. Are you building with make and have some make directory logging statements?

make[2]: Entering directory '/dir1/dir2/dir3'

I wonder why the original workspace is shown at /var/lib/docker/builder/repo/workspace/_Branch_build_updates-SDSYGOEWO53Z6ASKV6W4GSRWQU4DXCNNDGKGTWMQJ4O7LTMGYQVQ, but the warnings has the prefix /workspace

kerrhome (JIRA)

unread,
Feb 7, 2019, 9:36:02 AM2/7/19
to jenkinsc...@googlegroups.com

Yes we are using Make.  The original source is located in $WORKSPACE, but we use an overlay (see mount -t overlay) so we can share the same $WORKSPACE without writing to the $WORKSPACE (we write to /workspace instead, as you can see).  So, yes, the make 'Entering directory' paths look like:

make[1]: Entering directory '/workspace/TinyXml'

The odd thing is that the legacy warnings plugin handled this somehow, except for content that is only in the docker build container (not on the host machine), since the warnings processing, in our case, was being run after the build step completed.

 

The original $WORKSPACE would be the typical (something like):  /var/lib/docker/builder/repo/workspace/_Branch_build_updates-SDSYGOEWO53Z6ASKV6W4GSRWQU4DXCNNDGKGTWMQJ4O7LTMGYQVQ,

but that is not where we're actually building inside the docker container (instead the /workspace overlay).

ullrich.hafner@gmail.com (JIRA)

unread,
Feb 7, 2019, 10:12:02 AM2/7/19
to jenkinsc...@googlegroups.com

The old warnings plugin was allowed to read all files of the whole file system. I considered this as a security problem, so the new plugin is allowed to read sources only from the workspace.

kerrhome (JIRA)

unread,
Feb 7, 2019, 11:05:02 AM2/7/19
to jenkinsc...@googlegroups.com

OK, I understand.  So, we may lose the ability to see our warnings in context if we move to the Next Gen Warnings plugin because we use the overlay for our builds within docker containers?  Using the overlay allows us to checkout one time and share that checkout with 5 or sometimes more target builds.  I can't see us moving to checking out 5 times and I'm not even sure how we'd do that since all of the docker containers are used in parallel from the same host.

I'm using the updated plugins from last night and, as you would expect, I have the same issue with my Linux builds.  I'm not sure what we're going to do here except maybe stay with the legacy plugin for our gcc warnings processing.  If you have any ideas for me, I'm open to them.  If we had a way of telling the plugin to replace "/workspace" with "$WORKSPACE", that might work.

ullrich.hafner@gmail.com (JIRA)

unread,
Feb 7, 2019, 11:17:02 AM2/7/19
to jenkinsc...@googlegroups.com

Well, I wonder if we can tell
(1) either Jenkins to return /workspace as $WORKSPACE and not {{ /var/lib/docker/builder/repo/workspace/_Branch_build_updates-SDSYGOEWO53Z6ASKV6W4GSRWQU4DXCNNDGKGTWMQJ4O7LTMGYQVQ}}
(2) or make to print {{ /var/lib/docker/builder/repo/workspace/_Branch_build_updates-SDSYGOEWO53Z6ASKV6W4GSRWQU4DXCNNDGKGTWMQJ4O7LTMGYQVQ}} rather than /workspace

ullrich.hafner@gmail.com (JIRA)

unread,
Feb 7, 2019, 11:18:01 AM2/7/19
to jenkinsc...@googlegroups.com
Ulli Hafner edited a comment on Bug JENKINS-55998

ullrich.hafner@gmail.com (JIRA)

unread,
Feb 7, 2019, 11:19:01 AM2/7/19
to jenkinsc...@googlegroups.com
Ulli Hafner edited a comment on Bug JENKINS-55998
Well, I wonder if we can tell
(1) # either Jenkins to return {{/workspace}} as \ $WORKSPACE and not
{{ /var/lib/docker/builder/repo/workspace/_Branch_build_updates-SDSYGOEWO53Z6ASKV6W4GSRWQU4DXCNNDGKGTWMQJ4O7LTMGYQVQ}}
(2) # or make to print  {{ /var/lib/docker/builder/repo/workspace/_Branch_build_updates-SDSYGOEWO53Z6ASKV6W4GSRWQU4DXCNNDGKGTWMQJ4O7LTMGYQVQ}}  rather than {{/workspace}}

kerrhome (JIRA)

unread,
Feb 7, 2019, 12:24:04 PM2/7/19
to jenkinsc...@googlegroups.com

I don't think I can get option 2 to work. If I had a way to capture the log and run sed over it to do the replacement, I guess that could get me what I need, but that seems hacky.

kerrhome (JIRA)

unread,
Feb 11, 2019, 4:32:01 PM2/11/19
to jenkinsc...@googlegroups.com

If I wrap the recordIssues call in the docker.image.inside block, should recordIssues be able to see the filesystem within the docker container and see /workspace?

ullrich.hafner@gmail.com (JIRA)

unread,
Feb 13, 2019, 4:27:01 AM2/13/19
to jenkinsc...@googlegroups.com

I don't think that the wrapping is the problem. Currently the parser combines a path by concatenating the output of make and gcc. Copying of the files is then done by a call that is initiated by the master. This call does know nothing about a /workspace.

kerrhome (JIRA)

unread,
Feb 14, 2019, 12:47:02 PM2/14/19
to jenkinsc...@googlegroups.com

Ah, understood.  Please let me know if there is anything I can do.  I may just have to capture stdout/err from the build into a log file, use sed to change /workspace to $WORKSPACE, and then use recordIssues on that log file.  I really want to deploy the new Warnings plugin, but I can't lose this functionality.  Thanks!

ullrich.hafner@gmail.com (JIRA)

unread,
Feb 19, 2019, 8:47:02 AM2/19/19
to jenkinsc...@googlegroups.com

I think providing a source code prefix as suggested in JENKINS-48802 (and disabling the automatic path resolution) should help here as well.

ullrich.hafner@gmail.com (JIRA)

unread,
Mar 2, 2019, 8:17:02 AM3/2/19
to jenkinsc...@googlegroups.com
Ulli Hafner started work on Bug JENKINS-55998
 
Change By: Ulli Hafner
Status: Open In Progress

kerrhome (JIRA)

unread,
Mar 6, 2019, 11:13:02 AM3/6/19
to jenkinsc...@googlegroups.com

ullrich.hafner@gmail.com (JIRA)

unread,
Mar 6, 2019, 2:24:09 PM3/6/19
to jenkinsc...@googlegroups.com

The requirements of JENKINS-48802 are currently checked in a security review. Will need some time... It is not clear how this impacts security, since currently only workspace files are allowed to be shown.

kerrhome (JIRA)

unread,
Mar 11, 2019, 10:54:29 AM3/11/19
to jenkinsc...@googlegroups.com

kerrhome (JIRA)

unread,
Mar 11, 2019, 10:55:01 AM3/11/19
to jenkinsc...@googlegroups.com

ullrich.hafner@gmail.com (JIRA)

unread,
Sep 25, 2019, 9:10:02 AM9/25/19
to jenkinsc...@googlegroups.com
Ulli Hafner updated an issue
 
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

ullrich.hafner@gmail.com (JIRA)

unread,
Oct 23, 2019, 4:50:52 AM10/23/19
to jenkinsc...@googlegroups.com
Ulli Hafner resolved as Fixed
Change By: Ulli Hafner
Status: In Progress Resolved
Resolution: Fixed
Released As: 7.0.0
Reply all
Reply to author
Forward
0 new messages