[JIRA] (JENKINS-57129) java.io.IOException: Unexpected cgroup syntax (docker+jenkins+systemd)

6 views
Skip to first unread message

whans@web.de (JIRA)

unread,
Apr 22, 2019, 5:46:02 AM4/22/19
to jenkinsc...@googlegroups.com
werner hans updated an issue
 
Jenkins / Bug JENKINS-57129
java.io.IOException: Unexpected cgroup syntax (docker+jenkins+systemd)
Change By: werner hans
Summary: java.io.IOException: Unexpected cgroup syntax (docker+jenkins+systemd)
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

whans@web.de (JIRA)

unread,
Apr 22, 2019, 5:47:08 AM4/22/19
to jenkinsc...@googlegroups.com
werner hans updated an issue
Change By: werner hans
Environment: RHEL 8 beta
Jenkins 2.173 (
- rpm based installation, running Installation
- started as service, Service
- running as in docker container )
docker 18.09.2
Jenkins container/Docker host on same machine - using shared binary, socket via bind mounts

whans@web.de (JIRA)

unread,
Apr 22, 2019, 5:56:02 AM4/22/19
to jenkinsc...@googlegroups.com
werner hans updated an issue
Hi,

following behaviour is very similar to JENKINS-56888, with another cgroup pattern. Change [https://github.com/jenkinsci/docker-workflow-plugin/pull/169] might be too restrictive.

Szenario, cf. Field Environment with the following configuration in Jenkins:

- pipeline script (tried both with groovy sandbox unchecked and checked):
{code:java}
pipeline {
    agent {
        docker {
            image 'xxx'
        }
    }
    stages {
        stage('Build') {
            steps {
                sh 'echo "abc"'
            }
        }
    }
}{code}
- console output of a build:

{code:java}
Started by user yyy
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/lib/jenkins/workspace/abcd
[Pipeline] {
[Pipeline] sh
+ docker inspect -f . xxx
.
[Pipeline] withDockerContainer
[Pipeline] // withDockerContainer
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
java.io.IOException: Unexpected cgroup syntax /docker/f334457e283784c7aac6cbf6245c135037a0c1cd36fd983a285f67ac7d1e0383/system.slice/jenkins.service
at org.jenkinsci.plugins.docker.workflow.client.ControlGroup.getContainerId(ControlGroup.java:60)
at org.jenkinsci.plugins.docker.workflow.client.ControlGroup.getContainerId(ControlGroup.java:45)
at org.jenkinsci.plugins.docker.workflow.client.ControlGroup.getContainerId(ControlGroup.java:37)
at org.jenkinsci.plugins.docker.workflow.client.DockerClient.getContainerIdIfContainerized(DockerClient.java:336)
at org.jenkinsci.plugins.docker.workflow.WithContainerStep$Execution.start(WithContainerStep.java:159)
at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:268)
at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:176)
...
{code}
Does jenkins check its own cgroup?
{code:java}
ps -xp 1310:
1310 ?        Ssl    2:12 /etc/alternatives/java -Dcom.sun.akuma.Daemon=daemonized -Djava.awt.headless=true -DJENKINS_HOME=/var/lib/jenkins -jar /usr/lib...

cat /proc/1310/cgroup
12:memory:/docker/f334457e283784c7aac6cbf6245c135037a0c1cd36fd983a285f67ac7d1e0383/system.slice/jenkins.service
11:pids:/docker/f334457e283784c7aac6cbf6245c135037a0c1cd36fd983a285f67ac7d1e0383/system.slice/jenkins.service
10:rdma:/
9:hugetlb:/docker/f334457e283784c7aac6cbf6245c135037a0c1cd36fd983a285f67ac7d1e0383
8:blkio:/docker/f334457e283784c7aac6cbf6245c135037a0c1cd36fd983a285f67ac7d1e0383
7:perf_event:/docker/f334457e283784c7aac6cbf6245c135037a0c1cd36fd983a285f67ac7d1e0383
6:freezer:/docker/f334457e283784c7aac6cbf6245c135037a0c1cd36fd983a285f67ac7d1e0383
5:cpuset:/docker/f334457e283784c7aac6cbf6245c135037a0c1cd36fd983a285f67ac7d1e0383
4:cpu,cpuacct:/docker/f334457e283784c7aac6cbf6245c135037a0c1cd36fd983a285f67ac7d1e0383
3:devices:/docker/f334457e283784c7aac6cbf6245c135037a0c1cd36fd983a285f67ac7d1e0383/system.slice/jenkins.service
2:net_cls,net_prio:/docker/f334457e283784c7aac6cbf6245c135037a0c1cd36fd983a285f67ac7d1e0383
1:name=systemd:/docker/f334457e283784c7aac6cbf6245c135037a0c1cd36fd983a285f67ac7d1e0383/system.slice/jenkins.service
{code}

Seems that systemd adds some harm. Running Jenkins in a container without systemd is fine: without any cgroup having a suffix '/system.slice/jenkins.service'.

whans@web.de (JIRA)

unread,
Apr 22, 2019, 5:57:02 AM4/22/19
to jenkinsc...@googlegroups.com
werner hans updated an issue
Change By: werner hans
Priority: Minor Trivial

iabramov@gmail.com (JIRA)

unread,
Feb 25, 2020, 4:02:06 AM2/25/20
to jenkinsc...@googlegroups.com
Ivan Abramov commented on Bug JENKINS-57129
 
Re: java.io.IOException: Unexpected cgroup syntax (docker+jenkins+systemd)

This is the workaround.
docker-workflow-plugin/src/main/java/org/jenkinsci/plugins/docker/workflow/client/ControlGroup.java

package org.jenkinsci.plugins.docker.workflow.client;

import com.google.common.base.Optional;
import hudson.FilePath;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.Reader;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;

import java.util.regex.Matcher;
import java.util.regex.Pattern;

/**
 * @author <a href="mailto:nicolas...@gmail.com">Nicolas De Loof</a>
 */
public class ControlGroup {

    /** hierarchy ID number */
    public final int id;

    /** set of subsystems bound to the hierarchy */
    public final String subsystems;

    /** control group in the hierarchy to which the process belongs */
    public final String group;

    private ControlGroup(String line) throws NumberFormatException, IndexOutOfBoundsException {
        String[] fields = line.split(":");
        id = Integer.parseInt(fields[0]);
        subsystems = fields[1];
        group = fields[2];
    }


    public static Optional<String> getContainerId(FilePath procfile) throws IOException, InterruptedException {
        return getContainerId(new InputStreamReader(procfile.read(), StandardCharsets.UTF_8));
    }

    static Optional<String> getContainerId(Reader reader) throws IOException {
        try (BufferedReader r = new BufferedReader(reader)) {
            String line;
            while ((line = r.readLine()) != null) {
                final ControlGroup cgroup = new ControlGroup(line);
                final String containerId = cgroup.getContainerId();
                if (containerId != null) return Optional.of(containerId);
            }
        }
        return Optional.absent();
    }

    public String getContainerId() throws IOException {

        Pattern regex = Pattern.compile("([a-z0-9]{64})");
        Matcher matcher = regex.matcher(group);

        String containerId = null; 
        
        while (matcher.find()) {
          containerId = matcher.group();
        }

        if (containerId != null) {
          return containerId;
        } else {
          throw new IOException("Unexpected cgroup syntax "+group);
        }
    }

}
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

iabramov@gmail.com (JIRA)

unread,
Feb 25, 2020, 9:16:12 AM2/25/20
to jenkinsc...@googlegroups.com
Ivan Abramov edited a comment on Bug JENKINS-57129
This is the workaround.
docker-workflow-plugin/src/main/java/org/jenkinsci/plugins/docker/workflow/client/ControlGroup.java
{code:java}
        Pattern regex         // 4:cpuset:/system.slice/docker-3dd988081e7149463c043b5d9c57d7309e079c5e9290f91feba1cc45a04d6a5b.scope
        // 2:cpu:/docker/3dd988081e7149463c043b5d9c57d7309e079c5e9290f91feba1cc45a04d6a5b
        // 2:cpu:/docker-ce/docker/7cacbc548047c130ae50653548f037285806d49c0c4c1543925cffb8873ed213
        // 10:cpu,cpuacct:/docker/a9f3c3932cd81c4a74cc7e0a18c3300255159512f1d000545c42895adaf68932/docker/3dd988081e7149463c043b5d9c57d7309e079c5e9290f91feba1cc45a04d6a5b
        // 3:cpu:/docker/4193df6bcf5fce75f3fc77f303b2ac06fb664adeb269b959b7ae17b3f8dcf329/3dd988081e7149463c043b5d9c57d7309e079c5e9290f91feba1cc45a04d6a5b
        // 7:cpu:/ecs/0410eff2-7e59-4111-823e-1e0d98ef7f30/3dd988081e7149463c043b5d9c57d7309e079c5e9290f91feba1cc45a04d6a5b
        // 8:cpuset:/kubepods.slice/kubepods-pod9c26dfb6_b9c9_11e7_bfb9_02c6c1fc4861.slice/docker-3dd988081e7149463c043b5d9c57d7309e079c5e9290f91feba1cc45a04d6a5b.scope
        // 12:freezer:/actions_job/ddecc467e1fb3295425e663efb6531282c1c936f25a3eeb7bb64e7b0fc61a216
        // /kubepods/burstable/pod1fe52ba4-5709-11ea-9ee3-00505682780f/d65c8853fa45d139ce95d5c2b68a6e4aa8da83894d8eb0396cd6edd1c134c97c/user.slice

        Matcher matcher = Pattern.compile("([a-z0-9]{64})") ;
        Matcher matcher = regex .matcher(group);


        String containerId = null;
        
        while (matcher.find()) {
          containerId = matcher.group();
        }

        if ( containerId != null == containerId &&
            Pattern.compile("^(\\/docker\\/|\\/ecs\\/|\\/docker-|\\/kubepods\\/|\\/actions_job\\/ ) .*").matcher(group).matches() )
{
           return containerId;
        } else {
           throw new IOException("Unexpected cgroup syntax "+group);
        }

        return containerId;
     }

}
{code}
Reply all
Reply to author
Forward
0 new messages