Re: [kubernetes/kubernetes] Containerized subpath (#63143)
Kubernetes Submit Queue
[MILESTONENOTIFIER] Milestone Pull Request Needs Approval
@brendandburns @dchen1107 @jsafrane @lavalamp @msau42 @smarterclayton @thockin @kubernetes/sig-storage-misc
Action required: This pull request must have the status/approved-for-milestone label applied by a SIG maintainer. If the label is not applied within 7 days, the pull request will be moved out of the v1.11 milestone.
sig/storage: Pull Request will be escalated to these SIGs if needed.priority/important-soon: Escalate to the pull request owners and SIG owner; move out of milestone after several unsuccessful escalation attempts.kind/bug: Fixes a bug discovered during the current release.
—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
k8s-ci-robot
@jsafrane: The following tests failed, say /retest to rerun them all:
| Test name | Commit | Details | Rerun command |
|---|---|---|---|
| pull-kubernetes-local-e2e | fdb50b5 | link | /test pull-kubernetes-local-e2e |
| pull-kubernetes-integration | cb5eb25 | link | /test pull-kubernetes-integration |
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.
k8s-ci-robot
@jsafrane: The following tests failed, say /retest to rerun them all:
| Test name | Commit | Details | Rerun command |
|---|
| pull-kubernetes-local-e2e | fdb50b5 | link | /test pull-kubernetes-local-e2e |
| pull-kubernetes-integration | cb5eb25 | link | /test pull-kubernetes-integration |
| pull-kubernetes-verify | cb5eb25 | link | /test pull-kubernetes-verify |
fejta-bot
/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).
Review the full test history for this PR.
Silence the bot with an /lgtm cancel comment for consistent failures.
Davanum Srinivas
/status approved-for-milestone
Kubernetes Submit Queue
[MILESTONENOTIFIER] Milestone Pull Request: Up-to-date for process
Pull Request Labelssig/storage: Pull Request will be escalated to these SIGs if needed.priority/important-soon: Escalate to the pull request owners and SIG owner; move out of milestone after several unsuccessful escalation attempts.kind/bug: Fixes a bug discovered during the current release.
—
Kubernetes Submit Queue
Automatic merge from submit-queue (batch tested with PRs 63348, 63839, 63143, 64447, 64567). If you want to cherry-pick this change to another branch, please follow the instructions here.
Kubernetes Submit Queue
Merged #63143.
James Munnelly
@msau42 I'm running kubelet under rkt using the kubelet wrapper, with kube 1.11.0.
To fix this, there is one new requirement in containerized kubelet deployment that /rootfs needs to be mounted rw
My systemd unit does not currently have rootfs mounted into the kubelet container, so I have attempted to add it in an attempt to take advantage of this patch. When doing so, kubelet then refuses to start with:
failed to run Kubelet: could not detect clock speed from output: ""
I'm wondering if I'm missing something obvious here? I am able to mount standard host path volumes fine, but when using the prometheus operator (which enforces a subPath be used), it will not work.
Is there any more detailed info on how I should configure this? I'd imagine (hope!) that tectonic doesn't suffer from this same problem, however I'm unsure where to dig into to find this out, and I'd prefer not to have to spin up an entire tectonic cluster just to find a couple of arguments! 😄
Jan Šafránek
@munnerz, list of directories that must be in the container with kubelet is here:
Please report if there is anything missing.
lichuqiang
@jsafrane
I ran into a problem when running the unit test in "nsenter_mount_test.go",
and got error like this:
--- FAIL: TestNsenterExistsFile (0.01s)
nsenter_mount_test.go:343: Test "simple non-accessible file": expected error, got none
nsenter_mount_test.go:347: Test "simple non-accessible file": expected return value false, got true
E0719 09:26:23.220671 67952 mount_linux.go:495] format of disk "/dev/foo" failed: type:("ext4") target:("/tmp/mount681850900") options:(["defaults"])error:(formatting failed)
E0719 09:26:23.221027 67952 mount_linux.go:529] Could not determine if disk "/dev/foo" is formatted (exit 4)
FAIL
FAIL k8s.io/kubernetes/pkg/util/mount 0.077s
I switched to a non root user, and it got passed.
Is it required to manually switch to other users to run the case?
Justin Santa Barbara
@justinsb commented on this pull request.
> - // else. E.g. if the directory already exists, it may exists outside of the - // base due to symlinks. - SafeMakeDir(pathname string, base string, perm os.FileMode) error - // ExistsPath checks whether the path exists. - // Will operate in the host mount namespace if kubelet is running in a container - ExistsPath(pathname string) bool + // SafeMakeDir creates subdir within given base. It makes sure that the + // created directory does not escape given base directory mis-using + // symlinks. Note that the function makes sure that it creates the directory + // somewhere under the base, nothing else. E.g. if the directory already + // exists, it may exist outside of the base due to symlinks. + // This method should be used if the directory to create is inside volume + // that's under user control. User must not be able to use symlinks to + // escape the volume to create directories somewhere else. + SafeMakeDir(subdir string, base string, perm os.FileMode) error + // Will operate in the host mount namespace if kubelet is running in a container.
SafeFormatAndMount is a very delicate and tricky operation, and kops uses it (and this package). Another vote from me for keeping the layers well-factored here.
Michelle Au
@msau42 commented on this pull request.
> - // else. E.g. if the directory already exists, it may exists outside of the - // base due to symlinks. - SafeMakeDir(pathname string, base string, perm os.FileMode) error - // ExistsPath checks whether the path exists. - // Will operate in the host mount namespace if kubelet is running in a container - ExistsPath(pathname string) bool + // SafeMakeDir creates subdir within given base. It makes sure that the + // created directory does not escape given base directory mis-using + // symlinks. Note that the function makes sure that it creates the directory + // somewhere under the base, nothing else. E.g. if the directory already + // exists, it may exist outside of the base due to symlinks. + // This method should be used if the directory to create is inside volume + // that's under user control. User must not be able to use symlinks to + // escape the volume to create directories somewhere else. + SafeMakeDir(subdir string, base string, perm os.FileMode) error + // Will operate in the host mount namespace if kubelet is running in a container.
There is ongoing work to refactor the mount library to separate out k8s specific operations (like subpath processing), and general mounting/formatting utilities. #68513