Re: [kubernetes/kubernetes] Don't try to attach volumes which are already attached to other nodes (#40148)
Saad Ali
@codablock I'd like to better understand the problems that Azure volumes are having. It sounds like some bigger infrastructure changes maybe needed to fix the issues. We are about 10 minutes away from code freeze for 1.6. While we can can get small bug fixes in during code freeze, the big changes will have to wait for 1.7. But I really want to make sure we don't miss 1.7. So how about we set up a meeting early in the 1.7 dev cycle (I'm thinking 2nd week of April), to discuss what the pain points are, propose some changes, and hopefully get them implemented in 1.7?
CC @kubernetes/sig-storage-misc
—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
Alexander Block
@saad-ali As I won't have time to work on Kubernetes and/or Azure in the next few month I would suggest that @colemickens and @khenidak take over the discussion and do a meeting with you if required.
@khenidak is working on Azure managed disks and as I understood many of the problems I encountered with Azure Disks and which I tried to fix with PRs like these are also fixed in his work. So maybe this PR isn't even needed anymore.
Saad Ali
One of the items for Q2 2017 (v1.7) is improving Azure support. We should consider this PR as part of that effort.
CC @rootfs @chakri-nelluri for review
Saad Ali
Ack. Tracking both PRs
Alexander Block
REPLACED BY #45346
Alexander Block
Closed #40148.
smartVan
@zhonglin6666 commented on this pull request.
In pkg/controller/volume/attachdetach/reconciler/reconciler.go:
> @@ -206,16 +250,28 @@ func (rc *reconciler) reconcile() {
glog.V(5).Infof("Volume %q/Node %q is attached--touching.", volumeToAttach.VolumeName, volumeToAttach.NodeName)
rc.actualStateOfWorld.ResetDetachRequestTime(volumeToAttach.VolumeName, volumeToAttach.NodeName)
} else {
+ // Don't even try to start an operation if there is already one running
+ if rc.attacherDetacher.IsOperationPending(volumeToAttach.VolumeName, "") {
+ glog.V(10).Infof("Operation for volume %q is already running. Can't start attach for %q", volumeToAttach.VolumeName, volumeToAttach.NodeName)
+ continue
+ }
+
+ if rc.isMultiAttachForbidden(volumeToAttach.VolumeSpec) {
+ nodes := rc.actualStateOfWorld.GetNodesForVolume(volumeToAttach.VolumeName)
If set pv with accessMode 'ReadWriteOnce'，pod with pv running on node1.
when node1 is down，schedule pod to node2，new pod waitting for attach，but old pod is still not detach if node1 is still down
isMultiAttachForbidden and GetNodeForVolume will produce
attachdetach-controller Multi-Attach error for volume "pvc-d0fde86c-8661-11e9-b873-0800271c9f15" Volume is already used by pod