[go] path/filepath: re-implement windows EvalSymlinks

Alex Brainman (Gerrit)

unread,

Aug 12, 2017, 3:02:34 AM8/12/17

to Ian Lance Taylor, Rob Pike, Hiroshi Ioka, golang-co...@googlegroups.com

Alex Brainman has uploaded this change for review.

path/filepath: re-implement windows EvalSymlinks

CL 41834 used approach suggested by Raymond Chen in
https://blogs.msdn.microsoft.com/oldnewthing/20100212-00/?p=14963/
to implement os.Stat by getting Windows I/O manager
follow symbolic links.

Do the same for path/filepath.EvalSymlinks.

Old version of EvalSymlinks had special treatment for
symlinks with some relative paths. But new EvalSymlinks
always returns absolute paths. So adjust TestEvalSymlinks
to accommodate for this change.

Updates #19922
Fixes #20506

Change-Id: If2008ca0ebf639553d23bae0d6a5f77f96f0f621
---
M src/go/build/deps_test.go
M src/internal/syscall/windows/syscall_windows.go
M src/internal/syscall/windows/zsyscall_windows.go
M src/os/os_windows_test.go
M src/path/filepath/path_test.go
M src/path/filepath/path_windows_test.go
M src/path/filepath/symlink_windows.go
7 files changed, 147 insertions(+), 50 deletions(-)

diff --git a/src/go/build/deps_test.go b/src/go/build/deps_test.go
index 87abfba..d07debd 100644
--- a/src/go/build/deps_test.go
+++ b/src/go/build/deps_test.go
@@ -156,7 +156,7 @@
 
 	"internal/poll": {"L0", "internal/race", "syscall", "time", "unicode/utf16", "unicode/utf8"},
 	"os":            {"L1", "os", "syscall", "time", "internal/poll", "internal/syscall/windows"},
-	"path/filepath": {"L2", "os", "syscall"},
+	"path/filepath": {"L2", "os", "syscall", "internal/syscall/windows"},
 	"io/ioutil":     {"L2", "os", "path/filepath", "time"},
 	"os/exec":       {"L2", "os", "context", "path/filepath", "syscall"},
 	"os/signal":     {"L2", "os", "syscall"},
diff --git a/src/internal/syscall/windows/syscall_windows.go b/src/internal/syscall/windows/syscall_windows.go
index ec08a5a..af87416 100644
--- a/src/internal/syscall/windows/syscall_windows.go
+++ b/src/internal/syscall/windows/syscall_windows.go
@@ -165,3 +165,19 @@
 
 //sys  NetShareAdd(serverName *uint16, level uint32, buf *byte, parmErr *uint16) (neterr error) = netapi32.NetShareAdd
 //sys  NetShareDel(serverName *uint16, netName *uint16, reserved uint32) (neterr error) = netapi32.NetShareDel
+
+const (
+	FILE_NAME_NORMALIZED = 0x0
+	FILE_NAME_OPENED     = 0x8
+
+	VOLUME_NAME_DOS  = 0x0
+	VOLUME_NAME_GUID = 0x1
+	VOLUME_NAME_NONE = 0x4
+	VOLUME_NAME_NT   = 0x2
+)
+
+//sys	GetFinalPathNameByHandle(file syscall.Handle, filePath *uint16, filePathSize uint32, flags uint32) (n uint32, err error) = kernel32.GetFinalPathNameByHandleW
+
+func LoadGetFinalPathNameByHandle() error {
+	return procGetFinalPathNameByHandleW.Find()
+}
diff --git a/src/internal/syscall/windows/zsyscall_windows.go b/src/internal/syscall/windows/zsyscall_windows.go
index 7a2ffee..e882c89 100644
--- a/src/internal/syscall/windows/zsyscall_windows.go
+++ b/src/internal/syscall/windows/zsyscall_windows.go
@@ -41,21 +41,22 @@
 	modnetapi32 = syscall.NewLazyDLL(sysdll.Add("netapi32.dll"))
 	modadvapi32 = syscall.NewLazyDLL(sysdll.Add("advapi32.dll"))
 
-	procGetAdaptersAddresses  = modiphlpapi.NewProc("GetAdaptersAddresses")
-	procGetComputerNameExW    = modkernel32.NewProc("GetComputerNameExW")
-	procMoveFileExW           = modkernel32.NewProc("MoveFileExW")
-	procGetModuleFileNameW    = modkernel32.NewProc("GetModuleFileNameW")
-	procGetACP                = modkernel32.NewProc("GetACP")
-	procGetConsoleCP          = modkernel32.NewProc("GetConsoleCP")
-	procMultiByteToWideChar   = modkernel32.NewProc("MultiByteToWideChar")
-	procGetCurrentThread      = modkernel32.NewProc("GetCurrentThread")
-	procNetShareAdd           = modnetapi32.NewProc("NetShareAdd")
-	procNetShareDel           = modnetapi32.NewProc("NetShareDel")
-	procImpersonateSelf       = modadvapi32.NewProc("ImpersonateSelf")
-	procRevertToSelf          = modadvapi32.NewProc("RevertToSelf")
-	procOpenThreadToken       = modadvapi32.NewProc("OpenThreadToken")
-	procLookupPrivilegeValueW = modadvapi32.NewProc("LookupPrivilegeValueW")
-	procAdjustTokenPrivileges = modadvapi32.NewProc("AdjustTokenPrivileges")
+	procGetAdaptersAddresses      = modiphlpapi.NewProc("GetAdaptersAddresses")
+	procGetComputerNameExW        = modkernel32.NewProc("GetComputerNameExW")
+	procMoveFileExW               = modkernel32.NewProc("MoveFileExW")
+	procGetModuleFileNameW        = modkernel32.NewProc("GetModuleFileNameW")
+	procGetACP                    = modkernel32.NewProc("GetACP")
+	procGetConsoleCP              = modkernel32.NewProc("GetConsoleCP")
+	procMultiByteToWideChar       = modkernel32.NewProc("MultiByteToWideChar")
+	procGetCurrentThread          = modkernel32.NewProc("GetCurrentThread")
+	procNetShareAdd               = modnetapi32.NewProc("NetShareAdd")
+	procNetShareDel               = modnetapi32.NewProc("NetShareDel")
+	procGetFinalPathNameByHandleW = modkernel32.NewProc("GetFinalPathNameByHandleW")
+	procImpersonateSelf           = modadvapi32.NewProc("ImpersonateSelf")
+	procRevertToSelf              = modadvapi32.NewProc("RevertToSelf")
+	procOpenThreadToken           = modadvapi32.NewProc("OpenThreadToken")
+	procLookupPrivilegeValueW     = modadvapi32.NewProc("LookupPrivilegeValueW")
+	procAdjustTokenPrivileges     = modadvapi32.NewProc("AdjustTokenPrivileges")
 )
 
 func GetAdaptersAddresses(family uint32, flags uint32, reserved uintptr, adapterAddresses *IpAdapterAddresses, sizePointer *uint32) (errcode error) {
@@ -157,6 +158,19 @@
 	return
 }
 
+func GetFinalPathNameByHandle(file syscall.Handle, filePath *uint16, filePathSize uint32, flags uint32) (n uint32, err error) {
+	r0, _, e1 := syscall.Syscall6(procGetFinalPathNameByHandleW.Addr(), 4, uintptr(file), uintptr(unsafe.Pointer(filePath)), uintptr(filePathSize), uintptr(flags), 0, 0)
+	n = uint32(r0)
+	if n == 0 {
+		if e1 != 0 {
+			err = errnoErr(e1)
+		} else {
+			err = syscall.EINVAL
+		}
+	}
+	return
+}
+
 func ImpersonateSelf(impersonationlevel uint32) (err error) {
 	r1, _, e1 := syscall.Syscall(procImpersonateSelf.Addr(), 1, uintptr(impersonationlevel), 0, 0)
 	if r1 == 0 {
diff --git a/src/os/os_windows_test.go b/src/os/os_windows_test.go
index 04c4a4a..121fda7 100644
--- a/src/os/os_windows_test.go
+++ b/src/os/os_windows_test.go
@@ -520,10 +520,17 @@
 	if err != nil {
 		t.Fatal(err)
 	}
-
 	if got != target {
 		t.Errorf(`os.Readlink("%s"): got %v, want %v`, link, got, target)
 	}
+
+	got, err = filepath.EvalSymlinks(link)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if got != target {
+		t.Errorf(`filepath.EvalSymlinks("%s"): got %v, want %v`, link, got, target)
+	}
 }
 
 func TestStartProcessAttr(t *testing.T) {
diff --git a/src/path/filepath/path_test.go b/src/path/filepath/path_test.go
index 315f61e..4530c22 100644
--- a/src/path/filepath/path_test.go
+++ b/src/path/filepath/path_test.go
@@ -771,18 +771,6 @@
 	{"test/linkabs", "/"},
 }
 
-// findEvalSymlinksTestDirsDest searches testDirs
-// for matching path and returns correspondent dest.
-func findEvalSymlinksTestDirsDest(t *testing.T, testDirs []EvalSymlinksTest, path string) string {
-	for _, d := range testDirs {
-		if d.path == path {
-			return d.dest
-		}
-	}
-	t.Fatalf("did not find %q in testDirs slice", path)
-	return ""
-}
-
 // simpleJoin builds a file name from the directory and path.
 // It does not use Join because we don't want ".." to be evaluated.
 func simpleJoin(dir, path string) string {
@@ -842,8 +830,11 @@
 	for _, d := range tests {
 		path := simpleJoin(tmpDir, d.path)
 		dest := simpleJoin(tmpDir, d.dest)
-		if filepath.IsAbs(d.dest) || os.IsPathSeparator(d.dest[0]) {
+		if filepath.IsAbs(d.dest) {
 			dest = d.dest
+		} else if runtime.GOOS == "windows" && os.IsPathSeparator(d.dest[0]) {
+			// add C: in front of \...
+			dest = tmpDir[:2] + d.dest
 		}
 		if p, err := filepath.EvalSymlinks(path); err != nil {
 			t.Errorf("EvalSymlinks(%q) error: %v", d.path, err)
@@ -870,14 +861,10 @@
 				t.Errorf(`EvalSymlinks(".") in %q directory error: %v`, d.path, err)
 				return
 			}
-			if p == "." {
+			if filepath.Clean(p) != filepath.Clean(dest) {
+				t.Errorf(`EvalSymlinks(".") in %q directory returns %q, want %q`, d.path, p, dest)
 				return
 			}
-			want := filepath.Clean(findEvalSymlinksTestDirsDest(t, testdirs, d.path))
-			if p == want {
-				return
-			}
-			t.Errorf(`EvalSymlinks(".") in %q directory returns %q, want "." or %q`, d.path, p, want)
 		}()
 
 		// test EvalSymlinks("C:.") on Windows
@@ -903,14 +890,10 @@
 					t.Errorf(`EvalSymlinks("%s") in %q directory error: %v`, volDot, d.path, err)
 					return
 				}
-				if p == volDot {
+				if filepath.Clean(p) != filepath.Clean(dest) {
+					t.Errorf(`EvalSymlinks("%s") in %q directory returns %q, want %q`, volDot, d.path, p, dest)
 					return
 				}
-				want := filepath.Clean(findEvalSymlinksTestDirsDest(t, testdirs, d.path))
-				if p == want {
-					return
-				}
-				t.Errorf(`EvalSymlinks("%s") in %q directory returns %q, want %q or %q`, volDot, d.path, p, volDot, want)
 			}()
 		}
 
@@ -930,10 +913,6 @@
 			}
 
 			path := simpleJoin("..", d.path)
-			dest := simpleJoin("..", d.dest)
-			if filepath.IsAbs(d.dest) || os.IsPathSeparator(d.dest[0]) {
-				dest = d.dest
-			}
 
 			if p, err := filepath.EvalSymlinks(path); err != nil {
 				t.Errorf("EvalSymlinks(%q) error: %v", d.path, err)
@@ -958,8 +937,8 @@
 			}
 			if p, err := filepath.EvalSymlinks(d.path); err != nil {
 				t.Errorf("EvalSymlinks(%q) error: %v", d.path, err)
-			} else if filepath.Clean(p) != filepath.Clean(d.dest) {
-				t.Errorf("EvalSymlinks(%q)=%q, want %q", d.path, p, d.dest)
+			} else if filepath.Clean(p) != filepath.Clean(dest) {
+				t.Errorf("EvalSymlinks(%q)=%q, want %q", d.path, p, dest)
 			}
 		}()
 	}
diff --git a/src/path/filepath/path_windows_test.go b/src/path/filepath/path_windows_test.go
index d759a83..05fc143 100644
--- a/src/path/filepath/path_windows_test.go
+++ b/src/path/filepath/path_windows_test.go
@@ -458,3 +458,37 @@
 	testenv.MustHaveSymlink(t)
 	testWalkMklink(t, "D")
 }
+
+func TestNTNamespaceSymlink(t *testing.T) {
+	output, _ := exec.Command("cmd", "/c", "mklink", "/?").Output()
+	if !strings.Contains(string(output), " /J ") {
+		t.Skip("skipping test because mklink command does not support junctions")
+	}
+
+	tmpdir, err := ioutil.TempDir("", "TestNTNamespaceSymlink")
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer os.RemoveAll(tmpdir)
+
+	vol := filepath.VolumeName(tmpdir)
+	output, err = exec.Command("cmd", "/c", "mountvol", vol, "/L").CombinedOutput()
+	if err != nil {
+		t.Fatalf("failed to run mountvol %v /L: %v %q", vol, err, output)
+	}
+	target := strings.Trim(string(output), " \n\r")
+
+	link := filepath.Join(tmpdir, "link")
+	output, err = exec.Command("cmd", "/c", "mklink", "/J", link, target).CombinedOutput()
+	if err != nil {
+		t.Fatalf("failed to run mklink %v %v: %v %q", link, target, err, output)
+	}
+
+	got, err := filepath.EvalSymlinks(link)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if want := vol + `\`; got != want {
+		t.Errorf(`EvalSymlinks(%q): got %q, want %q`, link, got, want)
+	}
+}
diff --git a/src/path/filepath/symlink_windows.go b/src/path/filepath/symlink_windows.go
index f771fe3..19b4cc8 100644
--- a/src/path/filepath/symlink_windows.go
+++ b/src/path/filepath/symlink_windows.go
@@ -5,8 +5,11 @@
 package filepath
 
 import (
+	"errors"
+	"internal/syscall/windows"
 	"strings"
 	"syscall"
+	"unsafe"
 )
 
 // normVolumeName is like VolumeName, but makes drive letter upper case.
@@ -106,10 +109,54 @@
 	return volume + normPath, nil
 }
 
+// TODO(brainman): remove all windows related code from walkSymlinks
+
 func evalSymlinks(path string) (string, error) {
-	path, err := walkSymlinks(path)
+	if path == "" {
+		return path, nil
+	}
+
+	if windows.LoadGetFinalPathNameByHandle() != nil {
+		// we must be using old version of Windows
+		p, err := syscall.FullPath(path)
+		if err != nil {
+			return "", err
+		}
+		return toNorm(p, normBase)
+	}
+
+	// Use Windows I/O manager to dereference the symbolic link, as per
+	// https://blogs.msdn.microsoft.com/oldnewthing/20100212-00/?p=14963/
+	p, err := syscall.UTF16PtrFromString(path)
 	if err != nil {
 		return "", err
 	}
-	return toNorm(path, normBase)
+	h, err := syscall.CreateFile(p, 0, 0, nil,
+		syscall.OPEN_EXISTING, syscall.FILE_FLAG_BACKUP_SEMANTICS, 0)
+	if err != nil {
+		return "", err
+	}
+	defer syscall.CloseHandle(h)
+
+	buf := make([]uint16, 100)
+	for {
+		n, err := windows.GetFinalPathNameByHandle(h, &buf[0], uint32(len(buf)), windows.VOLUME_NAME_DOS)
+		if err != nil {
+			return "", err
+		}
+		if n < uint32(len(buf)) {
+			break
+		}
+		buf = make([]uint16, n)
+	}
+	s := syscall.UTF16ToString((*[1 << 29]uint16)(unsafe.Pointer(&buf[0]))[:])
+	if len(s) > 4 && s[:4] == `\\?\` {
+		s = s[4:]
+		if len(s) > 3 && s[:3] == `UNC` {
+			// return path like \\server\share\...
+			return Clean(`\` + s[3:]), nil
+		}
+		return Clean(s), nil
+	}
+	return "", errors.New("GetFinalPathNameByHandle returned unexpected path=" + s)
 }

To view, visit change 55250. To unsubscribe, or for help writing mail filters, visit settings.

Hiroshi Ioka (Gerrit)

unread,

Aug 13, 2017, 8:40:03 AM8/13/17

to Alex Brainman, golang-co...@googlegroups.com

I tried this CL.

C:\Users\Hiroshi\test>net use
New connections will be remembered.

Status Local Remote Network

-------------------------------------------------------------------------------
OK           Y:        \\localhost\Users\Hiroshi\test
                                                Microsoft Windows Network
The command completed successfully.
C:\Users\Hiroshi\test>dir
 Volume in drive C is Windows10_OS
 Volume Serial Number is 6203-7069

 Directory of C:\Users\Hiroshi\test

08/13/2017 09:17 PM <DIR> .
08/13/2017 09:17 PM <DIR> ..
08/13/2017 07:14 PM <JUNCTION> jlink [C:\Users\Hiroshi\jtarget]
08/13/2017 07:48 PM <JUNCTION> mount [\\?\Volume{b57ba7f7-a55e-41b2-84fa-074ca673b2e4}\]

C:\Users\Hiroshi\test>type x.go
package main

import (
        "fmt"
        "path/filepath"
)

func main() {
        fmt.Println(filepath.EvalSymlinks("y:jlink"))
        fmt.Println(filepath.EvalSymlinks(`\\localhost\Users\Hiroshi\test\jlink`))
        fmt.Println(filepath.EvalSymlinks(`y:mount\Users\Hiroshi`))
        fmt.Println(filepath.EvalSymlinks(`mount\Users\Hiroshi`))
}

C:\Users\Hiroshi\test>x.exe
\\localhost\Users\Hiroshi\test\jlink <nil>
\\localhost\Users\Hiroshi\test\jlink <nil>
\\localhost\Users\Hiroshi\test\mount\Users\Hiroshi <nil>
C:\Users\Hiroshi <nil>

Looking by the result, I think we can emulate some GetFinalPathNameByHandle behaviors without breaking compatibility.

I see GetFinalPathNameByHandle has two characteristics.
 1. If the given drive is network path, it give up evaluations in any case the network is local or remote. which is the same behavior I proposed in #19922.
 2. recognize NT namespace format, then call appropriate drivers per protocols. I have a hypothesis about NT namespace format. In my observation, those have always same format `\\?\` + relative path. I guess that is the way GetFinalPathNameByHandle recognize NT namespace. If this assumption is correct, we could process protocols ourselves. In above case, get real path from drive GUID.

[go] path/filepath: re-implement windows EvalSymlinks

Alex Brainman (Gerrit)

Hiroshi Ioka (Gerrit)

Alex Brainman (Gerrit)

Hiroshi Ioka (Gerrit)

Alex Brainman (Gerrit)

Alex Brainman (Gerrit)

Alex Brainman (Gerrit)

Alex Brainman (Gerrit)

Alex Brainman (Gerrit)

Alex Brainman (Gerrit)

Alex Brainman (Gerrit)

Hiroshi Ioka (Gerrit)

Alex Brainman (Gerrit)

Hiroshi Ioka (Gerrit)

Alex Brainman (Gerrit)

Hiroshi Ioka (Gerrit)

Alex Brainman (Gerrit)

Hiroshi Ioka (Gerrit)

Hiroshi Ioka (Gerrit)

Hiroshi Ioka (Gerrit)

Alex Brainman (Gerrit)

Alex Brainman (Gerrit)

Hiroshi Ioka (Gerrit)

Hiroshi Ioka (Gerrit)

Alex Brainman (Gerrit)

Hiroshi Ioka (Gerrit)

Alex Brainman (Gerrit)

Hiroshi Ioka (Gerrit)

Alex Brainman (Gerrit)

Hiroshi Ioka (Gerrit)

Alex Brainman (Gerrit)

Alex Brainman (Gerrit)

Alex Brainman (Gerrit)

Alex Brainman (Gerrit)

Ian Lance Taylor (Gerrit)

Alex Brainman (Gerrit)

Alex Brainman (Gerrit)

Ian Lance Taylor (Gerrit)

Alex Brainman (Gerrit)