filepath.walk in Go 1.19

[Robert Solomon]

On ubuntu 22.04, I would like the walk function to NOT follow symlinks to other filesystems.  The find command uses the -xdev switch to achieve this.

How can I get walk to behave like the -xdev switch to find?

Thx

[Marvin Renich]

* Robert Solomon <drro...@gmail.com> [221028 07:36]:

> On ubuntu 22.04, I would like the walk function to NOT follow symlinks to
> other filesystems. The find command uses the -xdev switch to achieve this.
>
> How can I get walk to behave like the -xdev switch to find?

On Linux:

---- getdevid_linux.go

package main

import (
"io/fs"
"syscall"
)

type DevId uint64

// GetDevice returns the Device ID on which the given file resides.
func GetDevice(path string, fi fs.FileInfo) DevId {
var stat = fi.Sys().(*syscall.Stat_t)
return DevId(stat.Dev)
}

----

Then before calling filepath.Walk, filepath.WalkDir (more efficient), or
fs.WalkDir, obtain the device ID of the root. In the call to WalkDir,
pass this device ID to your walk function:

err = filepath.WalkDir(root, func(path string, d fs.DirEntry, err
error) error { return MyWalkFn(devId, path, d, err) })

In MyWalkFn, use d to obtain the device ID of the current path, and
return fs.SkipDir if the device IDs do not match.

...Marvin

[Robert Solomon]

Thank you very much 

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/sBHhJydS66w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/Y1vHqjKVfQqM3gZy%40basil.wdw.

[Robert Solomon]

I now have this working as I want using filepath/walk. 

But if I try to use a concurrent walk operation, like the one originally written by Michael T Jones, I get behavior that seems to treat any return of SkipDir as a fatal error and all walking stops.  filepath/walk does not do this; it continues to the next target correctly.

I feel I'm missing something.  Am I?

[Konstantin Khomoutov]

On Sat, Oct 29, 2022 at 02:54:48PM -0700, Robert Solomon wrote:

[...]

>>> On ubuntu 22.04, I would like the walk function to NOT follow symlinks to
>>> other filesystems. The find command uses the -xdev switch to achieve
>>> this.
>>>
>>> How can I get walk to behave like the -xdev switch to find?

[...]

>> // GetDevice returns the Device ID on which the given file resides.
>> func GetDevice(path string, fi fs.FileInfo) DevId {
>> var stat = fi.Sys().(*syscall.Stat_t)
>> return DevId(stat.Dev)
>> }

[...]

>> Then before calling filepath.Walk, filepath.WalkDir (more efficient), or
>> fs.WalkDir, obtain the device ID of the root. In the call to WalkDir,
>> pass this device ID to your walk function:
>>
>> err = filepath.WalkDir(root, func(path string, d fs.DirEntry, err
>> error) error { return MyWalkFn(devId, path, d, err) })
>>
>> In MyWalkFn, use d to obtain the device ID of the current path, and
>> return fs.SkipDir if the device IDs do not match.

> I now have this working as I want using filepath/walk.
>
> But if I try to use a concurrent walk operation, like the one originally
> written by Michael T Jones, I get behavior that seems to treat any return
> of SkipDir as a fatal error and all walking stops. filepath/walk does not
> do this; it continues to the next target correctly.
>
> I feel I'm missing something. Am I?

It appears you have switched the topic to discuss some custom code which walks
a filesystem which is not in the standard library, have you?

If yes, this is sort of useless without presenting the actual code and the
problem you are having with it. The reason is simple: io/fs.SkipDir is just a
value without any magic attached to it (as there is with certain builtins like
`copy` and `append` or compiler intrinsics like the functions in the
sync/atomic package). That is, returning io/fs.SkipDir from your callback
function can only "work" if the code which calls that callback is prepared to
treat that value in some special manner; if it is not, it will likely treat it
like any other value implementing the standard interface `error`.
This seems to perfectly explain the

| <...> seems to treat any return of SkipDir as a fatal error <...>

bit.