Bup depends on fuse-2.9.9

[Jack]

Hey all, 

bup depends on fuse-python, which in turn depends on fuse-2.9.9. Since fuse2 is unmaintained, NixOS (and other operating systems IIRC) have chosen to move towards deprecating and dropping it, and its dependents, in favor of fuse3. Since bup is a cool tool, we would prefer not to drop it from nixpkgs! Would it be possible to use an alternative to fuse-python, which uses fuse3?

Warm regards,

Jack Rosenberg

[Greg Troxel]

"'Jack' via bup-list" <bup-...@googlegroups.com> writes:

> bup depends on fuse-python, which in turn depends on fuse-2.9.9. Since
> fuse2 is unmaintained, NixOS (and other operating systems IIRC) have
> chosen to move towards deprecating and [dropping

> it](https://github.com/NixOS/nixpkgs/pull/521536), [and its
> dependents](https://github.com/NixOS/nixpkgs/issues/522986), in favor
> of fuse3.

I maintain the pkgsrc entry, and have long found that naming of python
fuse things is very confusing, but I see in the bup README.md this is

https://github.com/libfuse/python-fuse

which has

https://github.com/libfuse/python-fuse/issues/100

> Since bup is a cool tool, we would prefer not to drop it from nixpkgs!
> Would it be possible to use an alternative to fuse-python, which uses
> fuse3?

fuse support in bup is optional and very much not part of the core
functionality. So dropping it doens't make sense; you should instead
just deconfigure the fuse dependency. bup will build and work fine
(pkgsrc has the fuse dependency as an option which is default off.)
Except of course

$ bup -d /path/to-bup-repo ls
machine1 machine2

$ bup -d /path/to-bup-repo fuse
error: cannot find the python "fuse" module; please install it



Which alternative do you suggest?

[Stefan Monnier]

> fuse support in bup is optional and very much not part of the core
> functionality.

While I can see why you'd say that it's technically not part of the core
functionality, I wouldn't use Bup if it didn't have something equivalent
to `bup fuse`.


=== Stefan

[Greg Troxel]

Up to you. I use it every week without!

(I do understand why it's nice to have.)

[Nathaniel Hourt]

I support adopting a more widely supported fuse dependency. The Python
fuse module bup depends on is not available on any of the platforms I
deploy on, and I've had to package it everywhere I deploy bup. The fuse
module is hugely inconvenient, and I've been thinking I'll need to start
going through git directly to read my backups so I can drop the
unsupported fuse dependency.

—
Nathaniel

[Greg Troxel]

Great, but you left out the part where you gave a pointer to one that
can be used!

[Nathaniel Hourt]

Oh, well I always find this one when searching for the one bup wants:
https://github.com/libfuse/pyfuse3

It's widely available, including on Debian stable and Nix. Is it
suitable for bup?

—
Nathaniel

[Greg Troxel]

It's async, so that seems awkward, but it points to

https://github.com/mxmlnkn/mfusepy

which has a table listing python-fuse-ish things.

[Rob Browning]

Hmm, don't know. I'll plan to take a look at fuse support after 0.34.

--
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4

[Rob Browning]

Rob Browning <r...@defaultvalue.org> writes:

> Hmm, don't know. I'll plan to take a look at fuse support after 0.34.

...I've wondered in the past about a libfuse lowlevel API (inode) based
implementation, and looks like I may have a "does nothing" skeleton of
that working (just the skeleton was straightforward) with libfuse 3. So
I might poke at it in the background for a bit, see where it leads (but
even if it worked out, it would be for after 0.34).

[Greg Troxel]

Rob Browning <r...@defaultvalue.org> writes:

> Rob Browning <r...@defaultvalue.org> writes:
>
>> Hmm, don't know. I'll plan to take a look at fuse support after 0.34.
>
> ...I've wondered in the past about a libfuse lowlevel API (inode) based
> implementation, and looks like I may have a "does nothing" skeleton of
> that working (just the skeleton was straightforward) with libfuse 3. So
> I might poke at it in the background for a bit, see where it leads (but
> even if it worked out, it would be for after 0.34).

On NetBSD, there is puffs(4) "Pass-to-Userspace Framework File System"
and refuse(3) which implements the FUSE high-level API using the
(slightly different) puffs kernel facilities.

My understanding is that the low-level API is very much "open /dev/fuse
and speak the protocol". NetBSD has a daemon that provides a /dev/fuse
socket and translates, which I believed was written for glusterfs.

I'm unclear on the state of FUSE of macOS, and other non-Linux platforms
in general.

My understanding from watching FUSE over many years is that doctrine is
that one should use the high-level API as simpler and easier all around,
unless there is a good reason why that can't be done. I expect that
using the high-level API will be much better in terms of portability.

I would expect, so far, without having gone down the path, that mfusepy
would be a relatively easy adaptation, staying with the high-level API.

[Rob Browning]

Greg Troxel <g...@lexort.com> writes:

> On NetBSD, there is puffs(4) "Pass-to-Userspace Framework File System"
> and refuse(3) which implements the FUSE high-level API using the
> (slightly different) puffs kernel facilities.
>
> My understanding is that the low-level API is very much "open /dev/fuse
> and speak the protocol". NetBSD has a daemon that provides a /dev/fuse
> socket and translates, which I believed was written for glusterfs.

I haven't looked in to any of it carefully yet, but to clarify, I meant
the fuse_lowlevel.h API, and the main difference I believe, is that it's
inode based rather than path based. I'm not even sure it'd be a good
idea, I was mostly just curious.

I believe it primarily expects you to provide some subset of these
operations:

https://libfuse.github.io/doxygen/structfuse__lowlevel__ops.html#ab6fa0b9edb5b002cd1502c969c887329

much like the higher level api (which is path based), though here you
have to call "this is an ENOENT" or "here's the stat info" reply
functions instead of returning data from a callback.

If the other platforms that have FUSE support only handle the higher
level API, then that does favor it. I also see a suggestion that NetBSD
may now support the lower level api via a newer "perfuse" layer.

> I'm unclear on the state of FUSE of macOS, and other non-Linux platforms
> in general.

Some discussion here:
https://en.wikipedia.org/wiki/Filesystem_in_Userspace

[Greg Troxel]

Rob Browning <r...@defaultvalue.org> writes:

> If the other platforms that have FUSE support only handle the higher
> level API, then that does favor it. I also see a suggestion that NetBSD
> may now support the lower level api via a newer "perfuse" layer.

Yes, it does, but it's more moving parts and more complicated. I have
always had the impression that the standard approach is to use the "high
level API" unless there's a good reason why that isn't a good idae, and
my impression is that the only (popular) system for which that's true is
glusterfs.

I just looked at all the packages in pkgsrc:filesystems and 34 use the
high-level API.

- ltfs requires perfused explicitly.
- gluster needs /dev/fuse which it interacts with without using a
library.
- fuse-ntfs-3g should be able to use the high-level API from librefuse
but it doesn't build and thus is linked against libfuse

which is consistent with use of low-level API being pretty rare.

I don't have data about the rest of the world, but it seems far easier
for a platform to support the high-level API.

I will check in with our FUSE expert, to make sure I'm not blurring
"using the low-level API from libfuse" and "direct accces to /dev/fuse",
or rather that those have the same practical consequences.

[Rob Browning]

Greg Troxel <g...@lexort.com> writes:

> which is consistent with use of low-level API being pretty rare.

Sure. I assume we wouldn't/won't use it unless we have a justification.
Was mostly just curious at the moment.