Trouble getting chirp with fuse

[Ángel de Vicente]

Hello,

I'm trying to get chirp (ideally chirp_fuse or chirp + parrot) to work on our Ubuntu 22.04 servers, but so far no luck.

I cloned the cctools repository and got the 2.9 version of libfuse installed. When compiling cctools,

When configuring cctools, all seems to be good:

Optional external systems configured:

General libraries
curl             no
ext2fs          yes
fuse            yes
openssl         yes
openssl_static   no
readline        yes

Language support:
swig            yes
perl             no
python3         yes

External support configured for parrot:
cvmfs            no
globus           no
irods            no
mysql            no
xrootd           no

and compilation goes apparently without issues, but when I try to run chirp_fuse, I get:

$ chirp_fuse
chirp_fuse: sorry, fuse support was not built in

If I try to run parrot_run, then the problem is as follows:

$ parrot_run bash
2024/10/21 17:03:24.89 parrot_run[464378] notice: parrot_run 8.0.0 DEVELOPMENT has not been tested on Linux 6.8.0-45-generic yet, this may not work
bash: error while loading shared libraries: libtinfo.so.6: cannot stat shared object: Error 20

Though is available at:

/usr/lib/x86_64-linux-gnu/libtinfo.so.6

Any hints/ideas what could be going on and how to move forward?

Cheers,

[Douglas Thain]

Hello Angel -

If you have the permissions setup to use fuse, then chirp+fuse is likely the better path to use.

I can see the same problem here on the main branch, so I think something was recently broken

in the build flags setup.  Let me take a look at it and get back to you...

Doug

--
You received this message because you are subscribed to the Google Groups "Cooperative Computing Tools" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cctools-nd+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cctools-nd/09de4593-aa23-42bb-9c30-8d8340c22ab1n%40googlegroups.com.

[Douglas Thain]

Yep, that's what I thought, here is a PR that fixes the problem that will be merged and deployed shortly:

https://github.com/cooperative-computing-lab/cctools/pull/3960

If you want to hack a directly in your tree, just edit chirp/src/chirp_fuse.c and change the line "#ifdef HAS_FUSE" to "#if 1".

Now, our experience is that chirp_fuse works fine once you have your permissions set as desired.  But the behavior can be a bit opaque if the permissions are not set properly. So, I would suggest that you set up a chirp_server and first use the "chirp" command line tool to make sure that you can set permissions, get and put files, etc.  Then, once that is set up properly, start using chirp_fuse.

A particularly useful way to understand the behavior of the system is to run chirp_fuse in one terminal window like this:

./chirp_fuse -f -d all /path/to/mount/location

And then open a different window to use the filesystem.  You will see log messages from the fuse process streaming by in the first window, and it gives you more insight into what's happening.

Let us know if that works for you -

Doug

[Ángel de Vicente]

I applied the PR and now I can use chirp_fuse without issues, great!

But I would like to use chirp_server and chirp_fuse in the following way:

User_A starts a chirp_server in Host_X 

User_B starts a chirp_server in Host_X

User_A uses chirp_fuse from Host_Y to access his data from Host_X

User_B uses chirp_fuse from Host_Z to access her data from Host_X

When I use chirp_fuse I see many servers. Ideally I would like that our users only see the servers from our domain, and hopefully that the servers can be named somehow, so that each user can easily recognize his/her stuff.

Is something like this possible?

I see that the port to be used is 9094, right? If two users want to create a server in the same host, do they need different ports?

I will try to read the documentation properly, but I was wondering if my plan is feasible with cctools at all?

Many thanks,

AdV

[Douglas Thain]

Hello Angel -

Some quick orientation answers:

1 - Yes, each server needs to run on a different port.  One way to accomplish this is to specify port zero (meaning use any available port) and then write the port out to a file for another program to read:

chirp_server --port 0 --port-file my.port.file --root my.root.dir

2 - The chirp servers advertise themselves to a catalog server.  By default, they use the "global" catalog server at Notre Dame, and so everyone can see all of them.  But if you run your own catalog server (and tell everyone to use it with the CATALOG_HOST environment variable) then you will only see things that you control.

3 - The server listing currently shows the domain name and port.  We don't currently have an easy way to make that appear as a listing of logical names.  But if Chirp otherwise meets your needs, then let's set up a time to talk about your goals and see what we can do.

Sincerely,

Doug

--
You received this message because you are subscribed to the Google Groups "Cooperative Computing Tools" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cctools-nd+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cctools-nd/8677cbc3-28d5-4d77-b530-5d12f916587dn%40googlegroups.com.

[Ángel de Vicente]

Hello,

many thanks for the answers. I will be looking to this during the coming days and see how far I get (not sure if I will be able to use chirp_fuse due to security reasons, and I'm not able to run parrot_run in Ubuntu 22.04 for the moment, but I will persevere :-)

I will get back in touch when I converge into a workflow configuration that could be useful for our site.

Many thanks,

AdV

[Ángel de Vicente]

Hello again,

the trouble I'm getting right now is that CCTools seems to require an old version of Fuse (I managed to compile it with fuse 2.9.9), but we set up all our workstations and laptops with CVMFS, which requires Fuse version 3. Is it possible to get CCTools to play along nicely with fuse v3?

Cheers,

AdV

[Douglas Thain]

Hello Angel - 

I think this is fixable, but let me open up an issue and get someone from my team to look into

this properly.  We will get back to you this week, I think:

https://github.com/cooperative-computing-lab/cctools/issues/3963

Sincerely,

Doug

To view this discussion visit https://groups.google.com/d/msgid/cctools-nd/2118af11-46c2-4b38-b0ac-82f505fbdd5fn%40googlegroups.com.

[Ángel de Vicente]

Great, many thanks. Looking forward to it.

Let me know if I can help in any way with this.

Cheers,

AdV

[Ángel de Vicente]

Hello,

I was trying the latest code from Github, and it looks like it is working just fine. I managed to get the chirp_server and chirp_fuse working in our workstations with minimal fuss. Thanks a lot.

But now I have to see if we can get it to work for our purposes. I've been playing with the ACLs, and I would like mainly to provide some shared storage, where users can create their own private directories, but we don't have any shared storage for the "unix" method to authenticate, so I'm not sure if it will be doable.

[well, we do have shared storage, but this is mounted via Kerberos authentication, and unmounted after a certain amount of time. My use case has to do with Condor (and Slurm) jobs that can run after the Kerberos ticket has expired, so ideally I'd like to find some way of authenticating users that doesn't rely on Kerberos...]

Cheers,

AdV

[Ben Tovar]

Hi Angel,

Thanks for letting us know!

Chirp has another authentication method via tickets. The idea is to create an authentication ticket for given directories and ACLs, and then use that ticket to access the chirp server remotely:

 authentication

To view this discussion visit https://groups.google.com/d/msgid/cctools-nd/9648b235-2039-4f62-af70-3ffb5db417e0n%40googlegroups.com.

[Ben Tovar]

Angel,

I apologize, I inadvertently sent the last email without finishing it. I meant to include this link:

https://cctools.readthedocs.io/en/stable/chirp/#ticket-authentication

Please let us know if that works for you, and if it is something we can help you with.

Best wishes!,

Ben

[Ángel de Vicente]

Aww, I should've checked the Advanced part of the documentation. Definitely looks promising. I'll give it a go this week and will report back.

Many thanks!

[Ángel de Vicente]

I just tried, and the ticket seems to be working fine if I use chirp:

$ /opt/SIE/CCTools/bin/chirp -i ticket.c4c3b96c31833d50afc7bad957081023 161.72.xxx.xxx:9094                                                                                                                                        

connected to 161.72.xxx.xxx:9094 as unix:<user>                                                                                
 chirp:161.72.xxx.xxx:9094:/> ls                                                                                                

ho                                    

But I was hoping it would also work with chirp_fuse:

$ /opt/SIE/CCTools/bin/chirp_fuse -i ticket.c4c3b96c31833d50afc7bad957081023 /tmp/chirp/                                                                                                                                        
chirp_fuse: mounted chirp on /tmp/chirp/                                                                                      
chirp_fuse: to unmount: fusermount -u /tmp/chirp/                                                                              

$ ls -ltr /tmp/chirp/161.72.xxx.xxx\:9094/            
ls: reading directory '/tmp/chirp/161.72.xxx.xxx:9094/': Permission denied                                                                                                                                                                                      
total 0                                    

Am I doing something wrong here or do I have to read the documentation more carefully :-)?

Cheers,

[Ben Tovar]

Angel,

It should work with chirp_fuse. A thing to try is to run chirp_fuse with the -dall and -f flags. That may print helpful debug information.

Here is an example that just worked for me:

In one terminal:
mkdir -p /tmp/chirp/hello/ /tmp/chirpf
chirp_server -r /tmp/chirp -a ticket -dall --inherit-default-acl --port 9095

In a second terminal:
chirp localhost:9095 ticket_create -output myticket.ticket -subject unix:$USER -bits 1024 -duration 864000 / rl /hello write
chirp_fuse -dall -f  -aticket /tmp/chirpf  -i myticket.ticket

In a third terminal:
echo hello > /tmp/chirpf/localhost:9095/hello.txt           # (should fail)
echo hello > /tmp/chirpf/localhost:9095/hello/hello.txt     # (should work)

Please let us know if this works for you!

Ben

To view this discussion visit https://groups.google.com/d/msgid/cctools-nd/70b220bc-cef7-463a-8db3-9c22e840a888n%40googlegroups.com.

[Ángel de Vicente]

Hi,

this is very weird, but I tried three times, and always with the same result. If I add "-f" then it works, otherwise I get permission denied.

$ /opt/SIE/CCTools/bin/chirp_fuse -i ticket.c4c3b96c31833d50afc7bad957081023 -f /tmp/chirp/ &
[1] 640656
$ chirp_fuse: mounted chirp on /tmp/chirp/

chirp_fuse: to unmount: fusermount -u /tmp/chirp/

$ ls -ltr /tmp/chirp/161.72.xxx.xxx\:9094/

total 4
-rw-r--r-- 1 angelv sieinves 11 nov 12 13:49 ho

$ fusermount -u /tmp/chirp/

$ /opt/SIE/CCTools/bin/chirp_fuse -i ticket.c4c3b96c31833d50afc7bad957081023 /tmp/chirp/
chirp_fuse: mounted chirp on /tmp/chirp/
chirp_fuse: to unmount: fusermount -u /tmp/chirp/
$ ls -ltr /tmp/chirp/161.72.xxx.xxx\:9094/
ls: reading directory '/tmp/chirp/161.72.xxx.xxx:9094/': Permission denied
total 0

$ fusermount -u /tmp/chirp/

To view this discussion visit https://groups.google.com/d/msgid/cctools-nd/CAMik99XC34W4drV0GEJ59T6sg07dGSWsnAWxTntMbYvYkogE5w%40mail.gmail.com.

[Ben Tovar]

Hi Angel,

I can confirm the error without the -f. One easy workaround is to give the whole absolute path of the ticket in the command line, like:

chirp_fuse -dall -aticket /tmp/chirpf  -i $(realpath myticket.ticket)

Ben

To view this discussion visit https://groups.google.com/d/msgid/cctools-nd/CAHTG2ftQqba%2BRy8f19X05b3hAtON3892%3DMJWHAO1H3bRLLSHOw%40mail.gmail.com.

[Ángel de Vicente]

Perfect, that way works fine.

And I think I'm getting very close to what I had in mind for our institute!

Many thanks!