second mount problem

[Maurizio Atza]

Hello to everyone, 

I'm pretty new to beegfs and I'm trying to use two mounts. 

I have prepared the two environments and with a single mount everything works fine. 

When I write a beegfs-mount.conf with:

/fast   /etc/beegfs/beegfs-client-fast.conf

/big   /etc/beegfs/beegfs-client-big.conf

the two entries pointing to two different beegfs-mgmt servers. 

The second mount gets an error and the beegfs-client service is aborted. 

Could you give me some advice ?

Thanks

M

[Harry Mangalam]

You didn't give us much info besides "it doesn't work", but I'd guess (based on my own mistakes) that maybe you didn't change the port shift parameters (connPortShift)?  ie each metadata server needs to communicate with its own set of clients and storage servers on an otherwise clear channel/port number, so if they work OK individually, but not at the same time, I'd guess that they're trying to use the same set of ports.

I won't guarantee that it's completely internally consistent, but I wrote this up a while ago as a guide for our other admins when they're adding BeeGFS systems (especially important now that we are using about 5 or 6 of them; some a single RAID as an NFS replacement; 3 big ones.)

http://hpc-trac.oit.uci.edu/wiki/AddingANewBeeGFS

esp this section:

http://hpc-trac.oit.uci.edu/wiki/AddingANewBeeGFS#ModifyingtheBrennersetup-beegfs.shconfigfiles

If it's not this, maybe your DNS or /etc/hosts file may not be as clear as it should be.

As always, check the logs of the mgmnt and metadata servers  - they are suprisingly  good for resolving these kinds of errors.

hjm

[Maurizio Atza]

Hello Harry 

thanks for you kind answer. 

The problem was really silly.

It was caused by a wrong port assignment on the client for the helperd service. 

Having two completely separated environment:

A-Management service: 192.168.1.100:8008

A-Meta service: 192.168.1.101:8005

A-Storage service: 192.168.1.103:8003, 192.168.1.104:8003, 192.168.1.105:8003, 192.168.1.106:8003

B-Management service 192.168.1.115:8018

B-Meta service 192.168.1.102:8015

B-Storage service 192.168.1.107:8013, 192.168.1.108:8013, 192.168.1.109:8013, 192.168.1.110:8013

The client had to maintain a single helperd-service instance so it is necessary to set properly the ports it in the client.conf files:

beegfs-mounts.conf

---------------------------

/mnt/beegfs/fast /etc/beegfs/beegfs-client-fast.conf

/mnt/beegfs/big  /etc/beegfs/beegfs-client-big.conf

/etc/beegfs/beegfs-client-fast.conf

--------------------------------------------

sysMgmtHost = 192.168.1.100

connClientPortUDP = 8004

connHelperdPortTCP = 8006

connMgmtdPortTCP = 8008

connMgmtdPortUDP = 8008

connPortShift = 0

/etc/beegfs/beegfs-client-big.conf

--------------------------------------------

sysMgmtHost = 192.168.1.115

connClientPortUDP = 8014

connHelperdPortTCP = 8006 <==

connMgmtdPortTCP = 8018

connMgmtdPortUDP = 8018

connPortShift = 0

reading your doc I got the hint from the connPortShift and everything came clear. 

Thanks again for your help. 

Maurizio

[yz...@tensyr.com]

Hi, Maurizio

Which kernel version are you running? 

I could do two BeeGFS mounts on the same client machine running on kernel 4.10 (Ubuntu 16.04) but had a kernel panic when doing so on kernel 4.13 (same Ubuntu 16.04). The kernel had the message "sysfs: cannot create duplicate filename '/devices/virtual/bdi/beegfs'" as I reported in the post