[lxc-users] cluster node with different storage config

[Aleksandar Ivanisevic]

Hi,

I’m trying to join a node to a lxd cluster that has different storage configuration. My cluster uses storage pool ‘local’ in zfs pool ‘ee’ and the new node uses ‘rpool/lxd’. Although the documentation (https://linuxcontainers.org/lxd/docs/master/clustering.html) suggests that should be possible, no matter what I put in the preseed yaml it always either tries to import the zfs pool ‘ee’ or create an already existing pool ‘local'

Does anyone know what is the correct syntax or is it possible at all?

$ lxd init --preseed < /tmp/cluster.yaml
$ cat /tmp/cluster.yaml
cluster:
enabled: true
server_name: ${HOSTNAME%%.*}
server_address: ${HOSTNAME%%.*}:8443
cluster_address: lxd1:8443
cluster_certificate: "-----BEGIN CERTIFICATE——
...
-----END CERTIFICATE-----
"
cluster_password: “..."
member_config:

# no matter what i put after this line it is always the same errors
# Error: Failed to join cluster: Failed to initialize member: Failed to initialize storage pools and networks: Failed to create storage pool 'local': Storage pool directory "/var/snap/lxd/common/lxd/storage-pools/local" already exists
# or, if the new member’s local pool is not called ‘local’ then it complains about zpool import ‘ee’ failed.

- entity: storage-pool
name: cluster
key: source
value: "rpool/lxd"
key: zfs.pool_name
value: "rpool/lxd"

[Aleksandar Ivanisevic]

Hi,

at the end this worked for me:

:; cat<<EOF|lxd init --preseed

cluster:

  enabled: true

  server_name: ${HOSTNAME%%.*}

  server_address: ${HOSTNAME%%.*}:8443

  cluster_address: cluster:8443

  cluster_certificate: "$(sed ':a;N;$!ba;s/\n/\n\n/g' /tmp/cluster.crt)

"

  cluster_password: “password"

  member_config:

  - entity: storage-pool

    name: your-storage-pool-name

    key: source

    value: yourzfspool/yourzfsfs

EOF

the trick is that you have to first migrate or delete all the existing containers on the joining node and destroy the storage pool if it has the same name as the cluster pool. New node can only join the cluster completely empty, I don’t know why I was somehow expecting that the existing containers will be merged with the ones in the cluster. It is a nice feature request though ;)

[Stéphane Graber]

Joining nodes should be empty, they should not have any existing
network or storage pool configured in LXD.

> _______________________________________________
> lxc-users mailing list
> lxc-...@lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users

--
Stéphane