Multi-master example?

[Jos Backus]

Hi,

It would be great if the documentation showed what a setup looks like with multiple hosts running a master each, instead of using localhost.

Reason I ask: with 0.61 I keep getting assertion panics of type leader.elected.at.same.term,N, where N is an integer that keeps going up. Any suggestions how to fix this? Removing /tmp/conf doesn't help, that state seems to be kept somewhere else.

Thanks,
Jos

[Chris Lu]

Let's resolve your issue first. Let us know what's your setup, and the error messages.

Chris

--
You received this message because you are subscribed to the Google Groups "Weed File System" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weed-file-syst...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Chris Lu]

I think in your case, the servers could be started at about the same time, so they all tried to be leaders. Maybe start one first, wait a few seconds or minutes, then start another one.

Chris

[Jos Backus]

Hi Chris,

Thanks for your response.

You may be right that that is the issue. Also, I sort worked around my problem by removing /tmp/log before starting the master, which seems to be some kind of transaction log? Is it generally unsafe to remove this file before restarting the master?

I'm running weed-fs on 4 hosts, 2 of which are masters, under daemontools-encore, a process supervision tool. My configuration is as follows:

Master host:

[root@vertica5 weed-master]# pwd

/service/weed-master

[root@vertica5 weed-master]# ls -l

total 24

drwxr-xr-x 2 root root 4096 Jun 19 11:27 env

drwxr-xr-x 4 root root 4096 Jun 19 11:32 log

-rwxr-xr-x 1 root root  131 Jun 20 10:36 run

drwxr-xr-x 3 root root 4096 Jun 20 10:40 state

drwx------ 2 root root 4096 Jun 19 11:32 supervise

-rw-r--r-- 1 root root  284 Jun 20 10:39 weedfs.conf

[root@vertica5 weed-master]# ls -l env/

total 0

[root@vertica5 weed-master]# ls -l state/

total 12

-rw------- 1 root root   94 Jun 20 10:32 conf

-rw------- 1 root root  628 Jun 20 10:32 log

drwx------ 2 root root 4096 Jun 20 10:24 snapshot

[root@vertica5 weed-master]# cat weedfs.conf

<Configuration>

  <Topology>

    <DataCenter name="dc1">

      <Rack name="rack1">

        <Ip>192.168.50.180</Ip>

        <Ip>192.168.50.181</Ip>

        <Ip>192.168.50.182</Ip>

        <Ip>192.168.50.183</Ip>

      </Rack>

    </DataCenter>

  </Topology>

</Configuration>

[root@vertica5 weed-master]# cat run

#!/bin/sh

exec 2>&1

exec envdir env weed -logtostderr=true -v=16 master -conf=weedfs.conf -mdir=state -publicIp="$(hostname -s)"

[root@vertica5 weed-master]# cat log/run

#!/bin/sh

user=root

test -d main || install -o $user -d main

exec setuidgid $user multilog t ./main

[root@vertica5 weed-master]#

[root@vertica5 weed-volume-1]# pwd

/service/weed-volume-1

[root@vertica5 weed-volume-1]# ls -l

total 16

drwxr-xr-x 2 root root 4096 Jun 19 11:27 env

drwxr-xr-x 4 root root 4096 Jun 19 11:46 log

-rwxr-xr-x 1 root root  146 Jun 19 12:26 run

drwx------ 2 root root 4096 Jun 19 11:46 supervise

[root@vertica5 weed-volume-1]# cat run

#!/bin/sh

exec 2>&1

exec envdir env weed volume -mserver="vertica5:9333" -ip="$(hostname -s)" -port=8081 -dir=/tmp/1 -max=100 -dataCenter=dc1

[root@vertica5 weed-volume-1]#

Second master host is the same except for the obvious hostname changes and the -peers option:

[root@vertica6 weed-master]# cat run

#!/bin/sh

exec 2>&1

exec envdir env weed -logtostderr=true -v=2 master -conf=weedfs.conf -mdir=state -publicIp="$(hostname -s)" -peers="vertica5:9333"

[root@vertica6 weed-master]#

The other 2 machines just have the weed-volume-1 service.

Thanks,

Jos