電源無預警斷電後，造成虛擬伺服器failed

[謝阿剛]

控制端伺服器 : ASUS RS100-E8     Intel Xeon E3 1230 v3(3.3GHz)

叢集伺服器 : Asus RS300-E8 *4

下列為建立新虛擬伺服器的log:

Fri Oct 24 11:19:10 2014 [DiM][I]: New VM state is ACTIVE.
Fri Oct 24 11:19:10 2014 [LCM][I]: New VM state is PROLOG.
Fri Oct 24 11:19:10 2014 [TM][I]: Command execution fail: /var/lib/one/remotes/tm/moosefs/clone ezilla-master:/var/lib/one/datastores/1/b1ce92c224c354defde20551301bd7cd 192.168.0.91:/var/lib/one//datastores/0/53/disk.0 53 1
Fri Oct 24 11:19:10 2014 [TM][I]: clone: Cloning ../../1/b1ce92c224c354defde20551301bd7cd in 192.168.0.91:/var/lib/one//datastores/0/53/disk.0
Fri Oct 24 11:19:10 2014 [TM][E]: clone: Command "cd /var/lib/one/datastores/0/53; /usr/bin/mfsmakesnapshot -o ../../1/b1ce92c224c354defde20551301bd7cd /var/lib/one/datastores/0/53/disk.0" failed:
Fri Oct 24 11:19:10 2014 [TM][E]: Error copying ezilla-master:/var/lib/one/datastores/1/b1ce92c224c354defde20551301bd7cd to 192.168.0.91:/var/lib/one//datastores/0/53/disk.0
Fri Oct 24 11:19:10 2014 [TM][I]: ExitCode: 255
Fri Oct 24 11:19:10 2014 [TM][E]: Error executing image transfer script: Error copying ezilla-master:/var/lib/one/datastores/1/b1ce92c224c354defde20551301bd7cd to 192.168.0.91:/var/lib/one//datastores/0/53/disk.0
Fri Oct 24 11:19:11 2014 [DiM][I]: New VM state is FAILED

下列為舊的伺服器開啟時的log:

Wed Oct 22 10:44:18 2014 [LCM][I]: New VM state is BOOT
Wed Oct 22 10:44:18 2014 [VMM][I]: Generating deployment file: /var/lib/one/50/deployment.0
Wed Oct 22 10:44:18 2014 [VMM][I]: ExitCode: 0
Wed Oct 22 10:44:18 2014 [VMM][I]: Successfully execute network driver operation: pre.
Wed Oct 22 10:44:19 2014 [VMM][I]: Command execution fail: cat << EOT | /var/tmp/one/vmm/kvm/deploy /var/lib/one//datastores/0/50/deployment.0 192.168.0.89 50 192.168.0.89
Wed Oct 22 10:44:19 2014 [VMM][I]: error: Failed to create domain from /var/lib/one//datastores/0/50/deployment.0
Wed Oct 22 10:44:19 2014 [VMM][I]: error: internal error process exited while connecting to monitor: kvm: -drive file=/var/lib/one//datastores/0/50/disk.0,if=none,id=drive-ide0-0-0,format=raw: could not open disk image /var/lib/one//datastores/0/50/disk.0: No such file or directory
Wed Oct 22 10:44:19 2014 [VMM][I]: 
Wed Oct 22 10:44:19 2014 [VMM][E]: Could not create domain from /var/lib/one//datastores/0/50/deployment.0
Wed Oct 22 10:44:19 2014 [VMM][I]: ExitCode: 255
Wed Oct 22 10:44:19 2014 [VMM][I]: Failed to execute virtualization driver operation: deploy.
Wed Oct 22 10:44:19 2014 [VMM][E]: Error deploying virtual machine: Could not create domain from /var/lib/one//datastores/0/50/deployment.0
Wed Oct 22 10:44:20 2014 [DiM][I]: New VM state is FAILED

不知道如何才能修復這個問題?謝謝

[Jonathan Chen]

使用者您好：

請麻煩確定moose file system 服務是否正常運作，

從您的log看出來 似乎分散式檔案系統沒有正常運作。

再麻煩請找一下相關的資訊

Jonathan

[謝阿剛]

Jonathan您好

我已經參考https://groups.google.com/forum/#!searchin/nchc-ezilla/moose$20file$20system/nchc-ezilla/NvVgwA9O4cE/R361zTKiH_EJ

進行重開機的步驟處理過，但是在最後一個步驟出現問題如下:

[root@ezilla-master ~]# mfsmount / var/lib/one/datastores -H ezilla-masterfs

mfsmaster accepted connection with parameters : read-write, restricted_ip; root mapped to root : root

fuse : mountpoint is not empty

fuse : if you are sure this save, use the 'nonempty' mount option error in fuse mount

此外已經做過 #mfsmetarestore -a

與#mfsmaster start

因為我算是這方面的初學者，不知道有甚麼方法能夠解決呢?  謝謝

[謝阿剛]

不好意思，之後我再根據  無法啟動VM（Clone Error？）這篇來做參考

我把步驟與資訊進行統整如下 :

開新虛擬機器時遇到的問題

Thu Nov 27 17:06:16 2014 [DiM][I]: New VM state is ACTIVE.

Thu Nov 27 17:06:16 2014 [LCM][I]: New VM state is PROLOG.

Thu Nov 27 17:06:17 2014 [TM][I]: Command execution fail: /var/lib/one/remotes/tm/moosefs/clone ezilla-master:/var/lib/one/datastores/1/b1ce92c224c354defde20551301bd7cd 192.168.0.91:/var/lib/one//datastores/0/68/disk.0 68 1

Thu Nov 27 17:06:17 2014 [TM][I]: clone: Cloning ../../1/b1ce92c224c354defde20551301bd7cd in 192.168.0.91:/var/lib/one//datastores/0/68/disk.0

Thu Nov 27 17:06:17 2014 [TM][E]: clone: Command "cd /var/lib/one/datastores/0/68; /usr/bin/mfsmakesnapshot -o ../../1/b1ce92c224c354defde20551301bd7cd /var/lib/one/datastores/0/68/disk.0" failed:

Thu Nov 27 17:06:17 2014 [TM][E]: Error copying ezilla-master:/var/lib/one/datastores/1/b1ce92c224c354defde20551301bd7cd to 192.168.0.91:/var/lib/one//datastores/0/68/disk.0

Thu Nov 27 17:06:17 2014 [TM][I]: ExitCode: 255

Thu Nov 27 17:06:17 2014 [TM][E]: Error executing image transfer script: Error copying ezilla-master:/var/lib/one/datastores/1/b1ce92c224c354defde20551301bd7cd to 192.168.0.91:/var/lib/one//datastores/0/68/disk.0

Thu Nov 27 17:06:17 2014 [DiM][I]: New VM state is FAILED

1.首先確認是否有正常掛載moosefs 檔案系統

# df -h

已確認 /var/lib/one/datastores 有掛載

且確認各子伺服器都有掛載

2.檢查mfsmaster 是否有運作

# /etc/init.d/mfsmaster status

回應 mfsmaster (pid 9525) is running ...

3.確認mfs chunkserver是否有正常運作

# /etc/init.d/mfschunkserver status

回應 mfschunkserver (pid 8188) is running...

4.可能因為電腦斷電重開。沒有先將此檔案系統關閉。

導致其紀錄metadata的檔案損壞。

所以使用

# cd /var/mfs

# cp metadata.mfs.empty metadata.mfs

完成後除新啟動

# mfsmaster star

回應 can't start : lockfile is already locked by another process

可能是因為有處理程序正在使用，所以先將其停止後啟動

# mfsmaster stop

sending SIGTERM to lock owner (pid:9525)

waiting for termination ... terminated

# mfsmaster start

之後出現以下訊息

rking directory: /var/mfs

lockfile created and locked

initializing mfsmaster modules ...

loading sessions ... ok

sessions file has been loaded

exports file has been loaded

mfstopology configuration file (/etc/mfs/mfstopology.cfg) not found - using defaults

loading metadata ...

loading objects (files,directories,etc.) ... ok

loading names ... ok

loading deletion timestamps ... ok

checking filesystem consistency ... ok

loading chunks data ... ok

connecting files and chunks ... ok

all inodes: 5

directory inodes: 5

file inodes: 0

chunks: 0

metadata file has been loaded

stats file has been loaded

master <-> metaloggers module:listen on *:9419

master <-> chunkservers module: listen on *:9420

main master server module: listen on *:9421

mfsmaster daemon initialized properly

之後還是無法正常開啟新的虛擬伺服器

再來要如何解決呢?  謝謝