Debian standalonevm always enters emergency mode
18 views
Skip to first unread message
Fernando
Jan 1, 2020, 9:00:03 AM1/1/20
to qubes-users
Happy new year to everyone!
I'm taking advantage of holydays to try to fix an issue that has been bothering me for a while now. I have a debian based standalonevm which recently started presented this issue after running out of space.
The issue is that the vm never starts on it own. In order to be able to start it, I have to run: sudo xl console -t pv <standalonevm>.
Then it enters emergency mode and after exiting it, it starts and works normally.
From the output of xl console:
From the output of journalctl in the standalone vm:
Jan 01 10:29:31 localhost mount-dirs.sh[303]: Private device management: checking /dev/xvdb
Jan 01 10:29:31 localhost resize-rootfs-if-needed.sh[302]: dumpe2fs 1.43.4 (31-Jan-2017)
Jan 01 10:29:31 localhost mount-dirs.sh[303]: Private device management: fsck.ext4 /dev/xvdb failed:
Jan 01 10:29:31 localhost mount-dirs.sh[303]: /dev/xvdb: recovering journal
Jan 01 10:29:31 localhost mount-dirs.sh[303]: /dev/xvdb contains a file system with errors, check forced.
Jan 01 10:29:31 localhost mount-dirs.sh[303]: /dev/xvdb: Inodes that were part of a corrupted orphan linke
Jan 01 10:29:31 localhost mount-dirs.sh[303]: /dev/xvdb: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
Jan 01 10:29:31 localhost mount-dirs.sh[303]: (i.e., without -a or -p options)
Jan 01 10:29:31 localhost kernel: EXT4-fs (xvdb): warning: mounting fs with errors, running e2fsck is reco
Jan 01 10:29:31 localhost kernel: EXT4-fs (xvdb): mounted filesystem with ordered data mode. Opts: discard
Jan 01 10:29:31 localhost mount-dirs.sh[303]: Checking /rw
Jan 01 10:29:31 localhost mount-dirs.sh[303]: Private device size management: enlarging /dev/xvdb
Jan 01 10:29:31 localhost resize-rootfs-if-needed.sh[302]: dumpe2fs 1.43.4 (31-Jan-2017)
Jan 01 10:29:31 localhost mount-dirs.sh[303]: Private device management: fsck.ext4 /dev/xvdb failed:
Jan 01 10:29:31 localhost mount-dirs.sh[303]: /dev/xvdb: recovering journal
Jan 01 10:29:31 localhost mount-dirs.sh[303]: /dev/xvdb contains a file system with errors, check forced.
Jan 01 10:29:31 localhost mount-dirs.sh[303]: /dev/xvdb: Inodes that were part of a corrupted orphan linke
Jan 01 10:29:31 localhost mount-dirs.sh[303]: /dev/xvdb: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
Jan 01 10:29:31 localhost mount-dirs.sh[303]: (i.e., without -a or -p options)
Jan 01 10:29:31 localhost kernel: EXT4-fs (xvdb): warning: mounting fs with errors, running e2fsck is reco
Jan 01 10:29:31 localhost kernel: EXT4-fs (xvdb): mounted filesystem with ordered data mode. Opts: discard
Jan 01 10:29:31 localhost mount-dirs.sh[303]: Checking /rw
Jan 01 10:29:31 localhost mount-dirs.sh[303]: Private device size management: enlarging /dev/xvdb
...
Jan 01 10:29:31 localhost kernel: EXT4-fs (xvda3): warning: mounting fs with errors, running e2fsck is rec
Jan 01 10:29:31 localhost kernel: EXT4-fs (xvda3): recovery complete
Jan 01 10:29:31 localhost kernel: EXT4-fs (xvda3): mounted filesystem with ordered data mode. Opts: (null)
Jan 01 10:29:31 localhost kernel: EXT4-fs (xvdd): mounting ext3 file system using the ext4 subsystem
Jan 01 10:29:31 localhost kernel: EXT4-fs (xvdd): mounted filesystem with ordered data mode. Opts: (null)
Jan 01 10:29:31 localhost kernel: EXT4-fs (xvda3): re-mounted. Opts: (null)
Jan 01 10:29:31 localhost kernel: EXT4-fs (xvda3): recovery complete
Jan 01 10:29:31 localhost kernel: EXT4-fs (xvda3): mounted filesystem with ordered data mode. Opts: (null)
Jan 01 10:29:31 localhost kernel: EXT4-fs (xvdd): mounting ext3 file system using the ext4 subsystem
Jan 01 10:29:31 localhost kernel: EXT4-fs (xvdd): mounted filesystem with ordered data mode. Opts: (null)
Jan 01 10:29:31 localhost kernel: EXT4-fs (xvda3): re-mounted. Opts: (null)
...
-- The start-up result is done.
Jan 01 10:29:32 localhost systemd-fsck[213]: /dev/xvda3: Inodes that were part of a corrupted orphan linke
Jan 01 10:29:32 localhost systemd-fsck[213]: /dev/xvda3: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
Jan 01 10:29:32 localhost systemd-fsck[213]: (i.e., without -a or -p options)
Jan 01 10:29:32 localhost systemd-fsck[213]: fsck failed with error code 4.
Jan 01 10:29:32 localhost systemd-fsck[213]: Running request emergency.target/start/replace
Jan 01 10:29:32 localhost systemd-fsck[213]: /dev/xvda3: Inodes that were part of a corrupted orphan linke
Jan 01 10:29:32 localhost systemd-fsck[213]: /dev/xvda3: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
Jan 01 10:29:32 localhost systemd-fsck[213]: (i.e., without -a or -p options)
Jan 01 10:29:32 localhost systemd-fsck[213]: fsck failed with error code 4.
Jan 01 10:29:32 localhost systemd-fsck[213]: Running request emergency.target/start/replace
...
-- Unit qubes-mount-dirs.service has begun starting up.
Jan 01 10:29:31 localhost mount-dirs.sh[303]: Private device management: checking /dev/xvdb
Jan 01 10:29:31 localhost resize-rootfs-if-needed.sh[302]: dumpe2fs 1.43.4 (31-Jan-2017)
Jan 01 10:29:31 localhost mount-dirs.sh[303]: Private device management: fsck.ext4 /dev/xvdb failed:
Jan 01 10:29:31 localhost mount-dirs.sh[303]: /dev/xvdb: recovering journal
Jan 01 10:29:31 localhost mount-dirs.sh[303]: /dev/xvdb contains a file system with errors, check forced.
Jan 01 10:29:31 localhost mount-dirs.sh[303]: /dev/xvdb: Inodes that were part of a corrupted orphan linke
Jan 01 10:29:31 localhost mount-dirs.sh[303]: /dev/xvdb: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
Jan 01 10:29:31 localhost mount-dirs.sh[303]: (i.e., without -a or -p options)
Jan 01 10:29:31 localhost kernel: EXT4-fs (xvdb): warning: mounting fs with errors, running e2fsck is reco
Jan 01 10:29:31 localhost kernel: EXT4-fs (xvdb): mounted filesystem with ordered data mode. Opts: discard
Jan 01 10:29:31 localhost mount-dirs.sh[303]: Checking /rw
Jan 01 10:29:31 localhost mount-dirs.sh[303]: Private device management: checking /dev/xvdb
Jan 01 10:29:31 localhost resize-rootfs-if-needed.sh[302]: dumpe2fs 1.43.4 (31-Jan-2017)
Jan 01 10:29:31 localhost mount-dirs.sh[303]: Private device management: fsck.ext4 /dev/xvdb failed:
Jan 01 10:29:31 localhost mount-dirs.sh[303]: /dev/xvdb: recovering journal
Jan 01 10:29:31 localhost mount-dirs.sh[303]: /dev/xvdb contains a file system with errors, check forced.
Jan 01 10:29:31 localhost mount-dirs.sh[303]: /dev/xvdb: Inodes that were part of a corrupted orphan linke
Jan 01 10:29:31 localhost mount-dirs.sh[303]: /dev/xvdb: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
Jan 01 10:29:31 localhost mount-dirs.sh[303]: (i.e., without -a or -p options)
Jan 01 10:29:31 localhost kernel: EXT4-fs (xvdb): warning: mounting fs with errors, running e2fsck is reco
Jan 01 10:29:31 localhost kernel: EXT4-fs (xvdb): mounted filesystem with ordered data mode. Opts: discard
Jan 01 10:29:31 localhost mount-dirs.sh[303]: Checking /rw
I tried running fsck/e2fsck, but the command fails cause those partitions are mounted and cannot be unmounted (they are in use). With current issue, everytime I start the standalonevm I need to run xlconsole and exit emergency mode.
Any suggestions?
Thanks and have a great 2020.
Fernando.
Fernando
Jan 1, 2020, 9:04:38 AM1/1/20
to qubes-users
One more detail: this seems to be closely related to https://github.com/QubesOS/qubes-issues/issues/979
Reply all
Reply to author
Forward
0 new messages