Hi Everyone,
I've been running a panda instance (ami-05d7336c) for a client over
the last 6 months. It seemed to be locked up and non-responsive this
morning, so I issued a reboot from the ec2 console, and the instance
rebooted, but it never did respond again. I wound up setting up a new
instance, but I'd like to know if anyone else has run into this
before?
I've attached the last console log I could get from the instance. I
truncated the top of it, it was all fsck's repair process.
Thank You,
Troy
-------
/dev/sda2: |========================================================-
99.6%
/dev/sda2: |========================================================|
100.0%
/dev/sda2: 12/44564480 files (8.3% non-contiguous), 1448314/89121408
blocks
fsck died with exit status 1
done.
Setting kernel variables...done.
Mounting local filesystems...kjournald starting. Commit interval 5
seconds
EXT3 FS on sda2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
mount: none already mounted or /sys busy
mount: according to mtab, sysfs is already mounted on /sys
[31mfailed.
[39;49m
Activating swapfile swap...done.
Setting up networking....
Configuring network interfaces...Internet Systems Consortium DHCP
Client V3.0.4
Copyright 2004-2006 Internet Systems Consortium.
All rights reserved.
For info, please visit
http://www.isc.org/sw/dhcp/
Listening on LPF/eth0/12:31:39:01:6c:f3
Sending on LPF/eth0/12:31:39:01:6c:f3
Sending on Socket/fallback
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 5
DHCPOFFER from 169.254.1.0
DHCPREQUEST on eth0 to 255.255.255.255 port 67
DHCPACK from 169.254.1.0
bound to 10.255.115.1 -- renewal in 36793 seconds.
done.
INIT: Entering runlevel: 4
Starting system log daemon: syslogd.
Starting kernel log daemon: klogd.
Starting nginx: nginx.
[33m*
[39;49m Not starting internet superserver: no services enabled.
Starting Postfix Mail Transport Agent: postfix.
Starting OpenBSD Secure Shell server: sshd.
Starting periodic command scheduler: crond.
Starting daemon monitor: monit.
% Total % Received % Xferd Average Speed Time Time
Time Current
Dload Upload Total Spent
Left Speed
100 392 100 392 0 0 6653 0 --:--:-- --:--:--
--:--:-- 6653
100 392 100 392 0 0 6626 0 --:--:-- --:--:--
--:--:-- 0
Debian GNU/Linux 4.0
ec2-67-202-24-49.compute-1.amazonaws.com tty1
ec2-67-202-24-49.compute-1.amazonaws.com login:
INIT: Switching to runlevel: 6
INIT: Sending processes the TERM signal
Stopping daemon monitor: monit.
Stopping periodic command scheduler: crond.
Stopping nginx: Stopping internet superserver: inetd.
Stopping Postfix Mail Transport Agent: postfix.
Stopping rsync daemon: rsync.
Stopping OpenBSD Secure Shell server: sshd.
Saving the system clock..
Stopping kernel log daemon: klogd.
Stopping system log daemon: syslogd.
Asking all remaining processes to terminate...done.
Killing all remaining processes...done.
Deconfiguring network interfaces...There is already a pid file /var/
run/dhclient.eth0.pid with pid 858
removed stale PID file
Internet Systems Consortium DHCP Client V3.0.4
Copyright 2004-2006 Internet Systems Consortium.
All rights reserved.
For info, please visit
http://www.isc.org/sw/dhcp/
Listening on LPF/eth0/12:31:39:01:6c:f3
Sending on LPF/eth0/12:31:39:01:6c:f3
Sending on Socket/fallback
DHCPRELEASE on eth0 to 169.254.1.0 port 67
done.
Cleaning up ifupdown....
Deactivating swap...done.
Unmounting local filesystems...done.
Will now restart.
md: stopping all md devices.
md: md0 switched to read-only mode.
Restarting system.
.
Linux version 2.6.16-xenU (bui...@xenbat.amazonsa) (gcc version 4.0.1
20050727 (Red Hat 4.0.1-5)) #1 SMP Mon May 28 03:41:49 SAST 2007
BIOS-provided physical RAM map:
Xen: 0000000000000000 - 000000006cc00000 (usable)
1020MB HIGHMEM available.
727MB LOWMEM available.
NX (Execute Disable) protection: active
IRQ lockup detection disabled
Built 1 zonelists
Kernel command line: root=/dev/sda1 ro 4
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 65536 bytes)
Xen reported: 2327.498 MHz processor.
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Software IO TLB disabled
vmalloc area: ee000000-f53fe000, maxmem 2d7fe000
Memory: 1759360k/1789952k available (1958k kernel code, 21308k
reserved, 620k data, 144k init, 1044488k highmem)
Checking if this processor honours the WP bit even in supervisor
mode... Ok.
Calibrating delay using timer specific routine.. 4659.57 BogoMIPS
(lpj=23297884)
Mount-cache hash table entries: 512
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 4096K
CPU: L3 cache: 4096K
Checking 'hlt' instruction... OK.
Brought up 1 CPUs
migration_cost=0
Grant table initialized
NET: Registered protocol family 16
Enabling SMP...
Initializing CPU#1
migration_cost=15754
Brought up 2 CPUs
xen_mem: Initialising balloon driver.
highmem bounce pool size: 64 pages
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
Initializing Cryptographic API
io scheduler noop registered
io scheduler anticipatory registered (default)
io scheduler deadline registered
io scheduler cfq registered
i8042.c: No controller found.
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
Xen virtual console successfully installed as tty1
Event-channel device installed.
netfront: Initialising virtual ethernet driver.
Registering block device major 8
mice: PS/2 mouse device common for all mice
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
NET: Registered protocol family 2
IP route cache hash table entries: 65536 (order: 6, 262144 bytes)
TCP established hash table entries: 262144 (order: 9, 2097152 bytes)
TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
TCP bic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
NET: Registered protocol family 15
Using IPI No-Shortcut mode
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
kjournald starting. Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
Freeing unused kernel memory: 144k freed
INIT: version 2.86 booting
[33m*
[39;49m Mount point '/lib/init/rw' does not exist. Skipping mount.
touch: cannot touch `/lib/init/rw/.ramfs': No such file or directory
Activating swap...Adding 917496k swap on /dev/sda3. Priority:-1
extents:1 across:917496k
done.
Checking root file system...fsck 1.40-WIP (14-Nov-2006)
/dev/sda1: clean, 80987/1313280 files, 365718/2621440 blocks
done.
EXT3 FS on sda1, internal journal
Setting the system clock..
Cleaning up ifupdown....
Loading kernel modules...done.
Loading device-mapper supportdm_mod: no version for "struct_module"
found: kernel tainted.
device-mapper: 4.5.0-ioctl (2005-10-04) initialised: dm-
de...@redhat.com
.
Checking file systems...fsck 1.40-WIP (14-Nov-2006)
/dev/sda2: clean, 12/44564480 files, 1448314/89121408 blocks
done.
Setting kernel variables...done.
Mounting local filesystems...kjournald starting. Commit interval 5
seconds
EXT3 FS on sda2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
mount: none already mounted or /sys busy
mount: according to mtab, sysfs is already mounted on /sys
[31mfailed.
[39;49m
Activating swapfile swap...done.
Setting up networking....
Configuring network interfaces...Internet Systems Consortium DHCP
Client V3.0.4
Copyright 2004-2006 Internet Systems Consortium.
All rights reserved.
For info, please visit
http://www.isc.org/sw/dhcp/
Listening on LPF/eth0/12:31:39:01:6c:f3
Sending on LPF/eth0/12:31:39:01:6c:f3
Sending on Socket/fallback
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 3
DHCPOFFER from 169.254.1.0
DHCPREQUEST on eth0 to 255.255.255.255 port 67
DHCPACK from 169.254.1.0
bound to 10.255.115.1 -- renewal in 41604 seconds.
done.
INIT: Entering runlevel: 4
Starting system log daemon: syslogd.
Starting kernel log daemon: klogd.
Starting nginx: nginx.
[33m*
[39;49m Not starting internet superserver: no services enabled.
Starting Postfix Mail Transport Agent: postfix.
Starting OpenBSD Secure Shell server: sshd.
Starting periodic command scheduler: crond.
Starting daemon monitor: monit.
% Total % Received % Xferd Average Speed Time Time
Time Current
Dload Upload Total Spent
Left Speed
100 392 100 392 0 0 49676 0 --:--:-- --:--:--
--:--:-- 49676
100 392 100 392 0 0 48353 0 --:--:-- --:--:--
--:--:-- 0
Debian GNU/Linux 4.0
ec2-67-202-24-49.compute-1.amazonaws.com tty1
ec2-67-202-24-49.compute-1.amazonaws.com login: fuse init (API version
7.8)
fuse distribution version: 2.7.3
Linux version 2.6.16-xenU (bui...@xenbat.amazonsa) (gcc version 4.0.1
20050727 (Red Hat 4.0.1-5)) #1 SMP Mon May 28 03:41:49 SAST 2007
BIOS-provided physical RAM map:
Xen: 0000000000000000 - 000000006cc00000 (usable)
1020MB HIGHMEM available.
727MB LOWMEM available.
NX (Execute Disable) protection: active
IRQ lockup detection disabled
Built 1 zonelists
Kernel command line: root=/dev/sda1 ro 4
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 65536 bytes)
Xen reported: 2327.500 MHz processor.
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Software IO TLB disabled
vmalloc area: ee000000-f53fe000, maxmem 2d7fe000
Memory: 1759360k/1789952k available (1958k kernel code, 21308k
reserved, 620k data, 144k init, 1044488k highmem)
Checking if this processor honours the WP bit even in supervisor
mode... Ok.
Calibrating delay using timer specific routine.. 4657.09 BogoMIPS
(lpj=23285493)
Mount-cache hash table entries: 512
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 4096K
CPU: L3 cache: 4096K
Checking 'hlt' instruction... OK.
Brought up 1 CPUs
migration_cost=0
Grant table initialized
NET: Registered protocol family 16
Enabling SMP...
Initializing CPU#1
migration_cost=15383
Brought up 2 CPUs
xen_mem: Initialising balloon driver.
highmem bounce pool size: 64 pages
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
Initializing Cryptographic API
io scheduler noop registered
io scheduler anticipatory registered (default)
io scheduler deadline registered
io scheduler cfq registered
i8042.c: No controller found.
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
Xen virtual console successfully installed as tty1
Event-channel device installed.
netfront: Initialising virtual ethernet driver.
Registering block device major 8
mice: PS/2 mouse device common for all mice
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
NET: Registered protocol family 2
IP route cache hash table entries: 65536 (order: 6, 262144 bytes)
TCP established hash table entries: 262144 (order: 9, 2097152 bytes)
TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
TCP bic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
NET: Registered protocol family 15
Using IPI No-Shortcut mode
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
EXT3-fs: INFO: recovery required on readonly filesystem.
EXT3-fs: write access will be enabled during recovery.
kjournald starting. Commit interval 5 seconds
EXT3-fs: sda1: orphan cleanup on readonly fs
EXT3-fs: sda1: 1 orphan inode deleted
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
Freeing unused kernel memory: 144k freed
INIT: version 2.86 booting
[33m*
[39;49m Mount point '/lib/init/rw' does not exist. Skipping mount.
touch: cannot touch `/lib/init/rw/.ramfs': No such file or directory
Activating swap...Adding 917496k swap on /dev/sda3. Priority:-1
extents:1 across:917496k
done.
Checking root file system...fsck 1.40-WIP (14-Nov-2006)
/dev/sda1: clean, 82352/1313280 files, 698892/2621440 blocks
done.
EXT3 FS on sda1, internal journal
Setting the system clock..
Cleaning up ifupdown....
Loading kernel modules...done.
Loading device-mapper supportdm_mod: no version for "struct_module"
found: kernel tainted.
device-mapper: 4.5.0-ioctl (2005-10-04) initialised: dm-
de...@redhat.com
.
Checking file systems...fsck 1.40-WIP (14-Nov-2006)
/dev/sda2: recovering journal
/dev/sda2: clean, 13/44564480 files, 1448315/89121408 blocks
done.
Setting kernel variables...done.
Mounting local filesystems...kjournald starting. Commit interval 5
seconds
EXT3 FS on sda2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
mount: none already mounted or /sys busy
mount: according to mtab, sysfs is already mounted on /sys
[31mfailed.
[39;49m
Activating swapfile swap...done.
Setting up networking....
Configuring network interfaces...Internet Systems Consortium DHCP
Client V3.0.4
Copyright 2004-2006 Internet Systems Consortium.
All rights reserved.
For info, please visit
http://www.isc.org/sw/dhcp/
Listening on LPF/eth0/12:31:39:01:6c:f3
Sending on LPF/eth0/12:31:39:01:6c:f3
Sending on Socket/fallback
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 4
DHCPOFFER from 169.254.1.0
DHCPREQUEST on eth0 to 255.255.255.255 port 67
DHCPACK from 169.254.1.0
bound to 10.255.115.1 -- renewal in 34844 seconds.
done.
INIT: Entering runlevel: 4
Starting system log daemon: syslogd.
Starting kernel log daemon: klogd.
Starting nginx: nginx.
[33m*
[39;49m Not starting internet superserver: no services enabled.
Starting Postfix Mail Transport Agent: postfix.
Starting OpenBSD Secure Shell server: sshd.
Starting periodic command scheduler: crond.
Starting daemon monitor: monit.
% Total % Received % Xferd Average Speed Time Time
Time Current
Dload Upload Total Spent
Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:--
--:--:-- 0
100 392 100 392 0 0 292 0 0:00:01 0:00:01
--:--:-- 1146
Debian GNU/Linux 4.0
ec2-67-202-24-49.compute-1.amazonaws.com tty1
ec2-67-202-24-49.compute-1.amazonaws.com login:
--
You received this message because you are subscribed to the Google Groups "Panda" group.
To post to this group, send email to
panda...@googlegroups.com.
To unsubscribe from this group, send email to
pandastream...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/pandastream?hl=en.