Thanks for finding and reporting this. I've fixed the build script so
the next release will have correct ownership.
The cause of this bug was the steps I took to reduce the size of the
Amazon provided kernel modules from 135 MB down to 14 MB. I had
tested modprobe of various kernel modules, which worked, but I didn't
review file ownership.
Other than the potential security risk of the first created user being
able to modify kernel modules, are there any additional problems
caused by this?
Does anybody need a new cut of the AMIs right away or might this wait
for a future update?
--
Eric Hammond
On Oct 12, 1:14 pm, Dmitriy <dmitriy04...@gmail.com> wrote:
This module is pretty important in an AWS environment (e.g., s3fs) so
I'll be compiling the fuse kernel module from source and releasing new
AMIs when I get a chance.
--
Eric Hammond
On Oct 12, 6:47 am, Eric Hammond <ehamm...@thinksome.com> wrote:
Actually I posted this as a comment on your blog but probably would
have been better to post here.
I noticed the fuse module doesn't exist anymore. In the previous
Jaunty 32bit AMI fuse worked fine after running modprobe fuse. Is
there anyway to have this included in 32 and 64 bit versions again or
was this removed for specific reasons?
Thanks
On Oct 12, 7:07 pm, Eric Hammond <ehamm...@thinksome.com> wrote:
> Thanks for finding and reporting this. I've fixed the build script so
> the next release will have correct ownership.
> The cause of this bug was the steps I took to reduce the size of the
> Amazon provided kernel modules from 135 MB down to 14 MB. I had
> tested modprobe of various kernel modules, which worked, but I didn't
> review file ownership.
> Other than the potential security risk of the first created user being
> able to modify kernel modules, are there any additional problems
> caused by this?
> Does anybody need a new cut of the AMIs right away or might this wait
> for a future update?
> --
> Eric Hammond
> On Oct 12, 1:14 pm, Dmitriy <dmitriy04...@gmail.com> wrote:
On Wed, Oct 14, 2009 at 11:14 PM, Eric Hammond <ehamm...@thinksome.com>wrote:
> In the 20091011 batch of AMIs published on Alestic.com I switched to > Amazon's kernel modules instead of building my own from source.
I'm getting kernel crashes with XFS on the newest Alestic Intrepid images: ami-c0f615a9 32bit/m1.small instance default aki/ari, reported as 2.6.21.7-2.fc8xen-ec2-v1.0
Boot with the following user-data script: #!/bin/bash apt-get update apt-get install xfsprogs umount /mnt mkfs.xfs /dev/sda2 -f FSTAB_ENTRY='/dev/sda2 /mnt xfs defaults,relatime 0 0' perl -pi -e "s%^.* /mnt .*%${FSTAB_ENTRY}%" /etc/fstab mount /mnt touch /mnt/some-file reboot
kapow! kernel oops every time when the system starts up 2nd time (and subsequently)
I guess it's related to one of the other recent threads, but i thought they were all 2.6.18 related...
Thanks for providing such an easy way to reproduce this problem.
In retrospect, I guess we should have seen this coming. With Amazon's
previous 2.6.21 kernel, the original xfs kernel module worked fine with
Hardy and Intrepid, but caused problems with Jaunty. Because of this,
Amazon build a special xfs.ko which I installed with Jaunty images.
With Amazon's newest 2.6.21 kernel and kernel modules, XFS seems to work
fine on Jaunty but breaks with Hardy and Intrepid (after reboot).
> Thanks for providing such an easy way to reproduce this problem.
> In retrospect, I guess we should have seen this coming. With Amazon's
> previous 2.6.21 kernel, the original xfs kernel module worked fine with
> Hardy and Intrepid, but caused problems with Jaunty. Because of this,
> Amazon build a special xfs.ko which I installed with Jaunty images.
> With Amazon's newest 2.6.21 kernel and kernel modules, XFS seems to work
> fine on Jaunty but breaks with Hardy and Intrepid (after reboot).
Quick follow up: going back to ami-ed46a784 (the 20090804 32 bit
Jaunty image from Eric) doesn't exhibit this behavior. XFS seems to
behave fine using that image.
Mark
On Oct 19, 2:46 pm, Gorism <mark.go...@gmail.com> wrote:
> On Oct 15, 1:21 am, Eric Hammond <ehamm...@thinksome.com> wrote:
> > Robert:
> > Thanks for providing such an easy way to reproduce this problem.
> > In retrospect, I guess we should have seen this coming. With Amazon's
> > previous 2.6.21 kernel, the original xfs kernel module worked fine with
> > Hardy and Intrepid, but caused problems with Jaunty. Because of this,
> > Amazon build a special xfs.ko which I installed with Jaunty images.
> > With Amazon's newest 2.6.21 kernel and kernel modules, XFS seems to work
> > fine on Jaunty but breaks with Hardy and Intrepid (after reboot).
Yep, just be aware that reverting to the previous AMI downgrades the kernel to one which includes the security hole where users on the system can gain root.
At the moment we're stuck with two kernel choices from Amazon for 2.6.21, both of which have problems (XFS vs. security).
If your application has any chance of running on Karmic, then I recommend trying out the Beta AMIs from Canonical which run on 2.6.31.
There has been a little movement on Hardy from Canonical, but I don't expect a lot to get done there in the short term since Karmic is launching in a couple weeks.
I have also heard some folks having success running the Alestic.com Jaunty on the 2.6.27 kernel from Canonical, but that kernel was built a while back and I don't know how reliable or secure it is.
Gorism wrote: > Quick follow up: going back to ami-ed46a784 (the 20090804 32 bit > Jaunty image from Eric) doesn't exhibit this behavior. XFS seems to > behave fine using that image.
> Mark
> On Oct 19, 2:46 pm, Gorism <mark.go...@gmail.com> wrote: >> I'm getting a crash with the base 32 bit image (ami-ccf615a5) by >> simply:
>> 1. Starting a fresh one >> 2. Either mounting an existing -or- creating a new XFS based volume >> 3. Unmounting the volume
>> At that point the instance is unresponsive and must be terminated. No >> reboot required.
>> This doesn't occur with ext3 based volumes.
>> A kernel bug shows up in the console, same one as Robert provided >> above.
>> On Oct 15, 1:21 am, Eric Hammond <ehamm...@thinksome.com> wrote:
>>> Robert: >>> Thanks for providing such an easy way to reproduce this problem. >>> In retrospect, I guess we should have seen this coming. With Amazon's >>> previous 2.6.21 kernel, the original xfs kernel module worked fine with >>> Hardy and Intrepid, but caused problems with Jaunty. Because of this, >>> Amazon build a special xfs.ko which I installed with Jaunty images. >>> With Amazon's newest 2.6.21 kernel and kernel modules, XFS seems to work >>> fine on Jaunty but breaks with Hardy and Intrepid (after reboot). >>> I've posted a request for help from Amazon: >>> http://developer.amazonwebservices.com/connect/thread.jspa?threadID=3... >>> -- >>> Eric Hammond >>> Robert Coup wrote: >>>> Hey folks, >>>> On Wed, Oct 14, 2009 at 11:14 PM, Eric Hammond <ehamm...@thinksome.com >>>> <mailto:ehamm...@thinksome.com>> wrote: >>>> In the 20091011 batch of AMIs published on Alestic.com I switched to >>>> Amazon's kernel modules instead of building my own from source. >>>> I'm getting kernel crashes with XFS on the newest Alestic Intrepid images: >>>> ami-c0f615a9 >>>> 32bit/m1.small instance >>>> default aki/ari, reported as 2.6.21.7-2.fc8xen-ec2-v1.0 >>>> Boot with the following user-data script: >>>> #!/bin/bash >>>> apt-get update >>>> apt-get install xfsprogs >>>> umount /mnt >>>> mkfs.xfs /dev/sda2 -f >>>> FSTAB_ENTRY='/dev/sda2 /mnt xfs defaults,relatime 0 0' >>>> perl -pi -e "s%^.* /mnt .*%${FSTAB_ENTRY}%" /etc/fstab >>>> mount /mnt >>>> touch /mnt/some-file >>>> reboot >>>> kapow! kernel oops every time when the system starts up 2nd time (and >>>> subsequently) >>>> I guess it's related to one of the other recent threads, but i thought >>>> they were all 2.6.18 related... >>>> Console log (initial + reboot):http://pastebin.com/fc54044b >>>> Should i be running a different aki/ari? >>>> Rob :)
I saw that you reached out to Amazon with a request for help. In that
request you indicated that XFS worked fine with Jaunty with the 2.6.21
kernel, which wasn't the case when I tried it. That's why I did the
follow up post; in case you weren't aware of that.
Sounds like you are aware of the problems with Jaunty + 2.6.21 + XFS,
though.
Have you heard anything back from Amazon regarding your suggestion to
compile a new xfs driver? I didn't see any replies to your original
post, so I'm guessing not.
Mark
On Oct 19, 6:16 pm, Eric Hammond <ehamm...@thinksome.com> wrote:
> Yep, just be aware that reverting to the previous AMI downgrades the
> kernel to one which includes the security hole where users on the system
> can gain root.
> At the moment we're stuck with two kernel choices from Amazon for
> 2.6.21, both of which have problems (XFS vs. security).
> If your application has any chance of running on Karmic, then I
> recommend trying out the Beta AMIs from Canonical which run on 2.6.31.
> There has been a little movement on Hardy from Canonical, but I don't
> expect a lot to get done there in the short term since Karmic is
> launching in a couple weeks.
> I have also heard some folks having success running the Alestic.com
> Jaunty on the 2.6.27 kernel from Canonical, but that kernel was built a
> while back and I don't know how reliable or secure it is.
> --
> Eric Hammond
> Gorism wrote:
> > Quick follow up: going back to ami-ed46a784 (the 20090804 32 bit
> > Jaunty image from Eric) doesn't exhibit this behavior. XFS seems to
> > behave fine using that image.
> > Mark
> > On Oct 19, 2:46 pm, Gorism <mark.go...@gmail.com> wrote:
> >> I'm getting a crash with the base 32 bit image (ami-ccf615a5) by
> >> simply:
> >> 1. Starting a fresh one
> >> 2. Either mounting an existing -or- creating a new XFS based volume
> >> 3. Unmounting the volume
> >> At that point the instance is unresponsive and must be terminated. No
> >> reboot required.
> >> This doesn't occur with ext3 based volumes.
> >> A kernel bug shows up in the console, same one as Robert provided
> >> above.
> >> On Oct 15, 1:21 am, Eric Hammond <ehamm...@thinksome.com> wrote:
> >>> Robert:
> >>> Thanks for providing such an easy way to reproduce this problem.
> >>> In retrospect, I guess we should have seen this coming. With Amazon's
> >>> previous 2.6.21 kernel, the original xfs kernel module worked fine with
> >>> Hardy and Intrepid, but caused problems with Jaunty. Because of this,
> >>> Amazon build a special xfs.ko which I installed with Jaunty images.
> >>> With Amazon's newest 2.6.21 kernel and kernel modules, XFS seems to work
> >>> fine on Jaunty but breaks with Hardy and Intrepid (after reboot).
> >>> I've posted a request for help from Amazon:
> >>>http://developer.amazonwebservices.com/connect/thread.jspa?threadID=3...
> >>> --
> >>> Eric Hammond
> >>> Robert Coup wrote:
> >>>> Hey folks,
> >>>> On Wed, Oct 14, 2009 at 11:14 PM, Eric Hammond <ehamm...@thinksome.com
> >>>> <mailto:ehamm...@thinksome.com>> wrote:
> >>>> In the 20091011 batch of AMIs published on Alestic.com I switched to
> >>>> Amazon's kernel modules instead of building my own from source.
> >>>> I'm getting kernel crashes with XFS on the newest Alestic Intrepid images:
> >>>> ami-c0f615a9
> >>>> 32bit/m1.small instance
> >>>> default aki/ari, reported as 2.6.21.7-2.fc8xen-ec2-v1.0
> >>>> Boot with the following user-data script:
> >>>> #!/bin/bash
> >>>> apt-get update
> >>>> apt-get install xfsprogs
> >>>> umount /mnt
> >>>> mkfs.xfs /dev/sda2 -f
> >>>> FSTAB_ENTRY='/dev/sda2 /mnt xfs defaults,relatime 0 0'
> >>>> perl -pi -e "s%^.* /mnt .*%${FSTAB_ENTRY}%" /etc/fstab
> >>>> mount /mnt
> >>>> touch /mnt/some-file
> >>>> reboot
> >>>> kapow! kernel oops every time when the system starts up 2nd time (and
> >>>> subsequently)
> >>>> I guess it's related to one of the other recent threads, but i thought
> >>>> they were all 2.6.18 related...
> >>>> Console log (initial + reboot):http://pastebin.com/fc54044b > >>>> Should i be running a different aki/ari?
> >>>> Rob :)
> I saw that you reached out to Amazon with a request for help. In that
> request you indicated that XFS worked fine with Jaunty with the 2.6.21
> kernel, which wasn't the case when I tried it. That's why I did the
> follow up post; in case you weren't aware of that.
> Sounds like you are aware of the problems with Jaunty + 2.6.21 + XFS,
> though.
> Have you heard anything back from Amazon regarding your suggestion to
> compile a new xfs driver? I didn't see any replies to your original
> post, so I'm guessing not.
> Mark
> On Oct 19, 6:16 pm, Eric Hammond <ehamm...@thinksome.com> wrote:
>> Mark:
>> Yep, just be aware that reverting to the previous AMI downgrades the
>> kernel to one which includes the security hole where users on the system
>> can gain root.
>> At the moment we're stuck with two kernel choices from Amazon for
>> 2.6.21, both of which have problems (XFS vs. security).
>> If your application has any chance of running on Karmic, then I
>> recommend trying out the Beta AMIs from Canonical which run on 2.6.31.
>> There has been a little movement on Hardy from Canonical, but I don't
>> expect a lot to get done there in the short term since Karmic is
>> launching in a couple weeks.
>> I have also heard some folks having success running the Alestic.com
>> Jaunty on the 2.6.27 kernel from Canonical, but that kernel was built a
>> while back and I don't know how reliable or secure it is.
>> --
>> Eric Hammond
>> Gorism wrote:
>>> Quick follow up: going back to ami-ed46a784 (the 20090804 32 bit
>>> Jaunty image from Eric) doesn't exhibit this behavior. XFS seems to
>>> behave fine using that image.
>>> Mark
>>> On Oct 19, 2:46 pm, Gorism <mark.go...@gmail.com> wrote:
>>>> I'm getting a crash with the base 32 bit image (ami-ccf615a5) by
>>>> simply:
>>>> 1. Starting a fresh one
>>>> 2. Either mounting an existing -or- creating a new XFS based volume
>>>> 3. Unmounting the volume
>>>> At that point the instance is unresponsive and must be terminated. No
>>>> reboot required.
>>>> This doesn't occur with ext3 based volumes.
>>>> A kernel bug shows up in the console, same one as Robert provided
>>>> above.
>>>> Mark
>>>> ------------[ cut here ]------------
>>>> kernel BUG at arch/i386/mm/hypervisor.c:185!
>>>> invalid opcode: 0000 [#1]
>>>> SMP
>>>> last sysfs file: /block/sda1/uevent
>>>> Modules linked in: ipv6(F)(U) xfs(F)(U) xennet(F)(U) xenblk(F)(U) ext3
>>>> (F)(U) jbd(F)(U) mbcache(F)(U)
>>>> CPU: 0
>>>> EIP: 0061:[<c101bf0e>] Tainted: GF VLI
>>>> EFLAGS: 00210282 (2.6.21.7-2.fc8xen-ec2-v1.0 #2)
>>>> EIP is at xen_pgd_pin+0x54/0x5e
>>>> eax: ffffffea ebx: c143eef8 ecx: 00000001 edx: 00000000
>>>> esi: 00007ff0 edi: 00000000 ebp: c19cef70 esp: c143eef8
>>>> ds: 007b es: 007b fs: 00d8 gs: 0033 ss: 0069
>>>> Process bash (pid: 1200, ti=c143e000 task=c140cef0 task.ti=c143e000)
>>>> Stack: 00000002 0020aa88 016f6000 0002dec0 00000000 c101816d c14b0480
>>>> c1018196
>>>> c15e31d4 c102250e c15e38d0 c143efb8 c106e111 c143efb8 bfd0a71c
>>>> 01200011
>>>> 00000000 c19cef70 c14b0480 c1346640 c15e31e0 c15e31f4 c15e31f0
>>>> c14b0480
>>>> Call Trace:
>>>> [<c101816d>] __pgd_pin+0x2f/0x3c
>>>> [<c1018196>] mm_pin+0x1c/0x23
>>>> [<c102250e>] copy_process+0xac3/0x10bc
>>>> [<c106e111>] kmem_cache_alloc+0x23/0x98
>>>> [<c1022b58>] do_fork+0x51/0x13a
>>>> [<c100321e>] sys_clone+0x36/0x3b
>>>> [<c1005688>] syscall_call+0x7/0xb
>>>> =======================
>>>> Code: eb fe a1 24 c2 38 c1 8b 14 90 81 e2 ff ff ff 7f 89 54 24 04 89
>>>> e3 b9 01 00 00 00 31 d2 be f0 7f 00 00 e8 36 54 fe ff 85 c0 79 04 <0f>
>>>> 0b eb fe 83 c4 0c 5b 5e c3 56 89 c2 53 83 ec 0c c1 ea 0c 80
>>>> EIP: [<c101bf0e>] xen_pgd_pin+0x54/0x5e SS:ESP 0069:c143eef8
>>>> ------------[ cut here ]------------
>>>> kernel BUG at arch/i386/mm/hypervisor.c:185!
>>>> invalid opcode: 0000 [#2]
>>>> SMP
>>>> last sysfs file: /block/sda1/uevent
>>>> Modules linked in: ipv6(F)(U) xfs(F)(U) xennet(F)(U) xenblk(F)(U) ext3
>>>> (F)(U) jbd(F)(U) mbcache(F)(U)
>>>> CPU: 0
>>>> EIP: 0061:[<c101bf0e>] Tainted: GF VLI
>>>> EFLAGS: 00210282 (2.6.21.7-2.fc8xen-ec2-v1.0 #2)
>>>> EIP is at xen_pgd_pin+0x54/0x5e
>>>> eax: ffffffea ebx: c1411ef8 ecx: 00000001 edx: 00000000
>>>> esi: 00007ff0 edi: 00000000 ebp: c140f490 esp: c1411ef8
>>>> ds: 007b es: 007b fs: 00d8 gs: 0033 ss: 0069
>>>> Process syslogd (pid: 1031, ti=c1411000 task=c140d450
>>>> task.ti=c1411000)
>>>> Stack: 00000002 002916f9 2d176000 005a2ec0 00000000 c101816d c14b0280
>>>> c1018196
>>>> c15f10cc c102250e c15f15b8 c1411fb8 c106e111 c1411fb8 bff296b4
>>>> 01200011
>>>> 00000000 c140f490 c14b0280 c14b0080 c15f10d8 c15f10ec c15f10e8
>>>> c14b0280
>>>> Call Trace:
>>>> [<c101816d>] __pgd_pin+0x2f/0x3c
>>>> [<c1018196>] mm_pin+0x1c/0x23
>>>> [<c102250e>] copy_process+0xac3/0x10bc
>>>> [<c106e111>] kmem_cache_alloc+0x23/0x98
>>>> [<c1022b58>] do_fork+0x51/0x13a
>>>> [<c10e48a8>] copy_to_user+0x3c/0x50
>>>> [<c100321e>] sys_clone+0x36/0x3b
>>>> [<c1005688>] syscall_call+0x7/0xb
>>>> =======================
>>>> Code: eb fe a1 24 c2 38 c1 8b 14 90 81 e2 ff ff ff 7f 89 54 24 04 89
>>>> e3 b9 01 00 00 00 31 d2 be f0 7f 00 00 e8 36 54 fe ff 85 c0 79 04 <0f>
>>>> 0b eb fe 83 c4 0c 5b 5e c3 56 89 c2 53 83 ec 0c c1 ea 0c 80
>>>> EIP: [<c101bf0e>] xen_pgd_pin+0x54/0x5e SS:ESP 0069:c1411ef8
>>>> On Oct 15, 1:21 am, Eric Hammond <ehamm...@thinksome.com> wrote:
>>>>> Robert:
>>>>> Thanks for providing such an easy way to reproduce this problem.
>>>>> In retrospect, I guess we should have seen this coming. With Amazon's
>>>>> previous 2.6.21 kernel, the original xfs kernel module worked fine with
>>>>> Hardy and Intrepid, but caused problems with Jaunty. Because of this,
>>>>> Amazon build a special xfs.ko which I installed with Jaunty images.
>>>>> With Amazon's newest 2.6.21 kernel and kernel modules, XFS seems to work
>>>>> fine on Jaunty but breaks with Hardy and Intrepid (after reboot).
>>>>> I've posted a request for help from Amazon:
>>>>> http://developer.amazonwebservices.com/connect/thread.jspa?threadID=3...
>>>>> --
>>>>> Eric Hammond
>>>>> Robert Coup wrote:
>>>>>> Hey folks,
>>>>>> On Wed, Oct 14, 2009 at 11:14 PM, Eric Hammond <ehamm...@thinksome.com
>>>>>> <mailto:ehamm...@thinksome.com>> wrote:
>>>>>> In the 20091011 batch of AMIs published on Alestic.com I switched to
>>>>>> Amazon's kernel modules instead of building my own from source.
>>>>>> I'm getting kernel crashes with XFS on the newest Alestic Intrepid images:
>>>>>> ami-c0f615a9
>>>>>> 32bit/m1.small instance
>>>>>> default aki/ari, reported as 2.6.21.7-2.fc8xen-ec2-v1.0
>>>>>> Boot with the following user-data script:
>>>>>> #!/bin/bash
>>>>>> apt-get update
>>>>>> apt-get install xfsprogs
>>>>>> umount /mnt
>>>>>> mkfs.xfs /dev/sda2 -f
>>>>>> FSTAB_ENTRY='/dev/sda2 /mnt xfs defaults,relatime 0 0'
>>>>>> perl -pi -e "s%^.* /mnt .*%${FSTAB_ENTRY}%" /etc/fstab
>>>>>> mount /mnt
>>>>>> touch /mnt/some-file
>>>>>> reboot
>>>>>> kapow! kernel oops every time when the system starts up 2nd time (and
>>>>>> subsequently)
>>>>>> I guess it's related to one of the other recent threads, but i thought
>>>>>> they were all 2.6.18 related...
>>>>>> Console log (initial + reboot):http://pastebin.com/fc54044b >>>>>> Should i be running a different aki/ari?
>>>>>> Rob :)
Thanks to you both for this thread, and related traffic.
I'm preparing to try the canonical karmic image -- canonical-testing- eu/karmic-i386-20090929.manifest.xml -- for use in EU-west, and I'm
doing pre-tests on the AWS Management Console.
When I try to run image ami-404f6434 that requires kernel
aki-6e4f641a, I get
"Nonexistent default AKI 'aki-6e4f641a' for AMI 'ami-404f6434'."
But running in us-east the counterpart ami-145ebe7d that uses
aki-841efeed there is no problem.
Is this related to Eric's thread relating to ambiguity in selection of
AKIs?:
> Thanks for the reminder. I've posted an update on the forum thread.
> --
> Eric Hammond
> Gorism wrote:
>> Thank for the follow up, Eric.
>> I saw that you reached out to Amazon with a request for help. In
>> that
>> request you indicated that XFS worked fine with Jaunty with the
>> 2.6.21
>> kernel, which wasn't the case when I tried it. That's why I did the
>> follow up post; in case you weren't aware of that.
>> Sounds like you are aware of the problems with Jaunty + 2.6.21 + XFS,
>> though.
>> Have you heard anything back from Amazon regarding your suggestion to
>> compile a new xfs driver? I didn't see any replies to your original
>> post, so I'm guessing not.
>> Mark
>> On Oct 19, 6:16 pm, Eric Hammond <ehamm...@thinksome.com> wrote:
>>> Mark:
>>> Yep, just be aware that reverting to the previous AMI downgrades the
>>> kernel to one which includes the security hole where users on the
>>> system
>>> can gain root.
>>> At the moment we're stuck with two kernel choices from Amazon for
>>> 2.6.21, both of which have problems (XFS vs. security).
>>> If your application has any chance of running on Karmic, then I
>>> recommend trying out the Beta AMIs from Canonical which run on
>>> 2.6.31.
>>> There has been a little movement on Hardy from Canonical, but I
>>> don't
>>> expect a lot to get done there in the short term since Karmic is
>>> launching in a couple weeks.
>>> I have also heard some folks having success running the Alestic.com
>>> Jaunty on the 2.6.27 kernel from Canonical, but that kernel was
>>> built a
>>> while back and I don't know how reliable or secure it is.
>>> --
>>> Eric Hammond
>>> Gorism wrote:
>>>> Quick follow up: going back to ami-ed46a784 (the 20090804 32 bit
>>>> Jaunty image from Eric) doesn't exhibit this behavior. XFS seems
>>>> to
>>>> behave fine using that image.
>>>> Mark
>>>> On Oct 19, 2:46 pm, Gorism <mark.go...@gmail.com> wrote:
>>>>> I'm getting a crash with the base 32 bit image (ami-ccf615a5) by
>>>>> simply:
>>>>> 1. Starting a fresh one
>>>>> 2. Either mounting an existing -or- creating a new XFS based
>>>>> volume
>>>>> 3. Unmounting the volume
>>>>> At that point the instance is unresponsive and must be
>>>>> terminated. No
>>>>> reboot required.
>>>>> This doesn't occur with ext3 based volumes.
>>>>> A kernel bug shows up in the console, same one as Robert provided
>>>>> above.
>>>>> Mark
>>>>> ------------[ cut here ]------------
>>>>> kernel BUG at arch/i386/mm/hypervisor.c:185!
>>>>> invalid opcode: 0000 [#1]
>>>>> SMP
>>>>> last sysfs file: /block/sda1/uevent
>>>>> Modules linked in: ipv6(F)(U) xfs(F)(U) xennet(F)(U) xenblk(F) >>>>> (U) ext3
>>>>> (F)(U) jbd(F)(U) mbcache(F)(U)
>>>>> CPU: 0
>>>>> EIP: 0061:[<c101bf0e>] Tainted: GF VLI
>>>>> EFLAGS: 00210282 (2.6.21.7-2.fc8xen-ec2-v1.0 #2)
>>>>> EIP is at xen_pgd_pin+0x54/0x5e
>>>>> eax: ffffffea ebx: c143eef8 ecx: 00000001 edx: 00000000
>>>>> esi: 00007ff0 edi: 00000000 ebp: c19cef70 esp: c143eef8
>>>>> ds: 007b es: 007b fs: 00d8 gs: 0033 ss: 0069
>>>>> Process bash (pid: 1200, ti=c143e000 task=c140cef0
>>>>> task.ti=c143e000)
>>>>> Stack: 00000002 0020aa88 016f6000 0002dec0 00000000 c101816d
>>>>> c14b0480
>>>>> c1018196
>>>>> c15e31d4 c102250e c15e38d0 c143efb8 c106e111 c143efb8
>>>>> bfd0a71c
>>>>> 01200011
>>>>> 00000000 c19cef70 c14b0480 c1346640 c15e31e0 c15e31f4
>>>>> c15e31f0
>>>>> c14b0480
>>>>> Call Trace:
>>>>> [<c101816d>] __pgd_pin+0x2f/0x3c
>>>>> [<c1018196>] mm_pin+0x1c/0x23
>>>>> [<c102250e>] copy_process+0xac3/0x10bc
>>>>> [<c106e111>] kmem_cache_alloc+0x23/0x98
>>>>> [<c1022b58>] do_fork+0x51/0x13a
>>>>> [<c100321e>] sys_clone+0x36/0x3b
>>>>> [<c1005688>] syscall_call+0x7/0xb
>>>>> =======================
>>>>> Code: eb fe a1 24 c2 38 c1 8b 14 90 81 e2 ff ff ff 7f 89 54 24
>>>>> 04 89
>>>>> e3 b9 01 00 00 00 31 d2 be f0 7f 00 00 e8 36 54 fe ff 85 c0 79
>>>>> 04 <0f>
>>>>> 0b eb fe 83 c4 0c 5b 5e c3 56 89 c2 53 83 ec 0c c1 ea 0c 80
>>>>> EIP: [<c101bf0e>] xen_pgd_pin+0x54/0x5e SS:ESP 0069:c143eef8
>>>>> ------------[ cut here ]------------
>>>>> kernel BUG at arch/i386/mm/hypervisor.c:185!
>>>>> invalid opcode: 0000 [#2]
>>>>> SMP
>>>>> last sysfs file: /block/sda1/uevent
>>>>> Modules linked in: ipv6(F)(U) xfs(F)(U) xennet(F)(U) xenblk(F) >>>>> (U) ext3
>>>>> (F)(U) jbd(F)(U) mbcache(F)(U)
>>>>> CPU: 0
>>>>> EIP: 0061:[<c101bf0e>] Tainted: GF VLI
>>>>> EFLAGS: 00210282 (2.6.21.7-2.fc8xen-ec2-v1.0 #2)
>>>>> EIP is at xen_pgd_pin+0x54/0x5e
>>>>> eax: ffffffea ebx: c1411ef8 ecx: 00000001 edx: 00000000
>>>>> esi: 00007ff0 edi: 00000000 ebp: c140f490 esp: c1411ef8
>>>>> ds: 007b es: 007b fs: 00d8 gs: 0033 ss: 0069
>>>>> Process syslogd (pid: 1031, ti=c1411000 task=c140d450
>>>>> task.ti=c1411000)
>>>>> Stack: 00000002 002916f9 2d176000 005a2ec0 00000000 c101816d
>>>>> c14b0280
>>>>> c1018196
>>>>> c15f10cc c102250e c15f15b8 c1411fb8 c106e111 c1411fb8
>>>>> bff296b4
>>>>> 01200011
>>>>> 00000000 c140f490 c14b0280 c14b0080 c15f10d8 c15f10ec
>>>>> c15f10e8
>>>>> c14b0280
>>>>> Call Trace:
>>>>> [<c101816d>] __pgd_pin+0x2f/0x3c
>>>>> [<c1018196>] mm_pin+0x1c/0x23
>>>>> [<c102250e>] copy_process+0xac3/0x10bc
>>>>> [<c106e111>] kmem_cache_alloc+0x23/0x98
>>>>> [<c1022b58>] do_fork+0x51/0x13a
>>>>> [<c10e48a8>] copy_to_user+0x3c/0x50
>>>>> [<c100321e>] sys_clone+0x36/0x3b
>>>>> [<c1005688>] syscall_call+0x7/0xb
>>>>> =======================
>>>>> Code: eb fe a1 24 c2 38 c1 8b 14 90 81 e2 ff ff ff 7f 89 54 24
>>>>> 04 89
>>>>> e3 b9 01 00 00 00 31 d2 be f0 7f 00 00 e8 36 54 fe ff 85 c0 79
>>>>> 04 <0f>
>>>>> 0b eb fe 83 c4 0c 5b 5e c3 56 89 c2 53 83 ec 0c c1 ea 0c 80
>>>>> EIP: [<c101bf0e>] xen_pgd_pin+0x54/0x5e SS:ESP 0069:c1411ef8
>>>>> On Oct 15, 1:21 am, Eric Hammond <ehamm...@thinksome.com> wrote:
>>>>>> Robert:
>>>>>> Thanks for providing such an easy way to reproduce this problem.
>>>>>> In retrospect, I guess we should have seen this coming. With
>>>>>> Amazon's
>>>>>> previous 2.6.21 kernel, the original xfs kernel module worked
>>>>>> fine with
>>>>>> Hardy and Intrepid, but caused problems with Jaunty. Because
>>>>>> of this,
>>>>>> Amazon build a special xfs.ko which I installed with Jaunty
>>>>>> images.
>>>>>> With Amazon's newest 2.6.21 kernel and kernel modules, XFS
>>>>>> seems to work
>>>>>> fine on Jaunty but breaks with Hardy and Intrepid (after reboot).
>>>>>> I've posted a request for help from Amazon:
>>>>>> http://developer.amazonwebservices.com/connect/thread.jspa?threadID=3 >>>>>> ...
>>>>>> --
>>>>>> Eric Hammond
>>>>>> Robert Coup wrote:
>>>>>>> Hey folks,
>>>>>>> On Wed, Oct 14, 2009 at 11:14 PM, Eric Hammond
>>>>>>> <ehamm...@thinksome.com
>>>>>>> <mailto:ehamm...@thinksome.com>> wrote:
>>>>>>> In the 20091011 batch of AMIs published on Alestic.com I
>>>>>>> switched to
>>>>>>> Amazon's kernel modules instead of building my own from
>>>>>>> source.
>>>>>>> I'm getting kernel crashes with XFS on the newest Alestic
>>>>>>> Intrepid images:
>>>>>>> ami-c0f615a9
>>>>>>> 32bit/m1.small instance
>>>>>>> default aki/ari, reported as 2.6.21.7-2.fc8xen-ec2-v1.0
>>>>>>> Boot with the following user-data script:
>>>>>>> #!/bin/bash
>>>>>>> apt-get update
>>>>>>> apt-get install xfsprogs
>>>>>>> umount /mnt
>>>>>>> mkfs.xfs /dev/sda2 -f
>>>>>>> FSTAB_ENTRY='/dev/sda2 /mnt xfs defaults,relatime 0 0'
>>>>>>> perl -pi -e "s%^.* /mnt .*%${FSTAB_ENTRY}%" /etc/fstab
>>>>>>> mount /mnt
>>>>>>> touch /mnt/some-file
>>>>>>> reboot
>>>>>>> kapow! kernel oops every time when the system starts up 2nd
>>>>>>> time (and
>>>>>>> subsequently)
>>>>>>> I guess it's related to one of the other recent threads, but i
>>>>>>> thought
>>>>>>> they were all 2.6.18 related...
>>>>>>> Console log (initial + reboot):http://pastebin.com/fc54044b >>>>>>> Should i be running a different aki/ari?
>>>>>>> Rob :)
Russ Ferriday - Topia Systems - Open Source content management with
Plone and Zope
ru...@topia.com - mobile: +1 805 234 6915 - skype: ferriday
On Tue, 20 Oct 2009, Russ Ferriday wrote: > Thanks to you both for this thread, and related traffic.
> I'm preparing to try the canonical karmic image -- canonical-testing- > eu/karmic-i386-20090929.manifest.xml -- for use in EU-west, and I'm > doing pre-tests on the AWS Management Console.
> When I try to run image ami-404f6434 that requires kernel > aki-6e4f641a, I get
> "Nonexistent default AKI 'aki-6e4f641a' for AMI 'ami-404f6434'."
> But running in us-east the counterpart ami-145ebe7d that uses > aki-841efeed there is no problem.
The above is an error, probably on my part. The error is correctly stating that, while aki-6e4f641a is referenced by ami-404f6434, there is no such registered kernel. I've removed the AMI in question to remove any confusion. There will probably be further cleanup that needs to be done at some point in the near future.
I've been getting automated publishing of our daily builds working over the last 3 weeks or so.
We're fairly well set now. Automated daily build output is published to 'ubuntu-images-testing-us' and 'ubuntu-images-testing-eu' at the same time. You can see builds named such as: ubuntu-images-testing-eu/ubuntu-karmic-daily-i386-server-20091020.1.manifes t.xml ubuntu-images-testing-eu/ubuntu-karmic-daily-amd64-server-20091020.1.manife st.xml
There is more information on the publishing policy at https://wiki.ubuntu.com/UEC/Images/NamingConvention . The above represent our RC candidate builds. If successful testing, they'll be promoted this thursday (10/22) to the 'ubuntu-images-milestone-eu' bucket as: ubuntu-images-milestone-eu/ubuntu-karmic-daily-i386-server-20091020.1.manif est.xml ubuntu-images-milestone-eu/ubuntu-karmic-daily-amd64-server-20091020.1.mani fest.xml
On promotion, their AMI ids and name will change, but their content will not. The new AMIs will be released as RC, and will not be removed. The nightly AMI's further above *will* be removed.
> And am I wasting my time trying bleeding-edge tests in the EU region?
I surely hope not. I apologize for the snafu that you've seen and the lack of ext4 support in the karmic kernel. I can only promise that we're getting better, and the NamingConvention doc above should be consistent from here out.
> I surely hope not. I apologize for the snafu that you've seen and the > lack of ext4 support in the karmic kernel. I can only promise that we're > getting better, and the NamingConvention doc above should be consistent > from here out.
Regarding the ext4 support in karmic, that bug 428692 is set for 'karmic updates' , so it is expected that at some point soon after release we will get ext4 support in AMIs. https://bugs.launchpad.net/ubuntu/+bug/428692
Scott, thank you for your prompt response, and for clarifying your
policies.
I had just gone back to testing a Hardy version for use in eu, but
will now give it another shot with one of the these ubuntu daily
karmic builds. I'll check that I can get XFS working, ready for a
switch to a milestone image.
>> Thanks to you both for this thread, and related traffic.
>> I'm preparing to try the canonical karmic image -- canonical-testing-
>> eu/karmic-i386-20090929.manifest.xml -- for use in EU-west, and I'm
>> doing pre-tests on the AWS Management Console.
>> When I try to run image ami-404f6434 that requires kernel
>> aki-6e4f641a, I get
>> "Nonexistent default AKI 'aki-6e4f641a' for AMI
>> 'ami-404f6434'."
>> But running in us-east the counterpart ami-145ebe7d that uses
>> aki-841efeed there is no problem.
> The above is an error, probably on my part. The error is correctly
> stating that, while aki-6e4f641a is referenced by ami-404f6434,
> there is
> no such registered kernel. I've removed the AMI in question to
> remove
> any confusion. There will probably be further cleanup that needs to
> be
> done at some point in the near future.
> I've been getting automated publishing of our daily builds working
> over
> the last 3 weeks or so.
> We're fairly well set now. Automated daily build output is
> published to
> 'ubuntu-images-testing-us' and 'ubuntu-images-testing-eu' at the same
> time. You can see builds named such as:
> ubuntu-images-testing-eu/ubuntu-karmic-daily-i386- > server-20091020.1.manifest.xml
> ubuntu-images-testing-eu/ubuntu-karmic-daily-amd64- > server-20091020.1.manifest.xml
> There is more information on the publishing policy at
> https://wiki.ubuntu.com/UEC/Images/NamingConvention . The above
> represent
> our RC candidate builds. If successful testing, they'll be promoted
> this
> thursday (10/22) to the 'ubuntu-images-milestone-eu' bucket as:
> ubuntu-images-milestone-eu/ubuntu-karmic-daily-i386- > server-20091020.1.manifest.xml
> ubuntu-images-milestone-eu/ubuntu-karmic-daily-amd64- > server-20091020.1.manifest.xml
> On promotion, their AMI ids and name will change, but their content
> will
> not. The new AMIs will be released as RC, and will not be removed. > The
> nightly AMI's further above *will* be removed.
>> And am I wasting my time trying bleeding-edge tests in the EU region?
> I surely hope not. I apologize for the snafu that you've seen and the
> lack of ext4 support in the karmic kernel. I can only promise that
> we're
> getting better, and the NamingConvention doc above should be
> consistent
> from here out.
> This module is pretty important in an AWS environment (e.g., s3fs) so
> I'll be compiling the fuse kernel module from source and releasing new
> AMIs when I get a chance.
> --
> Eric Hammond
> On Oct 12, 6:47 am, Eric Hammond <ehamm...@thinksome.com> wrote:
> > New updates have been released for the Ubuntu and Debian AMIs (EC2
> > images) published on:
Nearly 2 months without clear closure on this topic is fairly
worrisome. Did I miss an update that actually resolves the reported
XFS-related kernel instability?
Randy
On Oct 20, 9:10 am, Eric Hammond <ehamm...@thinksome.com> wrote:
> Thanks for the reminder. I've posted an update on the forum thread.
> --
> Eric Hammond
> Gorism wrote:
> > Thank for the follow up, Eric.
> > I saw that you reached out to Amazon with a request for help. In that
> > request you indicated that XFS worked fine with Jaunty with the 2.6.21
> > kernel, which wasn't the case when I tried it. That's why I did the
> > follow up post; in case you weren't aware of that.
> > Sounds like you are aware of the problems with Jaunty + 2.6.21 + XFS,
> > though.
> > Have you heard anything back from Amazon regarding your suggestion to
> > compile a new xfs driver? I didn't see any replies to your original
> > post, so I'm guessing not.
> > Mark
> > On Oct 19, 6:16 pm, Eric Hammond <ehamm...@thinksome.com> wrote:
> >> Mark:
> >> Yep, just be aware that reverting to the previous AMI downgrades the
> >> kernel to one which includes the security hole where users on the system
> >> can gain root.
> >> At the moment we're stuck with two kernel choices from Amazon for
> >> 2.6.21, both of which have problems (XFS vs. security).
> >> If your application has any chance of running on Karmic, then I
> >> recommend trying out the Beta AMIs from Canonical which run on 2.6.31.
> >> There has been a little movement on Hardy from Canonical, but I don't
> >> expect a lot to get done there in the short term since Karmic is
> >> launching in a couple weeks.
> >> I have also heard some folks having success running the Alestic.com
> >> Jaunty on the 2.6.27 kernel from Canonical, but that kernel was built a
> >> while back and I don't know how reliable or secure it is.
> >> --
> >> Eric Hammond
> >> Gorism wrote:
> >>> Quick follow up: going back to ami-ed46a784 (the 20090804 32 bit
> >>> Jaunty image from Eric) doesn't exhibit this behavior. XFS seems to
> >>> behave fine using that image.
> >>> Mark
> >>> On Oct 19, 2:46 pm, Gorism <mark.go...@gmail.com> wrote:
> >>>> I'm getting a crash with the base 32 bit image (ami-ccf615a5) by
> >>>> simply:
> >>>> 1. Starting a fresh one
> >>>> 2. Either mounting an existing -or- creating a new XFS based volume
> >>>> 3. Unmounting the volume
> >>>> At that point the instance is unresponsive and must be terminated. No
> >>>> reboot required.
> >>>> This doesn't occur with ext3 based volumes.
> >>>> A kernel bug shows up in the console, same one as Robert provided
> >>>> above.
> >>>> Mark
> >>>> ------------[ cut here ]------------
> >>>> kernel BUG at arch/i386/mm/hypervisor.c:185!
> >>>> invalid opcode: 0000 [#1]
> >>>> SMP
> >>>> last sysfs file: /block/sda1/uevent
> >>>> Modules linked in: ipv6(F)(U) xfs(F)(U) xennet(F)(U) xenblk(F)(U) ext3
> >>>> (F)(U) jbd(F)(U) mbcache(F)(U)
> >>>> CPU: 0
> >>>> EIP: 0061:[<c101bf0e>] Tainted: GF VLI
> >>>> EFLAGS: 00210282 (2.6.21.7-2.fc8xen-ec2-v1.0 #2)
> >>>> EIP is at xen_pgd_pin+0x54/0x5e
> >>>> eax: ffffffea ebx: c143eef8 ecx: 00000001 edx: 00000000
> >>>> esi: 00007ff0 edi: 00000000 ebp: c19cef70 esp: c143eef8
> >>>> ds: 007b es: 007b fs: 00d8 gs: 0033 ss: 0069
> >>>> Process bash (pid: 1200, ti=c143e000 task=c140cef0 task.ti=c143e000)
> >>>> Stack: 00000002 0020aa88 016f6000 0002dec0 00000000 c101816d c14b0480
> >>>> c1018196
> >>>> c15e31d4 c102250e c15e38d0 c143efb8 c106e111 c143efb8 bfd0a71c
> >>>> 01200011
> >>>> 00000000 c19cef70 c14b0480 c1346640 c15e31e0 c15e31f4 c15e31f0
> >>>> c14b0480
> >>>> Call Trace:
> >>>> [<c101816d>] __pgd_pin+0x2f/0x3c
> >>>> [<c1018196>] mm_pin+0x1c/0x23
> >>>> [<c102250e>] copy_process+0xac3/0x10bc
> >>>> [<c106e111>] kmem_cache_alloc+0x23/0x98
> >>>> [<c1022b58>] do_fork+0x51/0x13a
> >>>> [<c100321e>] sys_clone+0x36/0x3b
> >>>> [<c1005688>] syscall_call+0x7/0xb
> >>>> =======================
> >>>> Code: eb fe a1 24 c2 38 c1 8b 14 90 81 e2 ff ff ff 7f 89 54 24 04 89
> >>>> e3 b9 01 00 00 00 31 d2 be f0 7f 00 00 e8 36 54 fe ff 85 c0 79 04 <0f>
> >>>> 0b eb fe 83 c4 0c 5b 5e c3 56 89 c2 53 83 ec 0c c1 ea 0c 80
> >>>> EIP: [<c101bf0e>] xen_pgd_pin+0x54/0x5e SS:ESP 0069:c143eef8
> >>>> ------------[ cut here ]------------
> >>>> kernel BUG at arch/i386/mm/hypervisor.c:185!
> >>>> invalid opcode: 0000 [#2]
> >>>> SMP
> >>>> last sysfs file: /block/sda1/uevent
> >>>> Modules linked in: ipv6(F)(U) xfs(F)(U) xennet(F)(U) xenblk(F)(U) ext3
> >>>> (F)(U) jbd(F)(U) mbcache(F)(U)
> >>>> CPU: 0
> >>>> EIP: 0061:[<c101bf0e>] Tainted: GF VLI
> >>>> EFLAGS: 00210282 (2.6.21.7-2.fc8xen-ec2-v1.0 #2)
> >>>> EIP is at xen_pgd_pin+0x54/0x5e
> >>>> eax: ffffffea ebx: c1411ef8 ecx: 00000001 edx: 00000000
> >>>> esi: 00007ff0 edi: 00000000 ebp: c140f490 esp: c1411ef8
> >>>> ds: 007b es: 007b fs: 00d8 gs: 0033 ss: 0069
> >>>> Process syslogd (pid: 1031, ti=c1411000 task=c140d450
> >>>> task.ti=c1411000)
> >>>> Stack: 00000002 002916f9 2d176000 005a2ec0 00000000 c101816d c14b0280
> >>>> c1018196
> >>>> c15f10cc c102250e c15f15b8 c1411fb8 c106e111 c1411fb8 bff296b4
> >>>> 01200011
> >>>> 00000000 c140f490 c14b0280 c14b0080 c15f10d8 c15f10ec c15f10e8
> >>>> c14b0280
> >>>> Call Trace:
> >>>> [<c101816d>] __pgd_pin+0x2f/0x3c
> >>>> [<c1018196>] mm_pin+0x1c/0x23
> >>>> [<c102250e>] copy_process+0xac3/0x10bc
> >>>> [<c106e111>] kmem_cache_alloc+0x23/0x98
> >>>> [<c1022b58>] do_fork+0x51/0x13a
> >>>> [<c10e48a8>] copy_to_user+0x3c/0x50
> >>>> [<c100321e>] sys_clone+0x36/0x3b
> >>>> [<c1005688>] syscall_call+0x7/0xb
> >>>> =======================
> >>>> Code: eb fe a1 24 c2 38 c1 8b 14 90 81 e2 ff ff ff 7f 89 54 24 04 89
> >>>> e3 b9 01 00 00 00 31 d2 be f0 7f 00 00 e8 36 54 fe ff 85 c0 79 04 <0f>
> >>>> 0b eb fe 83 c4 0c 5b 5e c3 56 89 c2 53 83 ec 0c c1 ea 0c 80
> >>>> EIP: [<c101bf0e>] xen_pgd_pin+0x54/0x5e SS:ESP 0069:c1411ef8
> >>>> On Oct 15, 1:21 am, Eric Hammond <ehamm...@thinksome.com> wrote:
> >>>>> Robert:
> >>>>> Thanks for providing such an easy way to reproduce this problem.
> >>>>> In retrospect, I guess we should have seen this coming. With Amazon's
> >>>>> previous 2.6.21 kernel, the original xfs kernel module worked fine with
> >>>>> Hardy and Intrepid, but caused problems with Jaunty. Because of this,
> >>>>> Amazon build a special xfs.ko which I installed with Jaunty images.
> >>>>> With Amazon's newest 2.6.21 kernel and kernel modules, XFS seems to work
> >>>>> fine on Jaunty but breaks with Hardy and Intrepid (after reboot).
> >>>>> I've posted a request for help from Amazon:
> >>>>>http://developer.amazonwebservices.com/connect/thread.jspa?threadID=3...
> >>>>> --
> >>>>> Eric Hammond
> >>>>> Robert Coup wrote:
> >>>>>> Hey folks,
> >>>>>> On Wed, Oct 14, 2009 at 11:14 PM, Eric Hammond <ehamm...@thinksome.com
> >>>>>> <mailto:ehamm...@thinksome.com>> wrote:
> >>>>>> In the 20091011 batch of AMIs published on Alestic.com I switched to
> >>>>>> Amazon's kernel modules instead of building my own from source.
> >>>>>> I'm getting kernel crashes with XFS on the newest Alestic Intrepid images:
> >>>>>> ami-c0f615a9
> >>>>>> 32bit/m1.small instance
> >>>>>> default aki/ari, reported as 2.6.21.7-2.fc8xen-ec2-v1.0
> >>>>>> Boot with the following user-data script:
> >>>>>> #!/bin/bash
> >>>>>> apt-get update
> >>>>>> apt-get install xfsprogs
> >>>>>> umount /mnt
> >>>>>> mkfs.xfs /dev/sda2 -f
> >>>>>> FSTAB_ENTRY='/dev/sda2 /mnt xfs defaults,relatime 0 0'
> >>>>>> perl -pi -e "s%^.* /mnt .*%${FSTAB_ENTRY}%" /etc/fstab
> >>>>>> mount /mnt
> >>>>>> touch /mnt/some-file
> >>>>>> reboot
> >>>>>> kapow! kernel oops every time when the system starts up 2nd time (and
> >>>>>> subsequently)
> >>>>>> I guess it's related to one of the other recent threads, but i thought
> >>>>>> they were all 2.6.18 related...
> >>>>>> Console log (initial + reboot):http://pastebin.com/fc54044b > >>>>>> Should i be running a different aki/ari?
> >>>>>> Rob :)
(1) Stick with the 20090804 AMIs in the "alestic" buckets (not
directly listed on http://alestic.com). These work well with XFS but
the kernel has a security hole which allows normal users to become
root.
(2) Switch to the Ubuntu Karmic and Hardy AMIs published by
Canonical. These use official Ubuntu kernels which have no bug
reports against them on EC2.
I recommend (2) since those AMIs are likely to be better supported
going forward (including by me).
Advanced users may wish to try "(3) Run Alestic AMIs with the
Canonical kernels" but that's not much different from (2) since the
Alestic AMIs are so close to the official Canonical AMIs except for
the kernels.
--
Eric Hammond
On Dec 15, 9:58 am, Randy Harmon <rjharmon0...@gmail.com> wrote:
> Nearly 2 months without clear closure on this topic is fairly
> worrisome. Did I miss an update that actually resolves the reported
> XFS-related kernel instability?
> Randy
> On Oct 20, 9:10 am, Eric Hammond <ehamm...@thinksome.com> wrote:
> > Gorism:
> > Thanks for the reminder. I've posted an update on the forum thread.
> > --
> > Eric Hammond
> > Gorism wrote:
> > > Thank for the follow up, Eric.
> > > I saw that you reached out to Amazon with a request for help. In that
> > > request you indicated that XFS worked fine with Jaunty with the 2.6.21
> > > kernel, which wasn't the case when I tried it. That's why I did the
> > > follow up post; in case you weren't aware of that.
> > > Sounds like you are aware of the problems with Jaunty + 2.6.21 + XFS,
> > > though.
> > > Have you heard anything back from Amazon regarding your suggestion to
> > > compile a new xfs driver? I didn't see any replies to your original
> > > post, so I'm guessing not.
> > > Mark
> > > On Oct 19, 6:16 pm, Eric Hammond <ehamm...@thinksome.com> wrote:
> > >> Mark:
> > >> Yep, just be aware that reverting to the previous AMI downgrades the
> > >> kernel to one which includes the security hole where users on the system
> > >> can gain root.
> > >> At the moment we're stuck with two kernel choices from Amazon for
> > >> 2.6.21, both of which have problems (XFS vs. security).
> > >> If your application has any chance of running on Karmic, then I
> > >> recommend trying out the Beta AMIs from Canonical which run on 2.6.31.
> > >> There has been a little movement on Hardy from Canonical, but I don't
> > >> expect a lot to get done there in the short term since Karmic is
> > >> launching in a couple weeks.
> > >> I have also heard some folks having success running the Alestic.com
> > >> Jaunty on the 2.6.27 kernel from Canonical, but that kernel was built a
> > >> while back and I don't know how reliable or secure it is.
> > >> --
> > >> Eric Hammond
> > >> Gorism wrote:
> > >>> Quick follow up: going back to ami-ed46a784 (the 20090804 32 bit
> > >>> Jaunty image from Eric) doesn't exhibit this behavior. XFS seems to
> > >>> behave fine using that image.
> > >>> Mark
> > >>> On Oct 19, 2:46 pm, Gorism <mark.go...@gmail.com> wrote:
> > >>>> I'm getting a crash with the base 32 bit image (ami-ccf615a5) by
> > >>>> simply:
> > >>>> 1. Starting a fresh one
> > >>>> 2. Either mounting an existing -or- creating a new XFS based volume
> > >>>> 3. Unmounting the volume
> > >>>> At that point the instance is unresponsive and must be terminated. No
> > >>>> reboot required.
> > >>>> This doesn't occur with ext3 based volumes.
> > >>>> A kernel bug shows up in the console, same one as Robert provided
> > >>>> above.
> > >>>> Mark
> > >>>> ------------[ cut here ]------------
> > >>>> kernel BUG at arch/i386/mm/hypervisor.c:185!
> > >>>> invalid opcode: 0000 [#1]
> > >>>> SMP
> > >>>> last sysfs file: /block/sda1/uevent
> > >>>> Modules linked in: ipv6(F)(U) xfs(F)(U) xennet(F)(U) xenblk(F)(U) ext3
> > >>>> (F)(U) jbd(F)(U) mbcache(F)(U)
> > >>>> CPU: 0
> > >>>> EIP: 0061:[<c101bf0e>] Tainted: GF VLI
> > >>>> EFLAGS: 00210282 (2.6.21.7-2.fc8xen-ec2-v1.0 #2)
> > >>>> EIP is at xen_pgd_pin+0x54/0x5e
> > >>>> eax: ffffffea ebx: c143eef8 ecx: 00000001 edx: 00000000
> > >>>> esi: 00007ff0 edi: 00000000 ebp: c19cef70 esp: c143eef8
> > >>>> ds: 007b es: 007b fs: 00d8 gs: 0033 ss: 0069
> > >>>> Process bash (pid: 1200, ti=c143e000 task=c140cef0 task.ti=c143e000)
> > >>>> Stack: 00000002 0020aa88 016f6000 0002dec0 00000000 c101816d c14b0480
> > >>>> c1018196
> > >>>> c15e31d4 c102250e c15e38d0 c143efb8 c106e111 c143efb8 bfd0a71c
> > >>>> 01200011
> > >>>> 00000000 c19cef70 c14b0480 c1346640 c15e31e0 c15e31f4 c15e31f0
> > >>>> c14b0480
> > >>>> Call Trace:
> > >>>> [<c101816d>] __pgd_pin+0x2f/0x3c
> > >>>> [<c1018196>] mm_pin+0x1c/0x23
> > >>>> [<c102250e>] copy_process+0xac3/0x10bc
> > >>>> [<c106e111>] kmem_cache_alloc+0x23/0x98
> > >>>> [<c1022b58>] do_fork+0x51/0x13a
> > >>>> [<c100321e>] sys_clone+0x36/0x3b
> > >>>> [<c1005688>] syscall_call+0x7/0xb
> > >>>> =======================
> > >>>> Code: eb fe a1 24 c2 38 c1 8b 14 90 81 e2 ff ff ff 7f 89 54 24 04 89
> > >>>> e3 b9 01 00 00 00 31 d2 be f0 7f 00 00 e8 36 54 fe ff 85 c0 79 04 <0f>
> > >>>> 0b eb fe 83 c4 0c 5b 5e c3 56 89 c2 53 83 ec 0c c1 ea 0c 80
> > >>>> EIP: [<c101bf0e>] xen_pgd_pin+0x54/0x5e SS:ESP 0069:c143eef8
> > >>>> ------------[ cut here ]------------
> > >>>> kernel BUG at arch/i386/mm/hypervisor.c:185!
> > >>>> invalid opcode: 0000 [#2]
> > >>>> SMP
> > >>>> last sysfs file: /block/sda1/uevent
> > >>>> Modules linked in: ipv6(F)(U) xfs(F)(U) xennet(F)(U) xenblk(F)(U) ext3
> > >>>> (F)(U) jbd(F)(U) mbcache(F)(U)
> > >>>> CPU: 0
> > >>>> EIP: 0061:[<c101bf0e>] Tainted: GF VLI
> > >>>> EFLAGS: 00210282 (2.6.21.7-2.fc8xen-ec2-v1.0 #2)
> > >>>> EIP is at xen_pgd_pin+0x54/0x5e
> > >>>> eax: ffffffea ebx: c1411ef8 ecx: 00000001 edx: 00000000
> > >>>> esi: 00007ff0 edi: 00000000 ebp: c140f490 esp: c1411ef8
> > >>>> ds: 007b es: 007b fs: 00d8 gs: 0033 ss: 0069
> > >>>> Process syslogd (pid: 1031, ti=c1411000 task=c140d450
> > >>>> task.ti=c1411000)
> > >>>> Stack: 00000002 002916f9 2d176000 005a2ec0 00000000 c101816d c14b0280
> > >>>> c1018196
> > >>>> c15f10cc c102250e c15f15b8 c1411fb8 c106e111 c1411fb8 bff296b4
> > >>>> 01200011
> > >>>> 00000000 c140f490 c14b0280 c14b0080 c15f10d8 c15f10ec c15f10e8
> > >>>> c14b0280
> > >>>> Call Trace:
> > >>>> [<c101816d>] __pgd_pin+0x2f/0x3c
> > >>>> [<c1018196>] mm_pin+0x1c/0x23
> > >>>> [<c102250e>] copy_process+0xac3/0x10bc
> > >>>> [<c106e111>] kmem_cache_alloc+0x23/0x98
> > >>>> [<c1022b58>] do_fork+0x51/0x13a
> > >>>> [<c10e48a8>] copy_to_user+0x3c/0x50
> > >>>> [<c100321e>] sys_clone+0x36/0x3b
> > >>>> [<c1005688>] syscall_call+0x7/0xb
> > >>>> =======================
> > >>>> Code: eb fe a1 24 c2 38 c1 8b 14 90 81 e2 ff ff ff 7f 89 54 24 04 89
> > >>>> e3 b9 01 00 00 00 31 d2 be f0 7f 00 00 e8 36 54 fe ff 85 c0 79 04 <0f>
> > >>>> 0b eb fe 83 c4 0c 5b 5e c3 56 89 c2 53 83 ec 0c c1 ea 0c 80
> > >>>> EIP: [<c101bf0e>] xen_pgd_pin+0x54/0x5e SS:ESP 0069:c1411ef8
> > >>>> On Oct 15, 1:21 am, Eric Hammond <ehamm...@thinksome.com> wrote:
> > >>>>> Robert:
> > >>>>> Thanks for providing such an easy way to reproduce this problem.
> > >>>>> In retrospect, I guess we should have seen this coming. With Amazon's
> > >>>>> previous 2.6.21 kernel, the original xfs kernel module worked fine with
> > >>>>> Hardy and Intrepid, but caused problems with Jaunty. Because of this,
> > >>>>> Amazon build a special xfs.ko which I installed with Jaunty images.
> > >>>>> With Amazon's newest 2.6.21 kernel and kernel modules, XFS seems to work
> > >>>>> fine on Jaunty but breaks with Hardy and Intrepid (after reboot).
> > >>>>> I've posted a request for help from Amazon:
> > >>>>>http://developer.amazonwebservices.com/connect/thread.jspa?threadID=3...
> > >>>>> --
> > >>>>> Eric Hammond
> > >>>>> Robert Coup wrote:
> > >>>>>> Hey folks,
> > >>>>>> On Wed, Oct 14, 2009 at 11:14 PM, Eric Hammond <ehamm...@thinksome.com
> > >>>>>> <mailto:ehamm...@thinksome.com>> wrote:
> > >>>>>> In the 20091011 batch of AMIs published on Alestic.com I switched to
> > >>>>>> Amazon's kernel modules instead of building my own from source.
> > >>>>>> I'm getting kernel crashes with XFS on the newest Alestic Intrepid images:
> > >>>>>> ami-c0f615a9
> > >>>>>> 32bit/m1.small instance
> > >>>>>> default aki/ari, reported as 2.6.21.7-2.fc8xen-ec2-v1.0
> > >>>>>> Boot with the following user-data script:
> > >>>>>> #!/bin/bash
> > >>>>>> apt-get update
> > >>>>>> apt-get install xfsprogs
> > >>>>>> umount /mnt
> > >>>>>> mkfs.xfs /dev/sda2 -f
> > >>>>>> FSTAB_ENTRY='/dev/sda2 /mnt xfs defaults,relatime 0 0'
> > >>>>>> perl -pi -e "s%^.* /mnt .*%${FSTAB_ENTRY}%" /etc/fstab
> > >>>>>> mount /mnt
> > >>>>>> touch /mnt/some-file
> > >>>>>> reboot
> > >>>>>> kapow! kernel oops every time when the system starts up 2nd time (and
> > >>>>>> subsequently)
> > >>>>>> I guess it's related to one of the other recent threads, but i thought
> > >>>>>> they were all 2.6.18 related...
> > >>>>>> Console log (initial + reboot):http://pastebin.com/fc54044b > > >>>>>> Should i be running a different aki/ari?
> > >>>>>> Rob :)
> (1) Stick with the 20090804 AMIs in the "alestic" buckets (not
> directly listed on http://alestic.com). These work well with XFS but
> the kernel has a security hole which allows normal users to become
> root.
> (2) Switch to the Ubuntu Karmic and Hardy AMIs published by
> Canonical. These use official Ubuntu kernels which have no bug
> reports against them on EC2.
> I recommend (2) since those AMIs are likely to be better supported
> going forward (including by me).
> Advanced users may wish to try "(3) Run Alestic AMIs with the
> Canonical kernels" but that's not much different from (2) since the
> Alestic AMIs are so close to the official Canonical AMIs except for
> the kernels.
> --
> Eric Hammond
> On Dec 15, 9:58 am, Randy Harmon <rjharmon0...@gmail.com> wrote:
> > Nearly 2 months without clear closure on this topic is fairly
> > worrisome. Did I miss an update that actually resolves the reported
> > XFS-related kernel instability?
> > Randy
> > On Oct 20, 9:10 am, Eric Hammond <ehamm...@thinksome.com> wrote:
> > > Gorism:
> > > Thanks for the reminder. I've posted an update on the forum thread.
> > > --
> > > Eric Hammond
> > > Gorism wrote:
> > > > Thank for the follow up, Eric.
> > > > I saw that you reached out to Amazon with a request for help. In
> that
> > > > request you indicated that XFS worked fine with Jaunty with the
> 2.6.21
> > > > kernel, which wasn't the case when I tried it. That's why I did the
> > > > follow up post; in case you weren't aware of that.
> > > > Sounds like you are aware of the problems with Jaunty + 2.6.21 + XFS,
> > > > though.
> > > > Have you heard anything back from Amazon regarding your suggestion to
> > > > compile a new xfs driver? I didn't see any replies to your original
> > > > post, so I'm guessing not.
> > > > Mark
> > > > On Oct 19, 6:16 pm, Eric Hammond <ehamm...@thinksome.com> wrote:
> > > >> Mark:
> > > >> Yep, just be aware that reverting to the previous AMI downgrades the
> > > >> kernel to one which includes the security hole where users on the
> system
> > > >> can gain root.
> > > >> At the moment we're stuck with two kernel choices from Amazon for
> > > >> 2.6.21, both of which have problems (XFS vs. security).
> > > >> If your application has any chance of running on Karmic, then I
> > > >> recommend trying out the Beta AMIs from Canonical which run on
> 2.6.31.
> > > >> There has been a little movement on Hardy from Canonical, but I
> don't
> > > >> expect a lot to get done there in the short term since Karmic is
> > > >> launching in a couple weeks.
> > > >> I have also heard some folks having success running the Alestic.com
> > > >> Jaunty on the 2.6.27 kernel from Canonical, but that kernel was
> built a
> > > >> while back and I don't know how reliable or secure it is.
> > > >> --
> > > >> Eric Hammond
> > > >> Gorism wrote:
> > > >>> Quick follow up: going back to ami-ed46a784 (the 20090804 32 bit
> > > >>> Jaunty image from Eric) doesn't exhibit this behavior. XFS seems
> to
> > > >>> behave fine using that image.
> > > >>> Mark
> > > >>> On Oct 19, 2:46 pm, Gorism <mark.go...@gmail.com> wrote:
> > > >>>> I'm getting a crash with the base 32 bit image (ami-ccf615a5) by
> > > >>>> simply:
> > > >>>> 1. Starting a fresh one
> > > >>>> 2. Either mounting an existing -or- creating a new XFS based
> volume
> > > >>>> 3. Unmounting the volume
> > > >>>> At that point the instance is unresponsive and must be terminated.
> No
> > > >>>> reboot required.
> > > >>>> This doesn't occur with ext3 based volumes.
> > > >>>> A kernel bug shows up in the console, same one as Robert provided
> > > >>>> above.
> > > >>>> Mark
> > > >>>> ------------[ cut here ]------------
> > > >>>> kernel BUG at arch/i386/mm/hypervisor.c:185!
> > > >>>> invalid opcode: 0000 [#1]
> > > >>>> SMP
> > > >>>> last sysfs file: /block/sda1/uevent
> > > >>>> Modules linked in: ipv6(F)(U) xfs(F)(U) xennet(F)(U) xenblk(F)(U)
> ext3
> > > >>>> (F)(U) jbd(F)(U) mbcache(F)(U)
> > > >>>> CPU: 0
> > > >>>> EIP: 0061:[<c101bf0e>] Tainted: GF VLI
> > > >>>> EFLAGS: 00210282 (2.6.21.7-2.fc8xen-ec2-v1.0 #2)
> > > >>>> EIP is at xen_pgd_pin+0x54/0x5e
> > > >>>> eax: ffffffea ebx: c143eef8 ecx: 00000001 edx: 00000000
> > > >>>> esi: 00007ff0 edi: 00000000 ebp: c19cef70 esp: c143eef8
> > > >>>> ds: 007b es: 007b fs: 00d8 gs: 0033 ss: 0069
> > > >>>> Process bash (pid: 1200, ti=c143e000 task=c140cef0
> task.ti=c143e000)
> > > >>>> Stack: 00000002 0020aa88 016f6000 0002dec0 00000000 c101816d
> c14b0480
> > > >>>> c1018196
> > > >>>> c15e31d4 c102250e c15e38d0 c143efb8 c106e111 c143efb8
> bfd0a71c
> > > >>>> 01200011
> > > >>>> 00000000 c19cef70 c14b0480 c1346640 c15e31e0 c15e31f4
> c15e31f0
> > > >>>> c14b0480
> > > >>>> Call Trace:
> > > >>>> [<c101816d>] __pgd_pin+0x2f/0x3c
> > > >>>> [<c1018196>] mm_pin+0x1c/0x23
> > > >>>> [<c102250e>] copy_process+0xac3/0x10bc
> > > >>>> [<c106e111>] kmem_cache_alloc+0x23/0x98
> > > >>>> [<c1022b58>] do_fork+0x51/0x13a
> > > >>>> [<c100321e>] sys_clone+0x36/0x3b
> > > >>>> [<c1005688>] syscall_call+0x7/0xb
> > > >>>> =======================
> > > >>>> Code: eb fe a1 24 c2 38 c1 8b 14 90 81 e2 ff ff ff 7f 89 54 24 04
> 89
> > > >>>> e3 b9 01 00 00 00 31 d2 be f0 7f 00 00 e8 36 54 fe ff 85 c0 79 04
> <0f>
> > > >>>> 0b eb fe 83 c4 0c 5b 5e c3 56 89 c2 53 83 ec 0c c1 ea 0c 80
> > > >>>> EIP: [<c101bf0e>] xen_pgd_pin+0x54/0x5e SS:ESP 0069:c143eef8
> > > >>>> ------------[ cut here ]------------
> > > >>>> kernel BUG at arch/i386/mm/hypervisor.c:185!
> > > >>>> invalid opcode: 0000 [#2]
> > > >>>> SMP
> > > >>>> last sysfs file: /block/sda1/uevent
> > > >>>> Modules linked in: ipv6(F)(U) xfs(F)(U) xennet(F)(U) xenblk(F)(U)
> ext3
> > > >>>> (F)(U) jbd(F)(U) mbcache(F)(U)
> > > >>>> CPU: 0
> > > >>>> EIP: 0061:[<c101bf0e>] Tainted: GF VLI
> > > >>>> EFLAGS: 00210282 (2.6.21.7-2.fc8xen-ec2-v1.0 #2)
> > > >>>> EIP is at xen_pgd_pin+0x54/0x5e
> > > >>>> eax: ffffffea ebx: c1411ef8 ecx: 00000001 edx: 00000000
> > > >>>> esi: 00007ff0 edi: 00000000 ebp: c140f490 esp: c1411ef8
> > > >>>> ds: 007b es: 007b fs: 00d8 gs: 0033 ss: 0069
> > > >>>> Process syslogd (pid: 1031, ti=c1411000 task=c140d450
> > > >>>> task.ti=c1411000)
> > > >>>> Stack: 00000002 002916f9 2d176000 005a2ec0 00000000 c101816d
> c14b0280
> > > >>>> c1018196
> > > >>>> c15f10cc c102250e c15f15b8 c1411fb8 c106e111 c1411fb8
> bff296b4
> > > >>>> 01200011
> > > >>>> 00000000 c140f490 c14b0280 c14b0080 c15f10d8 c15f10ec
> c15f10e8
> > > >>>> c14b0280
> > > >>>> Call Trace:
> > > >>>> [<c101816d>] __pgd_pin+0x2f/0x3c
> > > >>>> [<c1018196>] mm_pin+0x1c/0x23
> > > >>>> [<c102250e>] copy_process+0xac3/0x10bc
> > > >>>> [<c106e111>] kmem_cache_alloc+0x23/0x98
> > > >>>> [<c1022b58>] do_fork+0x51/0x13a
> > > >>>> [<c10e48a8>] copy_to_user+0x3c/0x50
> > > >>>> [<c100321e>] sys_clone+0x36/0x3b
> > > >>>> [<c1005688>] syscall_call+0x7/0xb
> > > >>>> =======================
> > > >>>> Code: eb fe a1 24 c2 38 c1 8b 14 90 81 e2 ff ff ff 7f 89 54 24 04
> 89
> > > >>>> e3 b9 01 00 00 00 31 d2 be f0 7f 00 00 e8 36 54 fe ff 85 c0 79 04
> <0f>
> > > >>>> 0b eb fe 83 c4 0c 5b 5e c3 56 89 c2 53 83 ec 0c c1 ea 0c 80
> > > >>>> EIP: [<c101bf0e>] xen_pgd_pin+0x54/0x5e SS:ESP 0069:c1411ef8
> > > >>>> On Oct 15, 1:21 am, Eric Hammond <ehamm...@thinksome.com> wrote:
> > > >>>>> Robert:
> > > >>>>> Thanks for providing such an easy way to reproduce this problem.
> > > >>>>> In retrospect, I guess we should have seen this coming. With
> Amazon's
> > > >>>>> previous 2.6.21 kernel, the original xfs kernel module worked
> fine with
> > > >>>>> Hardy and Intrepid, but caused problems with Jaunty. Because of
> this,
> > > >>>>> Amazon build a special xfs.ko which I installed with Jaunty
> images.
> > > >>>>> With Amazon's newest 2.6.21 kernel and kernel modules, XFS seems
> to work
> > > >>>>> fine on Jaunty but breaks with Hardy and Intrepid (after reboot).
> > > >>>>> I've posted a request for help from Amazon:
> http://developer.amazonwebservices.com/connect/thread.jspa?threadID=3...
> > > >>>>> --
> > > >>>>> Eric Hammond
> > > >>>>> Robert Coup wrote:
> > > >>>>>> Hey folks,
> > > >>>>>> On Wed, Oct 14, 2009 at 11:14 PM, Eric Hammond <
> ehamm...@thinksome.com
> > > >>>>>> <mailto:ehamm...@thinksome.com>> wrote:
> > > >>>>>> In the 20091011 batch of AMIs published on Alestic.com I
> switched to
> > > >>>>>> Amazon's kernel modules instead of building my own from
> source.
> > > >>>>>> I'm getting kernel crashes with XFS on the newest Alestic
> Intrepid images:
> > > >>>>>> ami-c0f615a9
> > > >>>>>> 32bit/m1.small instance
> > > >>>>>> default aki/ari, reported as 2.6.21.7-2.fc8xen-ec2-v1.0
> > > >>>>>> Boot with the following user-data script:
> > > >>>>>> #!/bin/bash
> > > >>>>>> apt-get update
When launching in us-west I notice there's a spot in the instance launch where you can specify a different AKI. I don't remember seeing this field before yesterday, but it might allow trying a different AKI without rebundling - and if the launch works right, I would expect that bundling an AMI from that running instance would point by default at the AKI you intend.
> so it seems and in-place kernel upgrade is impossible on an ec2 > instance? kind of annoying, yet good to know!
> I will start from a fresh instance and see how things go,
> Thanks again,
> On Tue, Dec 15, 2009 at 10:14 AM, Eric Hammond <ehamm...@thinksome.com > <mailto:ehamm...@thinksome.com>> wrote:
> Randy:
> You have a couple options:
> (1) Stick with the 20090804 AMIs in the "alestic" buckets (not
> directly listed on http://alestic.com). These work well with XFS but
> the kernel has a security hole which allows normal users to become
> root.
> (2) Switch to the Ubuntu Karmic and Hardy AMIs published by
> Canonical. These use official Ubuntu kernels which have no bug
> reports against them on EC2.
> I recommend (2) since those AMIs are likely to be better supported
> going forward (including by me).
> Advanced users may wish to try "(3) Run Alestic AMIs with the
> Canonical kernels" but that's not much different from (2) since the
> Alestic AMIs are so close to the official Canonical AMIs except for
> the kernels.
> --
> Eric Hammond
> On Dec 15, 9:58 am, Randy Harmon <rjharmon0...@gmail.com
> <mailto:rjharmon0...@gmail.com>> wrote:
> > Nearly 2 months without clear closure on this topic is fairly
> > worrisome. Did I miss an update that actually resolves the reported
> > XFS-related kernel instability?
> > Randy
> > On Oct 20, 9:10 am, Eric Hammond <ehamm...@thinksome.com
> <mailto:ehamm...@thinksome.com>> wrote:
> > > Gorism:
> > > Thanks for the reminder. I've posted an update on the forum
> thread.
> > > --
> > > Eric Hammond
> > > Gorism wrote:
> > > > Thank for the follow up, Eric.
> > > > I saw that you reached out to Amazon with a request for
> help. In that
> > > > request you indicated that XFS worked fine with Jaunty with
> the 2.6.21
> > > > kernel, which wasn't the case when I tried it. That's why I
> did the
> > > > follow up post; in case you weren't aware of that.
> > > > Sounds like you are aware of the problems with Jaunty +
> 2.6.21 + XFS,
> > > > though.
> > > > Have you heard anything back from Amazon regarding your
> suggestion to
> > > > compile a new xfs driver? I didn't see any replies to your
> original
> > > > post, so I'm guessing not.
> > > > Mark
> > > > On Oct 19, 6:16 pm, Eric Hammond <ehamm...@thinksome.com
> <mailto:ehamm...@thinksome.com>> wrote:
> > > >> Mark:
> > > >> Yep, just be aware that reverting to the previous AMI
> downgrades the
> > > >> kernel to one which includes the security hole where users
> on the system
> > > >> can gain root.
> > > >> At the moment we're stuck with two kernel choices from
> Amazon for
> > > >> 2.6.21, both of which have problems (XFS vs. security).
> > > >> If your application has any chance of running on Karmic, then I
> > > >> recommend trying out the Beta AMIs from Canonical which run
> on 2.6.31.
> > > >> There has been a little movement on Hardy from Canonical,
> but I don't
> > > >> expect a lot to get done there in the short term since
> Karmic is
> > > >> launching in a couple weeks.
> > > >> I have also heard some folks having success running the
> Alestic.com
> > > >> Jaunty on the 2.6.27 kernel from Canonical, but that kernel
> was built a
> > > >> while back and I don't know how reliable or secure it is.
> When launching in us-west I notice there's a spot in the instance launch
> where you can specify a different AKI. I don't remember seeing this field
> before yesterday, but it might allow trying a different AKI without
> rebundling - and if the launch works right, I would expect that bundling an
> AMI from that running instance would point by default at the AKI you intend.
This is more or less option 3 that Eric mentioned above. It was
available before the us-west zone was available. I've never tried it
before, though.
> 2009/12/15 brian bulkowski <br...@bulkowski.org>:
>> Devon, et al,
>> When launching in us-west I notice there's a spot in the instance launch
>> where you can specify a different AKI. I don't remember seeing this field
>> before yesterday, but it might allow trying a different AKI without
>> rebundling - and if the launch works right, I would expect that bundling an
>> AMI from that running instance would point by default at the AKI you intend.
> This is more or less option 3 that Eric mentioned above. It was
> available before the us-west zone was available. I've never tried it
> before, though.
Of course, this doesn't help with upgrading an existing instance.
Only when starting new instances, in which case you may as well start
with one of the Canonical instances that already have the correct
kernel/initrd.
On Wed, 16 Dec 2009, Michael Wood wrote:
> 2009/12/15 brian bulkowski <br...@bulkowski.org>:
> > Devon, et al,
> > When launching in us-west I notice there's a spot in the instance launch
> > where you can specify a different AKI. I don't remember seeing this field
> > before yesterday, but it might allow trying a different AKI without
> > rebundling - and if the launch works right, I would expect that bundling an
> > AMI from that running instance would point by default at the AKI you intend.
> This is more or less option 3 that Eric mentioned above. It was
> available before the us-west zone was available. I've never tried it
> before, though.
What *is* new, is that ec2-register will now take '--kernel' and
'--ramdisk'. Previously you could only supply ramdisk id and kernel id at
bundle-image time or launch time. Now, you can register a new image with
a different kernel/ramdisk as the "default".
