Re: Qemu fails on code generated by ASAN even for -march=atom

[Denis Glotov]

Hi! 

The binary contained "movbe" instruction which caused #UD on virtual machine (although my real Alex device worked fine with it). That is why x86-generic-asan is failing. So here are 2 solutions:

1. Make binary for ASAN compile with -march=i686, not atom. I tried it locally and it works. Or

2. Make qemu kvm emulate more modern cpu. I tried -cpu=atom, pentium4, coreduo, core2duo, but it all did not help and #UD kept happening on "movbe".

What do you think is a preferable solution? If (2) - how to make it work?

-- 
Thank you, 
Denis

[Mike Frysinger]

this is why we build all code with "-mno-movbe -mno-ssse3" and why
we're trying to move away from vmtests for target boards and to
hardware tests

you should build the same way for now
-mike

[asharif]

Should we add "-mno-movbe -mno-ssse3" to the target make.conf so at least the packages that respect those flags (like Chrome) get built properly with asan?

[Bhaskar Janakiraman]

Its insane that we have to restrict code generation for real hardware so that we can run tests on the VM, definitely need a better solution long term. 

I'm not sure if ASAN (ie LLVM) support these flags....

Bhaskar

--
Chromium OS Developers mailing list: chromiu...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-os-dev?hl=en

[Mike Frysinger]

it would make sense if we added these flags to the x86-generic
profile, but not to any of the others

in the meantime, Denis posted a CL to extend the existing Chrome/Clang
logic to build with those flags

i'd really like it if we focused on not needing these flags at all though ...
-mike

[Chris Masone]

On Mon, May 14, 2012 at 11:22 AM, Bhaskar Janakiraman <bjanak...@chromium.org> wrote:

Its insane that we have to restrict code generation for real hardware so that we can run tests on the VM, definitely need a better solution long term. 

We have one, and it's moving the "smoke" tests to run on actual hardware.  Work is actively underway; the canary bots already do this.  If you'd like to assist in this effort, ping scottz@ to find out what you can do to help out.

[Mike Frysinger]

On Mon, May 14, 2012 at 3:27 PM, Geoff Pike <gp...@google.com> wrote:
>> We have one, and it's moving the "smoke" tests to run on actual hardware.
>>  Work is actively underway; the canary bots already do this.  If you'd like
>> to assist in this effort, ping scottz@ to find out what you can do to help
>> out.
>

> An alternative is to take the fault, emulate the instruction in
> software, and keep going. This is work... not sure how much work it is
> vs. the other options. It would also make the VM tests slower, but
> probably not too much since the instructions in question are not
> likely to be frequent.

trouble is that this is the KVM code paths falling down. my
understanding is that inserting insn emulation there isn't nearly as
easy as a qemu system since KVM is just a hypervisor ... it isn't
doing dynamic translation.
-mike

[Sonny Rao]

Not 100% sure, but you might "only" have to add the emulation to
Generic x86 kernel and then KVM would get it for free.

> -mike

[Geoff Pike]

Yes, I meant doing the emulation in the app or the guest OS.

Another alternative is to use atom-based machines for the host, if the
guest contains stuff that requires movbe and such.

I don't know what's best, just mentioning some ideas...

Geoff

[Mike Frysinger]

On Mon, May 14, 2012 at 3:41 PM, Sonny Rao <sonn...@chromium.org> wrote:
> On Mon, May 14, 2012 at 12:38 PM, Mike Frysinger <vap...@chromium.org> wrote:
>> On Mon, May 14, 2012 at 3:27 PM, Geoff Pike <gp...@google.com> wrote:
>>>> We have one, and it's moving the "smoke" tests to run on actual hardware.
>>>>  Work is actively underway; the canary bots already do this.  If you'd like
>>>> to assist in this effort, ping scottz@ to find out what you can do to help
>>>> out.
>>>
>>> An alternative is to take the fault, emulate the instruction in
>>> software, and keep going. This is work... not sure how much work it is
>>> vs. the other options. It would also make the VM tests slower, but
>>> probably not too much since the instructions in question are not
>>> likely to be frequent.
>>
>> trouble is that this is the KVM code paths falling down.  my
>> understanding is that inserting insn emulation there isn't nearly as
>> easy as a qemu system since KVM is just a hypervisor ... it isn't
>> doing dynamic translation.
>
> Not 100% sure, but you might "only" have to add the emulation to
> Generic x86 kernel and then KVM would get it for free.

yes, i think that's where it would go -- the arch/x86/ undefined insn
exception path would load up the insn, decode it, and see if it has to
do emulation on it (like many other arches do for missing insns e.g.
arm and floating point). is there any support for that today in the
x86 kernel ?
-mike

[Mike Frysinger]

we discussed this in the past and that won't scale. we would need to
make sure all our build machines have a cpu that is a superset of all
the possible ISA extensions that we want to build ChromiumOS for, and
it isn't inconceivable that that's impossible :(. with many of our
build machines being AMD systems, you can see how this won't work even
today :).
-mike

[Rahul Chaturvedi]

The original intention for VM's was to support only the x86-generic board - the fact that we're testing qemu images for other boards on KVM is already pushing it.

In the future, only x86-generic should be run on VM's (and it's important that we keep this functional) but images built for other hardware, should run on _that hardware only (which I believe is the current direction).

[Mike Frysinger]

x86-generic and amd64-generic should be vmtested. the rest make sense
for restricting to hardware devices.
-mike

[Mike Frysinger]

On Mon, May 14, 2012 at 2:22 PM, Mike Frysinger <vap...@chromium.org> wrote:
> it would make sense if we added these flags to the x86-generic
> profile, but not to any of the others

so i say we change x86-generic to mean a baseline i686 that supports
MMX, SSE, SSE2, and SSE3:
MARCH_TUNE="-march=i686 -mmmx -msse -msse2 -msse3"
CFLAGS="${CFLAGS} ${MARCH_TUNE}"
CXXFLAGS="${CXXFLAGS} ${MARCH_TUNE}"

are there any other extensions we care about ?

similarly, change amd64-generic to mean a 64bit system that supports
MMX, SSE, SSE2, and SSE3:
MARCH_TUNE="-march=x86-64 -msse3"
CFLAGS="${CFLAGS} ${MARCH_TUNE}"
CXXFLAGS="${CXXFLAGS} ${MARCH_TUNE}"
-mike

[Geoff Pike]

> are there any other extensions we care about ?

I would like to have -msse42 also, but if the amd64-generic VMs need
to run on AMD CPUs that don't have SSE4.2 then we could add it later.
It isn't urgent.

Geoff

[Mike Frysinger]

we def run on AMD CPUs (i think our 48 core builders are), but they
might be new enough to support SSE4A

David: could you comment ?
-mike

[Rahul Chaturvedi]

Are there any tests that depend on the kernel/binaries being compiled to use these extensions? Or any other driving reason to use these extensions?

If not, it would make sense to keep x86/amd64-generic as simple as possible and compile the kernel will the absolute minimal set of common supported extensions between AMD/Intel.

[Mike Frysinger]

afaik, the kernel could care less about the extensions available. as
long as the ABI doesn't imply extra regs need to be saved/restored
across processes (and i believe the SIMD instructions just re-use
existing FP registers to make that happen, although my x86 asm is
rusty), then the kernel build is irrelevant.
-mike

[Geoff Pike]

> Are there any tests that depend on the kernel/binaries being compiled to use
> these extensions? Or any other driving reason to use these extensions?
> If not, it would make sense to keep x86/amd64-generic as simple as possible
> and compile the kernel will the absolute minimal set of common supported
> extensions between AMD/Intel.

We want performance, and with SSE (or other extensions), we can
sometimes get more.

All future x86-64 chips from AMD and Intel will probably have SSE2,
SSE3, SSE4.2, etc. Nehalem (2008ish) had those.

It'd be surprising if we ship an x86-64 product for hardware missing
any of those.

Geoff

[Chris Masone]

We also won't ship an x86-64 product with the *-generic build on it :-)

 

Geoff

[Rahul Chaturvedi]

On Mon, May 14, 2012 at 3:48 PM, Geoff Pike <gp...@chromium.org> wrote:

It'd be surprising if we ship an x86-64 product for hardware missing
any of those.

This isn't about shipping hardware though :-\

The question is, will people be running VM's on hardware that don't support these.

Also I am not certain how much non-media web performance will really be affected by these extensions. SSE mostly helps with streaming data - it is actually rather slow to use on most regular code.

I don't believe the performance benefit will justify the possible lack of compatibility.

We want the 'generic' boards to be as 'generic' as possible :)

[Mike Frysinger]

i've posted:
https://gerrit.chromium.org/gerrit/22703

i've left out SSE4 because:
- atom doesn't support it
- only recent AMD's support SSE4A
- corei7 is first to support SSE4.1/SSE4.2

since these are for x86/amd64 generic only, i'm not worried about sse4
-mike