configure failure inside kvm 64 bit virtual machine

[Gonzalo Tornaria]

I reported this issue in sage trac
(http://trac.sagemath.org/sage_trac/ticket/5516), but I don't have an
account in mpir trac.

When running a 64 bit virtual cpu in kvm (72+dfsg-4, debian/lenny),
the virtual cpuid reports the cpu as:

vendor_id : GenuineIntel
cpu family : 6
model : 2
model name : QEMU Virtual CPU version 0.9.1
stepping : 3

AFAICT, from intel's cpuid documentation in
http://download.intel.com/design/processor/applnots/24161832.pdf, such
a family/model combination doesn't actually exist (pentiumpro is model
1, pentium II is model 3 and 5, celeron is model 6, etc).

On the other hand the config.guess script in mpir considers anything
models 2 to 6 as being pentium2. Misdetection is already bad because
core 2 optimizations should be much better than pentium II ones. The
build then fails because for a pentium II, the configure logic forces
ABI=32, and this is wrong (configure complains about sizeof(long) not
being 4, indeed it is 8).

----

Of course, this seems to be a bug in kvm; but maybe it should be
workarounded? I don't think it is relevant, but just in case the host
cpu reports family=6, model=23 (it's a core 2 quad Q9550).

After the fact I discovered that one can use the following command
line switch when running kvm:

-cpu qemu64,family=6,model=15,+ssse3

to set family/model to a core2, and also set ssse3 cpu flag (disabled
by default). Unfortunately, this comand line switch doesn't support
models higher than 15, nor the sse4_1 flag.

Best, Gonzalo

[Gonzalo Tornaria]

Here's a proposed fix. The rationale is to separate the decision logic
for intel family 6 in the two cases (32bit / 64bit). This is the same
as it's done for family 15.

In other words, with this patch, any intel processor running in 64 bit
mode is identified as either "nocona" (family 15) or "core2" (family
6). In particular, kvm 64 bits will be decided as "core2". Also, note
that if this patch had been in place before fixing atom and nehalem
model cases, they would have been decided as "core2" also when running
in 64 bit mode.

This (core2) seems a sensible default for intel family 6 processors
running in 64 bit mode (future proof for new models in this family)

Another question is: maybe the default for newer models in 32 bits
should also be set to something more sensible?

Right now, family 15 in 32 bits mode defaults to "p4???", and family 6
in 32 bits mode defaults to "p6???". The former family is most
probably dead already. But family 6 seems to be thriving... would it
make sense to default to "core2" instead?

Note that my patch doesn't change default for 32 bits mode as
suggested above... The changes only affect processors running in 64
bit mode.

Best, Gonzalo

[Bill Hart]

I agree with having all new 64 bit intel family 6 processors default to Core 2.

Not so clear what 32 bit ones should default to. I would say leave
that for now. Are there even that many new 32 bit processors arriving
on the scene these days?

The logical fix for the kvm issue appears to be to remove the
non-existent model from config.guess so that it will simply default
right through to Core 2. I'm nervous about doing this, but if the
Intel docs say there is no such processor, then it should be safe. A
quick google for that family and model might be in order. There are
oodles of people who post their /proc/cpuinfo online. If such a
processor really exists it should be easy to find one.

Is that what your patch does, remove the non-existent model and let it
default through to Core 2? Perhaps you could help me see what it does
by posting the relevant section of config.guess here, now that you
have modified it. I find it hard to read patch files. Essentially it
sounds like your patch is good and we should apply it.

Bill.

2009/3/14 Gonzalo Tornaria <torn...@math.utexas.edu>:

[Bill Hart]

I applied Gonzalo's patch. I looked over it carefully. Essentially it
assumes all 64 bit Intel chips fall into either family 6 (core 2) or
family 15 (nocona).

Checking wikipedia, this appears to be a correct assumption from the
point of view of mpir where it currently is, though we are eventually
going to want to distinguish nehalem and atom.

All core duo's are 32 bit (and long dead).

There seems to be no point identifying atoms and nehalems in the 32
bit section as they will never identify themselves as such. I think
setting abi = 32 doesn't affect config.guess.

But we can do a whole pile of cleaning up in a service release.

Maybe for unidentified 64 bit family we default to x86_64 and for
unidentified 32 bit chip we default to p4. But again let's fix that in
the next release.

Bill.

2009/3/14 Bill Hart <goodwi...@googlemail.com>:

[Gonzalo Tornaria]

On Sat, Mar 14, 2009 at 2:41 PM, Bill Hart <goodwi...@googlemail.com> wrote:
> Not so clear what 32 bit ones should default to. I would say leave
> that for now. Are there even that many new 32 bit processors arriving
> on the scene these days?

The issue is not new 32 bit processors, but new 64 bit processors
running in 32 bit mode. Say, a new atom running in 32 bit mode (it
would be odd to have an i7 running 32 bit OS).

OTOH, defaulting a new processor in 32 bit mode to whatever won't
affect setting of ABI=32, but defaulting an unknown processor in 64
bit mode to "pentium2" sets ABI=32 instead of ABI=64, which causes a
build failure.

For example, in kvm in 32 bit mode in my core2, config.guess would
return "pentium2", which is probably non-optimal, but still works.

On Sat, Mar 14, 2009 at 5:37 PM, Bill Hart <goodwi...@googlemail.com> wrote:
> I applied Gonzalo's patch. I looked over it carefully. Essentially it
> assumes all 64 bit Intel chips fall into either family 6 (core 2) or
> family 15 (nocona).

Not quite; it assumes intel chips running in 64 bit mode are either
core 2 or nocona. For processors running in 32 bit mode, the behaviour
is not changed.

(unknown processors in 32 bit mode are still labeled as "p6???" or
"p4???" depending on family).

> Checking wikipedia, this appears to be a correct assumption from the
> point of view of mpir where it currently is, though we are eventually
> going to want to distinguish nehalem and atom.

Sure, but that's a longer patch (have config.guess return "atom" or
"nehalem", and have "configure" grok that).

> All core duo's are 32 bit (and long dead).
>
> There seems to be no point identifying atoms and nehalems in the 32
> bit section as they will never identify themselves as such. I think
> setting abi = 32 doesn't affect config.

An atom running in 32 bit mode will identify itself as an atom running
in 32 bit mode, i.e. "uname -m" will report "i686" or something like
that, "configfsf.guess" will pass that back to "config.guess", and the
32 bit sections of that file will apply... Am I getting this right? (I
don't have a 64 bit cpu running 32 bit linux to check)

> But we can do a whole pile of cleaning up in a service release.
>
> Maybe for unidentified 64 bit family we default to x86_64 and for
> unidentified 32 bit chip we default to p4. But again let's fix that in
> the next release.

It makes sense to default an unknown family=15 to "pentium4" or
"prescott", but an unknown family=6 may be better defaulted to
"pentium3" or "core2" ???

Gonzalo

[Bill Hart]

Yes, I hadn't thought about 64 bit processors running in 32 bit mode.
My windows machine does that for example, so yeah, we have to leave
the 32 bit stuff in, even for 64 bit processors.

As for defaults, I definitely think we will change them in the next
release. Currently what the defaults are, break the build altogether.

p3 and p4 would be safe presumably, but not good if you have a new
chip which is unrecognised. It is more likely to have something in
common with core2.

It's a case of damned if you do and damned if you don't. At least if
it breaks, someone reports it and you get to fix it properly.

Bill.

2009/3/15 Gonzalo Tornaria <torn...@gmail.com>:

[Gonzalo Tornaria]

On Sat, Mar 14, 2009 at 10:29 PM, Bill Hart <goodwi...@googlemail.com> wrote:
>
> Yes, I hadn't thought about 64 bit processors running in 32 bit mode.
> My windows machine does that for example, so yeah, we have to leave
> the 32 bit stuff in, even for 64 bit processors.

A lot of people run linux 32bit in 64 bit processors. E.g. according
to michael sage binaries for linux 32bit are downloaded much more than
64 bit binaries -- despite most processors from the past 3 years (at
least) having 64 bit mode.

We recently got a new quad core 2 system (Q9550) with 8Gb of ram to
install a sage notebook server (that's me playing with kvm). Our
sysadmin was very kind to install the OS for me, and he installed
debian/lenny..... 32bits... :-)

So, yes, 64 bit processors running in 32 bit mode do exist.

> As for defaults, I definitely think we will change them in the next
> release. Currently what the defaults are, break the build altogether.
>
> p3 and p4 would be safe presumably, but not good if you have a new
> chip which is unrecognised. It is more likely to have something in
> common with core2.
>
> It's a case of damned if you do and damned if you don't. At least if
> it breaks, someone reports it and you get to fix it properly.

Safe is better than broken. However, it may be a good idea to print a
warning to stderr for an unknown processor... Something like

Unknown intel processor, family=FF, model=MM, default to "core2" --
please report to mpir-...@googlegroups.com.

Gonzalo

[Jason Moxham]

I volunteer to to do the rewrite.I want to split the x86_64 into these
microarchitectures

mpn/x86_64/k8
mpn/x86_64/k8/k10
mpn/x86_64/netburst
mpn/x86_64/netburst/lahf
mpn/x86_64/core2
mpn/x86_64/core2/penryn
mpn/x86_64/atom
mpn/x86_64/nehalem

Some of these may just have a gmp-mparam.h in, or perhaps a few asm files
copied strategically from elsewhere .
There is also the Phenom II which is slightly different from K10 , may be
worth another directory , and another "notebook?" AMD cpu family 17

http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/41256.pdf

Don't know if it exists?

Feel free to argue about the names I have chosen :)

the mpn/x86_64/ directory is the fallback when we dont recognize the cpu , ie
the lowest common denominator for x86_64 , which just happens to be the same
as netburst , but that will not always be the case , so we might as well
separate them now.

I believe any new x86_64 chip running in 32 bit mode should do the same .
ie fallback to 386 , it's the safe option, and we can print a message
saying "untested configuration , please report or upgrade"

[Gonzalo Tornaria]

On Sat, Mar 14, 2009 at 11:07 PM, Jason Moxham
<ja...@njkfrudils.plus.com> wrote:
>
>
> I volunteer to to do the rewrite.I want to split the x86_64 into these
> microarchitectures
>
> mpn/x86_64/k8
> mpn/x86_64/k8/k10

While we are at this... The kvm issue I originally reported, for intel
cpus, is now fixed in trunk. There's also the same (similar) issue for
kvm 64 bit using amd cpus. This is sage ticket #5186
(http://trac.sagemath.org/sage_trac/ticket/5186), and I think it was
also discussed in this mailing list some time ago.

Short story: kvm virtual cpu 64 bits reports family=6, model=2. Either
that could be special cased when in 64 bit mode, or else the default
for a cpu in 64 bit mode should be x86_64, IOW, don't return "athlon"
when in 64 bit mode, even if the cpuid says so...

Gonzalo

[Jason Moxham]

It seems strange that they return family 6 , you would thought they would use
family 15 , anyway , they also return authenticamd , perhaps they should
return the vendor_id as "qemu virtual cpu" . If your goner emulate a cpu do
it consistently!

Once we know we are using x86_64 then if we dont recognized the cpu (as athlon
is unknown) then we should use the default C or x86_64_base (whatever the
name is). If we release on Monday , I can do the new arrangement in a day or
two.

> Gonzalo
>

[Bill Hart]

I made a ticket for this issue. We'll look at it as part of the
cleanup of config.guess, configfsf.sub and configure.in.

Bill.

2009/3/15 Gonzalo Tornaria <torn...@gmail.com>:
>