[9fans] AMD64 system

[Strake]

Hello all.

I just lately installed Plan 9, but the stock system is built for
32-bit x86, and I have an amd64 computer.

I found this diff: http://9legacy.org/9legacy/patch/nix.diff
which seems to have all needed system libraries, and its own kernel,
but the kernel seems to lack basic functionality, such as graphics and
mouse, and I can't find the local bootloader for it — the stock
bootloader chokes with message "bad kernel format", and on the nix web
site it says only to build pxe bootloader, which is not what I need.
It seems that nix is meant for cpu servers, not terminals. I need, if
my ken of 9jargon is not wrong, a terminal kernel.

In an earlier thread, "9vx instability", F. J. Ballesteros said this:
"Jim, Charles, and others made an excellent port for amd64 ... We used
that as a starting point for nix."
Is this the legendary amd64 port? Is it available?

I feel a bit lost. In the documentation, the authours emphasize its
portability, yet to actually build for another architecture seems
quite a bother, regrettably, since I was quite enthusiastic to use it
as my primary system.

Anyhow, I would be glad of any pointers to an amd64 port or
instructions to do same.

Cheers,
strake

[Nemo]

nix does not have graphics yet. sorry.
we are using a changed 9pxeload and
are switching to the new 9boot.
the loader can be found in the distrib.
if you can't wait.

--
iphone kbd. excuse typos :)

[erik quanstrom]

> I just lately installed Plan 9, but the stock system is built for
> 32-bit x86, and I have an amd64 computer.

the stock system will work find for you.

> I found this diff: http://9legacy.org/9legacy/patch/nix.diff
> which seems to have all needed system libraries, and its own kernel,
> but the kernel seems to lack basic functionality, such as graphics and
> mouse, and I can't find the local bootloader for it — the stock

unfortunately, it can't be run easily on a terminal yet due to the lack
of graphics, as you note.

/n/sources/contrib/quanstro/root/sys/src/boot/pc-e820 will boot amd64
and pc kernels.

the source is at
http://code.google.com/p/nix-os (hopefully correctly typed)
and
/n/sources.lsub.org/nix

the official bootloader is at

/n/sources.lsub.org/nix/sys/src/nix/w/pxeload

> I feel a bit lost. In the documentation, the authours emphasize its
> portability, yet to actually build for another architecture seems
> quite a bother, regrettably, since I was quite enthusiastic to use it
> as my primary system.

the fact that the amd64 port is immature doesn't mean that the
system isn't portable. it may mean that x86 is a complicated place
to work. :-)

- erik

[David du Colombier]

> I found this diff: http://9legacy.org/9legacy/patch/nix.diff
> which seems to have all needed system libraries, and its own kernel,
> but the kernel seems to lack basic functionality, such as graphics and
> mouse, and I can't find the local bootloader for it — the stock
> bootloader chokes with message "bad kernel format", and on the nix web
> site it says only to build pxe bootloader, which is not what I need.
> It seems that nix is meant for cpu servers, not terminals. I need, if
> my ken of 9jargon is not wrong, a terminal kernel.

You should use Erik Quanstrom's bootloader.

It is available here:

/n/sources/contrib/quanstro/root/sys/src/boot/pc-e820

Or here:

http://www.9legacy.org/9legacy/patch/boot-pc-e820.diff

--
David du Colombier

[Christoph Lohmann]

Greetings.

On Wed, 25 Apr 2012 16:31:42 +0200 Nemo <ne...@lsub.org> wrote:
> iphone kbd. excuse typos :)

Nix is running on iPhones?


Sincerely,

Christoph Lohmann

[Nemo]

No. But sofware capable of using nix services is.
As you can see by looking at my signature in the previous mail.

Enjoy.

[Strake]

On 25/04/2012, erik quanstrom <quan...@quanstro.net> wrote:
>> I just lately installed Plan 9, but the stock system is built for
>> 32-bit x86, and I have an amd64 computer.
>
> the stock system will work find for you.

Assume s/find/fine/.

32-bit is not fine.

Four billion is not enough.

>> I found this diff: http://9legacy.org/9legacy/patch/nix.diff
>> which seems to have all needed system libraries, and its own kernel,
>> but the kernel seems to lack basic functionality, such as graphics and
>> mouse, and I can't find the local bootloader for it — the stock
>
> unfortunately, it can't be run easily on a terminal yet due to the lack
> of graphics, as you note.
>
> /n/sources/contrib/quanstro/root/sys/src/boot/pc-e820 will boot amd64
> and pc kernels.

Ah, thanks.

> the fact that the amd64 port is immature doesn't mean that the
> system isn't portable. it may mean that x86 is a complicated place
> to work. :-)

The majority charge carrier in an x86 chip is the moron, not the electron.
I mean to get a Loongson system, when one such is available and not
memory-crippled. I'd just have to port Plan 9 again, to MIPS...

Cheers,
strake

[erik quanstrom]

> On 25/04/2012, erik quanstrom <quan...@quanstro.net> wrote:

> >> I just lately installed Plan 9, but the stock system is built for
> >> 32-bit x86, and I have an amd64 computer.
> >
> > the stock system will work find for you.
>

> Assume s/find/fine/.
>
> 32-bit is not fine.
>
> Four billion is not enough.

can you be more specific? what do you need exactly?

- erik

[Lyndon Nerenberg]

On 2012-04-25, at 11:04 AM, Strake wrote:

> Four billion is not enough.

Not enough what? This cat's curiosity is raised.

[Matthew Veety]

Numbers obviously.

--
Veety

[erik quanstrom]

> > On 2012-04-25, at 11:04 AM, Strake wrote:
> >
> > > Four billion is not enough.
> >
> > Not enough what? This cat's curiosity is raised.
> >
>

> Numbers obviously.

i think you mean the maximum value of an integer
rather than a count. assuming this, vlongs are
still 64 bits with 8c and the 32-bit architecture.

what's wrong with them?

- erik

[Strake]

This. A limit on cryptography, physical simulation, ...
which are computation-bound, so bignum arithmetic would be slow.

Also logical memory addresses, timestamps, ...

Oh, and 8 registers are far too few.

[John Floren]

If you're doing cryptography and physical simulation, computation
bound stuff, why not set up a 64-bit CPU server? I've got one at work,
all you should need to do is get the 64-bit binaries on your
fileserver.

John

[Strake]

On 25/04/2012, erik quanstrom <quan...@quanstro.net> wrote:

> i think you mean the maximum value of an integer
> rather than a count. assuming this, vlongs are
> still 64 bits with 8c and the 32-bit architecture.
>
> what's wrong with them?

Twice as many instructions, if I'm not mistaken, and a waste of good
64-bit registers.

[Lyndon Nerenberg]

On 2012-04-25, at 12:09 PM, Strake wrote:

> This. A limit on cryptography, physical simulation, ...
> which are computation-bound, so bignum arithmetic would be slow.
>
> Also logical memory addresses, timestamps, ...

Don't vlongs cover this? Perhaps the physical simulation example would like 64 bit addressing, but sparse arrays could be a viable alternative.

> Oh, and 8 registers are far too few.

Unless you're writing assembler, the compilers hide that.

Anyway, I was just curious to see what specific real case you had for needing 64 bits. Proprietary considerations often get in the way of that.

--lyndon

[erik quanstrom]

On Wed Apr 25 15:17:15 EDT 2012, stra...@gmail.com wrote:
> On 25/04/2012, erik quanstrom <quan...@quanstro.net> wrote:

> > i think you mean the maximum value of an integer
> > rather than a count. assuming this, vlongs are
> > still 64 bits with 8c and the 32-bit architecture.
> >
> > what's wrong with them?
>

> Twice as many instructions, if I'm not mistaken, and a waste of good
> 64-bit registers.

it's not like the registers are real on a modern x86 machine in any mode
after renaming, etc. and this is also offset somewhat by the fact that
pointers are now twice as big.

so best case for 64-bit, is that you are adding 64-bit numbers in a tight
loop with almost no memory access. i get only a 2-3x speedup for this
case

32-bit machine (not idle):
minooka; time 8.addv
3.08u 0.00s 3.11r 8.addv # status= main
minooka; aux/cpuid -i
Intel(R) Xeon(R) CPU E5540 @ 2.53GHz

64-bit machine (idle, but slower)
bonanza; time 6.addv
1.55u 0.00s 1.58r 6.addv # status= main
bonanza; aux/cpuid -i
Intel(R) Xeon(R) CPU E5504 @ 2.00GHz

by amdahl's law, you're going to have to be doing a hell of a lot of
vlong arithmetic to make this pay.

also, in case you missed it sizeof(int)==sizeof(long)==4 on both 32
and 64 bit plan 9, so recompiled programs won't get bigger integers
just for the recompiling.

- erik

-----
bonanza; cat addv.c
#include <u.h>
#include <libc.h>

void
main(void)
{
int i;
vlong acc;

acc = 0;
for(i = 0; i < 1000000000; i++)
acc += i;
}

[erik quanstrom]

> Also logical memory addresses, timestamps, ...

the tsc (timestamp counter) is 64 bits regardless of processor mode.

- erik

[Strake]

On 25/04/2012, John Floren <jo...@jfloren.net> wrote:
> If you're doing cryptography and physical simulation, computation
> bound stuff, why not set up a 64-bit CPU server? I've got one at work,
> all you should need to do is get the 64-bit binaries on your
> fileserver.

Then I have a CPU server with very nice on-board sound, and powerful
graphics card, both idle, the latter since I have no other machine
that can take it, I have no driver, and even if I had,
(32 b/pixel)(1920x1080 pixel)(60.0 Hz) = 4 Gb/s > network data rate = 1 Gb/s.

[Strake]

On 25/04/2012, Lyndon Nerenberg <lyn...@orthanc.ca> wrote:
> Anyway, I was just curious to see what specific real case you had for
> needing 64 bits.

The main one is this: I have a 64-bit machine, and I'll be damned if
my programs won't use every last one of them (^_~)

[Gorka Guardiola]

>
> The main one is this: I have a 64-bit machine, and I'll be damned if
> my programs won't use every last one of them (^_~)
>

We are going to be grateful to you saving yourself by writing
drivers...

G.

[John Floren]

There are 3 options:

1. Suck it up and use the 64-bit system that is available
2. Write drivers for your hardware (this is the comedy option)
3. Complain on 9fans for a while before eventually giving up (this is
the popular option)

I don't even know what you're attempting to imply with that
calculation at the end, though. What does the onboard graphics card
have to do with network bandwidth? If you run a big drawterm/cpu
window, it won't be that high of a data rate, and it won't use the
graphics card anyway.

john

[Strake]

On 25/04/2012, erik quanstrom <quan...@quanstro.net> wrote:
> it's not like the registers are real on a modern x86 machine in any mode
> after renaming, etc. and this is also offset somewhat by the fact that
> pointers are now twice as big.

It can rename them but I can't name them, so I can't keep any more
variables in the core at a time.

> also, in case you missed it sizeof(int)==sizeof(long)==4 on both 32
> and 64 bit plan 9, so recompiled programs won't get bigger integers
> just for the recompiling.

What the hell? This is a waste and a fault. long at least ought to be
at least a machine word.

[Lyndon Nerenberg]

On 2012-04-25, at 1:01 PM, Strake wrote:

> The main one is this: I have a 64-bit machine, and I'll be damned if
> my programs won't use every last one of them (^_~)

Hookers?

[Lyndon Nerenberg]

On 2012-04-25, at 1:13 PM, Strake wrote:

> What the hell? This is a waste and a fault

Yup :-P

[Strake]

On 25/04/2012, John Floren <jo...@jfloren.net> wrote:
> There are 3 options:
>
> 1. Suck it up and use the 64-bit system that is available
> 2. Write drivers for your hardware (this is the comedy option)
> 3. Complain on 9fans for a while before eventually giving up (this is
> the popular option)

4. Keep to Linux and curse the world in wrath.

I'd shut up if no one _asked_ me about it, but some did.

> I don't even know what you're attempting to imply with that
> calculation at the end, though. What does the onboard graphics card
> have to do with network bandwidth?

It doesn't; however graphics are drawn, whether in hardware or
software, they must be sent to terminal.

> If you run a big drawterm/cpu
> window, it won't be that high of a data rate

How?

> and it won't use the
> graphics card anyway.

Then it will be slow. Software graphics are slow.

[John Floren]

On Wed, Apr 25, 2012 at 1:30 PM, Strake <stra...@gmail.com> wrote:
> On 25/04/2012, John Floren <jo...@jfloren.net> wrote:
>> There are 3 options:
>>
>> 1. Suck it up and use the 64-bit system that is available
>> 2. Write drivers for your hardware (this is the comedy option)
>> 3. Complain on 9fans for a while before eventually giving up (this is
>> the popular option)
> 4. Keep to Linux and curse the world in wrath.
>

I forgot about #4. We almost all end up going with #4 at some point,
to a greater or lesser extent.

> I'd shut up if no one _asked_ me about it, but some did.

You still haven't clarified what exactly you want to do with your
64-bit system, besides win dicksize wars. Reasons for using a 64-bit
system include, for example, *needing* more than 4 GB of RAM. If you
want to do stuff like Ron and Nemo have done, where you stick your
entire filesystem in 64 GB of memory or so, then yeah it's important.
On the other hand, I've never had a Plan 9 system with more than 4 GB
of RAM, excepting our NIX test box, and everything has been fine--you
don't need a lot for this OS!

>> I don't even know what you're attempting to imply with that
>> calculation at the end, though. What does the onboard graphics card
>> have to do with network bandwidth?
>
> It doesn't; however graphics are drawn, whether in hardware or
> software, they must be sent to terminal.
>
>> If you run a big drawterm/cpu
>> window, it won't be that high of a data rate

Through the magic of compression, and other things like realizing that
you don't have to redraw the *entire* screen 60 times a second when
displaying a mostly-static desktop. You just send the chunks that have
changed, *when* they change.

>> and it won't use the
>> graphics card anyway.
>
> Then it will be slow. Software graphics are slow.
>

I'm not that familiar with how the Plan 9 graphics system works, but
we're not talking about hardware vs software OpenGL. There is no
OpenGL to be had here. This is writing bits into a framebuffer and
having them appear on the screen. It's pretty damn fast to write
things to main memory.

john

[Strake]

On 25/04/2012, John Floren <jo...@jfloren.net> wrote:
> On Wed, Apr 25, 2012 at 1:30 PM, Strake <stra...@gmail.com> wrote:
>> On 25/04/2012, John Floren <jo...@jfloren.net> wrote:
>>> There are 3 options:
>>>
>>> 1. Suck it up and use the 64-bit system that is available
>>> 2. Write drivers for your hardware (this is the comedy option)
>>> 3. Complain on 9fans for a while before eventually giving up (this is
>>> the popular option)
>> 4. Keep to Linux and curse the world in wrath.
>>
>
> I forgot about #4. We almost all end up going with #4 at some point,
> to a greater or lesser extent.

Alas, fame brings drivers.

>> I'd shut up if no one _asked_ me about it, but some did.
>
> You still haven't clarified what exactly you want to do with your
> 64-bit system, besides win dicksize wars.

This is a major reason.

Me: Yeah, well, mine is 2^32 +1 units long!
Other: *Arithmetic Overflow* Curses!

> Reasons for using a 64-bit
> system include, for example, *needing* more than 4 GB of RAM. If you
> want to do stuff like Ron and Nemo have done, where you stick your
> entire filesystem in 64 GB of memory or so, then yeah it's important.
> On the other hand, I've never had a Plan 9 system with more than 4 GB
> of RAM, excepting our NIX test box, and everything has been fine--you
> don't need a lot for this OS!

Yes — the OS takes less, so the computations can have more.
Anyhow, this is not my worry — I have only 4 GB.

> Through the magic of compression, and other things like realizing that
> you don't have to redraw the *entire* screen 60 times a second when
> displaying a mostly-static desktop.
> You just send the chunks that have
> changed, *when* they change.

And when watching full-screen video, or playing full-screen 3D games?
Then it must redraw nearly the whole screen, nearly every frame.

> I'm not that familiar with how the Plan 9 graphics system works, but
> we're not talking about hardware vs software OpenGL. There is no
> OpenGL to be had here.

Not yet. It seems to be in the works:
http://plan9.bell-labs.com/wiki/plan9/todo/index.html

> This is writing bits into a framebuffer and
> having them appear on the screen. It's pretty damn fast to write
> things to main memory.

Yes, which works iff the video output is local. This I wrote in
response to the idea that I make one machine a 64-bit devoted CPU
server, which I doubt would be appropriate for my usage case and
available hardware.

[andrey mirtchovski]

> Not yet. It seems to be in the works:
> http://plan9.bell-labs.com/wiki/plan9/todo/index.html

i think that work "stalled" in 2004 :)

[andy zerger]

And the link moved to somewhere like http://bellard.org/TinyGL/ ? "L'eurreur de 404" at the plan9.bell link.

[John Floren]

On Wed, Apr 25, 2012 at 6:11 PM, Strake <stra...@gmail.com> wrote:
> On 25/04/2012, John Floren <jo...@jfloren.net> wrote:
>> Through the magic of compression, and other things like realizing that
>> you don't have to redraw the *entire* screen 60 times a second when
>> displaying a mostly-static desktop.
>> You just send the chunks that have
>> changed, *when* they change.
>
> And when watching full-screen video, or playing full-screen 3D games?
> Then it must redraw nearly the whole screen, nearly every frame.

I thought you wanted this to do your uber computations, not watch movies?

And if you have full-screen 3D games for Plan 9, share!

>> I'm not that familiar with how the Plan 9 graphics system works, but
>> we're not talking about hardware vs software OpenGL. There is no
>> OpenGL to be had here.
>
> Not yet. It seems to be in the works:
> http://plan9.bell-labs.com/wiki/plan9/todo/index.html
>
>> This is writing bits into a framebuffer and
>> having them appear on the screen. It's pretty damn fast to write
>> things to main memory.
>
> Yes, which works iff the video output is local. This I wrote in
> response to the idea that I make one machine a 64-bit devoted CPU
> server, which I doubt would be appropriate for my usage case and
> available hardware.
>

You still haven't told us your usage case. Wild speculation about what
is possible, impossible, desirable, necessary, etc. is cheap on 9fans,
I'm sure you've seen that.

john

[Russ Cox]

On Wed, Apr 25, 2012 at 4:13 PM, Strake <stra...@gmail.com> wrote:
> What the hell? This is a waste and a fault. long at least ought to be
> at least a machine word.

Use vlong. Why does it matter what it's called?

> The main one is this: I have a 64-bit machine, and I'll be damned if
> my programs won't use every last one of them (^_~)

They certainly won't. The address space is really only 48 bits wide,
and 47 for user space on most kernels. Sorry to disappoint.

More generally, you showed up demanding things and basically
being a jerk. People have explained the situation, you didn't pay
anything for any of this, and we don't owe you anything. If you're
not happy about the state of the Plan 9 world, write some code
or stop whining.

Russ

[Strake]

On 25/04/2012, John Floren <jo...@jfloren.net> wrote:
> I thought you wanted this to do your uber computations, not watch movies?

No, I want this to do both! Likely not simultaneously.

> And if you have full-screen 3D games for Plan 9, share!

When I do, I shall.

> You still haven't told us your usage case. Wild speculation about what
> is possible, impossible, desirable, necessary, etc. is cheap on 9fans,
> I'm sure you've seen that.

Ah, the specifics, then:
* Browse the Web
* Read and write e-mail
* Write and build programs and libraries, mostly in C and Haskell:
Haskell compiler, VCS, various little utilites
* Build various other programs: kernels, compilers, Web browsers,
e-mail clients, window systems, others
* Encrypt and decrypt data: PGP, SSH, TLS
* Do symbolic computations: solve equations and such
* Do numeric computations: now light; potentially very heavy in near
future, e.g. computational fluid dynamics
* Potentially CAD: mechanical, if I ever find or write CADware not
grievous to use; and possibly electrical
* Play music
* Watch movies
* Play games: mostly first-person shooters, flight simulators, arcade games
* Typeset documents, mostly with LaTeX

I know that much that I wish to do is not yet feasible on stock P9,
but I hope to either do it myself or procrastinate until someone else
does (^_^)

Cheers,
strake

[andrey mirtchovski]

...but whining feels so righteous :(

[Devon H. O'Dell]

True story. If I wasn't on a phone i'd elaborate more.

[Strake]

On 25/04/2012, Russ Cox <r...@swtch.com> wrote:
> On Wed, Apr 25, 2012 at 4:13 PM, Strake <stra...@gmail.com> wrote:
>> What the hell? This is a waste and a fault. long at least ought to be
>> at least a machine word.
>
> Use vlong. Why does it matter what it's called?

Not all programs that I use, I write.

>> The main one is this: I have a 64-bit machine, and I'll be damned if
>> my programs won't use every last one of them (^_~)
>
> They certainly won't. The address space is really only 48 bits wide,
> and 47 for user space on most kernels. Sorry to disappoint.

Machine words are nonetheless 64 bits wide.

> More generally, you showed up demanding things and basically
> being a jerk.

No. I demanded nothing. I simply said what I seek, and asked whether
it might be available. Clearly, it's not. I thought it might, since in
a prior thread there was vague reference to such. I never told anyone
to do it for me, and I never asked anyone to help me do it.

Some then asked me plainly what I need and why, so I responded; for
this, it seems, you call me jerk. My second message, I meant to be the
end of it. I said thanks for the link, and made some comments, which
were not imperative.

Some then asked me further questions, some of which asked me directly
what is wrong with X, for some X. When one asks me what is wrong with
X, I can only assume that one wishes to know my opinion on the matter,
so I declare it. Many were grievances against Plan 9 and its parts,
which I would otherwise keep to myself.

> People have explained the situation, you didn't pay
> anything for any of this, and we don't owe you anything.

All true. I never said otherwise.

> If you're
> not happy about the state of the Plan 9 world, write some code
> or stop whining.

I mean to do both.

Please remember what I said earlier:

I'd shut up if no one _asked_ me about it, but some did.

Please note this amendment: s/ if / iff /

Thanks to everyone who sent links to the amd64 bootloader and other help.

Cheers,
strake

[Ethan Grammatikidis]

Relax. I've been seeing this thread as a comedy since about the 10th
post or so.

[dexen deVries]

On Wednesday 25 of April 2012 15:32:06 erik quanstrom wrote:
> also, in case you missed it sizeof(int)==sizeof(long)==4 on both 32
> and 64 bit plan 9, so recompiled programs won't get bigger integers
> just for the recompiling.

pardon silly question, but... why, on 64bit machine, P9 uses 32bit ints and
longs?

my impression is, int was supposed to match machine's preferred (best
performance etc.) integeral datatype, and long was supposed to be enough to
hold a pointer? (i.e., sizeof(long) >= sizeof(void*))


--
dexen deVries

Until real software engineering is developed, the next best practice is to
develop with a dynamic system that has extreme late binding in all aspects.
-- Alan Kay

[David du Colombier]

> my impression is, int was supposed to match machine's preferred (best
> performance etc.) integeral datatype, and long was supposed to be enough to
> hold a pointer? (i.e., sizeof(long) >= sizeof(void*))

No, long is always 32-bit on Plan 9. uintptr is big enough to hold a pointer.

--
David du Colombier

[erik quanstrom]

> pardon silly question, but... why, on 64bit machine, P9 uses 32bit ints and
> longs?
>
> my impression is, int was supposed to match machine's preferred (best
> performance etc.) integeral datatype, and long was supposed to be enough to
> hold a pointer? (i.e., sizeof(long) >= sizeof(void*))

charles could provide a more complete answer, but
http://en.wikipedia.org/wiki/C_data_types
is a good summary of the general rules. long is only
>= 32 bits. there are no other restrictions. it does
not have to be big enough to hold a pointer.

while the original idea for int was that it was the natural
word size, things are no longer quite that simple. int
must be >= 16 bits, so if you were to do c on, say, a
15 bit machine, int would need to be simulated to be
standard, but i doubt anyone would bother slaving to
the standard so.

and on a x86_64 what is the "natural size" of an integer?
since it's not risc, there is no penalty for doing 32-bit calcuations.
heck, the docs call a 4-byte integer a "double word" and
an 8-byte integer a "quad word". i suppose you could make
an argument for 16-bit int on intel!

- erik

[Charles Forsyth]

many existing C programs (and not just on Plan 9) think long is 32 bits, or don't care.

those that do care don't work. for those that don't care, it doesn't matter.

it's quite difficult to decide automatically (eg, in a compiler) into which category a program falls.

a compiler can't easily detect that a program expected 32 bits only and will malfunction with 64.

libflate was full of that, but it wasn't the only one. of course, in time, those can be changed to using

u32int (or u64int) to make intentions clear.

by contrast, if long remains 32 bits (thus satisfying most existing programs for integer operations),

but pointers are 64 bits, it's easy for a compiler to spot conversion of a pointer to an integer type that

is too small, and the plan 9 compilers will do that. there are very few such cases in the plan 9 code, and

thanks to the compiler diagnostic, we know exactly where they are and whether they matter, and have

converted them to use uintptr.

if it's performance you're worried about, for programs that don't care about width, i'd expect 32 bits at least

to match performance with 64 bits (if there's a measurable difference). for one thing, cache lines will contain

more values, and several will be fetched at once when cache lines are filled.

[dexen deVries]

On Saturday 05 of May 2012 18:48:37 Charles Forsyth wrote:
> many existing C programs (and not just on Plan 9) think long is 32 bits, or
> don't care.
> those that do care don't work. for those that don't care, it doesn't matter.
> it's quite difficult to decide automatically (eg, in a compiler) into which
> category a program falls.
> a compiler can't easily detect that a program expected 32 bits only and
> will malfunction with 64.

> (..snip..)

thank you and erik for the explanations, makes perfect sense now :-)

[Comeau At9Fans]

On Sat, May 5, 2012 at 11:33 AM, dexen deVries <dexen....@gmail.com> wrote:

On Wednesday 25 of April 2012 15:32:06 erik quanstrom wrote:
> also, in case you missed it sizeof(int)==sizeof(long)==4 on both 32
> and 64 bit plan 9, so recompiled programs won't get bigger integers
> just for the recompiling.


pardon silly question, but... why, on 64bit machine, P9 uses 32bit ints and
longs?

my impression is, int was supposed to match machine's preferred (best
performance etc.) integeral datatype, and long was supposed to be enough to
hold a pointer? (i.e., sizeof(long) >= sizeof(void*))

Re "supposed" if you mean according to say Standard C, no.  Even so-called K&R C was not black and white regarding this per se.  Standard C only provided minimum sizes.  Indeed, there is often preferences, but that's usually up to vendors, and lots of it yield *-defined behaviors.  As it should.

There is also the issue of that, if, you really want a 64-bit integer, then, using int is certainly moving towards a direction of a programming error, since int does not often yield such a beast, even on so-called systems which could provide it.  C99 provides for stuff such as int_least64_t for those who really need such.

--

Greg Comeau / 4.3.10.1 with C++0xisms now in beta!

Comeau C/C++ ONLINE ==>     http://www.comeaucomputing.com/tryitout

World Class Compilers:  Breathtaking C++, Amazing C99, Fabulous C90.

Comeau C/C++ with Dinkumware's Libraries... Have you tried it?

[Comeau At9Fans]

On Sat, May 5, 2012 at 1:48 PM, Charles Forsyth <charles...@gmail.com> wrote:

if it's performance you're worried about, for programs that don't care about width, i'd expect 32 bits at least

to match performance with 64 bits (if there's a measurable difference). for one thing, cache lines will contain

more values, and several will be fetched at once when cache lines are filled.

And for programs that do care about this, C99 provides things such as int_fast64_t (which IIRC 8c et al does not currently support).  In a way, expecting something out of 'int' is sloppy, despite it's wide use.

[Joel C. Salomon]

On 05/05/2012 05:06 PM, Comeau At9Fans wrote:

> On Sat, May 5, 2012 at 1:48 PM, Charles Forsyth wrote:
>
> if it's performance you're worried about, for programs that don't
> care about width, i'd expect 32 bits at least
> to match performance with 64 bits (if there's a measurable
> difference). for one thing, cache lines will contain
> more values, and several will be fetched at once when cache lines
> are filled.
>
> And for programs that do care about this, C99 provides things such as
> int_fast64_t (which IIRC 8c et al does not currently support).

<stdint.h> is just a bunch of typedef's, some of which have been
included, under different names (u{8,16,32,64}int), in <u.h>. Wouldn't
be hard to provide for APE if anyone wants it.

--Joel

[Bruce Ellis]

the consensus at the sunshine club is that to take advantage of the
wasted bits in a 64 bit pointer you should RENT THEM OUT!

brucee
--
Don't meddle in the mouth -- MVS (0416935147, +1-513-3BRUCEE)

[Comeau At9Fans]

Yes, the u{...}int names are important.  Those "different names" are normally for other (though obviously related) purposes (conceptually int_exact_*), and the int_least_* and int_fast_* names, usually are built from the exact ones.  BTW, more generally re the greater issue raised earlier by an OP, if ones intent is X bits, then int is not expressing that intent, and even if your assumption happens to be correct, it might also create a portability faux pas if that is a concern.  Also, if int was X bits underneath, then arrays of int, now in the case of X == 64 effectively become arrays of long stuff since not only is there now maybe wasted memory, but cache management and such can be a concern.  This all has architecture dependencies though.

[steve]

i think this is an often misunderstood fact, 32bit ints are, in my experience,
a significant win compared with 64bit when doing memory intensive work
- image processing in my case.

-Steve


On 5 May 2012, at 06:48 PM, Charles Forsyth <charles...@gmail.com>
.

[Comeau At9Fans]

I've heard that 64-bit is not an immediate win over 32 for graphics and such, but then again also heard that 32 bit is not the pits, and that it more or less becomes a wash.  So, in your case, what is is about 32 that makes it a *significant* win, given that the space would have been used in either case (I don't know if that's true, but assuming it is)?   (And sorry if I've connected "graphics and such" with image processing since it may be you're doing something I'm not realizing and so we might be talking about different things.)

[erik quanstrom]

> Yes, the u{...}int names are important. Those "different names" are
> normally for other (though obviously related) purposes (conceptually
> int_exact_*), and the int_least_* and int_fast_* names, usually are built

the int_least_* names appear to me to be completely wasted, given
this is how the normal types are defined and that you couldn't depend
on the extra bits. also, how do you debug code written like this?
one complete test for each possible definition of int_least_*?
of course, that leads to the question, is that set known?

one also wonders about int_fast_* as well. fast /for what/ exactly?

it seems that all this is in respose to the fact that the c-std version
of u32int is not required. (the names are somewhat offensive to
good taste, so i refer to them as one does carlin's list, by suggestion.)
rather than fix that issue, layering on another set of types was the
solution. really?

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n868.htm :

“ It can also be argued that programmers that use the 'intN_t' more than
likely really meant to use the 'int_leastN_t' types. Such programmers
probably want a type with a guaranteed number of bits rather than an
exact number of bits.

It can be argued further that use of the 'intN_t' names is not portable
because they may not exist in all implementations. (An implementation
is not required to provide these types.)

unfortunately, there are still programmers who use the definition
more complicated ≡ more better. evidently believing in the
transitive nature of "more" over false premises.

- erik

[Comeau At9Fans]

On Sun, May 6, 2012 at 9:10 AM, erik quanstrom <quan...@quanstro.net> wrote:

> Yes, the u{...}int names are important.  Those "different names" are
> normally for other (though obviously related) purposes (conceptually
> int_exact_*), and the int_least_* and int_fast_* names, usually are built

the int_least_* names appear to me to be completely wasted, given
this is how the normal types are defined and that you couldn't depend
on the extra bits.

Kind of.  The thing is there is the minimums required by the standard and then there is the minimums provided by the implementation, the latter of which at least has to match the former.   But yes, the least names can be viewed to just thrown a wrench into it all.

 

 also, how do you debug code written like this?

Not easily especially when you begin to look at its interaction with I/O as well.

 

one complete test for each possible definition of int_least_*?
of course, that leads to the question, is that set known?

one also wonders about int_fast_* as well.  fast /for what/ exactly?

IIRC the Standard even specifically leaves that question open.  I think the goal was to try to codify/match some existing practice at the time realizing that it wasn't perfect but at least gave a fight chance in some situation and in those it couldn't, there was probably going to be bumps in the road either way.  I know it can lead to a why bother situation then.

 

it seems that all this is in respose to the fact that the c-std version
of u32int is not required.

IIRC the full set is not required but I think say uint32_t is.  But it's been so long you might just be right.

 

 (the names are somewhat offensive to
good taste, so i refer to them as one does carlin's list, by suggestion.)
rather than fix that issue, layering on another set of types was the
solution.  really?

IIRC the naming was raised a number of times.

 

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n868.htm :

“       It can also be argued that programmers that use the 'intN_t' more than
       likely really meant to use the 'int_leastN_t' types.  Such programmers
       probably want a type with a guaranteed number of bits rather than an
       exact number of bits.

       It can be argued further that use of the 'intN_t' names is not portable
       because they may not exist in all implementations.  (An implementation
       is not required to provide these types.)

unfortunately, there are still programmers who use the definition
more complicated ≡ more better.  evidently believing in the
transitive nature of "more" over false premises.

Personally I could never buy the whole argument for all those names and such, and probably would be happy with just the exact ones.  I'm trying to recall the exact etymology of it but it's too long ago to recall.

[steve]

sorry for being vague.

treating pixels as 64bit on amd64 as that is the natural size for the machine,

vs using 32bits per pixel - 10 bits of r, g, and b or y, u, and v plus 2 spare

leads to a significant speedup; where significant is a number lost in the mists of time.

i believe this speedup is due to the reduction in the rate of cache line refills,

as forsyth described.

-Steve

[dexen deVries]

On Monday 07 of May 2012 09:53:01 steve wrote:
> sorry for being vague.
>
> treating pixels as 64bit on amd64 as that is the natural size for the
> machine, vs using 32bits per pixel - 10 bits of r, g, and b or y, u, and v
> plus 2 spare leads to a significant speedup; where significant is a number
> lost in the mists of time.
>
> i believe this speedup is due to the reduction in the rate of cache line
> refills, as forsyth described.

on RISC, there's usually significant penalty for accessing data units smaller
than machine word (`unaligned access'), but it ain't so on the benevolent x86
CISC.

both handling pixel graphics and transferring to graphic card are special
cases.
speedup may be due to better prefetch during sequential memory access, but
larger data size should not help much here.
more data causes FSB and PCIe contention, and cache trashing. oops?

when i asked about int and long size on amd64, i was more concerned with
ability to cast between pointer and integer and handling offset of large files.

--
dexen deVries

[[[↓][→]]]

Weightless and alone
you speed through the eerie nothingness of space
you circle 'round the Moon
and journey back
to face the punishing torment of re-entry

-- LUNA-C, ``Supaset8 (full release)'', #24m52s

[Charles Forsyth]

The MIPS, PowerPC and SPARC all grew to 64 bits from 32 bits, so 32 bit quantities

(and usually but not always 16 bit and 8 bit quantities) have suitable instructions to fetch them.

Exceptions include ARM, where the original 32 bit architecture made byte access (in the C style at least)

expensive, but they corrected that in (I think) v4 by adding byte instructions. Alpha might have

been an exception too, but that's dead.

accesses aren't "unaligned" when they are smaller than the machine word,

but when the address isn't a multiple of the length accessed. Plan 9 generally keeps

things properly aligned. Processors don't usually have trouble with accesses smaller than

the machine word.

[Charles Forsyth]

uintptr for pointer/integer (uintptr_t in ANSI_t "t for type" style).

the problem with large file offset doesn't exist in Plan 9, because the type

of seek changed, instead of Unix's messing about trying to conserve the type

of lseek (because it's got an "l" in it, I suppose), or worse, on some systems making

it dependent on a series of #defines and weak pragmas to select lseek64 vs lseek32.

honestly. change the type of off_t and be done with it. as Plan 9 did, you can change

the system call number to allow old executables to work for seek/stat/fstat and so on

when the file offset changes. put another way, why shouldn't executables running

in 32 bit mode be able to access large files? that's therefore independent of the physical rep. of "long".

On 7 May 2012 11:01, dexen deVries <dexen....@gmail.com> wrote:

[dexen deVries]

On Monday 07 of May 2012 11:27:29 Charles Forsyth wrote:
> The MIPS, PowerPC and SPARC all grew to 64 bits from 32 bits, so 32 bit
> quantities
> (and usually but not always 16 bit and 8 bit quantities) have suitable
> instructions to fetch them.

fwiw, back in my uni days we were using some old 64bit Solaris on Sun's
UltraSparcs, and those had sizeof(int) == 8. i remember distinctly how my lame
programs (developed on x86 and assuming sizeof(int) == 4) tripped on it.

can't recall exact versions, but the CPUs were somewhere around 400MHz or so,
and GCC was the compiler of choice.

[erik quanstrom]

> instructions. Alpha might have
> been an exception too, but that's dead.

alpha was originally, but later added them.
http://en.wikipedia.org/wiki/DEC_Alpha#Byte-Word_Extensions_.28BWX.29

- erik

[erik quanstrom]

> both handling pixel graphics and transferring to graphic card are special
> cases.
> speedup may be due to better prefetch during sequential memory access, but
> larger data size should not help much here.
> more data causes FSB and PCIe contention, and cache trashing. oops?

pci "memory" is not prefetched. if you're stuffing bytes in you can use
the write-combining memory type to get pretty good performance for
writes (there's no similar trick for reads). but generally dma is used to
move large chunks where performance matters.

regardless of dma, larger data sizes *do* help. like any other network
protocol, there's a header and whatnot. the minimum tlp for a write is 4 bytes.
ignoring other overhead, that's 25% data for 4-byte integers and ~6% data
for byte writes. since pcie-3 is 128/130 encoded, the minimum is now
4 bytes. (quiz: why could this make keeping the plls synced difficult?)

all the 10gbe vendors crank it up to 11 and use 4kb transfers when possible.
all that i've seen can't hit their theoretical maximum frame rate with 60-byte
frames. too much overhead.

then there's the latency. in the kernel i use, i keep the cumulative time
spend in irq handlers. this is useful to see if changes help or hurt irq
latency. in one case, i found that going from 1 to 2 pcie 4-byte register
reads doubled the time in that irq handler.

- erik

[erik quanstrom]

On Mon May 7 04:54:23 EDT 2012, st...@quintile.net wrote:

> sorry for being vague.
>
> treating pixels as 64bit on amd64 as that is the natural size for the
> machine, vs using 32bits per pixel - 10 bits of r, g, and b or y, u,
> and v plus 2 spare leads to a significant speedup; where significant
> is a number lost in the mists of time.
>
> i believe this speedup is due to the reduction in the rate of cache
> line refills, as forsyth described.

i'm confused. (asserting parity error.) which one's faster?
given your problem one would assume that in the absense of
any real gotchas,

processor bw >> memory bw ==> smaller integers faster
memory bw >> processor bw ==> natural integers faster

(this is yet another reason that int_fast* are a half-baked idea.
how does the compiler know this relation for the target machine
ahead of time?)

don't get me wrong, i can easily believe there are gotchas, that's
why i'm confused.

- erik

[Comeau At9Fans]

On Mon, May 7, 2012 at 6:36 AM, dexen deVries <dexen....@gmail.com> wrote:

On Monday 07 of May 2012 11:27:29 Charles Forsyth wrote:
> The MIPS, PowerPC and SPARC all grew to 64 bits from 32 bits, so 32 bit
> quantities
> (and usually but not always 16 bit and 8 bit quantities) have suitable
> instructions to fetch them.

fwiw, back in my uni days we were using some old 64bit Solaris on Sun's
UltraSparcs, and those had sizeof(int) == 8. i remember distinctly how my lame
programs (developed on x86 and assuming sizeof(int) == 4) tripped on it.

can't recall exact versions, but the CPUs were somewhere around 400MHz or so,
and GCC was the compiler of choice.

I may be recalling this incorrectly, but I think you're right about the 8, but didn't they also eventually bring it to 4 as well (or maybe I'm thinking of Sun C)?

 

[Comeau At9Fans]

On Mon, May 7, 2012 at 8:44 AM, erik quanstrom <quan...@quanstro.net> wrote:

...

       processor bw >> memory bw  ==> smaller integers faster
       memory bw >> processor bw  ==> natural integers faster

(this is yet another reason that int_fast* are a half-baked idea.
how does the compiler know this relation for the target machine

ahead of time?) ...

I think that is all so, and I'm not necessarily trying to defend its problems, but on the same note, similar assumptions are also make about int, and it can be argued that int_fast* etc just become par for that course, especially given that there may be command line arguments allowing the user to give some of these things a poke so to speak.  And this is not alone in those kinds of things (which are probably not even really engineering compromises), for instance, take malloc.