Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Windows 7 and all my Java stuff.

19 views
Skip to first unread message

DuncanIdaho

unread,
Nov 22, 2009, 11:46:00 AM11/22/09
to
Hi

Today my faithful old Toshiba backup laptop finally gave up the ghost.
The disk is dead, long live the USB stick (which I managed to backup to
before it finally died)

Anyway, I want to buy a new laptop. The best deals from Dell all come
with Windows 7 this is apparently a 64 bit thing

The question is, will all my Java stuff (Tomcat 6, Eclipse/My Eclipse
5.1/ Java 1.6)that I currently have installed on my remaining XP SP2
machine still work or do I need to get everything in new, spanky 64 bit
versions. I have all the archives of my current Java setup safely tucked
away and I'd really like to just copy them over and carry on working.

Can I do this or do I have many hours/days/weeks of misery ahead.

TIA

Idaho

Qu0ll

unread,
Nov 22, 2009, 2:47:56 PM11/22/09
to
"DuncanIdaho" <Duncan.I...@googlemail.com> wrote in message
news:SfedndhcJ80w95TW...@bt.com...

It will all work but I have only tested with the 32-bit JVM. I have not
tried installing a 64-bit JVM but all my apps, libraries and code work fine
on Windows 7 64-bit.

--
And loving it,

-Qu0ll (Rare, not extinct)
_________________________________________________
Qu0llS...@gmail.com
[Replace the "SixFour" with numbers to email me]

DuncanIdaho

unread,
Nov 22, 2009, 4:03:34 PM11/22/09
to
Qu0ll wrote:

...

>> The question is, will all my Java stuff (Tomcat 6, Eclipse/My Eclipse
>> 5.1/ Java 1.6)that I currently have installed on my remaining XP SP2
>> machine still work or do I need to get everything in new, spanky 64
>> bit versions. I have all the archives of my current Java setup safely
>> tucked away and I'd really like to just copy them over and carry on
>> working.
>>
>> Can I do this or do I have many hours/days/weeks of misery ahead.
>
> It will all work but I have only tested with the 32-bit JVM. I have not
> tried installing a 64-bit JVM but all my apps, libraries and code work
> fine on Windows 7 64-bit.

OK, thanks for that, much appreciated ... now where is my credit card.

Idaho

Roedy Green

unread,
Nov 22, 2009, 5:09:49 PM11/22/09
to
On Sun, 22 Nov 2009 16:46:00 +0000, DuncanIdaho
<Duncan.I...@googlemail.com> wrote, quoted or indirectly quoted
someone who said :

>Anyway, I want to buy a new laptop. The best deals from Dell all come
>with Windows 7 this is apparently a 64 bit thing

There are two versions of Windows 7, and at least in the upgrade, you
get both. 32 and 64 bit. However, new machines come with the 64-bit
version pre-installed. Vendors usually don't give you an OS DVD
unless you hound them. (MS eventually caved and gave me a Vista CD.)
I think they hope to sell you a whole new computer the first time the
OS crashes.

It would be suicidal of MS to come out with an OS that needed all new
64-bit software. So it seems highly probable the 64-bit OS can run
32-bit apps. You can download 32 or 64 bit JDKs from Sun. Your
source code and class files stay the same. The 64 bit version chews up
RAM since it uses 64 bit addresses. Unless you are using some app
that overflows 32-bit addressing, you probably want the 32 bit JDK.

I ran the beta Windows 7 32 bit version with 32-bit JDK fine, but I
have no other practical experience, so I defer to those that have.
--
Roedy Green Canadian Mind Products
http://mindprod.com
Finding a bug is a sign you were asleep a the switch when coding. Stop debugging, and go back over your code line by line.

Steve Sobol

unread,
Nov 22, 2009, 5:46:51 PM11/22/09
to
In article <69djg5l32m0lga969...@4ax.com>,
see_w...@mindprod.com.invalid says...

> There are two versions of Windows 7, and at least in the upgrade, you
> get both. 32 and 64 bit. However, new machines come with the 64-bit
> version pre-installed. Vendors usually don't give you an OS DVD
> unless you hound them. (MS eventually caved and gave me a Vista CD.)
> I think they hope to sell you a whole new computer the first time the
> OS crashes.

Nah, you usually have the option of making a "recovery disk" set
yourself. The recovery disk set, if you need to use it, will restore
your computer to the condition it was in when you bought it; this means
a reinstall of Windows, your computer's specific drivers and all of the
software originally bundled with the computer.


--
Steve Sobol, Victorville, California, USA
sjs...@JustThe.net

Roedy Green

unread,
Nov 22, 2009, 7:18:19 PM11/22/09
to
On Sun, 22 Nov 2009 14:46:51 -0800, Steve Sobol <sjs...@JustThe.net>

wrote, quoted or indirectly quoted someone who said :

>Nah, you usually have the option of making a "recovery disk" set

>yourself. The recovery disk set, if you need to use it, will restore
>your computer to the condition it was in when you bought it; this means
>a reinstall of Windows, your computer's specific drivers and all of the
>software originally bundled with the computer.

IIRC this is not that useful. Your partitions are undone. You lose
all your data too. It also requires that a special partition be
undamaged.

There things you can recover with the Windows OS disk that don't
require a complete reinstall. I think they are trying to prevent
piracy, and they are not willing two enforce unique activation keys.

What bugs me is vendors advertise that "Windows 7" is included. It
isn't if you don't get a Windows 7 DVD.

Acer does this. If you have any problem that want you to ship the
entire machine to be fixed. You go without your machine for two
months or so. You have no choice but to buy a Windows CD, or a whole
new computer.

Steve Sobol

unread,
Nov 22, 2009, 10:06:53 PM11/22/09
to
In article <qukjg5918bo20o9er...@4ax.com>,
see_w...@mindprod.com.invalid says...


> IIRC this is not that useful. Your partitions are undone. You lose
> all your data too. It also requires that a special partition be
> undamaged.

Depends on the manufacturer. I've never had to do a System Recovery on
my Dells, but my HP allows me to do a non-destructive System Recovery.
You need to reinstall your apps, but your data is left alone.



> Acer does this. If you have any problem that want you to ship the
> entire machine to be fixed. You go without your machine for two
> months or so. You have no choice but to buy a Windows CD, or a whole
> new computer.

Then don't buy Acer. Dell sells PC's with other service plans, and I
have no appreciable amount of experience with other brands, but I'd
imagine they do too.

Thomas Pornin

unread,
Nov 23, 2009, 8:54:01 AM11/23/09
to
According to DuncanIdaho <Duncan.I...@googlemail.com>:

> The question is, will all my Java stuff (Tomcat 6, Eclipse/My Eclipse
> 5.1/ Java 1.6)that I currently have installed on my remaining XP SP2
> machine still work or do I need to get everything in new, spanky 64 bit
> versions.

The 64-bit versions of Windows are quite good at running the 32-bit
applications. 32-vs-64 problems occur when trying to mix compiled binary
code in the same address space. In the Java world, this means that if
you run a 64-bit JVM then the JVM will dynamically translate the code
into a 64-bit address space, which is fine, until you try to load a
32-bit DLL for native code, in which case the loading fails.

Therefore it is most probable that running all your 32-bit apps "as is"
in Windows will "just work", but you will shun the advantages of 64-bit
computing; also, some applications may somehow complain about being
roughly copied and not "installed" in the Windows sense of the term.

Java code itself (the .class and .jar) are bit-size-neutral, they are
loaded unchanged regardless of whether the JVM is 32-bit or 64-bit or
whatever else. That is the whole point of Java. Hence, unless you
explicitly use native code, Tomcat should run fine on a 64-bit JVM.


The 64-bit address space may imply a slight increase in memory
consumption because internal pointers get larger. But this is unlikely
to have any real impact in most situations (most of the memory used in a
typical application is for non-pointer data, e.g. byte[] or char[],
which keeps the same size in a 64-bit JVM); also, Sun's JVM has an
option to use 32-bit pointers in 64-bit mode ('-XX:+UseCompressedOops').
On the other hand, the extra and larger registers unlocked by 64-bit
mode give a performance boost to some specific operations (in particular
BigInteger, which runs about 3 times faster). And anyway, you will have
to go 64-bit at some time, so better do it now.


--Thomas Pornin

Roedy Green

unread,
Nov 23, 2009, 1:14:32 PM11/23/09
to
On 23 Nov 2009 13:54:01 GMT, Thomas Pornin <por...@bolet.org> wrote,

quoted or indirectly quoted someone who said :

>most of the memory used in a


>typical application is for non-pointer data, e.g. byte[] or char[],

In my own code, Strings and arrays of pointers to Strings would be the
most common ram hog.

I wonder if someone could cook up a simple tool to predict the effect
of going to 64 bit on any given app.

David Lamb

unread,
Nov 23, 2009, 7:39:59 PM11/23/09
to
Thomas Pornin wrote:
>(most of the memory used in a
> typical application is for non-pointer data, e.g. byte[] or char[],
> which keeps the same size in a 64-bit JVM)

Do you have any studies that show this (ie. that most memory use is
large arrays of this sort)? I came from an environment where
tree-structured data (with lots of pointers) was common.

Lew

unread,
Nov 23, 2009, 8:42:37 PM11/23/09
to
Roedy Green wrote:
> In my own code, Strings and arrays of pointers to Strings would be the
> most common ram hog.

Your code would probably benefit greatly from the '-XX:+UseCompressedOops'
option in a 64-bit environment, then.

--
Lew

Nigel Wade

unread,
Nov 24, 2009, 4:12:53 AM11/24/09
to
On Mon, 23 Nov 2009 10:14:32 -0800, Roedy Green wrote:

> On 23 Nov 2009 13:54:01 GMT, Thomas Pornin <por...@bolet.org> wrote,
> quoted or indirectly quoted someone who said :
>
>>most of the memory used in a
>>typical application is for non-pointer data, e.g. byte[] or char[],
>
> In my own code, Strings and arrays of pointers to Strings would be the
> most common ram hog.
>
> I wonder if someone could cook up a simple tool to predict the effect of
> going to 64 bit on any given app.

Given that going 64bit lifts you out of the 3GB straight-jacket, I
predict the effect of going 64bit would be to free you from worries about
the amount of RAM your application requires and concentrate on other,
more important issues.

--
Nigel Wade

Arved Sandstrom

unread,
Nov 24, 2009, 5:17:00 AM11/24/09
to
Leaving aside the specific numbers, how many times over the past thirty
years (rough timeframe for the Age of PCs [1]) have we said the same
thing? :-)

AHS

[1] defining that "Age" by at least reasonably large use.

Thomas Pornin

unread,
Nov 24, 2009, 6:58:35 AM11/24/09
to
According to David Lamb <dal...@cs.queensu.ca>:

> Do you have any studies that show this (ie. that most memory use is
> large arrays of this sort)?

Only my own experience -- which includes fitting a Java-based server
into my own JVM for an embedded system, where I wrote the GC and saw
first-hand what RAM was used for. Although there were fairly complex
data structures in there (including complete validation of X.509
certificates), most of the memory used turned out to be I/O buffers (a
compliant SSL server requires a 18437-byte input buffer per connection,
and that's quite a lot actually).

On anything vaguely graphical, I would expect bitmap pictures to eat up
quite a lot of RAM as well.


--Thomas Pornin

DuncanIdaho

unread,
Nov 24, 2009, 9:31:17 AM11/24/09
to
Thomas Pornin wrote:
> According to DuncanIdaho <Duncan.I...@googlemail.com>:
>> The question is, will all my Java stuff (Tomcat 6, Eclipse/My Eclipse
>> 5.1/ Java 1.6)that I currently have installed on my remaining XP SP2
>> machine still work or do I need to get everything in new, spanky 64 bit
>> versions.
>
...

> And anyway, you will have
> to go 64-bit at some time, so better do it now.

Well most of my time is spent developing business systems that usually
run as webapps on Tomcat. Consequently I need to be aware of which
versions of the servlet engine I am working with (or rather which
versions the clients are using) and I certainly can't 'go 64 bit' on a
whim.

I can see that the address size might be a slight issue memory wise but
there is more to 64 bit that bigger addresses isn't there. Do 64 bit
registers mean a 64 bit memory bus ... do we really need to fetch 64
bits for a short (oops, sorry Short) ... so many years since university
so much forgotten. Anyway, many thanks for the very complete
answers to my question


-- Idaho

Patricia Shanahan

unread,
Nov 24, 2009, 10:02:36 AM11/24/09
to

I spend a lot less time on memory issues now than I did in 1970. I
remember being allowed a couple of weeks for adding a new input record
type to a program. I would do the same task in minutes in my current
Java program.

The main difficulty was that the program was close to no longer fitting
in a 16KB machine. Most of the time went on rearranging overlays and
reusing buffers to make space to add new code and data. Working in
assembly language on punch cards with one machine time slot per day
didn't help, but was less of a problem than the memory.

However, there is a definite tendency for programming to stay hard.
Features get added until programs are almost, but not quite, impossible
to write. It's just the type of hardness that changes. I think
multi-threading to get full performance on many core computers has a lot
of potential.

Patricia

Thomas Pornin

unread,
Nov 24, 2009, 10:21:50 AM11/24/09
to
According to DuncanIdaho <Duncan.I...@googlemail.com>:

> Do 64 bit registers mean a 64 bit memory bus ... do we really need to
> fetch 64 bits for a short (oops, sorry Short) ...

CPU already work by cache lines. A CPU fetches data only(*) from its
innermost cache (the L1 cache), which it fills by "lines" from the
L2 cache, and so on. The lines in the L1 cache have a size which
depends on the specific CPU architecture; typical sizes are 32 or
64 bytes (note: bytes, not bits).

64-bit mode means that you will need a bit more cache to handle the
same set of references. Cache pressure may reduce performance. On the
other hand, in 64-bit mode on x86 processors, registers are not only
larger, there are also more of them, which helps the JVM keep things
in registers, i.e. not using cache at all.

On my own experience, most of the time, going 64-bit does not measurably
modify performance. When performance _is_ modified in a non-negligeable
way, then most of the time the 64-bit version is faster. For some very
specific usages, performance gain with 64-bit mode can be dramatic (in
particular BigInteger.modPow(); also, the SHA-512 hash function). It is
probable that one can find some very specific usages where 32-bit mode
beats 64-bit mode by a large margin, but I have not come accross any.


--Thomas Pornin

(*) That is, except when the CPU fetches data from main memory directly.
This happens on some models when using synchronization features such as
'volatile'.

Arved Sandstrom

unread,
Nov 24, 2009, 3:44:28 PM11/24/09
to
Patricia Shanahan wrote:
> Arved Sandstrom wrote:
[ SNIP ]

>> Leaving aside the specific numbers, how many times over the past
>> thirty years (rough timeframe for the Age of PCs [1]) have we said the
>> same thing? :-)
>>
>
> I spend a lot less time on memory issues now than I did in 1970. I
> remember being allowed a couple of weeks for adding a new input record
> type to a program. I would do the same task in minutes in my current
> Java program.
>
> The main difficulty was that the program was close to no longer fitting
> in a 16KB machine. Most of the time went on rearranging overlays and
> reusing buffers to make space to add new code and data. Working in
> assembly language on punch cards with one machine time slot per day
> didn't help, but was less of a problem than the memory.
>
> However, there is a definite tendency for programming to stay hard.
> Features get added until programs are almost, but not quite, impossible
> to write. It's just the type of hardness that changes. I think
> multi-threading to get full performance on many core computers has a lot
> of potential.
>
> Patricia

I was being somewhat tongue in cheek about the memory thing. :-) I have
to agree, even with application bloat I spend a much smaller fraction of
my time these days worrying about memory than I did 20 or 30 years ago.
Well, to be accurate, I still worry about memory but now it's not RAM
I'm thinking about.

Interesting observation about programming tending to stay hard, with
only the type of hardness changing. I agree. I suspect that neither one
of us is referring to "hardness" at the architectural and design levels,
which despite the flurry of new software development methodologies (and
an explosion of tool support) still doesn't seem to be getting much
better, but rather to "hardness" at the implementation level. I think
you're right. 15 years ago I would have considered one of my harder
problems doing up a desktop GUI app, and now that's relatively easy. I
think that right now, and for the past while, if I had to identify one
"hard" area it would probably be threading. As you point out, threading
and parallelism is probably going to remain a "hard" area for a while.

AHS

Thomas Pornin

unread,
Nov 24, 2009, 4:16:42 PM11/24/09
to
According to Arved Sandstrom <dce...@hotmail.com>:

> Well, to be accurate, I still worry about memory but now it's not RAM
> I'm thinking about.

When I worry about RAM, it is mostly for a few routines where performance
happens to be critical, and there I worry about "fast RAM". Fast RAM is
the RAM which is accessed without any extra delay. On a recent CPU this
means L1 cache. My relatively recent Intel CPU has a 32 kB L1 cache
for data.

The home computer I was using in 1984 (that's 25 years ago) also had
32 kB of fast RAM. To be fair, it had only 32 kB of RAM, but at least
all of it was as fast as the machine could handle.

I tend to think that problems which humans want to handle with a
computer have a rather constant size, about 32 kB. Computers get faster
and faster, and have much more RAM, but problem size does not expand.
Human patience, however, shrinks fast.


--Thomas Pornin

DuncanIdaho

unread,
Nov 25, 2009, 4:28:40 AM11/25/09
to
Thomas Pornin wrote:
> According to DuncanIdaho <Duncan.I...@googlemail.com>:
>> Do 64 bit registers mean a 64 bit memory bus ... do we really need to
>> fetch 64 bits for a short (oops, sorry Short) ...
>

snip


> It is
> probable that one can find some very specific usages where 32-bit mode
> beats 64-bit mode by a large margin, but I have not come accross any.
>

OK, the responses to my original post have really got my head buzzing
with all this low level stuff now, it's been years and I forgot how
obssessive I can get about these nitty gritty little bits and pieces
however the bottom line for me, in a practical sense is (Ignoring
execution issues for the time being) ...


If I develop against a 64bit JVM (say because I gotta have the 'latest
and greatest' no other reason) and then compile the code on a client
machine against a 32bit JVM then the only possible problem could be that
I'm using some humoungous great data structure that needs bigger that 32
bit addresses in which case I should probably be sacked anyway ... or am
I wide of the mark.


-- Idaho

Thomas Pornin

unread,
Nov 25, 2009, 8:33:52 AM11/25/09
to
According to DuncanIdaho <Duncan.I...@googlemail.com>:

> If I develop against a 64bit JVM (say because I gotta have the 'latest
> and greatest' no other reason) and then compile the code on a client
> machine against a 32bit JVM

Java source code is actually never compiled against a 64-bit or 32-bit
JVM; it is compiled against _the_ Java Virtual Machine, i.e. a
theoretical architecture which has no "register size". A Java compiler
should produce, for a given set of source file, the exact same .class
files, regardless of whether the compiler runs on a 64-bit, 32-bit or
whatever system.

The compiled code, i.e. the .class files (usually stored in .jar
archives), can be fed unchanged to a 64-bit or a 32-bit JVM, and will
run to produce the exact same results(*) on both. I routinely compile
Java code on a 64-bit Linux system, and the .jar files are deployed to
all kinds of systems, including 32-bit Windows and the like. This is
the point of using a virtual machine as compilation target.


> then the only possible problem could be that I'm using some humoungous
> great data structure that needs bigger that 32 bit addresses in which
> case I should probably be sacked anyway ...

It is not really a problem of address size -- it is a problem of RAM
size. If your code requires more than 4 GB of RAM, then it will not run
on a 32-bit system since 32-bit systems can use only 4 GB(**). A system
has registers wide enough to address all the RAM it can use, so you will
not be limited by the register size, but by the RAM size.

And if your code needs 4 GB, chances are that you will notice it...


--Thomas Pornin

(*) Except for performance, i.e. how much time is used to perform said
operations; also, there can be some rounding issues with floating-point
types in non-strictfp code.

(**) With PAE, available since the Pentium Pro (that's more than twelve
years ago), a 32-bit x86 system may use up to 64 GB of RAM, but not
simultaneously: each single process may use only 4 GB.

Lew

unread,
Nov 25, 2009, 9:19:16 AM11/25/09
to
Thomas Pornin wrote:
> (**) With PAE, available since the Pentium Pro (that's more than twelve
> years ago), a 32-bit x86 system may use up to 64 GB of RAM, but not
> simultaneously: each single process may use only 4 GB.

I'm aware of no 32b JVM that can specify a heap over about 1.8 GB; most are
lower. 32b Windows only allows up to 2GB per process or 3GB with a switch.
The JVM and permanent generation use some of that, leaving less for the heap.

--
Lew

Nigel Wade

unread,
Nov 25, 2009, 10:13:16 AM11/25/09
to

$ java -version
java version "1.6.0_15"
Java(TM) SE Runtime Environment (build 1.6.0_15-b03)
Java HotSpot(TM) Server VM (build 14.1-b02, mixed mode)

$ java -jar -Xmx3700m some.jar
...

$ java -jar -Xmx3800m some.jar
Error occurred during initialization of VM
Could not reserve enough space for object heap
Could not create the Java virtual machine.

So, the limit on this particular Java application is somewhere between
3.7GB and 3.8GB of heap.

I should point out that this is a 32bit JVM running on a 64bit system
with 32GB of RAM. Nonetheless, the limit on a 32bit application is still
4GB despite the 64bit OS. What it does mean, though, is that any
individual application has got a much greater chance of getting the max.
address space because the entire system isn't limited to 4GB.

The same may be true of a PAE system, I don't know and can't test it.

--
Nigel Wade

Thomas Pornin

unread,
Nov 25, 2009, 11:24:51 AM11/25/09
to
According to Lew <no...@lewscanon.com>:

> I'm aware of no 32b JVM that can specify a heap over about 1.8 GB;
> most are lower.

Sun's JVM can. Right now, with Sun's JDK 1.6.0_16, 32-bit version for
Linux, I can specify a 3.68 GB heap with '-Xmx3683m', and use it
(I made a simple Java code which successfully stored 3683 byte[],
each newly allocated of size 1048576, into an ArrayList).

The trick here is that while the JDK is a 32-bit binary and runs
in a 32-bit system, the Linux kernel itself is a 64-bit kernel. The
limit on 2 or 3 GB per process is due to the need for the kernel
code to be able to access its own code and the user address space
with special access rights (not those seen from user space), which
basically means that memory seen by the application is mapped twice
(for the 3 GB limit, the kernel must employ various trickeries
such as messing with the MMU tables, which are cumbersome and
potentially expensive). When the kernel itself is a 64-bit kernel,
it can map the full 32-bit address space (4 GB) twice in its own,
much wider 64-bit address space; hence, application may then have
their (almost) full complement of 4 GB.

As far as I know, this is also what happens when a 64-bit Windows
system runs a 32-bit application.


Running a full 32-bit system with a 64-bit kernel is a possible
compromise which is actually recommended when there are many 32-bit
legacy, binary only applications to run, but the machine has more
than 2 or 3 GB of RAM and the CPU supports the 64-bit amd64 mode.


--Thomas Pornin

DuncanIdaho

unread,
Nov 25, 2009, 11:27:18 AM11/25/09
to
Nigel Wade wrote:
> On Wed, 25 Nov 2009 09:19:16 -0500, Lew wrote:
>
>> Thomas Pornin wrote:
>>> (**) With PAE, available since the Pentium Pro (that's more than twelve
>>> years ago), a 32-bit x86 system may use up to 64 GB of RAM, but not
>>> simultaneously: each single process may use only 4 GB.
>> I'm aware of no 32b JVM that can specify a heap over about 1.8 GB; most
>> are lower.

snip

> I should point out that this is a 32bit JVM running on a 64bit system
> with 32GB of RAM. Nonetheless, the limit on a 32bit application is still
> 4GB despite the 64bit OS. What it does mean, though, is that any
> individual application has got a much greater chance of getting the max.
> address space because the entire system isn't limited to 4GB.
>
> The same may be true of a PAE system, I don't know and can't test it.
>

OK well I've ordered my new laptop with Windows 7

By the way http://java.sun.com/javase/downloads/widget/jdk6.jsp offers
JDK 6 Update 17 for Windows and Windows x64 where the archive for the 64
bit version is smaller than that for the 32 bit version by around 8MB,
interesting huh, also, apparently the 64 bit release can be used in 32
bit mode...

Anyway, it will be interesting to see just how big the heap can be with
a 64bit JVM running on a 64bit system. Roll on delivery day.

-- Idaho

Lew

unread,
Nov 25, 2009, 2:19:21 PM11/25/09
to
According to Lew :

>> I'm aware of no 32b JVM that can specify a heap over about 1.8 GB;
>> most are lower.
>

Thomas Pornin wrote:
> Sun's JVM can. Right now, with Sun's JDK 1.6.0_16, 32-bit version for
> Linux, I can specify a 3.68 GB heap with '-Xmx3683m', and use it
> (I made a simple Java code which successfully stored 3683 byte[],
> each newly allocated of size 1048576, into an ArrayList).
>
> The trick here is that while the JDK is a 32-bit binary and runs
> in a 32-bit system, the Linux kernel itself is a 64-bit kernel. The
>

Right to both of you. I should have specified that I meant a 32b JVM
on a 32b OS. My mistake.

--
Lew

Peter Duniho

unread,
Nov 25, 2009, 2:52:47 PM11/25/09
to
Thomas Pornin wrote:
> [...] When the kernel itself is a 64-bit kernel,

> it can map the full 32-bit address space (4 GB) twice in its own,
> much wider 64-bit address space; hence, application may then have
> their (almost) full complement of 4 GB.
>
> As far as I know, this is also what happens when a 64-bit Windows
> system runs a 32-bit application.

I believe with would only happen for Windows applications marked as
"LARGEADDRESSAWARE".

Unfortunately, there are application developers out there who take an
implementation detail like "all of your allocations will return an
address from 0 to 2^31-1" and stick dependencies on that detail into
their code. So Windows can't by default start handing out addresses in
the upper 2GB of the 4GB address space, lest those applications stop
working.

But yes, with the "LARGEADDRESSAWARE" switch enabled for an executable,
on 64-bit Windows, the OS can provide all of the 4GB address space a
32-bit application could theoretically use to that application, because
it no longer needs any of the application's lower 4GB of 64-bit address
space (the portion the 32-bit code can address) for its own needs.

Pete

Peter Duniho

unread,
Nov 25, 2009, 3:00:50 PM11/25/09
to
DuncanIdaho wrote:
> [...]

> Anyway, it will be interesting to see just how big the heap can be with
> a 64bit JVM running on a 64bit system. Roll on delivery day.

It will probably depend on the OS. But, for Windows and based on this
MSDN article (http://support.microsoft.com/default.aspx?scid=889654), I
would expect your 64-bit JVM process to max out at 8 terabytes (or just
under), the maximum virtual address space for a 64-bit process on Windows.

It's interesting to note that 64-bit Windows has essentially the same
"half and half" virtual address space approach 32-bit Windows does.
It's just that the numbers are bigger (16TB virtual address space, where
8TB is dedicated to the application).

Anyway, with a theoretical 8TB limit on application allocations, unless
you have some really huge disk or RAID volume, you'll run out of disk
space before you can hit the actual theoretical limit. That is,
observed behavior will be configuration-dependent. (But that's just for
Windows...my understanding is that with most versions of Linux, for
example, it will happily allow you to allocate as much memory is
theoretically possible for the process, and only fail due to lack of
system resources if your process actually tries to use the allocated
memory).

Pete

Thomas Pornin

unread,
Nov 25, 2009, 4:31:33 PM11/25/09
to
According to Peter Duniho <NpOeS...@NnOwSlPiAnMk.com>:

> my understanding is that with most versions of Linux, for example, it
> will happily allow you to allocate as much memory is theoretically
> possible for the process, and only fail due to lack of system
> resources if your process actually tries to use the allocated memory

It is called "overcommit". Any Linux from the last ten years or so can
be configured to overcommit memory as you describe, but default
behaviour on most Linux distributions is not to overcommit. Check the
contents of /proc/sys/vm/overcommit_memory: if that's a 0, then you are
not overcommitting.

A very large FTP site from last century, called ftp.cdrom.com (yeah,
that was before the DVD, when CDROMs where the quintessential hugeness)
saw its maximum number of concurrently connected clients drop from 3000
to 700 when it switched OS, because the new OS did not overcommit by
default, whereas the old one was overcommitting (the FTP server
allocated much more RAM per connected client than what it was actually
using). Hence there are situations where overcommitting is good.
However, the notion of overcommit tends to trigger irrational prophecies
of doom, hence it is deactivated by default. For most people,
oveercommitting makes no difference whatsoever: in order to hit the
point where an overcommitted page turns out not to be present, one must
first exhaust the whole swap space, a slow and painful process,
especially for a desktop system, in a typical configuration.

To sum up, overcommit is there but not used by default, and it does not
usually matter anyway.


--Thomas Pornin

Nigel Wade

unread,
Nov 26, 2009, 4:13:57 AM11/26/09
to

Ok, on a 32bit machine:

$ java -Xmx2700m ParseTime
Thu Jan 01 08:00:00 GMT 1970
...

$ java -Xmx2800m ParseTime


Error occurred during initialization of VM
Could not reserve enough space for object heap
Could not create the Java virtual machine.

So, here I'm limited to 2.7GB on a stock openSUSE desktop with 3GB RAM
and 4GB swap, running:

$ java -version
java version "1.6.0_15"

I thought you used Linux. What distro and memory configuration has
limited you to 1.8GB?

--
Nigel Wade

Lew

unread,
Nov 26, 2009, 8:12:23 PM11/26/09
to
Nigel Wade wrote:
> Ok, on a 32bit machine:
>
> $ java -Xmx2700m ParseTime
> Thu Jan 01 08:00:00 GMT 1970
> ...
>
> $ java -Xmx2800m ParseTime
> Error occurred during initialization of VM
> Could not reserve enough space for object heap
> Could not create the Java virtual machine.
>
> So, here I'm limited to 2.7GB on a stock openSUSE desktop with 3GB RAM
> and 4GB swap, running:
>
> $ java -version
> java version "1.6.0_15"

I couldn't be more glad to be proven wrong.

--
Lew

Nigel Wade

unread,
Nov 27, 2009, 4:53:13 AM11/27/09
to
On Wed, 25 Nov 2009 16:27:18 +0000, DuncanIdaho wrote:

> Nigel Wade wrote:
>> On Wed, 25 Nov 2009 09:19:16 -0500, Lew wrote:
>>
>>> Thomas Pornin wrote:
>>>> (**) With PAE, available since the Pentium Pro (that's more than
>>>> twelve years ago), a 32-bit x86 system may use up to 64 GB of RAM,
>>>> but not simultaneously: each single process may use only 4 GB.
>>> I'm aware of no 32b JVM that can specify a heap over about 1.8 GB;
>>> most are lower.
>
> snip
>
>> I should point out that this is a 32bit JVM running on a 64bit system
>> with 32GB of RAM. Nonetheless, the limit on a 32bit application is
>> still 4GB despite the 64bit OS. What it does mean, though, is that any
>> individual application has got a much greater chance of getting the
>> max. address space because the entire system isn't limited to 4GB.
>>
>> The same may be true of a PAE system, I don't know and can't test it.
>>
>>
> OK well I've ordered my new laptop with Windows 7
>
> By the way http://java.sun.com/javase/downloads/widget/jdk6.jsp offers
> JDK 6 Update 17 for Windows and Windows x64 where the archive for the 64
> bit version is smaller than that for the 32 bit version by around 8MB,
> interesting huh,

Well, the Linux 64bit version is about 1MB larger. It doesn't really tell
you anything about the unpacked size, or the size of running JVM.

> also, apparently the 64 bit release can be used in 32
> bit mode...

Maybe, maybe not:

$ /usr/java/jdk1.6.0_17/bin/java -version
java version "1.6.0_17"
Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01, mixed mode)

$ /usr/java/jdk1.6.0_17/bin/java -version -d32
Running a 32-bit JVM is not supported on this platform.

I don't know whether it works on Windows.

>
> Anyway, it will be interesting to see just how big the heap can be with
> a 64bit JVM running on a 64bit system. Roll on delivery day.
>

Whatever the limit of your VM is, presumably. Probably RAM+swap, or the
administrator/OS defined per-process VM limit. You can request max. heap
size much more than the available VM, but it's not actually physically
allocated until required, for example:

$ /usr/java/jdk1.6.0_17/bin/java -Xmx128000m -jar some.jar

works on a system with only 32GB RAM.

If, however, I restrict the per-process VM to 20GB using:

$ ulimit -v 20000000

I get:

$ /usr/java/jdk1.6.0_17/bin/java -Xmx20000m -jar dist/MP3FileCopier.jar .


Error occurred during initialization of VM
Could not reserve enough space for object heap
Could not create the Java virtual machine.

but I can request 19GB.

As I said elsethread, taking up a few MB of extra space for 64bit
addresses is not something to be overly concerned about.

--
Nigel Wade

Arne Vajhøj

unread,
Jan 3, 2010, 10:10:18 PM1/3/10
to

Besides the already mentioned case of 32 bit JVM on 64 bit OS,
then JRockit is said to be able to use the /3GB in Windows.

I think I have been told that 32 bit Solaris also allows
the JVM to go above the approx. 1.8 GB.

Arne

Arne Vajhøj

unread,
Jan 3, 2010, 10:11:12 PM1/3/10
to
On 25-11-2009 10:13, Nigel Wade wrote:
> I should point out that this is a 32bit JVM running on a 64bit system
> with 32GB of RAM. Nonetheless, the limit on a 32bit application is still
> 4GB despite the 64bit OS. What it does mean, though, is that any
> individual application has got a much greater chance of getting the max.
> address space because the entire system isn't limited to 4GB.
>
> The same may be true of a PAE system, I don't know and can't test it.

PAE increases physical address space not virtual address space,
so it should not make a difference.

Arne

Arne Vajhøj

unread,
Jan 3, 2010, 10:25:00 PM1/3/10
to
On 25-11-2009 15:00, Peter Duniho wrote:
> DuncanIdaho wrote:
>> [...]
>> Anyway, it will be interesting to see just how big the heap can be
>> with a 64bit JVM running on a 64bit system. Roll on delivery day.
>
> It will probably depend on the OS. But, for Windows and based on this
> MSDN article (http://support.microsoft.com/default.aspx?scid=889654), I
> would expect your 64-bit JVM process to max out at 8 terabytes (or just
> under), the maximum virtual address space for a 64-bit process on Windows.

Note that it is a Windows limitation.

The processors support 256 TB.

> It's interesting to note that 64-bit Windows has essentially the same
> "half and half" virtual address space approach 32-bit Windows does. It's
> just that the numbers are bigger (16TB virtual address space, where 8TB
> is dedicated to the application).

Cutler took that design with him from DEC.

> Anyway, with a theoretical 8TB limit on application allocations, unless
> you have some really huge disk or RAID volume, you'll run out of disk
> space before you can hit the actual theoretical limit. That is, observed
> behavior will be configuration-dependent. (But that's just for
> Windows...my understanding is that with most versions of Linux, for
> example, it will happily allow you to allocate as much memory is
> theoretically possible for the process, and only fail due to lack of
> system resources if your process actually tries to use the allocated
> memory).

Note even though Windows does not allow overcommit like *nix and VMS
does, then the restriction is not:
sum of virtual space <= size pagefile
but:
sum of virtual space <= size pagefile + size RAM + size memory mapped
files

In practice size RAM would be so much smaller than size of disks
that the bottom line is the smae.

Arne

Roedy Green

unread,
Jan 6, 2010, 3:39:17 PM1/6/10
to
On Sun, 03 Jan 2010 22:25:00 -0500, Arne Vajh�j <ar...@vajhoej.dk>

wrote, quoted or indirectly quoted someone who said :

>


>The processors support 256 TB.

The architecture may, but surely no chips on the market now have the
addressing pins to support that.
--
Roedy Green Canadian Mind Products
http://mindprod.com
The question of whether Machines Can Think... is about as relevant as the question of whether Submarines Can Swim.
~ Edsger Wybe Dijkstra (born: 1930-05-11 died: 2002-08-06 at age: 72)

Arne Vajhøj

unread,
Jan 6, 2010, 10:28:28 PM1/6/10
to
On 06-01-2010 15:39, Roedy Green wrote:
> On Sun, 03 Jan 2010 22:25:00 -0500, Arne Vajh�j<ar...@vajhoej.dk>
>> The processors support 256 TB.
>
> The architecture may, but surely no chips on the market now have the
> addressing pins to support that.

Actually they do.

Arne

Roedy Green

unread,
Jan 6, 2010, 11:57:45 PM1/6/10
to
On Wed, 06 Jan 2010 22:28:28 -0500, Arne Vajh�j <ar...@vajhoej.dk>

wrote, quoted or indirectly quoted someone who said :

>>


>> The architecture may, but surely no chips on the market now have the
>> addressing pins to support that.
>
>Actually they do.

Future shock. The biggest Asus MB I could find supports 24 GB, so I
guess even if the chips have the full 64 bit complement of addressing
lines, the motherboards only support something like 35 of them. I
wonder who has machines that need the high order lines. Maybe
somewhere out there are i/o devices that look like real ram to a CPU
but which manage virtual ram with a multilayers of cache, including
flash RAM, and high speed disks.

I wonder how long before mass-storage and virtual RAM are unified so
that mass storage is accessed as a dedicated part of virtual RAM.

Peter Duniho

unread,
Jan 7, 2010, 12:09:49 AM1/7/10
to
Roedy Green wrote:
> [...]

> I wonder how long before mass-storage and virtual RAM are unified so
> that mass storage is accessed as a dedicated part of virtual RAM.

You've been able to do that for years. It's called memory-mapped files.

Eric Sosman

unread,
Jan 7, 2010, 8:56:10 AM1/7/10
to
On 1/6/2010 11:57 PM, Roedy Green wrote:
>
> I wonder how long before mass-storage and virtual RAM are unified so
> that mass storage is accessed as a dedicated part of virtual RAM.

It'll take another minus twenty-two to minus forty-six
years, I'd guess.

http://en.wikipedia.org/wiki/Single_level_store

--
Eric Sosman
eso...@ieee-dot-org.invalid

Roedy Green

unread,
Jan 7, 2010, 9:19:29 AM1/7/10
to
On Wed, 06 Jan 2010 21:09:49 -0800, Peter Duniho
<NpOeS...@NnOwSlPiAnMk.com> wrote, quoted or indirectly quoted
someone who said :

>> I wonder how long before mass-storage and virtual RAM are unified so


>> that mass storage is accessed as a dedicated part of virtual RAM.
>
>You've been able to do that for years. It's called memory-mapped files.

That is not what I mean. I mean at a hardware level, that the
hardware CPU (not the application program) sees as what it thinks is
245TB bank of RAM. However the RAM "chips" fake this with a firmware
managed virtual RAM system of several layers of cache, perhaps even at
the lowest level accessing the net.

The idea is special purpose and proprietary hardware can go inside the
"RAM". It would allow even the clutziest OS to evolve rapidly to the
cleverest virtual RAM system.

The "RAM" would have to warn the CPU when it was going to take a long
time to satisfy a request, or the CPU might be so heavily
hyperthreaded it would not matter if most of the threads were waiting
on RAM.

It seems to me if it is possible to carve out any well-understood
chunk of functionality, and put in inside a peripheral, that should
encourage innovation through unusual proprietary, hardware-assisted
techniques, while simplifying the OS, and making it less vulnerable to
failure or attack.

Ideas along that line include the multi-line serial controller
(available since the DOS days), in-disk cache firmware (standard now),
the martha, an SQL appliance and the dark room.

Patricia Shanahan

unread,
Jan 7, 2010, 9:32:30 AM1/7/10
to
Roedy Green wrote:
> On Wed, 06 Jan 2010 21:09:49 -0800, Peter Duniho
> <NpOeS...@NnOwSlPiAnMk.com> wrote, quoted or indirectly quoted
> someone who said :
>
>>> I wonder how long before mass-storage and virtual RAM are unified so
>>> that mass storage is accessed as a dedicated part of virtual RAM.
>> You've been able to do that for years. It's called memory-mapped files.
>
> That is not what I mean. I mean at a hardware level, that the
> hardware CPU (not the application program) sees as what it thinks is
> 245TB bank of RAM. However the RAM "chips" fake this with a firmware
> managed virtual RAM system of several layers of cache, perhaps even at
> the lowest level accessing the net.
...

What advantage do you see to managing this from firmware rather than
operating system software, as is done in current implementations of
memory-mapped files?

Hardware and firmware are less flexible and harder to update than the
operating system. Hard wired decisions have to be kept simple.
Performance is the usual reason for moving memory hierarchy layer
management into hardware or firmware, despite those disadvantages. That
does not apply to the disk layer, given disk access times and block sizes.

Patricia

Martin Gregorie

unread,
Jan 7, 2010, 1:06:19 PM1/7/10
to
On Thu, 07 Jan 2010 06:19:29 -0800, Roedy Green wrote:

> On Wed, 06 Jan 2010 21:09:49 -0800, Peter Duniho
> <NpOeS...@NnOwSlPiAnMk.com> wrote, quoted or indirectly quoted
> someone who said :
>
>>> I wonder how long before mass-storage and virtual RAM are unified so
>>> that mass storage is accessed as a dedicated part of virtual RAM.
>>
>>You've been able to do that for years. It's called memory-mapped files.
>
> That is not what I mean. I mean at a hardware level, that the hardware
> CPU (not the application program) sees as what it thinks is 245TB bank
> of RAM. However the RAM "chips" fake this with a firmware managed
> virtual RAM system of several layers of cache, perhaps even at the
> lowest level accessing the net.
>

This was done years ago - around 1970 to be precise and was known as the
IBM Future Series.

The architecture was very different to the System/360 /370 mainframes and
it was canned because IBM chickened out over forcing their customer base
to migrating to something so radical: a huge amount of S/360 code was
written in assembler and/or used the IMS hierarchic database but Future
Series had no assembler (it was a PL/1 machine in the same way that
Unices run on C machines) and didn't support IMS.

However, the idea was good. It not only worked, but worked well and was
revived in the late 80s as the System/38. The second generation S/38 was
better known as the AS/400 and the various names the sales droids called
it more recently.

At a user and programming level you had files and libraries, but that was
only an implementation of the usual programing environment. Without it
programming in C. PL/I, COBOL and even RPG would have been hard because
virtually all programming languages except SQL expect to deal with files.

However, beneath that abstraction layer there was a single, flat storage
system that mapped virtual memory onto RAID 5 disks handled by good, fast
parallel tasking disk controllers. The downside was that you had
absolutely no control over the placement of the blocks making up a file.
The OS/400 load balancing algorithm took care of that. The upsides were
that all data was on redundant, hot-pluggable disks, random access
performance was good and serial access to a file was blindingly fast
thanks to the ability of controllers to stream in parallel off all the
disks in a RAID 5 set.

> The idea is special purpose and proprietary hardware can go inside the
> "RAM". It would allow even the clutziest OS to evolve rapidly to the
> cleverest virtual RAM system.
>

As I said, its been done. See above. Actually, under the covers the
AS/400 was a SNA LU6.2 network with processors, disk controllers, network
controllers, implemented as nodes strung on the network.

> The "RAM" would have to warn the CPU when it was going to take a long
> time to satisfy a request, or the CPU might be so heavily hyperthreaded
> it would not matter if most of the threads were waiting on RAM.
>

Didn't appear to be an issue, but then every process ran in its own VM
and the process scheduler was pretty good.

> It seems to me if it is possible to carve out any well-understood chunk
> of functionality, and put in inside a peripheral, that should encourage
> innovation through unusual proprietary, hardware-assisted techniques,
> while simplifying the OS, and making it less vulnerable to failure or
> attack.
>

Exactly so. The key is organising the system as a network of intelligent
nodes. Other cases worth looking at as ways of using this architecture
are the ICL System/39 and HP's Guardian NonStop fault tolerant computers.
Its worth pointing out, too, that any Unix based on a Mach kernel is
built on the same principle, though here all the nodes and the network
are inside the same chunk of RAM.


--
martin@ | Martin Gregorie
gregorie. | Essex, UK
org |

Martin Gregorie

unread,
Jan 7, 2010, 1:12:36 PM1/7/10
to
On Thu, 07 Jan 2010 08:56:10 -0500, Eric Sosman wrote:

> On 1/6/2010 11:57 PM, Roedy Green wrote:
>>
>> I wonder how long before mass-storage and virtual RAM are unified so
>> that mass storage is accessed as a dedicated part of virtual RAM.
>
> It'll take another minus twenty-two to minus forty-six
> years, I'd guess.
>
> http://en.wikipedia.org/wiki/Single_level_store
>

That is OK except that it left out IBM's first iteration, Future Series,
which was canned (see my previous post) around 1970. As this range was
never publicly announced, I guess it isn't generally known but it was
canned just before the S/370s (the first virtual memory IBM mainframes)
appeared.

Roedy Green

unread,
Jan 7, 2010, 3:17:38 PM1/7/10
to
On Thu, 07 Jan 2010 06:32:30 -0800, Patricia Shanahan <pa...@acm.org>

wrote, quoted or indirectly quoted someone who said :

>What advantage do you see to managing this from firmware rather than


>operating system software, as is done in current implementations of
>memory-mapped files?

1. It can be hardware assisted. Imagine if virtual RAM were done
entirely with software!

2. It can use proprietary techniques even in an open source OS.

3. Innovation can be used immediately. You don't have to wait for the
OS people to implement it.

4. machines are more modular. If you need more performance, you just
plug in higher performance peripherals. The OS does not need to impose
complexity and cost on those who don't need it.

5. The OS is simpler.

6. Malfunctioning of the something in one of these smart peripherals
can harm only itself. It cannot reach out and screw up unrelated
software.

7. By allowing vendors to specialise, they can be smaller and more
numerous. They should be thus more competition and evolution.

8. It is much like the advantage of having a high level standard video
driver interface rather than having the OS work directly with the
video iron.

>
>Hardware and firmware are less flexible and harder to update than the
>operating system. Hard wired decisions have to be kept simple.
>Performance is the usual reason for moving memory hierarchy layer
>management into hardware or firmware, despite those disadvantages. That
>does not apply to the disk layer, given disk access times and block sizes.

It theory that is correct, but when you have only one vendor in effect
for your OS, if they screw up, you are screwed. What I am hoping to
do is carve away chunks of the OS and parcel it out to multiple
implementors. The smart peripherals presumably would firmware in it,
that with proper design should be no harder to update than software,
but could be designed to prevent tampering by anyone but the hardware
maker. E.g. updates need to be digitally signed.

Roedy Green

unread,
Jan 7, 2010, 3:35:09 PM1/7/10
to
On Thu, 7 Jan 2010 18:06:19 +0000 (UTC), Martin Gregorie
<mar...@address-in-sig.invalid> wrote, quoted or indirectly quoted
someone who said :

>>


>As I said, its been done.

That is about half way to what I am proposing. The System 38 had what
amounted to a gigantic virtual RAM system with 128-bit addressing. The
CPU was completely "aware" of what was in real RAM and what was not.

I am floating the idea that the CPU not know that. The CPU HARDWARE
(not just the software) would think it has terabytes of real silicon
RAM. The RAM chips themselves manage the virtualness with layers of
cache of varying speed working down to disk and the internet. The RAM
"chips" are a miniature proprietary computer", in the simplest case
implemented with pure hardware with static ram cache and real RAM.

To the programmer, the machine would look much like a System 38 with a
blending of RAM for running and the file system into a single flat
address space.

RAM would be a bit smarter than now. It would have commands to
forget ranges of bytes, clear ranges of bytes, and copy ranges of
bytes.The RAM peripheral would be free to transparently implement
these operations with lazy logic. The peripheral could be in the
background defragging its various layers.

Roedy Green

unread,
Jan 7, 2010, 3:40:57 PM1/7/10
to
On Thu, 7 Jan 2010 18:06:19 +0000 (UTC), Martin Gregorie
<mar...@address-in-sig.invalid> wrote, quoted or indirectly quoted
someone who said :

>> The "RAM" would have to warn the CPU when it was going to take a long


>> time to satisfy a request, or the CPU might be so heavily hyperthreaded
>> it would not matter if most of the threads were waiting on RAM.
>>
>Didn't appear to be an issue, but then every process ran in its own VM
>and the process scheduler was pretty good.

The System 38 had no problem that way because whenever it accessed
virtual RAM not in real ram the virtual RAM hardware would trigger and
interrupt. The OS would look in its tables, schedule a read of the
missing page. When it arrived it flushed it translation lookaside
buffers and modified the lookup tables, and restart the stalled task.

In the system I am proposing, the response from the RAM peripheral
could be from sub nanoseconds to seconds. If it were going to be a
long time, the CPU would probably want to know so it could get on with
some other task, rather than waiting.

If you had enough hardware hyperthreads to give one to every running
thread, (I am talking a bit in to the future here. Computing seems to
advance primarily by figuring out new ways to waste once scarce
resources.) then you might not bother.

Roedy Green

unread,
Jan 7, 2010, 4:00:28 PM1/7/10
to
On Thu, 7 Jan 2010 18:12:36 +0000 (UTC), Martin Gregorie
<mar...@address-in-sig.invalid> wrote, quoted or indirectly quoted
someone who said :

>That is OK except that it left out IBM's first iteration, Future Series,

>which was canned (see my previous post) around 1970. As this range was
>never publicly announced, I guess it isn't generally known but it was
>canned just before the S/370s (the first virtual memory IBM mainframes)
>appeared.

Thinking back to the days of the IBM 360/370 MFT/MVT we had streams of
batch jobs for 70K, 149K, and 210K.

Most commercial jobs written in PL/1 fit in 70k.
A big FORTRAN job was 210K.

Today with Jet, the smallest Java apps are about 60K.

The smallest useful Java app jar is about 15K.

However, java.exe typically runs with about 128 MEGABYTES for the
heap, and that is not counting the room for java.exe, DLL native code
and classes.

Arne Vajhøj

unread,
Jan 7, 2010, 10:04:06 PM1/7/10
to
On 06-01-2010 23:57, Roedy Green wrote:
> On Wed, 06 Jan 2010 22:28:28 -0500, Arne Vajh�j<ar...@vajhoej.dk>
> wrote, quoted or indirectly quoted someone who said :
>>> The architecture may, but surely no chips on the market now have the
>>> addressing pins to support that.
>>
>> Actually they do.
>
> Future shock. The biggest Asus MB I could find supports 24 GB, so I
> guess even if the chips have the full 64 bit complement of addressing
> lines, the motherboards only support something like 35 of them. I
> wonder who has machines that need the high order lines. Maybe
> somewhere out there are i/o devices that look like real ram to a CPU
> but which manage virtual ram with a multilayers of cache, including
> flash RAM, and high speed disks.

You are mixing up virtual addresses and RAM. The CPU
supports virtual addresses. The MB supports RAM.

The MB you have been looking at is desktop MB's. Both HP and
IBM sell x86-64 servers with 256 GB RAM.

Arne

Arne Vajhøj

unread,
Jan 7, 2010, 10:07:32 PM1/7/10
to
On 07-01-2010 15:35, Roedy Green wrote:
> I am floating the idea that the CPU not know that. The CPU HARDWARE
> (not just the software) would think it has terabytes of real silicon
> RAM. The RAM chips themselves manage the virtualness with layers of
> cache of varying speed working down to disk and the internet. The RAM
> "chips" are a miniature proprietary computer", in the simplest case
> implemented with pure hardware with static ram cache and real RAM.

If the software don't know what is in real RAM and what is in rotating
RAM, then performance will be bad.

Arne

Arne Vajhøj

unread,
Jan 7, 2010, 10:08:16 PM1/7/10
to
On 07-01-2010 15:40, Roedy Green wrote:
> The System 38 had no problem that way because whenever it accessed
> virtual RAM not in real ram the virtual RAM hardware would trigger and
> interrupt. The OS would look in its tables, schedule a read of the
> missing page. When it arrived it flushed it translation lookaside
> buffers and modified the lookup tables, and restart the stalled task.

That is how virtual memory works on all systems.

Arne

Roedy Green

unread,
Jan 8, 2010, 2:00:16 AM1/8/10
to
On Thu, 07 Jan 2010 22:07:32 -0500, Arne Vajh�j <ar...@vajhoej.dk>

wrote, quoted or indirectly quoted someone who said :

>If the software don't know what is in real RAM and what is in rotating


>RAM, then performance will be bad.

That is not the CPU's concern in my proposal.. The RAM controller
works to ensure commonly used stuff is in higher performance layers.

Today, when a CPU is running an application, it does not know which
pages are in real RAM and which in backing store. Where apps allocate
pages is not based on whether the frames are in RAM or not. That is
not its concern. The OS handles it. In my proposed scheme, that
responsibility shifts to the RAM controller hardware.


--
Roedy Green Canadian Mind Products
http://mindprod.com

There is no end to what can be accomplished if you don�t care who gets the credit.
~ Art Rennison

Roedy Green

unread,
Jan 8, 2010, 2:02:55 AM1/8/10
to
On Thu, 07 Jan 2010 22:08:16 -0500, Arne Vajh�j <ar...@vajhoej.dk>

wrote, quoted or indirectly quoted someone who said :

>> The System 38 had no problem that way because whenever it accessed

The System 38 is different is that ALL files are permanently memory
mapped files.

The System 38 is also different is having such whacking huge virtual
addresses, it can't simply address the location of every piece of it
with a straight forward linear table. You have to use associative
lookup. the address space usage is very sparse.

--
Roedy Green Canadian Mind Products
http://mindprod.com

Roedy Green

unread,
Jan 8, 2010, 2:10:38 AM1/8/10
to
On Thu, 07 Jan 2010 22:04:06 -0500, Arne Vajh�j <ar...@vajhoej.dk>

wrote, quoted or indirectly quoted someone who said :

>


>You are mixing up virtual addresses and RAM. The CPU
>supports virtual addresses. The MB supports RAM.

It is correct that for current CPUs, the CPU supports virtual
addresses and the MB supports RAM, but that does not negate what I
said.


The CPU from the application programmer's point of view works with
virtual addresses. The physical CPU chip has pins that address a bus
into real RAM positioned somewhere off the CPU on the motherboard.
Someone was claiming today's CPUs have 64 addressing pins when the
motherboard supports only about 35.

The virtual address on the CPU chip looks up physical RAM addresses
from virtual ones. If there is no corresponding page, the chip
triggers and interrupt and the OS wakes up and schedules loading the
page in to a free page slot and setting up the virtual tables to point
to it, then restarts the task waiting for that page. The CPU chip
knows absolutely nothing about pages on backing store. That is an OS
function. It keeps tables in RAM and oddly in virtual RAM.


--
Roedy Green Canadian Mind Products
http://mindprod.com

Dirk Bruere at NeoPax

unread,
Jan 8, 2010, 2:40:42 PM1/8/10
to

I wouldn't mind betting that the NSA has machines with insane quantities
of RAM.

--
Dirk

http://www.transcendence.me.uk/ - Transcendence UK
http://www.theconsensus.org/ - A UK political party
http://www.blogtalkradio.com/onetribe - Occult Talk Show

Arne Vajhøj

unread,
Jan 8, 2010, 7:31:12 PM1/8/10
to
On 08-01-2010 02:10, Roedy Green wrote:
> On Thu, 07 Jan 2010 22:04:06 -0500, Arne Vajh�j<ar...@vajhoej.dk>
> wrote, quoted or indirectly quoted someone who said :
>> You are mixing up virtual addresses and RAM. The CPU
>> supports virtual addresses. The MB supports RAM.
>
> It is correct that for current CPUs, the CPU supports virtual
> addresses and the MB supports RAM, but that does not negate what I
> said.
>
> The CPU from the application programmer's point of view works with
> virtual addresses. The physical CPU chip has pins that address a bus
> into real RAM positioned somewhere off the CPU on the motherboard.
> Someone was claiming today's CPUs have 64 addressing pins when the
> motherboard supports only about 35.

No.

Someone (me) noted that processors today supported 256 TB of virtual
address space.

Somebody else (you) compared that to how much RAM a motherboard
supported.

Which is a useless comparison.

And a useless comparison based on wrong information since server
motherboards support a lot more RAM.

Arne

Arne Vajhøj

unread,
Jan 8, 2010, 7:40:51 PM1/8/10
to
On 08-01-2010 02:02, Roedy Green wrote:
> On Thu, 07 Jan 2010 22:08:16 -0500, Arne Vajh�j<ar...@vajhoej.dk>
> wrote, quoted or indirectly quoted someone who said :
>>> The System 38 had no problem that way because whenever it accessed
>>> virtual RAM not in real ram the virtual RAM hardware would trigger and
>>> interrupt. The OS would look in its tables, schedule a read of the
>>> missing page. When it arrived it flushed it translation lookaside
>>> buffers and modified the lookup tables, and restart the stalled task.
>>
>> That is how virtual memory works on all systems.
>
> The System 38 is different is that ALL files are permanently memory
> mapped files.

On virtual memory systems you generally have to have either
the memory management system use the IO system to access page file
or have the IO system use the memory mapping feature of the
memory management system.

The first is the most normal. Apparently system 38 did the second.
But that is not unique. Multics, CDC NOS/VE etc. did some similar.

Arne

Arne Vajhøj

unread,
Jan 8, 2010, 7:43:46 PM1/8/10
to
On 08-01-2010 02:00, Roedy Green wrote:
> On Thu, 07 Jan 2010 22:07:32 -0500, Arne Vajh�j<ar...@vajhoej.dk>
> wrote, quoted or indirectly quoted someone who said :
>> If the software don't know what is in real RAM and what is in rotating
>> RAM, then performance will be bad.
>
> That is not the CPU's concern in my proposal.. The RAM controller
> works to ensure commonly used stuff is in higher performance layers.
>
> Today, when a CPU is running an application, it does not know which
> pages are in real RAM and which in backing store. Where apps allocate
> pages is not based on whether the frames are in RAM or not. That is
> not its concern. The OS handles it. In my proposed scheme, that
> responsibility shifts to the RAM controller hardware.

Hmm.

Virtual memory and page file works great if the physical
memory is able to cover most of the virtual address usage.
Otherwise it is dog slow.

Your proposed system will also work fine if the same condition
is met. But is not today and will not be in the near future either.

Arne


Lew

unread,
Jan 8, 2010, 8:35:11 PM1/8/10
to
Dirk Bruere at NeoPax wrote:
> I wouldn't mind betting that the NSA has machines with insane quantities
> of RAM.

You'd lose that bet because you'd be unable to verify your assertion.

--
Lew

Arne Vajhøj

unread,
Jan 8, 2010, 9:29:44 PM1/8/10
to

Unable to verify and be able to tell us.

Arne

Martin Gregorie

unread,
Jan 9, 2010, 12:14:52 PM1/9/10
to

"I could tell you that but I'd have to kill you".

0 new messages