In general, DrRacket does not promise to avoid this situation. Let me
explain a little bit. This program is effectively equivalent to this
one:
#lang racket/base
(define (run l)
(run (cons 'x l)))
(run '())
(but running at compile time, which is why things happen
automatically, but lets leave that aside for now).
So when you run that program, DrRacket is asking the operating system
for all of its memory. At some point, the operating system says
"sorry, no, we're out of memory" and since the program running inside
DrRacket doesn't really care that we're out of memory, the operating
system responds with "die!". And so DrRacket dies.
There are two ways we try to do better. First: we can set limits on
the memory. But if the limit is set to be higher than the amount of
memory available on the machine, then it is as if the memory limit
isn't set. Also worth noting at this point: the way that DrRacket's
counting of "how much memory" is only loosely related to the way that
your machine's vendor or the OS counts. There are some constant
factors of difference there that depend on things like how the garbage
collector is configured and how many other programs are running on
your system and what they are doing (like the OS itself likes to have
some of the memory and it (quite reasonably) doesn't want to share
that memory).
The second way is that the OS will sometimes say "you are about to
cause me to exhaust all of the system's memory and if you do that, I
will kill you" at which point the GC inside Racket responds with
"okay, I'll just die first". The main advantage of this is that you
get an error message saying "out of memory" from Racket instead of an
error message saying "I killed your program" from the OS (which is the
message you're getting). It turns out to be remarkably difficult to
get this error message to work in all situations. We're better than we
used to be at it, but at some level, the best we can do in that case
is really just to change the text of the error message. When we're out
of memory like that, it's very very hard to recover if the limits are
not set up ahead of time (i.e. back to the previous paragraph).
I hope this clears up the issue somewhat.
In the meantime, I suggest you lower the memory limit for your machine.
Robby