segfault with large-ish array with GNAT
Jerry
array size set to 1_048_343 but not when it is set to 1_048_342.
What's going on and how do I fix it? If I put in a bit of text IO the
allowed array size is some what smaller.
GNATMAKE 4.4.0 20080314 (experimental) [trunk revision 133226]
OS X 10.5.8
4 GB RAM
procedure bomb is
type Float_Array_Type is array (Integer range <>) of Long_Float;
-- 1_048_343 causes segmentation fault, 1_048_342 does not.
x : Float_Array_Type(1 .. 1_048_343);
begin
x(1) := 1.0;
end bomb;
Jerry
Gautier write-only
in the memory :-).
In case of a stack issue you would have perhaps
raised STORAGE_ERROR : object too large
or
raised STORAGE_ERROR : EXCEPTION_STACK_OVERFLOW
(with -fstack-check)
G.
jonathan
Have you tried setting the stacksize in your shell. In bash the
command is
ulimit -s unlimited
in csh, (I seem to recall):
limit stacksize unlimited
Type ulimit or limit to see what the defaults are.
When I do it on Debian Linux, your program I get arrays up to
2 Gigabyte in size.
HTH ...
Jonathan
Georg Bauhaus
Anything to do with
http://en.wikibooks.org/wiki/Ada_Programming/Tips#Stack_Size ?
Any news on GCC stack checking and page(?) size?
Trampolines, if applicable?
Jerry
First,
ulimit -s unlimited
does not work on OS X:
-bash: ulimit: stack size: cannot modify limit: Operation not
permitted
but I understand that it works on Linux. And possibly the reason is
the difference in the way that Linux and OS X treat stack and heap
memory. (Don't be confused and think I know what I'm talking about but
I read that somewhere.)
ulimit allows querying the hard limit of stack space
ulimit -Hs
which on OS X reports 65532 = 2^16 -4 kilobytes, about 67 MB. The user
via ulimit can set the stack up to that size but not higher:
ulimit -s 65532
The default soft limit on OS X is 8192 kB, found by
ulimit -s
So here's me being naive: I would have thought that Ada (or GNAT
specifically) would be smart enough to allocate memory for large
objects such as my long array in a transparent way so that I don't
have to worry about it, thus (in the Ada spirit) making it harder to
screw up. (Like not having to worry about whether arguments to
subprograms are passed by value or by reference--it just happens.)
But it seems that I will have to allocate memory for large objects
using pointers (and thus take the memory from the heap). Is that
right?
In this context, is there any advantage to declaring the large object
inside a declare block? Would that force the memory to be allocated
from the heap?
Jerry
Ludovic Brenta
> I would have thought that Ada (or GNAT
> specifically) would be smart enough to allocate memory for large
> objects such as my long array in a transparent way so that I don't
> have to worry about it, thus (in the Ada spirit) making it harder to
> screw up. (Like not having to worry about whether arguments to
> subprograms are passed by value or by reference--it just happens.)
So, you would like the Ada run-time to bypass the operating system-
enforced, administrator-approved stack limit? If userspace programs
could do that, what would be the point of having a stack limit in the
first place?
--
Ludovic Brenta.
Jeffrey Creem
> Thanks for the helpful comments.
>
> So here's me being naive: I would have thought that Ada (or GNAT
> specifically) would be smart enough to allocate memory for large
> objects such as my long array in a transparent way so that I don't
> have to worry about it, thus (in the Ada spirit) making it harder to
> screw up. (Like not having to worry about whether arguments to
> subprograms are passed by value or by reference--it just happens.)
>
> But it seems that I will have to allocate memory for large objects
> using pointers (and thus take the memory from the heap). Is that
> right?
>
> In this context, is there any advantage to declaring the large object
> inside a declare block? Would that force the memory to be allocated
> from the heap?
>
> Jerry
If you want the memory to come from the heap, you need to declare the
variables inside of packages instead of inside procedures. You can then
avoid using access types.
declare blocks will not help.
As for wishing that the compiler would automatically switch between heap
and stack, that would probably be a terrible idea and render the
language quite unsuitable for embedded systems.
-- warning, not even compiled early morning code example below
package do_stuff is
procedure no_bomb;
end do_stuff;
package body do_stuff is
type Float_Array_Type is array (Integer range <>) of Long_Float;
-- 1_048_343 causes segmentation fault, 1_048_342 does not.
x : Float_Array_Type(1 .. 1_048_343);
procedure No_bomb is
begin
x(1) := 1.0;
end No_bomb;
end do_stuff;
with do_stuff;
procedure stuff is
begin
do_stuff.No_Bomb;
end stuff;
Ludovic Brenta
No, the array is not in the heap in this case; it is in the executable
program's data segment. This may increase the size of the binary file.
To ensure that the array is on the heap, it is necessary to use an
access type and an allocator, e.g.:
type Float_Array_Access_Type is access Float_Array_Type;
x : Float_Array_Access_Type := new Float_Array_Type (1 .. 1_048_343);
--
Ludovic Brenta.
John B. Matthews
<ac4bed10-f655-4fa5...@k6g2000prg.googlegroups.com>,
Jerry <lance...@qwest.net> wrote:
I think so. When I ran into this some years ago, I was pleasantly
surprised at how easy it was to change over to heap allocation for my
largest data structure. Under Mac OS 9, such allocations fragmented the
heap, but Mac OS X behaves more reasonably.
The menace listed below allocates megabyte-sized blocks right up to the
limit of wired memory, as shown by top:
-----
with Ada.Text_IO;
procedure Heap_Test is
Megabyte : constant Positive := 1024 * 1024;
type Block is array (0 .. Megabyte - 1) of Character;
type Block_Ptr is access all Block;
BPtr : Block_Ptr;
N : Natural := 1;
begin
Ada.Text_IO.Put_Line("*** Heap test...");
while True loop
BPtr := new Block;
Ada.Text_IO.Put (N'Img);
N := N + 1;
end loop;
Ada.Text_IO.New_Line;
end Heap_Test;
-----
This horror raises STORAGE_ERROR at the `ulimit -s` you showed, but only
when compiled with -fstack-check:
-----
with Ada.Text_IO;
procedure Stack_Test is
Megabyte : constant Positive := 1024 * 1024;
type Block is array (0 .. Megabyte - 1) of Character;
procedure Allocate_Stack (N : Positive) is
Local : Block := (others => Character'Val(0));
begin
Ada.Text_IO.Put (N'Img);
Allocate_Stack(N + 1);
end;
begin
Ada.Text_IO.Put_Line("*** Stack test...");
Allocate_Stack(1);
Ada.Text_IO.New_Line;
end Stack_Test;
-----
For reference, ulimit is a bash built-in, so `man bash` for details:
<http://linux.die.net/man/1/bash>
--
John B. Matthews
trashgod at gmail dot com
<http://sites.google.com/site/drjohnbmatthews>
Gautier write-only
> But it seems that I will have to allocate memory for large objects
> using pointers (and thus take the memory from the heap). Is that
> right?
Seems so.
Funnily I had a similar surprise ~12 years ago.
It was with a reputable Ada 83 compiler (DEC Ada), on the university's
main server, with a very reputable system (OpenVMS).
I had declared: A: Matrix(m,n); somewhere, and the m's and n's were
enough for 500MB.
And told myself the system would be smart enough...
Build, run... oh, frozen - and everybody was going out of the offices:
"what happened with the mail ? what happened with... ?"
G.
tmo...@acm.org
> specifically) would be smart enough to allocate memory for large
> objects such as my long array in a transparent way so that I don't
> have to worry about it, thus (in the Ada spirit) making it harder to
> screw up. (Like not having to worry about whether arguments to
> subprograms are passed by value or by reference--it just happens.)
>
> But it seems that I will have to allocate memory for large objects
> using pointers (and thus take the memory from the heap). Is that
> right?
data set. It used Janus Ada and ran in a 2 GB Windows system. I thought
about switching to Gnat (for faster floating point) but discovered that
would require changing all large arrays to heap allocation, so I dropped
that idea. IMO, that's a ridiculous limitation in this day and age.
Warren
@k6g2000prg.googlegroups.com:
..
> So here's me being naive: I would have thought that Ada (or GNAT
> specifically) would be smart enough to allocate memory for large
> objects such as my long array in a transparent way so that I don't
> have to worry about it, thus (in the Ada spirit) making it harder to
> screw up.
Programmer's do still have some responsibility in
design, Ada or otherwise. ;-)
> But it seems that I will have to allocate memory for large objects
> using pointers (and thus take the memory from the heap). Is that
> right?
..
> Jerry
The stack is a valuable resource and sometimes you need to
be careful with allocation. I forget how gnat implements
tasks (threads) precisely, but they too require a stack. So
your virtual address space for stack must also be carved up
for each additional task. A 2G address space seemed huge 20
years ago, but it seems rather small these days.
With the new focus on parallel cores etc., I've often pondered
what a future CPU without a stack might look like. Imagine
a CPU that somehow in microcode was able to do a fast-malloc
of a stack frame (as part of a call), linking the new
frame back to the calling frame. Then you could eliminate
the need for a "linear virtual stack region" altogether. This
would allow code to freely fork into many parallel
threads without the issue of pre-allocating stack [address]
space to each new thread. When the called procedure executed
a "return", it would (in microcode) free the current stack
frame and return to the prior one. The "call" allocate/free's
could be constrained to one general "stack heap".
[..back from the day-dream..]
Generally speaking, large objects are best allocated
from the heap. But don't forget to release them once
finished. Otherwise you'll only feed from that trough
a few times also. ;-)
Warren
Warren
Adam Beneschan
>
> As for wishing that the compiler would automatically switch between heap
> and stack, that would probably be a terrible idea and render the
> language quite unsuitable for embedded systems.
Ummm, you're mixing two things there. The *language* is suitable for
embedded systems. The language does not dictate anything about where
things are stored, and nobody is asking for the Ada standard to
require that every Ada implementation be able to allocate every large
local array it sees. Not every compiler that compiles the language
will be suitable for embedded systems, but that's not a bad thing---
not everyone who uses an Ada compiler is using it for embedded
systems, and those that aren't shouldn't mind if their particular Ada
*implementation* does things in a way that's not suitable for embedded
systems.
I don't see a problem with a compiler generating code to allocate
arrays on the heap that it knows will be large, or supporting a pragma
that tells it to put arrays on the heap.
-- Adam
Maciej Sobczak
> With the new focus on parallel cores etc., I've often pondered
> what a future CPU without a stack might look like. Imagine
> a CPU that somehow in microcode was able to do a fast-malloc
> of a stack frame
There is no need to do that in microcode - the compiler decides what
does it mean to call a subprogram and what does it mean to allocate
the "frame", so you might simply have a compiler that implements these
concepts in terms of a dynamically allocated linked list.
I vaguely remember reading about a C compiler that did exactly that a
while ago - but I fail to find it in Google due to the noise from
billions of tutorials with stacks and lists. :-)
Today you might find this idea implemented in just about any
*interpreter*.
--
Maciej Sobczak * www.inspirel.com
YAMI4 - Messaging Solution for Distributed Systems
http://www.inspirel.com/yami4/
Robert A Duff
Well, yeah, but what IS the point of having a stack limit in the
first place? As opposed to a limit on virtual memory use,
whether it be stack or heap or whatever.
It's useful to limit a process to a certain amount of virtual address
space. It prevents that process from hogging the whole system. And
it prevents infinite-recursion bugs from causing thrashing. And it
prevents infinite-"new" bugs from causing the same.
It seems to me, a process should be allowed to allocate its
memory however it likes. If it is allowed to allocate (say)
2 gigabytes of address space, then it should be allowed to
allocate (say) half of that to the main thread's stack,
if it likes.
- Bob
Jerry
> Jeffrey Creem wrote on comp.lang.ada:
> > If you want the memory to come from the heap, you need to declare the
> > variables inside of packages instead of inside procedures. You can then
> > avoid using access types.
This (using a package) works. I made two Long_Float arrays of
100_000_000 length. The binary size is 292 KB (I included Text_IO)
regardless of the size of the arrays.
I'm not sure if I like the programming style better than using access
variables. I would have to essentially embed my entire program in a
package. And it seems to become impossible to selectively release
memory as is possible with access types. But it does solve my problem.
I programmed Pascal on Mac OS < 10 for years using pointers to arrays.
IIRC the stack was 32 KB.
Jerry
Jeffrey R. Carter
> while True loop
> ...
> end loop;
In Ada, we write infinite loops as
loop
...
end loop;
--
Jeff Carter
"Now go away or I shall taunt you a second time."
Monty Python & the Holy Grail
07
John B. Matthews
"Jeffrey R. Carter" <spam.jrc...@spam.acm.org> wrote:
> John B. Matthews wrote:
> > while True loop
> > ...
> > end loop;
>
> In Ada, we write infinite loops as
>
> loop
> ...
> end loop;
I suppose this is worse! ;-)
loop
...
exit when False;
end loop;
Ludovic Brenta
> Imagine
> a CPU that somehow in microcode was able to do a fast-malloc
> of a stack frame (as part of a call), linking the new
> frame back to the calling frame. Then you could eliminate
> the need for a "linear virtual stack region" altogether. This
> would allow code to freely fork into many parallel
> threads without the issue of pre-allocating stack [address]
> space to each new thread. When the called procedure executed
> a "return", it would (in microcode) free the current stack
> frame and return to the prior one. The "call" allocate/free's
> could be constrained to one general "stack heap".
I wonder how that would work with processors that have register
windows specifically to reduce the need for a memory-based stack (i.e.
SPARC, IA64 and maybe others).
--
Ludovic Brenta.
Brian Drummond
>But it seems that I will have to allocate memory for large objects
>using pointers (and thus take the memory from the heap). Is that
>right?
I believe so.
But code using the array can be written as normal, with a "my_array ... renames
my_array_ptr.all" clause to hide the pointer.
- Brian
Warren
@e7g2000yqf.googlegroups.com:
I'm not sure I follow, but objects like small buffers would
presumably still occupy the current stack frame.
Warren
Charmed Snark
@q23g2000yqd.googlegroups.com:
> On 18 Mar, 18:03, Warren <ve3...@gmail.com> wrote:
>> With the new focus on parallel cores etc., I've often pondered
>> what a future CPU without a stack might look like. Imagine
>> a CPU that somehow in microcode was able to do a fast-malloc
>> of a stack frame
>
> There is no need to do that in microcode - the compiler decides what
> does it mean to call a subprogram and what does it mean to allocate
> the "frame", so you might simply have a compiler that implements these
> concepts in terms of a dynamically allocated linked list.
True enough..
> I vaguely remember reading about a C compiler that did exactly that a
> while ago - but I fail to find it in Google due to the noise from
> billions of tutorials with stacks and lists. :-)
Interesting. Do you recall the time frame? Was it DOS era
or post Win95?
> Today you might find this idea implemented in just about any
> *interpreter*.
> Maciej Sobczak * www.inspirel.com
Even interpreters use stacks. A basic interpreter needs to
allocate a stack for each thread of control. However, as
you say, it could be implemented differently as a linked
list of "frames".
Warren
tmo...@acm.org
branching would occur for new threads. And there was a Texas Instruments
microprocessor that had no real stack, but rather a set of registers
that could be saved to RAM on a subroutine call.
These preceded DOS.
Simon Wright
The PDP-8 worked like that; the JMS instruction stored the return
address in the first word of the subroutine, execution started at the
next word.
JMS FOO / CALL SUBROUTINE FOO
FOO, +0 / STORED RETURN ADDRESS
...
JMP I FOO / RETURN
(we only had capitals)
--S
Jerry
wrote:
> On Wed, 17 Mar 2010 23:45:25 -0700 (PDT), Jerry <lancebo...@qwest.net> wrote:
> >But it seems that I will have to allocate memory for large objects
> >using pointers (and thus take the memory from the heap). Is that
> >right?
>
> I believe so.
> But code using the array can be written as normal, with a "my_array ... renames
> my_array_ptr.all" clause to hide the pointer.
>
> - Brian
Cool tip. Thanks.
Jerry
Warren
> tmo...@acm.org writes:
>
>> IIRC, the Burroughs 6500 had what they called a "cactus stack" where
>> branching would occur for new threads. And there was a Texas
>> Instruments microprocessor that had no real stack, but rather a set
>> of registers that could be saved to RAM on a subroutine call.
>> These preceded DOS.
>
> The PDP-8 worked like that; the JMS instruction stored the return
> address in the first word of the subroutine, execution started at the
> next word.
That was the way with the IBM-1130 as well. Not too
good for recursive calls though.
Warren
Jeffrey R. Carter
>
> But code using the array can be written as normal, with a "my_array ... renames
> my_array_ptr.all" clause to hide the pointer.
Generally, after something like
A : constant Array_Ptr := new Array_Type [(Low .. High)];
You can use A just as if it were the array itself:
B := A (A'First);
A (A'First) := A (A'Last);
A (A'Last) := B;
--
Jeff Carter
"If you think you got a nasty taunting this time,
you ain't heard nothing yet!"
Monty Python and the Holy Grail
23
Warren
> IIRC, the Burroughs 6500 had what they called a "cactus stack" where
> branching would occur for new threads.
Looking up "cactus threads" reveals that it is also
known as a "spaghetti stack". Looking at the picture on
wikipedia seems to suggest exactly what I envisioned.
http://en.wikipedia.org/wiki/Spaghetti_stack
> And there was a Texas Instruments
> microprocessor that had no real stack, but rather a set of registers
> that could be saved to RAM on a subroutine call.
> These preceded DOS.
Wasn't that the TMS9900? It also had this neat X instruction,
that could execute one instruction pointed by another register,
but control returned to the instruction following the X instruction
itself. Effectively execute and return from an arbitrary instruction.
I'm not sure what happens if X invoked a "call" instruction of
some sort.
Thanks for the lead on "cactus threads".
Warren
Warren
@q23g2000yqd.googlegroups.com:
> On 18 Mar, 18:03, Warren <ve3...@gmail.com> wrote:
>> With the new focus on parallel cores etc., I've often pondered
>> what a future CPU without a stack might look like. Imagine
>> a CPU that somehow in microcode was able to do a fast-malloc
>> of a stack frame
>
> There is no need to do that in microcode - the compiler decides what
> does it mean to call a subprogram and what does it mean to allocate
> the "frame", so you might simply have a compiler that implements these
> concepts in terms of a dynamically allocated linked list.
> I vaguely remember reading about a C compiler that did exactly that a
> while ago - but I fail to find it in Google due to the noise from
> billions of tutorials with stacks and lists. :-)
> Maciej Sobczak * www.inspirel.com
Were you thinking Cilk (or Cilk++)? I saw this in passing this
afternoon. One reference indicates a Cilk project existing since
'94 using gcc. Looks like good reading material for this weekend.
Warren
Adam Beneschan
Isn't that approximately how all computers used to work? Or at least
a lot of them? When I was first learning how computers ran, it seemed
like the common thing was for jump instructions to save the address of
the next instruction in a register, and then the subroutine code would
store the register somewhere---perhaps in the location containing the
return jump, since I don't remember those computers having "indirect
jump" instructions either. It was a few years before I encountered a
machine that automatically saved return addresses on a stack---or even
had a concept of a stack built into it.
One thing I remember was a particular computer's implementation of
COBOL (Honeywell 200/2000 series---my dad used to work for Honeywell
so those were some of the computers I was most familiar with). COBOL
used PERFORM statements for subroutine calls, and since you could have
PERFORM A in one part of the program and PERFORM A THROUGH B in
another part, or you could just fall into A without a PERFORM, the
paragraph A couldn't tell whether it was supposed to return to the
caller at the end of the paragraph. So the code, before it did a
PERFORM A, would do a "move" to move a branch instruction into a
location at the end of A that would branch back to the caller. Then,
after it returned, it would do another "move" to move a no-op to that
location.
Of course, none of these were good with recursion but I don't think
the experts understood the importance of recursion back then.
-- Adam
Jerry
<spam.jrcarter....@spam.acm.org> wrote:
> Brian Drummond wrote:
>
> > But code using the array can be written as normal, with a "my_array ... renames
> > my_array_ptr.all" clause to hide the pointer.
>
> Generally, after something like
>
> A : constant Array_Ptr := new Array_Type [(Low .. High)];
>
> You can use A just as if it were the array itself:
>
> B := A (A'First);
> A (A'First) := A (A'Last);
> A (A'Last) := B;
>
> --
> Jeff Carter
That is true when accessing array elements and slices but not for
indicating the entire array as a single entity such as in a subroutine
argument, e.g.
y := cos(x);
or
z := x + y;
with
function "+" (Left, Right : Real_Vector) return Real_Vector;
which is from ARM G.3.1.
Using access variables without renaming, the above line would have to
be written
y := cos(x.all);
and
z.all := x.all + y.all;
and in the algebraic-like calculations that are possible using
overloads the .all's clutter up the equation a bit. It's just a matter
of where you want to put the eye pollution. I have an extensive
library of subroutines including overloads that work with Real_Vector
and Complex_Vector from G.3 and either way (renaming or .all-ing)
works fine but on balance I think renaming is easier to read (and
probably easier to forget to release memory 8^( .)
Jerry
Jeffrey R. Carter
>
> That is true when accessing array elements and slices but not for
> indicating the entire array as a single entity such as in a subroutine
> argument, e.g.
>
> y := cos(x);
>
> or
>
> z := x + y;
That's true. I haven't done a lot of code like that recently, so not doing a
renames tends to work fine for me; hence "generally". (Also, I haven't messed
with big arrays much recently, either.) If you do reference the whole array
frequently, it's a different story.
--
Jeff Carter
"I wave my private parts at your aunties."
Monty Python & the Holy Grail
13
Björn Persson
> ulimit allows querying the hard limit of stack space
> ulimit -Hs
> which on OS X reports 65532 = 2^16 -4 kilobytes, about 67 MB. The user
> via ulimit can set the stack up to that size but not higher:
> ulimit -s 65532
> The default soft limit on OS X is 8192 kB, found by
> ulimit -s
I read this discussion as it happened, and thought it should be possible for
a program to raise its own soft stack limit up to the hard limit, without a
shell script wrapper. Today I stumbled on it. The POSIX system calls are
getrlimit and setrlimit. It should be possible to import them with the usual
methods for interfacing to C. Try "man 2 setrlimit" on any Unix-like system.
Maybe this helps someone.
--
Bj�rn Persson
PGP key A88682FD