More AIX weirdness

[Charlie Bursell]

I posted a little on this before and did not get much. Let me try
again with a bit more info

AIX 6.1 Tcl 8.4.12 (Old, I know)

Part of Script:
---------------------------------------------------------------
set fidx [open $file_in_idx r]
set idx [split [read -nonewline $fidx] \n]
close $fidx

while {![lempty $idx]} {
set x [lvarpop idx]
set offset [keylget x OFFSET]
set length [keylget x LENGTH]
set time [keylget x TIME]
etc.
-----------------------------------------------------------------

Error: unable to alloc 284792760 bytes
File size: 128427291 bytes

AIX memory > 16 GB

Ulimits all set to unlimited

Any ideas as to why Tcl cannot allocate enough memory? The same
script runs without problem on my 4GB laptop
Is there a compile flag or something peuliar to AIX?

Appreciate any help

Charlie

[Alexandre Ferrieux]

On Feb 4, 3:44 pm, Charlie Bursell <cburs...@geusnet.com> wrote:
> I posted a little on this before and did not get much.  Let me try
> again with a bit more info

Being your sole responder I do appreciate the comment.
Did you at least try what I suggested ?

>
> AIX 6.1  Tcl 8.4.12  (Old, I know)
>
> Part of Script:
> ---------------------------------------------------------------
> set fidx [open $file_in_idx r]
> set idx [split [read -nonewline $fidx] \n]
> close $fidx
>
>     while {![lempty $idx]} {
>            set x [lvarpop idx]
>            set offset [keylget x OFFSET]
>            set length [keylget x LENGTH]
>            set time [keylget x TIME]
>            etc.
> -----------------------------------------------------------------
>
> Error:   unable to alloc 284792760 bytes
> File size: 128427291 bytes
>
> AIX  memory > 16 GB
>
> Ulimits all set to unlimited
>
> Any ideas as to why Tcl cannot allocate enough memory?  The same
> script runs without problem on my 4GB laptop
> Is there a compile flag or something peuliar to AIX?

The difference between OSes is puzzling, assuming it is the exact same
version of TclX. Do you confirm ?

Anyway, the way you are popping values one by one off the head of your
huge list is bound to be inefficient. Do you have reasons to do that,
like a need to push back things from time to time ?
If not, please replace [lempty]+[lvarpop] with [foreach], and report
about the outcome. Indeed I suspect a leak in one of the TclX
primitives; this test could at least tell us about [lvarpop]'s
responsibility.

-Alex

[Charlie Bursell]

On Feb 4, 12:00 pm, Alexandre Ferrieux <alexandre.ferri...@gmail.com>
wrote:

> -Alex- Hide quoted text -
>
> - Show quoted text -

Thanks Alex, I do appreciate your responses. yes, I did try what you
suggested last time.

I do realize the code is very inefficient. It was puposely shown that
way to show that, even though there are sufficient memory resources,
Tcl cannot handle large in-memory strings on AIX. The question is why
can these large in-memory strings be handled on my small laptop but
not on a *MUCH* larger AIX box that uses the same version of Tcl,
TclX, etc? The problem does happen when using only core commands. I
still think there much be some weird with the way it was all compiled
or something like that.


Again, thank you for your help

Charlie

[Alexandre Ferrieux]

And never reported back !

> I do realize the code is very inefficient.  It was puposely shown that
> way to show that, even though there are sufficient memory resources,
> Tcl cannot handle large in-memory strings on AIX.  The question is why
> can these large in-memory strings be handled on my small laptop but
> not on a *MUCH* larger AIX box that uses the same version of Tcl,
> TclX, etc?  The problem does happen when using only core commands.  I
> still think there much be some weird with the way it was all compiled
> or something like that.

Again, you're dodging the questions. So please proceed with the test
I'm asking for.

-Alex

[schlenk]

AIX limits the maximum memory a program via a compiler option or
envvar, even if you have lots of memory available.

Lookup the LDR_CNTRL environment variable or the appropriate compiler
options (-bmaxdata).
http://www.john-dudley.co.uk/2009/09/15/aix-compiler-defaults-cause-me-way-too-much-grief/

Michael