MCB question

[andy]

Hi All,

I have a TNS C code as below and it is going to dynamically create a
long array with 87666218 times. Afterwards, it will put the sequential
integers into such array. the creation code "seq_array=(unsigned long
*)malloc(sizeof(unsigned long)*units_a);" can be passed and the
seq_array = 0. Afterwards, executing "seq_array[i]=1+i;", when i =
2887, the whole process will be aborted with below error. I know this
error is because the pointer corrupted the MCB pointer G[0] but I
don't know how to sovle it. Everytime the dynamically array creation
will give the initial address 0 to seq_array. The run-time env is
Operating System H06, Release H06.25.00. Please advise. Thank you so
much.

\P3DEV.$:0:931:3647952864 - *** Run-time Error 013 ***
\P3DEV.$:0:931:3647952864 - MCB pointer corrupt (%0)
\P3DEV.$:0:931:3647952864 - From INITIALIZESEQ + %30, UC.00

#pragma refaligned 2
#pragma page
#pragma strict
#pragma symbols, inspect, saveabend, save
#pragma heap 32 pages
#pragma wide
#pragma highpin
#pragma env common
#pragma xmem
#pragma extensions

#include "=SYSHDR^S_STDIOH"
#include "=SYSHDR^S_STDLIBH"
#include "=SYSHDR^S_STRINGH"
#include "=SYSHDR^S_MATHH"
#include "=SYSHDR^S_TIMEH"
#include "=SYSHDR^S_ERRNOH"

#define fixedArraySize 1000

static unsigned long *seq_array;

int INITIALIZESEQ(unsigned long buffer[],unsigned long
units_a,unsigned long units_b,
unsigned long loop_control);

int INITIALIZESEQ(unsigned long buffer[],unsigned long
units_a,unsigned long units_b,
unsigned long loop_control)
{
unsigned long i,reply_code;
reply_code = 0;
if (loop_control == 0)
{
seq_array=(unsigned long *)malloc(sizeof(unsigned
long)*units_a);
for (i=0; i< units_a; i++)
seq_array[i]=1+i;
}
return 0;
}

int main()
{
unsigned long i,result,units_a,units_b;
unsigned long p[fixedArraySize*5];
i=0;
units_a = 87666218;
units_b = 555555;
result = INITIALIZESEQ (p,units_a,units_b,i);
return 0;
}

Thanks
Andy

[Doug Miller]

andy <zbz...@gmail.com> wrote in news:b6b0a968-9ef1-4051-a11f-c8755c0a4f57
@r3g2000yqe.googlegroups.com:

malloc() can fail, and returns NULL if it does. Before using the value returned by malloc()
you should test it to make sure it isn't NULL before you try to use it:

if (seq_array == NULL) {
/* display some sort of failure message */
/* return some value other than 0 so the calling procedure knows the difference */
return -1;
}
else
{

[Doug Miller]

Doug Miller <doug_at_mil...@example.com> wrote in
news:XnsA1BA4A3F4EF...@78.46.70.116:

> malloc() can fail, and returns NULL if it does. Before using the value returned by malloc()
> you should test it to make sure it isn't NULL before you try to use it:

One more thing: you need to free() the memory you allocated with malloc() too, or your
program will have a memory leak -- in this case, a massive one, since you're allocating over
300 MB.

[Keith Dick]

Doug explained that you must check the result of malloc() to be sure it was able to find enough memory to satisfy the request. That is correct. Now, let's look at why malloc() was not able to satisfy the request in the code you supplied.

You specified

#pragma heap 32 pages

which says to limit the size of the heap, which is the area from which malloc() allocations memory, to 32 pages, which is 64K bytes (2K bytes per page). If you remove that line, you will allow malloc() to increase the size of the heap up to the maximum size available, which is about 127MB. (The sum of global data and heap is limited to 127.5MB.)

Your code is requesting to allocate 4 times 87666218 bytes, which is a bit over 350MB. So even if you allowed malloc() to increase the size of the heap to as large as it can become, this program will not be able to allocate such a large request as a TNS C program. If you can reduce units_a to under 30 million, you might be able to get it to run. As this program doesn't do anything useful, I assume it is an example derived from a more complex program. Depending on the data needs of the full program, you might have to reduce the equivalent of units_a even further.

If you can switch to using native-mode C (CCOMP on H-series systems), the size of the heap can become larger than the 127.5MB limit of TNS C. I don't remember the typical limit and don't see it in the C/C++ Programmer's Guide. I know I have seen a description of memory layout for native processes, but I don't remember right now where I saw it. Maybe in the conversion guide. Anyway, I'm pretty sure it would be possible to get 350MB in a native-mode C program. Whether the full memory requirements of your actual program can be met, I can't say, since I don't know how much more memory your actual program would need.

If you cannot switch to using native-mode C and cannot substantially reduce the amount of memory you use, then I think there is no solution for your problem.

[Robert Hutchings]

Maybe this would be a good candidate for SEGMENT_ALLOCATE_(). I think you can get up to 1 Gig of memory (flat, unaliased) with this system call...

[Keith Dick]

Robert Hutchings wrote:
> Maybe this would be a good candidate for SEGMENT_ALLOCATE_(). I think you can get up to 1 Gig of memory (flat, unaliased) with this system call...

The original example program is a TNS C program, and I believe TNS processes still are limited to 127.5 MB in an extended segment, so I believe that calling SEGMENT_ALLOCATE_() directly probably won't help solve the original problem. If I'm wrong and a TNS extended segment created with SEGMENT_ALLOCATE_() can exceed 127.5 MB in size, then using it might be a solution to the original problem posted.

If the real program from which the posted example was derived makes several requests for very large blocks of memory, there might be some effort needed to make sure the several segments don't end up with overlapping virtual addresses. At least in a native-mode program, I think SEGMENT_ALLOCATE_() will attempt to avoid allocating overlapping segments, but I think that there are some circumstances in which it cannot avoid it, depending on where other data gets placed in the data address range available to a native process. How that translates to a TNS process, I don't know.

From what I recall (no time right now to try to find it), all of the data areas a native process uses come out of a single range of virtual addresses, and I think the total space available in that range is only a gigabyte or so. I think the OS has half of the 4 GB virtual memory space to use for code and data, and the user process has the other half of the 4 GB virtual memory space to use, and the user half is evenly divided -- 1 GB for code and 1 GB for data. But the details of how things are divided up are dim in my mind, so I might have them wrong.

Whatever portion of that virtual memory is given to user data in a native-mode process includes anything allocated using SEGMENT_ALLOCATE_(). So, if you could use SEGMENT_ALLOCATE_() to get the memory needed by the original example program converted to native mode, I'm not sure it would be any easier or better than using malloc().

I recall seeing that NSK now supports some form of 64-bit addressing. I haven't read anything about it, but I imagine that means that processes using that new feature should be able to access much more memory than is needed by the original example program. However, I have no idea what you have to do to gain access to 64-bit addressing, and it certainly won't help a TNS program.

[andy]

On 5月8日, 下午7时16分, Doug Miller <doug_at_milmac_dot_...@example.com>
wrote:
> Doug Miller <doug_at_milmac_dot_...@example.com> wrote innews:XnsA1BA4A3F4EF...@78.46.70.116:

>
> > malloc() can fail, and returns NULL if it does. Before using the value returned by malloc()
> > you should test it to make sure it isn't NULL before you try to use it:
>
> One more thing: you need to free() the memory you allocated with malloc() too, or your
> program will have a memory leak -- in this case, a massive one, since you're allocating over
> 300 MB.

you are correct. that's just the script of this code and i forgot to
paste the free() words. thanks

[wbreidbach]

The size of an extended segment depends on the OS-version, for all H- and J-series releases it is 1536 megabytes, so this should not be the problem. For G-series it is different, 1120 MB for all segments in total.
If you wnat to see the available HEAP you can use PSTATE <process>, here an example for $ZPNS:

pstate $zpns,sta
PSTATE - T0705AAH (H01) (05NOV2011)
(C)1981 Tandem (C)2004 Hewlett Packard Development Company, L.P.
PSTATE for Guardian T06 on May 10, 2013 10:42:41.03 (LCT)
Target process: pin 538 in cpu 0 (32 bit), \BVPROD0.$ZPNS:404029783 (Primary)

......

Process memory values and other counters.
PFS Current 15256, Size 34032, Max 33554432
Main Stack Origin x70000000, Size 262144, Max 2097152
Priv Stack Origin x6DFE0000, Size 65536, Max 901120
Heap Origin x0802B000, Size 791314432, Max 1610436608
Globals Origin x08000000, Size 176128
KMSF Guarantee 0
Resident memory pages 65534
Messages Sent 2975488, Received 11590579
$RECEIVE Current QLength 0
Page faults 48388

You can see the available and the used memory.
Maybe this helps to check if you have a memory problem.

Just another idea: I am not a C but a Tal guy. When I started using extended storage I had a lot of problems when I used 2 byte integers as indexes. Changing to 4 byte integers solved all those problems immediately. I do not know if there might occur similar problems using C.

[Keith Dick]

Wolfgang, Do you know whether the sizes you mention are correct for TNS processes? I imagine you are correct for native processes, but the original question was for TNS C, and the current manual seems to indicate that the 127.5 MB limit still stands for TNS C programs. In another post, I pointed out that if he can switch to native-mode C, the available memory is much larger.

[wbreidbach]

I had a look at "Guardian procedure calls" at segment_allocate_ and the description does not mention a difference between TNS and TNSE.
Anyway, if possible, the OP should use native C as using the native compilers can improve performance significantly especially when dealing with large internal tables.