Memory problems

[William Eveland]

Anyone,

I am running an aggregate command using a very large data set. I am using
SPSS for the VAX. Each time I run the command, I get the following error
message:

"Error #10963
There is not enough memory for all the cases in the aggregated file. The
aggregated file is missing some cases. Rerun with more memory. This
command not executed."

The output file tells me it was able to process about 42,000 cases. The
original system file contains about 180,000 cases. My question is, what
kind of "memory" does it need? My VAX tech support tells me that I have
ample disk space because I am using a scratch disk with "virtually
unlimited space".

Any ideas???

Thanks,

William Eveland
Associate Researcher
National Institute for Science Education

[Gloria Edwards]

I've run into a similar problem with SAS on the VAX, so the solution may
be similar. Disk space was not the problem--the problem was the amount of
memory allocated to the account, and the problem could not be solved
within SAS itself. Rather, I had to request that one of our VMS
programmers increase the page size quota for the account. If that doesn't
work by itself, you may also need to consider the virtual page count.

********************
Gloria Edwards
Instructional & Research Computing
235 Bryan Building
UNC Greensboro
(910)334-5350

[Bruce L. Peterson]

William Eveland wrote:
>
> Anyone,
>
> I am running an aggregate command using a very large data set. I am using
> SPSS for the VAX. Each time I run the command, I get the following error
> message:
>
> "Error #10963
> There is not enough memory for all the cases in the aggregated file. The
> aggregated file is missing some cases. Rerun with more memory. This
> command not executed."
>
> The output file tells me it was able to process about 42,000 cases. The
> original system file contains about 180,000 cases. My question is, what
> kind of "memory" does it need? My VAX tech support tells me that I have
> ample disk space because I am using a scratch disk with "virtually
> unlimited space".
>

I haven't seen any of my colleagues answer this so I will throw my $.02
in.

Actually the aggregate procedure is quite efficient in terms of memory.
However a sort is required. To minimize any chance of a #10963 error I
always do a Sort Cases on the Break Variables and then use the
/PRESORTED switch in aggregate.

With this procedure Aggregate runs in very little memory. This trick
works with any version of SPSS.

Good luck.

Bruce

--
Bruce L. Peterson * Internet bru...@spss.com
Consultant, Prof Services * Voice 312/329-3655
SPSS Inc * FAX 312/329-3668
Chicago IL * CompuServe 71303,3436

[Markus Quandt]

Bruce L. Peterson wrote:
>
> William Eveland wrote:
> >
> > Anyone,
> >
> > I am running an aggregate command using a very large data set. I am using
> > SPSS for the VAX. Each time I run the command, I get the following error
> > message:
> >
> > "Error #10963
> > There is not enough memory for all the cases in the aggregated file. The
> > aggregated file is missing some cases. Rerun with more memory. This
> > command not executed."
> >
> > The output file tells me it was able to process about 42,000 cases. The
> > original system file contains about 180,000 cases. My question is, what
> > kind of "memory" does it need? My VAX tech support tells me that I have
> > ample disk space because I am using a scratch disk with "virtually
> > unlimited space".
> >

This prpbably is a trivial suggestion, but did you think of the

set workspace=xxxx

statement?
This is usually at a default of 1024 or 512 kB, which serves fine most
of the time but NOT always (because some calculations can't be swapped
to disk, I think).
I don't know how SPSS on the VAX handles it but SPSS for UNIX and OS/2
have a command line option '-s' which serves the same purpose (for
command line operation, don't know how X-windows would behave). So to
run your analyses in 8 MB of physical memory you would enter (on UNIX):

spss -m -s8m inputfile >outputfile

where the '-m' option specifies command line/batch mode and the rest is
self explaining.
So try this or look for command line options on the VAX in your
manual...

HTH, M. Quandt

*****************************************************************
Markus Quandt e-mail: qua...@wiso.Uni-Koeln.DE
Institut fuer Angewandte Sozialforschung - Greinstr. 2
Universitaet zu Koeln - 50923 Koeln - GERMANY