CPLEX Error 1001: Out of memory.

[MatthiasW]

Hey everyone,

I'm currently trying to solve a very large scale MIP (currently 1.6
million decision variables).

In order to speed up the solution process, I allowed CPLEX to use
muliple threads. I set the Global thread limit to 4, as I'm working on
a machine with 4 CPUs. Parallel mode is set to 'opportunistic'.

Now, when I run the model, at some point, CPLEX terminates as it runs
out of memory:
CPLEX Error 1001: Out of memory.

Interestingly, this does not seem to be an issue of too little
physical memory. When I monitor the usage of my physical RAM, the
error message already occurs when there is plenty of RAM left that
could be filled.

Do you have any idea, what the reason for this might be? I played
around with node files and whatsoever but it didn't really help.


Thanks a lot in advance for your input, best
Matthias

[Luis Pinto]

Hello Matthias,

First thing that came to mind is: Are you running AIMMS/Windows on a 32-bits or 64-bits system?

A quick tip is to set the CPLEX memory emphasis option to 'on'. Check out project options.

Cheers,

Luis Pinto

www.unisoma.com.br

--
You received this message because you are subscribed to the Google Groups "AIMMS - The Modeling System" group.
To post to this group, send email to ai...@googlegroups.com.
To unsubscribe from this group, send email to aimms+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/aimms?hl=en.

[MatthiasW]

Hi Luis,

thanks for your comment! I'm running AIMMS on a 64-bits system. Why do
you think that this might be an issue?

Best
Matthias

On 26 Sep., 15:00, Luis Pinto <luisf...@gmail.com> wrote:
> Hello Matthias,
>
> First thing that came to mind is: Are you running AIMMS/Windows on a 32-bits
> or 64-bits system?
>
> A quick tip is to set the CPLEX memory emphasis option to 'on'. Check out
> project options.
>
> Cheers,
>
> Luis Pinto
>
> www.unisoma.com.br
>

> >http://groups.google.com/group/aimms?hl=en.- Zitierten Text ausblenden -
>
> - Zitierten Text anzeigen -

[MatthiasW]

Oh and thanks for the hint regarding memory emphasis. Setting CPLEX
memory emphasis option to 'on' doesn't entirely solve the problem but
it help a little, as the out of memory error now occurs a little
later.

Would you say that I might have less memory issues when running AIMMS
on a 32-bit system?

Cheers
Matthias

> > >http://groups.google.com/group/aimms?hl=en.-Zitierten Text ausblenden -
>
> > - Zitierten Text anzeigen -- Zitierten Text ausblenden -

[MatthiasW]

Just in case this helps understanding my problem, here are some
general indications of the size of the model:

Constraints: 645.613
Variables: 1.601.307 (960.200 integer)
Nonzeros: 5.125.013

Unfortnately, this is not the maximum model size I would like to work
with in the very end...

> > > >http://groups.google.com/group/aimms?hl=en.-ZitiertenText ausblenden -

[Leonardo Moraes]

Mathias,

what is the version of the CPLEX you are using to solve the problem?

AIMMS has some problems when using CPLEX 12.2 on a 64-bit Windows. If this is your case, as it was mine, I suggest you to try to solve your problem with CPLEX 12.1 or CPLEX 12.3, if available.

Best regards,

Leonardo.

2011/9/28 MatthiasW <mat...@gmx.net>

--
Enviado do meu gmail.

[MatthiasW]

Hi Leonardo,

I'm using CPLEX 12.3.

I have a general question to all of you:
If I run the Identifier Cardinality Tool and find certain identifiers
that use a lot of memory, what is the best thing to do with them?
I tried to remove these identifiers and include their definitions
directly into the definitions of the parameters that used to refer to
them. This approach seems to save up memory, however, computation time
suffers from it.

Often, these identifiers are in fact parameters whose values NEVER
change throughoput the optimization process. Is there maybe a way to
let CPLEX store them ONCE and GLOBALLY instead of storing them in
every single node of the solution process? As I said, these parameters
usually contain fixed values that are not affected by changes in the
decision avriables or whatsoever.


Best
Matthias

On 28 Sep., 12:24, Leonardo Moraes <lmorae...@gmail.com> wrote:
> Mathias,
>
> what is the version of the CPLEX you are using to solve the problem?
>
> AIMMS has some problems when using CPLEX 12.2 on a 64-bit Windows. If this
> is your case, as it was mine, I suggest you to try to solve your problem
> with CPLEX 12.1 or CPLEX 12.3, if available.
>
> Best regards,
>
> Leonardo.
>

> 2011/9/28 MatthiasW <matz...@gmx.net>

> Enviado do meu gmail.- Zitierten Text ausblenden -

[Luis Pinto]

Hello Matthias,

If I'm not mistaken (and please someone correct if I'm wrong) if you run AIMMS 32-bit, you have a windows limitation on memory.
We have optimised larger mathematical models (say dozens of million variables), but we used a 64-bit machine with 64-bit AIMMS and LOTS of ram.

Regarding the cardinality, it can help to limit your parameters domain. Also, check if the parameter has a high density, if so check out the default value and see if there is a better option (AIMMS doesn't use memory to store default values, so if you're parameter has more 1's than 0's, try changing the default to 1).

Lastly, you can try running the CPLEX Stub. It's basically the same as regular CPLEX (some cons) but it's called from outside of AIMMS, generating an independent process. This way, Windows can allocate memory for AIMMS and CPLEX independently. To do so, go to your solver select and configure screen (not project!!!) and add the cplex "dll" with the PS in the name. Then select it as default solver for your type of problem.

Hope it helps!

Cheers,

Luis Pinto

www.unisoma.com.br

[MatthiasW]

Hi Luis

thanks for your response!
When you say that there is a windows limitation on memory for the 32
bit AIMMS, does that mean it would not change anything if I increased
my physical memory because Windows won't let AIMMS access it?

Since I'm running a 64-bit Windows I will try and download the 64-bit
Version of AIMMS. I will also try out CPLEX Stub, thanks for this
advice!

I'm currently working with only 4 GB of RAM. Would you say that my
problem will become less severe if I uopgrade to 8 GB?

Best
Matthias

On 28 Sep., 14:28, Luis Pinto <luisf...@gmail.com> wrote:
> Hello Matthias,
>

> If I'm not mistaken (and please someone correct if I'm wrong) if you run
> AIMMS 32-bit, you have a windows limitation on memory.
> We have optimised larger mathematical models (say dozens of million
> variables), but we used a 64-bit machine with 64-bit AIMMS and LOTS of ram.
>
> Regarding the cardinality, it can help to limit your parameters domain.
> Also, check if the parameter has a high density, if so check out the default
> value and see if there is a better option (AIMMS doesn't use memory to store
> default values, so if you're parameter has more 1's than 0's, try changing
> the default to 1).
>
> Lastly, you can try running the CPLEX Stub. It's basically the same as
> regular CPLEX (some cons) but it's called from outside of AIMMS, generating
> an independent process. This way, Windows can allocate memory for AIMMS and
> CPLEX independently. To do so, go to your solver select and configure screen
> (not project!!!) and add the cplex "dll" with the PS in the name. Then
> select it as default solver for your type of problem.
>
> Hope it helps!
>
> Cheers,
>
> Luis Pinto
>
> www.unisoma.com.br
>

> >http://groups.google.com/group/aimms?hl=en.- Zitierten Text ausblenden -

[MatthiasW]

I switched to AIMMS x64 and the BIG difference is, that now, CPLEX
makes efficient use of teh virtual working memory that I defined on my
hard drive (which it didn't udner the x32 version).
However, AIMMS would not let me ad the CPLEX Stub Solver. It says it
was unable to find the corresponding module when I try to add the dll
of this CPLEX version.

It seems like the model might solve now without CPLEX terminating due
to a lack of memory. However, it takes AGES to solve. Would you expect
it to solve faster if I had more physical RAM?

Best
Matthias

> > >http://groups.google.com/group/aimms?hl=en.-Zitierten Text ausblenden -

[Luis Pinto]

Matthias,

I don't think the Stub is available for x64.

I would imagine it should take long! Its a huge model with lots of integer variables!
RAM will only help if windows is paging memory to your hard drive (when the current memory runs out).

Cheers,

Luis Pinto
www.unisoma.com.br

[Sergio Bruno]

Matthias,
Luís nailed it.there is no cplex stub in 64 bits because it would be pointless, since there is no memory limit on 64 bits.
The fast solution is to buy more memory, it is the cheapest thing to do (just think about the cost of one hour of your work x number of hours you will spend trying to figure out where to change your model to make less memory use).one alternative is to open the identifier cardinality tool (just before the solve statement) and check out what is taking more memory, maybe you have something easy to fix.
Best,
Sergio

Em 28/09/2011 11:42, "Luis Pinto" <luis...@gmail.com>escreveu:

Matthias,

I don't think the Stub is available for x64.

I would imagine it should take long! Its a huge model with lots of integer variables!
RAM will only help if windows is paging memory to your hard drive (when the current memory runs out).

Cheers,

Luis Pinto
www.unisoma.com.br

On 28 September 2011 11:16, MatthiasW <mat...@gmx.net> wrote:
>

> I switched to AIMMS x64 and the B...

--

You received this message because you are subscribed to the Google Groups "AIMMS - The Modeling Syst...

[Joned David Chica Llamas]

I am two short questions.

I have two sets; set1 of index i, and set2 of index j.   the set2 is a subset of set1. Now, I have a parameter Process and a variable X.

I need that Process and X have as index domain i or j.

Is the same: Process, index domain (i or j) and X index domain (i or j), because I have two constrains of the follow form:  Process(i) - Process (j) <= 1    and  X(i) -X(j) <= 0.

I NEED THAT PROCESS AND X TAKE THE INDEX DOMAIN I OR J.

Joned Chica

[Sergio Bruno]

Joned,

I guess that what you want is:

you have some set MySet with indexes m1 and m2 (you can declare more
than one index to refer to the set).
Then you will write your constraints as:
MyConstraint1(m1):
Process(m1) - Process (m2) <= 1

MyConstraint2(m1):
X(m1) -X(m2) <= 0.

[Guido Diepen]

Hi Wenyi,

which version of AIMMS are you using? Are you using the 32 bit or the 64 bit version of AIMMS?

Guido Diepen

AIMMS Specialist

On Saturday, March 23, 2013 11:39:42 AM UTC+1, Wenyi Chen wrote:

Hi Matthias,

"Interestingly, this does not seem to be an issue of too little 

physical memory. When I monitor the usage of my physical RAM, the 
error message already occurs when there is plenty of RAM left that 

could be filled."

I have encountered exactly the same problem as you did. Do you have an efficient way to solve it? Any what does it happen in the first place?

Thank you very much.

Wenyi

[Wenyi Chen]

Hi Guido,

 

I am using the 64 bit version of CPLEX 12.5. I code in java and call cplexqp from there.

My OS is Windows Server 2008 R2 standard. According to the gory details, the memory limit on X64 is 32GB. I allocated 3G memory to eclipse (64 bit) for the program but from the profiling result attached, the memory it used is just a tip of iceberg, isn't it?

Wenyi

--
You received this message because you are subscribed to a topic in the Google Groups "AIMMS - The Modeling System" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/aimms/uWd5S6R8O2c/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to aimms+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Wenyi

[Guido Diepen]

Hi Wenyi,

Note that this is an AIMMS forum and not a specific cplex forum :)

Not really sure what is causing the problem in your case when directly calling CPLEX from your Java program.

The only things I can think of are:

* Check if CPLEX is having issues with solving the rootnode or whether the problem occurs after the rootnode has been solved

* Try to set some memory related cplex settings

Guido Diepen

AIMMS Specialist