Guidelines for using the AMPL Matlab API in parallel?

[at]

I have a project written in AMPL with the Matlab API in which I want to solve the same problem under many (thousands) different choices of parameters.

The slowest aspect of this is the time that it takes AMPL to write the model file. The solve time itself is relatively fast.

Is it possible/advisable to try to parallelize these thousands of solves using the Matlab parallel computing toolbox?

I have in mind something simple like using Matlab's parallel for loop where in each iteration of the loop I change parameters and issue another solve.

However I am concerned with how this might interact with the "ampl" object in the API. This object seems to be "global" (not sure of the right phrase) in the sense that changes to it made within functions remain in place once leaving that function. This seems like it might cause problems for parallelization?

Is this concern reasonable or am I misunderstanding something about how the AMPL Matlab API works? Are there any solutions to what I want to do? Could I somehow start many different ampl instances within Matlab (say ampl(1)--ampl(16), one for each core) and then make sure that each core only ever touches one of these objects?

[Victor Zverovich]

You can create multiple AMPL objects provided that your license permit running multiple instances of AMPL:

>> ampl1 = AMPL

>> ampl2 = AMPL

>> ampl1.read('problem1.ampl')

>> ampl1.solve

>> ampl2.read('problem2.ampl')

>> ampl2.solve

These objects can be used in parallel, but you shouldn't simultaneously access the same AMPL object from multiple threads. The easiest option is to have one AMPL object per thread and access it exclusively from this thread.

HTH,

Victor

--
You received this message because you are subscribed to the Google Groups "AMPL Modeling Language" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ampl+uns...@googlegroups.com.
To post to this group, send email to am...@googlegroups.com.
Visit this group at http://groups.google.com/group/ampl.
For more options, visit https://groups.google.com/d/optout.

[at]

Hi Victor,

Thanks for the response. However, I am having trouble getting something like this to work.

I attach a simple MWE that works with a standard "for" loop but not a "parfor" loop. Could you tell me what I am doing wrong?

The error I get is:

is not serializable
> In parallel.internal.pool.serialize (line 21)
  In distcomp.remoteparfor (line 77)
  In parallel_function>iMakeRemoteParfor (line 1060)
  In parallel_function (line 444)
  In parallel (line 8)
Error using AMPLBase/solve (line 297)
Struct contents reference from a non-struct array object.

Error in parallel (line 8)
parfor i = 1:1:10

[Victor Zverovich]

Unfortunately the error message is incomplete and I don't have the parallel toolbox installed on my machine, but it is probably caused by MATLAB trying to serialize an AMPL object for transfer to a worker. Serialization of AMPL objects is not supported by the API.

What could be possible is creating a separate AMPL session on each worker:

parfor i = 1:1:10

    ampl = AMPL;

    ampl.read('model.mod');

    ampl.setOption('solver', 'cplex');

    ampl.solve;

end

I haven't tried it though.

HTH,

Victor

[at]

Hi Victor,

Sorry, I pasted that error message incorrectly. The full message is the following repeated 6 times:

Warning: AMPLAPI version 1.0.0

AMPL Version 20151130 (Linux x86_64)

Licensed to --(redacted)--

Maintenance expires with version 20161130.

Using license file "/usr/local/bin/amplapi/matlab/../../ampl.lic".

 is not serializable 

> In parallel.internal.pool.serialize (line 21)

  In distcomp.remoteparfor (line 77)

  In parallel_function>iMakeRemoteParfor (line 1060)

  In parallel_function (line 444)

  In parallel (line 8) 

Followed by:

Error using AMPLBase/solve (line 297)

Struct contents reference from a non-struct array object.

Error in parallel (line 8)

parfor i = 1:1:10

Using your suggested code, i.e.

evalc('run /usr/local/bin/amplapi/matlab/setUp.m');

% Works with a "for" loop but not with a "parfor" loop

parfor i = 1:1:10

    ampl = AMPL;

    ampl.read('model.mod');

    ampl.setOption('solver', 'cplex');

    ampl.solve;

end

I get the following error:

Error using AMPL (line 21)

Undefined variable "com" or class "com.ampl.AMPL".

Error in parallel2 (line 5)

parfor i = 1:1:10

(So, the code in setUp.m seems to not be propagating to the individual workers?)

I have a single machine academic license from AMPL, which says: "Each single-machine license allows the product to be run by any number of users in any number of simultaneous processes, on one designated computer." (See http://ampl.com/files/2014%20AMPL%20Academic%20Price%20List.pdf)

So this shouldn't be a licensing issue.

Thanks,

Alex

[Victor Zverovich]

Hi Alex,

The error

> Undefined variable "com" or class "com.ampl.AMPL".

indicates that the environment (class path) hasn't been set up for using AMPL API. This is done by the setUp script, so I guess you need to run it from the loop too:

parfor i = 1:1:10

    setUp

    ...

end

HTH,

Victor

Visit this group at https://groups.google.com/group/ampl.

[at]

Hi Victor,

A script like that also raises an error. In particular:

clear all;

addpath('/usr/local/bin/amplapi/matlab/');

parfor i = 1:1:10

    setUp
    ampl = AMPL;

    ampl.read('model.mod');
    ampl.setOption('solver', 'cplex');
    ampl.solve;
end

leads to

Error using setUp (line 2)
Transparency violation error.
 See Parallel Computing Toolbox documentation about Transparency

Error in parallel2 (line 5)
parfor i = 1:1:10

Thanks,
Alex

[Victor Zverovich]

Looks like the transparency violation error can be fixed by wrapping the content of setUp in a function as done in the attached setUp.m. If this works for you we can apply this change in the next revision of the API.

I also got warnings

   Objects of com/ampl/AMPL class exist - not clearing java 

and was able to get rid of them by assigning 0 to the ampl reference:

parfor ii = 1:4

    setUp

    ampl = AMPL

    ...

    ampl = 0

end

HTH,

Victor

[cjosue]

Dear Victor,
Thanks for your help on this issue.
Where should the SetUp.m file should be placed?

I have a very similar code but haven't been able to make it work.

Thanks



--
View this message in context: http://ampl.996311.n3.nabble.com/AMPL-11074-Guidelines-for-using-the-AMPL-Matlab-API-in-parallel-tp11834p11945.html
Sent from the AMPL mailing list archive at Nabble.com.

[at]

Hi Victor,

Turning setUp into a function worked on my end. I didn't see the warnings you mentioned.

This approach to parallelizing AMPL seems like it will only really be beneficial if the different "ampl.solve" commands to be issued are for quite different models. Often, I want to just change a couple of parameters and then resolve. With this type of loop I then have to initialize the AMPL process, load the data, do the initial genmod/presolve phases, and do a cold solve.

What would be nice is if there is a way to have say

for i = 1:1:NCores
    ampl(i) = AMPL;
    ampl(i).read('model.mod');
end

parfor i = 1:1:BigNumber
    [change some aspects of ampl(currentcore)]
    ampl(currentcore).solve
end

Do you think something like that is feasible either in the current version of the API or as a modification to a future version?

Thanks,
Alex

[at]

The SetUp.m file is included in the matlab subdirectory of the main amplapi directory. So you just replace the SetUp.m that is already there with Victor's modification.

[Victor Zverovich]

Hi Alex,

This could be possible if MATLAB supported communication between workers. Then you could run multiple solves in a nested loop in parfor, something like:

for i = 1:1:NCores

    setUp

    ampl = AMPL

    ampl.read('model.mod');

    [while there are solve tasks in the queue]

      [pop task from the queue]

      [change some aspects of ampl(currentcore)]

      ampl.solve

      [do processing and, if necessary, push solve task to the queue]

    [end]

end

But I'm not an expert in MATLAB and don't know if such communication is possible.

HTH,

Victor

[sleiman...@gmail.com]

Hi Alex,

I am running into the exact same problem. I have modified the "setUp.m" as suggested by Victor. I even tried creating the ampl instances as cells. i.e. "ampl{i}" instead of "ampl(i)".

parfor i=1:N
    setUp
    ampl{i}=AMPL;
    ampl{i}.read([basef  '/' 'Di.mod']);
end

However, in both cases (with "ampl{i}" and with "ampl(i)"), I still get the same warning you were getting. Namely,

"Using license file "C:\Program Files\ampl\amplapi\lib\..\..\ampl.lic".
 is not serializable"

Have you managed to figure it out?

Thanks.

Sleiman

[at]

This works for me after wrapping setUp.m in a function declaration:

clear all;

addpath('/usr/local/amplapi/matlab/');

parfor i = 1:1:10

    setUp;

    ampl = AMPL;

    ampl.read('model.mod');

    ampl.setOption('solver', 'cplex');

    ampl.solve;

    ampl = 0;

end

[lopez.ca...@gmail.com]

Hello again.
As I reported earlier, using the modified Setup.m function provided above worked for me too. Today I wanted to run my code and got the following error:

Error using setUp (line 13)

Cannot find AMPL API in
C:\Users\ADMINI~1\AppData\Local\Temp\2\tp07e8c5a6_d161_4860_bd9f_60bfb0bc4d8crp10428\a\tpaaab5098_c17b_4756_b08d_734579b78d0f\..\lib\ampl-1.2.2.jar

Error in cont1stage (line 17)
parfor t = 1:et

Error in TestSubproblemDecomp (line 158)
 [KK1, KK2, KK3, W, tEnd2] =  cont1stage(K, GENS, LINES, BUSES,...

Will appreciate your advice.

[Victor Zverovich]

setUp looks up for the ampl.jar relative to the location of setUp.m file where it is defined, so the error message indicates that this file was moved to the temp directory for some reason.

You can add the following command

  which('setUp')

before the call to setUp to see where it is defined.

HTH,

Victor

--

[sleiman...@gmail.com]

Hi guys,

 

I am using AMPL on MATLAB through the AMPL API. In one of my applications I intend to call AMPL in a “parfor” (parallelizing calling AMPL). So I start by setting up AMPL on each worker (core) as follows:

“myCluster = parcluster('local');

poolsize=myCluster.NumWorkers;

parfor i=1:poolsize

    setUp;

end

“

And then I run “parfor” (parallel “for” on MATLAB) as follows

“

basef = fileparts(which('example'));

u=-1:0.1:2;

parfor i=1:length(u)

    ampl{i}= AMPL;

    ampl{i}.read([basef  '/' 'example.mod']);

    ampl{i}.getParameter('u').setValues(u(i));

    ampl{i}.setOption('solver' ,'gurobi');

    ampl{i}.solve();

    p(i)=ampl{i}.getValue('Objective');

    ampl{i}.close;

%     i

end

“

This works perfectly and it is much faster than a regular “for”. Notice that in this case there is only one parameter (‘u’) but in other instances where there are other fixed parameters on top of the ones that are changing inside "parfor", I find myself having to use dataframes to setup these fixed parameters inside the parfor. This however slows the performance significantly as the ampl structure for each subproblems is created every time from inside the for loop (at each core). Is there a way to use the AMPL API more efficiently with parfor? Like for instance forming the AMPL structures beforehand as in a regular for loop? As far as i know "parfor" does not allow being passed an array of ampl structures. Is there a way around this issue?

 

Thanks a lot.

Sleiman

[AMPL Support]

Dear Sleiman,

Can you send us also two more examples? We would like to see (1) an example where you have to use dataframes, and as a result performance is significantly slower, and (2) an example of what you would like to do but that is not permitted by parfor.

AMPL Support Services
am...@googlegroups.com

=======

[sleiman...@gmail.com]

Dear AMPL support services,

 

Thank you for your prompt reply. Here’s an example where I use dataframes to form the AMPL objects inside the parfor.

 

Example 1:

“

myCluster = parcluster('local');

poolsize=myCluster.NumWorkers;

parfor i=1:poolsize

    setUp;

end

 

for k=1:maxiter;

    parfor l=1:nlines

        amplline=AMPL;

        amplline.read([basef  '/' 'Line.mod']);

        amplline.getParameter('rho').setValues(rho(k));

        amplline.getSet('lines').setValues(l);

       

        dfline=DataFrame(1,'terminals','lambdap');

        dfline.setColumn('terminals',num2cell(T_line(l,:)'));

        dfline.setColumn('lambdap',lp(k));

        amplline.setData(dfline,'terminals');

 

  dfline=DataFrame(1,'lines_fr','Gf','Bf','Gsf','Bsf','thetad','Smax','Vminf','Vmaxf');

        dfline.setColumn('lines_fr',[l,T_line(l,:)]);

        dfline.setColumn('Gf',Gf(l));

        dfline.setColumn('Bf',Bf(l));

        dfline.setColumn('Gsf',Gsf(l));

        dfline.setColumn('Bsf',Bsf(l));

        dfline.setColumn('thetad',Anglediff(l));

        dfline.setColumn('Smax',Smaxt(l));

        dfline.setColumn('Vminf',Vminf(l));

        dfline.setColumn('Vmaxf',Vmaxf(l));

        amplline.setData(dfline,'lines_fr');

 

 

        amplline.setOption('solver' ,'knitro');

 

        amplline.solve();

        objectiveline=amplline.getValue('subproblem');

        exitflag=amplline.getObjective('subproblem').exitcode;

        amplline.close;

    end

end

“

This approach works but, as you can see, the AMPL objects “amplline” are created each time parfor called. And I reality these AMPL objects can be larger than this (and more dataframes are used to form them) which results in an large overhead for creating them (and this leads to a performance decline). Notice that the only parameter that is changing from one parfor call to another is “'lambdap',lp(k)”. However, if I can form these AMPL objects beforehand and then pass them to each worker through the parfor, the overhead for forming them would not carry over to each call of parfor. This can be easily done with a regular “for” (on one worker) where the AMPL objects “amplline” can be created beforehand as follows.

 

“

for l=1:nlines;

    amplline(l)=AMPL;

    amplline(l).read([basef  '/' 'Line.mod']);

    amplline(l).getParameter('rho').setValues(rho);

    amplline(l).getSet('lines').setValues(l);

   

    dfline(l)=DataFrame(1,'terminals' 'lambdap');

    dfline(l).setColumn('terminals',num2cell(T_line(l,:)'));

    dfline(l).setColumn('lambdap',lp(k));

    amplline(l).setData(dfline(l),'terminals');

   

    dfline(l)=DataFrame(1,'lines_fr','Gf','Bf','Gsf','Bsf','thetad','Smax','Vminf','Vmaxf');

    dfline(l).setColumn('lines_fr',[l,T_line(l,:)]);

    dfline(l).setColumn('Gf',Gf(l));

    dfline(l).setColumn('Bf',Bf(l));

    dfline(l).setColumn('Gsf',Gsf(l));

    dfline(l).setColumn('Bsf',Bsf(l));

    dfline(l).setColumn('thetad',Anglediff(l));

    dfline(l).setColumn('Smax',Smaxt(l));

    dfline(l).setColumn('Vminf',Vminf(l));

    dfline(l).setColumn('Vmaxf',Vmaxf(l));

    amplline(l).setData(dfline(l),'lines_fr');

end

 

for k=1:maxiter;

    for l=1:nlines

        amplline(l).getParameter('lambdap').setValues(T_line(l,:)',lp(k));

        amplline(l).setOption('solver' ,'knitro');

        amplline(l).solve();

        Objectiveline=amplline(l).getValue('subproblem');

    end

end

“

 

To summarize, what I desire to be able to do is the following:

 

Example 2:

“

myCluster = parcluster('local');

poolsize=myCluster.NumWorkers;

parfor i=1:poolsize

    setUp;

end

 

for l=1:nlines;

    amplline(l)=AMPL;

    amplline(l).read([basef  '/' 'Line.mod']);

    amplline(l).getParameter('rho').setValues(rho);

    amplline(l).getSet('lines').setValues(l);

   

    dfline(l)=DataFrame(1,'terminals' 'lambdap');

    dfline(l).setColumn('terminals',num2cell(T_line(l,:)'));

    dfline(l).setColumn('lambdap',lp(k));

    amplline(l).setData(dfline(l),'terminals');

   

    dfline(l)=DataFrame(1,'lines_fr','Gf','Bf','Gsf','Bsf','thetad','Smax','Vminf','Vmaxf');

    dfline(l).setColumn('lines_fr',[l,T_line(l,:)]);

    dfline(l).setColumn('Gf',Gf(l));

    dfline(l).setColumn('Bf',Bf(l));

    dfline(l).setColumn('Gsf',Gsf(l));

    dfline(l).setColumn('Bsf',Bsf(l));

    dfline(l).setColumn('thetad',Anglediff(l));

    dfline(l).setColumn('Smax',Smaxt(l));

    dfline(l).setColumn('Vminf',Vminf(l));

    dfline(l).setColumn('Vmaxf',Vmaxf(l));

    amplline(l).setData(dfline(l),'lines_fr');

end

 

for k=1:maxiter;

    parfor l=1:nlines

        amplline(l).getParameter('lambdap').setValues(T_line(l,:)',lp(k));

        amplline(l).setOption('solver' ,'knitro');

        amplline(l).solve();

        Objectiveline=amplline(l).getValue('subproblem');

    end

end

“

 

However, doing so results in the following error:

 

“

AMPL Version 20160803 (Intel icl EMT64 10.1.029, 64-bit; expires midnight 20170131 GMT)

License 57b64d6d-1 for ELEC4712 Honours Thesis, Gregor Verbic, University of Sydney

 is not serializable

> In parallel.internal.pool.serialize (line 21)

  In distcomp.remoteparfor/addInterval (line 103)

  In parallel_function>distributed_execution (line 780)

  In parallel_function (line 587)

  In ADMMtest (line 306)

 

Error using AMPL/getValue (line 82)

Struct contents reference from a non-struct array object.

 

Error in example1 (line 306)

    parfor l=1:n

”

 

Your support is greatly appreciated.

 

Sincerely,

 

Sleiman

[Malcolm Bean]

Hi All,

I would also like to implement parfor loops with AMPL+MATLAB API, was this issue ever fully resolved?

Thank You,

Malcolm

[AMPL Google Group]

Hi Malcolm,

The MATLAB API now works on top of our C++ API with a thin layer of Java in the middle (instead of a complete Java API in the middle), this should have reduced the overhead of having to create multiple objects but still does not allow direct use with with parfor.

--
Filipe Brandão
am...@googlegroups.com

{#HS:836060449-42020#}

[Carlos J. López-Salgado]

Hello.

It worked for me. I implemented a Benders decomposition scheme for a hydrothermal scheduling problem; the Benders sub-problems were solved in parallel.

Just don't have my laptop with me right now, but will gladly share the files by evening.

Regards,

Carlos J. López

--
You received this message because you are subscribed to a topic in the Google Groups "AMPL Modeling Language" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ampl/PtfBRMRdzkU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ampl+uns...@googlegroups.com.

[Malcolm Bean]

Carlos,

That would be awesome, thank you so much. I am running the old API (ampl-1.4.0.0.jar), do I need to update to 2.0.0-3 (C++ version)? 

Another random question, do you run both the AMPL instances and the linear/non-linear solver on the same local machine? I was hoping this would allow me to write the problem locally with API and the send all of the problem instances to Gurobi Cloud. Please let me know If I am misunderstanding what capabilities the AMPL + MATLAB API parfor combination.

Thanks Again,

Malcolm

To unsubscribe from this group and all its topics, send an email to am...@googlegroups.com.

[AMPL Google Group]

Thank you Carlos for this information, we did not make any changes to fix the serialization issue and we really appreciate if you could share with us how did you achieve this. The main changes made to the API between 1.4 and 2.0 were internal and not related to this.

Regarding being able to solve each problem with Gurobi Cloud, you just need to set two options (see, e.g., slide 82 of https://ampl.com/MEETINGS/TALKS/2018_04_Baltimore_Cloud.pdf), which is something that you can do with the API as follows:

ampl.setOption('solver', 'gurobi');
ampl.setOption('gurobi_options', 'cloudid=... cloudkey=...');

--
Filipe Brandão
am...@googlegroups.com

{#HS:836060449-42020#}

On Wed, Apr 24, 2019 at 5:18 PM UTC, Ampl Modeling Language <am...@googlegroups.com> wrote:

Carlos,

That would be awesome, thank you so much. I am running the old API (ampl-1.4.0.0.jar), do I need to update to 2.0.0-3 (C++ version)?

Another random question, do you run both the AMPL instances and the linear/non-linear solver on the same local machine? I was hoping this would allow me to write the problem locally with API and the send all of the problem instances to Gurobi Cloud. Please let me know If I am misunderstanding what capabilities the AMPL + MATLAB API parfor combination.

Thanks Again,
Malcolm

On Wed, Apr 24, 2019 at 4:49 PM UTC, Ampl Modeling Language <am...@googlegroups.com> wrote:

Hello.

It worked for me. I implemented a Benders decomposition scheme for a hydrothermal scheduling problem; the Benders sub-problems were solved in parallel.

Just don't have my laptop with me right now, but will gladly share the files by evening.

Regards,

Carlos J. López

On Wed, Apr 24, 2019 at 4:04 PM UTC, AMPL Google Group <am...@googlegroups.com> wrote:

Hi Malcolm,

The MATLAB API now works on top of our C++ API with a thin layer of Java in the middle (instead of a complete Java API in the middle), this should have reduced the overhead of having to create multiple objects but still does not allow direct use with with parfor.

--
Filipe Brandão
am...@googlegroups.com

You received this message because you are subscribed to the Google Groups "AMPL Modeling Language" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ampl+uns...@googlegroups.com.

[Carlos J. López-Salgado]

Malcolm,

I am attaching the necessary files. The main script is UB_BD_DetLossless.m and the "parfor" instruction is called within the UB24SubDet.m function.

The code is in a rather primitive state and I'm sure it can be improved. Any suggestion or recommendation would be appreciated. It was intended to work in a 28-core machine, but today I tested in a 2-core i5 processor and it worked.

I too am still using ampl-1.4.0.0.jar, but I believe it should behave the same with the newest version. And for the second question, that is right, I run both the AMPL instance and the solver on the same machine.

Best regards,

Carlos J.

To unsubscribe from this group and all its topics, send an email to ampl+uns...@googlegroups.com.

[Malcolm Bean]

Thanks again, I'll give an update once I'm able to adapt the code to my optimization.

[Malcolm Bean]

Just as an update, I got my code to run in parallel. It seems like most of the restrictions are due to MATLAB's parfor restrictions. My workaround is writing text data file(s) rather than using dataframes to feed data into my model.