PaPILO for PDLP and CP-SAT

232 views
Skip to first unread message

Stuart Rogers

unread,
Feb 6, 2023, 5:58:57 PM2/6/23
to or-tools-discuss
Are there plans to integrate PaPILO as a presolver for PDLP and CP-SAT? My understanding is that PDLP and CP-SAT use GLOP as a presolver, which may not be ideal for very large problems. 

Laurent Perron

unread,
Feb 7, 2023, 12:08:39 AM2/7/23
to or-tools-discuss
No. I do not see why. 

Glop presolve is a very small part of CP-SAT presolve. 

Le lun. 6 févr. 2023, 23:59, Stuart Rogers <stum...@gmail.com> a écrit :
Are there plans to integrate PaPILO as a presolver for PDLP and CP-SAT? My understanding is that PDLP and CP-SAT use GLOP as a presolver, which may not be ideal for very large problems. 

--
You received this message because you are subscribed to the Google Groups "or-tools-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to or-tools-discu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/or-tools-discuss/6a6c3434-2791-45a8-9bcd-f8a656ebfa7en%40googlegroups.com.

Stuart Rogers

unread,
Feb 7, 2023, 12:18:35 AM2/7/23
to or-tools-discuss
Are there plans to integrate PaPILO as an optional presolver for PDLP?

Laurent Perron

unread,
Feb 7, 2023, 12:33:13 AM2/7/23
to or-tools-discuss
Currently no. 

Do you have evidence it is doing a better job ?

There are not that many things to do when presolving as a lp, compared to as a mip. 

Stuart Rogers

unread,
Feb 7, 2023, 12:49:59 AM2/7/23
to or-tools-discuss
For very large problems, the GLOP presolver doesn't work, whereas PaPILO does. Also, PaPILO will presolve a MIP. Then PDLP can solve the LP relaxation of the presolved MIP.

Laurent Perron

unread,
Feb 7, 2023, 1:09:56 AM2/7/23
to or-tools-discuss
Please define 'does not work's for glop.

Stuart Rogers

unread,
Feb 7, 2023, 3:35:20 AM2/7/23
to or-tools...@googlegroups.com
With the GLOP presolver turned off, PDLP solved a very large LP in 3.5 hours. With the GLOP presolver turned on, PDLP was killed after about 15 minutes, probably during presolve. PDLP was able to solve the PaPILO-presolved LP in less time than 3.5 hours (this was several months ago so I don't have the timing).

You received this message because you are subscribed to a topic in the Google Groups "or-tools-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/or-tools-discuss/TTqSgFu2eXs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to or-tools-discu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/or-tools-discuss/CABcmEeZS4WUp-0aMFnUB1Tz%3DeWJUjhWA%3D0DqtLXyp%3DLXx-QFcg%40mail.gmail.com.

Laurent Perron

unread,
Feb 7, 2023, 3:37:39 AM2/7/23
to or-tools-discuss

David Applegate

unread,
Feb 7, 2023, 12:05:33 PM2/7/23
to or-tools-discuss
For PDLP, we do have some interest in supporting PaPILO, especially for very large problems. However, it would add quite a few dependencies to or-tools (Boost, TBB, Catch2, Fmt, Lusol, Pdqsort, Ska, at least, I believe). An alternative might be letting the user integrate with PaPILO (i.e., use PaPILO and send the presolved instance to PDLP). As is, that wouldn't support checking termination criteria on the original problem, but we've also been considering adding a termination check callback to better support custom termination criteria.

Stuart Rogers

unread,
Feb 7, 2023, 12:08:22 PM2/7/23
to or-tools-discuss
Below is the error when the GLOP presolver is turned on, using PDLP in OR-Tools v9.4. The input MPS file is 9.6 GB.

$ solve --solver=pdlp --logtostderr --linear_solver_enable_verbose_output --input=input.mps --params="termination_criteria{eps_optimal_relative:1e-3}, presolve_options{use_glop:true}" --num_threads=24
I0207 08:56:38.110951 186917 solve.cc:150] Read input proto as an MPModelProto.
File        : '/data/stuart/gen_MPS_files/p10_fix2_MILP_binary_x.mps'
I0207 08:56:38.117798 186917 solve.cc:181] Set number of threads to 24.
Solver      : PDLP_LINEAR_PROGRAMMING
Dimension   : 47261300 x 47251695
I0207 08:59:01.050074 186917 primal_dual_hybrid_gradient.cc:1858] Problem stats before presolving and rescaling:
I0207 08:59:01.053852 186917 primal_dual_hybrid_gradient.cc:1298] There are 47251695 variables, 47261300 constraints, and 141744517 constraint matrix nonzeros.
I0207 08:59:01.053862 186917 primal_dual_hybrid_gradient.cc:1304] Absolute values of nonzero constraint matrix elements: largest=1.000000, smallest=1.000000, avg=1.000000
I0207 08:59:01.053876 186917 primal_dual_hybrid_gradient.cc:1309] Constraint matrix, infinity norm: max(row & col)=1.000000, min_col=1.000000, min_row=1.000000
I0207 08:59:01.053882 186917 primal_dual_hybrid_gradient.cc:1314] Constraint bounds statistics (max absolute value per row): largest=10.000000, smallest=1.000000, avg=0.000315, l2_norm=122.425488
I0207 08:59:01.053888 186917 primal_dual_hybrid_gradient.cc:1331] Absolute values of objective vector elements: largest=12800.000000, smallest=20.000000, avg=3561.474902, l2_norm=30055161.602451
I0207 08:59:01.053895 186917 primal_dual_hybrid_gradient.cc:1338] Gaps between variable upper and lower bounds: #finite=47251695 of 47251695, largest=1.000000, smallest=1.000000, avg=1.000000
Killed

Stuart Rogers

unread,
Feb 7, 2023, 12:41:37 PM2/7/23
to or-tools-discuss
When the GLOP presolver is turned off, PDLP is solving the problem.

$ solve --solver=pdlp --logtostderr --linear_solver_enable_verbose_output --input=input.mps --params="termination_criteria{eps_optimal_relative:1e-3}, presolve_options{use_glop:false}" --num_threads=24
I0207 09:13:37.567702 186946 solve.cc:150] Read input proto as an MPModelProto.
File        : 'input.mps'
I0207 09:13:37.578476 186946 solve.cc:181] Set number of threads to 24.

Solver      : PDLP_LINEAR_PROGRAMMING
Dimension   : 47261300 x 47251695
I0207 09:16:01.121251 186946 primal_dual_hybrid_gradient.cc:1858] Problem stats before rescaling:
I0207 09:16:01.125003 186946 primal_dual_hybrid_gradient.cc:1298] There are 47251695 variables, 47261300 constraints, and 141744517 constraint matrix nonzeros.
I0207 09:16:01.125011 186946 primal_dual_hybrid_gradient.cc:1304] Absolute values of nonzero constraint matrix elements: largest=1.000000, smallest=1.000000, avg=1.000000
I0207 09:16:01.125023 186946 primal_dual_hybrid_gradient.cc:1309] Constraint matrix, infinity norm: max(row & col)=1.000000, min_col=1.000000, min_row=1.000000
I0207 09:16:01.125028 186946 primal_dual_hybrid_gradient.cc:1314] Constraint bounds statistics (max absolute value per row): largest=10.000000, smallest=1.000000, avg=0.000315, l2_norm=122.425488
I0207 09:16:01.125032 186946 primal_dual_hybrid_gradient.cc:1331] Absolute values of objective vector elements: largest=12800.000000, smallest=20.000000, avg=3561.474902, l2_norm=30055161.602451
I0207 09:16:01.125036 186946 primal_dual_hybrid_gradient.cc:1338] Gaps between variable upper and lower bounds: #finite=47251695 of 47251695, largest=1.000000, smallest=1.000000, avg=1.000000
I0207 09:16:02.928667 186946 primal_dual_hybrid_gradient.cc:1927] Problem stats after rescaling:
I0207 09:16:02.928694 186946 primal_dual_hybrid_gradient.cc:1298] There are 47251695 variables, 47261300 constraints, and 141744517 constraint matrix nonzeros.
I0207 09:16:02.928700 186946 primal_dual_hybrid_gradient.cc:1304] Absolute values of nonzero constraint matrix elements: largest=0.840896, smallest=0.010618, avg=0.300228
I0207 09:16:02.928707 186946 primal_dual_hybrid_gradient.cc:1309] Constraint matrix, infinity norm: max(row & col)=0.840896, min_col=0.076125, min_row=0.025097
I0207 09:16:02.928711 186946 primal_dual_hybrid_gradient.cc:1314] Constraint bounds statistics (max absolute value per row): largest=1.172895, smallest=0.117290, avg=0.000045, l2_norm=18.228729
I0207 09:16:02.928715 186946 primal_dual_hybrid_gradient.cc:1331] Absolute values of objective vector elements: largest=10763.474115, smallest=16.817928, avg=2994.831478, l2_norm=25273277.651371
I0207 09:16:02.928718 186946 primal_dual_hybrid_gradient.cc:1338] Gaps between variable upper and lower bounds: #finite=47251695 of 47251695, largest=11.046289, smallest=1.189207, avg=1.190133
 iter# kkt_pass   time | rel_prim_res rel_dual_res      rel_gap |   prim_resid   dual_resid      obj_gap |     prim_obj     dual_obj |  prim_var_l2  dual_var_l2
     0      0.0    1.9 |     0.998790      0.00000      0.00000 |      122.278      0.00000      0.00000 |      0.00000      0.00000 |      1.41421      0.00000
    64     70.0   55.8 |    0.0280624     0.618417    -0.253533 |      3.43558  1.85866e+07 -2.02184e+07 |  2.97642e+07  4.99826e+07 |      6.22323  1.26147e+07
   128    139.0   92.8 |    0.0160167    0.0536887   -0.0107858 |      1.96086  1.61362e+06     -654584. |  3.00175e+07  3.06721e+07 |      7.23091  1.21464e+07
   192    208.0  129.8 |   0.00668027    0.0184950   -0.0133601 |     0.817842      555872.     -829742. |  3.06381e+07  3.14679e+07 |      8.01165  1.11868e+07
   256    275.0  162.4 |   0.00400867    0.0129691  -0.00605125 |     0.490767      389790.     -375009. |  3.07986e+07  3.11736e+07 |      8.25760  1.10829e+07
   320    342.0  198.7 |   0.00263937    0.0101943   0.00568137 |     0.323129      306393.      351557. |  3.11153e+07  3.07637e+07 |      9.11567  1.10669e+07
   384    409.0  231.4 |   0.00336836   0.00784146   0.00508904 |     0.412376      235676.      315286. |  3.11346e+07  3.08193e+07 |      9.38245  1.10504e+07
   448    477.0  264.5 |   0.00384927   0.00711738   0.00657943 |     0.471253      213914.      406885. |  3.11244e+07  3.07175e+07 |      9.65453  1.10236e+07
   512    544.0  297.4 |   0.00416169   0.00653913   0.00755084 |     0.509501      196535.      466019. |  3.10918e+07  3.06258e+07 |      9.89131  1.09921e+07
   576    610.0  333.6 |   0.00507244   0.00568525   0.00967830 |     0.621001      170871.      590991. |  3.08272e+07  3.02363e+07 |      11.2322  1.08502e+07
   640    676.0  366.3 |   0.00493565   0.00507531   0.00942726 |     0.604254      152539.      573869. |  3.07236e+07  3.01497e+07 |      11.5011  1.08227e+07
   704    742.0  398.8 |   0.00475720   0.00468064   0.00900656 |     0.582407      140677.      546691. |  3.06229e+07  3.00762e+07 |      11.7567  1.07966e+07
   768    808.0  431.3 |   0.00455310   0.00445373   0.00820788 |     0.557419      133858.      496964. |  3.05220e+07  3.00251e+07 |      11.9973  1.07714e+07
   832    874.0  463.8 |   0.00432830   0.00430867   0.00728883 |     0.529898      129498.      440274. |  3.04221e+07  2.99818e+07 |      12.2291  1.07479e+07
   896    939.0  497.2 |   0.00290021   0.00375786   0.00196999 |     0.355062      112943.      117263. |  2.98211e+07  2.97038e+07 |      13.7708  1.06364e+07
 iter# kkt_pass   time | rel_prim_res rel_dual_res      rel_gap |   prim_resid   dual_resid      obj_gap |     prim_obj     dual_obj |  prim_var_l2  dual_var_l2
   960   1004.0  529.3 |   0.00262924   0.00353591  0.000869503 |     0.321888      106272.      51639.9 |  2.97209e+07  2.96692e+07 |      14.0495  1.06248e+07
  1024   1068.0  562.2 |   0.00192380   0.00296895  -0.00158507 |     0.235524      89232.2     -93436.9 |  2.94274e+07  2.95208e+07 |      15.0478  1.06000e+07
  1088   1133.0  594.4 |   0.00173793   0.00277025  -0.00202687 |     0.212769      83260.2     -119184. |  2.93415e+07  2.94607e+07 |      15.4405  1.05952e+07
  1152   1198.0  626.6 |   0.00159988   0.00265910  -0.00231884 |     0.195867      79919.8     -136055. |  2.92690e+07  2.94051e+07 |      15.8226  1.05910e+07
  1216   1264.0  659.3 |   0.00150470   0.00248462  -0.00262411 |     0.184216      74675.8     -153703. |  2.92098e+07  2.93635e+07 |      16.1956  1.05871e+07
  1280   1329.0  692.7 |   0.00143892   0.00211410  -0.00474291 |     0.176162      63539.5     -275809. |  2.89380e+07  2.92138e+07 |      18.2309  1.05725e+07
  1344   1394.0  725.0 |   0.00145242   0.00198114  -0.00476626 |     0.177814      59543.4     -276837. |  2.89029e+07  2.91797e+07 |      18.8052  1.05704e+07
  1408   1460.0  757.7 |   0.00141101   0.00188438  -0.00476242 |     0.172745      56635.3     -276316. |  2.88719e+07  2.91482e+07 |      19.3431  1.05680e+07
  1472   1526.0  790.6 |   0.00142003   0.00182156  -0.00485047 |     0.173849      54747.4     -281183. |  2.88445e+07  2.91257e+07 |      19.8766  1.05653e+07
  1536   1590.0  822.6 |   0.00146611   0.00176715  -0.00484904 |     0.179490      53111.9     -280867. |  2.88207e+07  2.91016e+07 |      20.3720  1.05626e+07
  1600   1656.0  855.4 |   0.00152361   0.00171531  -0.00474580 |     0.186530      51553.9     -274636. |  2.87973e+07  2.90719e+07 |      20.8793  1.05597e+07
  1664   1721.0  888.0 |   0.00157459   0.00167162  -0.00471112 |     0.192772      50240.9     -272414. |  2.87756e+07  2.90480e+07 |      21.3559  1.05569e+07
  1728   1787.0  920.8 |   0.00162125   0.00163333  -0.00479021 |     0.198484      49090.0     -276798. |  2.87537e+07  2.90305e+07 |      21.8406  1.05541e+07
  1792   1852.0  953.3 |   0.00166127   0.00160652  -0.00485793 |     0.203383      48284.1     -280516. |  2.87317e+07  2.90122e+07 |      22.3126  1.05513e+07
  1856   1918.0  987.2 |   0.00179957   0.00163835  -0.00576537 |     0.220315      49240.9     -332571. |  2.86759e+07  2.90084e+07 |      23.4478  1.05502e+07
 iter# kkt_pass   time | rel_prim_res rel_dual_res      rel_gap |   prim_resid   dual_resid      obj_gap |     prim_obj     dual_obj |  prim_var_l2  dual_var_l2
  1920   1983.0 1019.8 |   0.00201262   0.00150792  -0.00541294 |     0.246399      45320.9     -311831. |  2.86482e+07  2.89601e+07 |      24.4289  1.05489e+07
  1984   2048.0 1052.8 |   0.00217206   0.00142667  -0.00494481 |     0.265917      42878.7     -284495. |  2.86248e+07  2.89093e+07 |      25.3764  1.05474e+07
  2048   2113.0 1085.6 |   0.00229857   0.00136409  -0.00423993 |     0.281406      40998.1     -243566. |  2.86010e+07  2.88446e+07 |      26.2823  1.05458e+07
  2112   2177.0 1117.9 |   0.00241215   0.00130823  -0.00344102 |     0.295311      39319.0     -197366. |  2.85797e+07  2.87771e+07 |      27.1466  1.05442e+07
  2176   2241.0 1150.3 |   0.00252043   0.00124513  -0.00161565 |     0.308567      37422.7     -92430.4 |  2.85585e+07  2.86510e+07 |      27.9985  1.05425e+07
  2240   2305.0 1182.8 |   0.00262269   0.00118832 -0.000464597 |     0.321086      35715.2     -26530.1 |  2.85385e+07  2.85650e+07 |      28.8337  1.05407e+07
  2304   2371.0 1216.0 |   0.00271645   0.00114950  0.000266402 |     0.332565      34548.4      15190.7 |  2.85184e+07  2.85032e+07 |      29.6446  1.05389e+07
  2368   2436.0 1248.7 |   0.00281238   0.00110998  0.000938904 |     0.344310      33360.7      53461.8 |  2.84970e+07  2.84436e+07 |      30.5092  1.05369e+07
  2432   2502.0 1283.1 |   0.00313395   0.00104309   0.00445495 |     0.383678      31350.1      252344. |  2.84479e+07  2.81955e+07 |      32.5592  1.05361e+07
  2496   2568.0 1316.3 |   0.00374818  0.000890457   0.00800549 |     0.458876      26762.8      451508. |  2.84256e+07  2.79741e+07 |      34.4600  1.05352e+07
  2560   2634.0 1349.4 |   0.00416604  0.000829613   0.00921991 |     0.510034      24934.2      518918. |  2.84006e+07  2.78817e+07 |      36.2645  1.05341e+07
  2624   2699.0 1381.9 |   0.00462079  0.000781113   0.00999460 |     0.565707      23476.5      561673. |  2.83797e+07  2.78180e+07 |      37.8992  1.05329e+07
  2688   2765.0 1415.3 |   0.00502777  0.000744529    0.0105881 |     0.615532      22376.9      594292. |  2.83612e+07  2.77669e+07 |      39.4438  1.05316e+07

Stuart Rogers

unread,
Feb 7, 2023, 1:55:58 PM2/7/23
to or-tools-discuss
When the GLOP presolver is turned off, PDLP solved the problem.

$ solve --solver=pdlp --logtostderr --linear_solver_enable_verbose_output --input=input.mps --params="termination_criteria{eps_optimal_relative:1e-3}, presolve_options{use_glop:false}" --num_threads=24
......
Status      : MPSOLVER_OPTIMAL
Objective   : 2.705765382474077e+07
BestBound   :             inf
Iterations  : 9920
Time        : 5266

Stuart Rogers

unread,
Feb 7, 2023, 2:45:07 PM2/7/23
to or-tools-discuss
For that particular problem with the relative error set to 1e-3, PDLP solves the
Original problem with GLOP on: Killed during GLOP presolve
Original problem with GLOP off: in 5266 [s] in 9920 iterations
PaPILO-presolved problem with GLOP off: in 307.1 [s] in 2240 iterations
PaPILO-presolved problem with GLOP on: in 266.4 [s] in 1940 iterations

Stuart Rogers

unread,
Feb 8, 2023, 1:40:21 AM2/8/23
to or-tools-discuss
PaPILO is bundled with SCIP, which is provided with OR-Tools. Could OR-Tools, e.g., PDLP and CP-SAT, use the version of PaPILO bundled with SCIP? SCIP's license was updated to Apache 2.0 in November, starting with v8.0.3. https://scipopt.org/

watchdogs132

unread,
Feb 8, 2023, 4:14:09 AM2/8/23
to or-tools-discuss
From here it seems PaPILO has LGPL-3.0 license.  Reading the answer from here, it states " you can include Apache 2 licensed code in a LGPLv3 project, but not LGPLv3 licensed code in a Apache 2 project." This maybe an issue, but I'm not sure!.

Laurent Perron

unread,
Feb 8, 2023, 5:03:40 AM2/8/23
to or-tools-discuss
We are not using papilo in ortools/scip.
We cannot easily deliver an release artifact that contains papilo because of the licence.  

Stuart Rogers

unread,
Apr 21, 2025, 1:05:38 PMApr 21
to or-tools-discuss
It looks like a future version (3.0.0) of PaPILO will change the license from LGPL-3.0 to Apache License 2.0. The links below are from the develop branch of PaPILO:  https://github.com/scipopt/papilo/tree/develop

Laurent Perron

unread,
Apr 21, 2025, 1:21:58 PMApr 21
to or-tools...@googlegroups.com
Interesting.
Laurent Perron | Operations Research | lpe...@google.com | (33) 1 42 68 53 00



Stuart Rogers

unread,
Dec 3, 2025, 8:40:50 PM (3 days ago) Dec 3
to or-tools-discuss
PaPILO v3.0.0 has been released, which changed the license to Apache License 2.0. https://github.com/scipopt/papilo/releases/tag/v3.0.0

Laurent Perron

unread,
Dec 5, 2025, 7:57:02 AM (2 days ago) Dec 5
to or-tools...@googlegroups.com
Yes, still a nightmare to integrate.
Bazel has no support, I have not yet managed to make it work.
CMake is unfinished.



--
--Laurent

Reply all
Reply to author
Forward
0 new messages