Update on expr_dev modifications

[Bill Hart]

Following several days of expr_dev reviews, I wanted to give an update.  I think I've resolved all of the "bug" issues and "rename this" issues that we've discussed.

There are a variety of "documentation" issues that we've discussed that I have not resolved.  I have notes, and I'll work on those later this week.

I've also created tickets for issues/bugs that we've discussed that I didn't see an immediate resolution for.

--Bill

[Bill Hart]

I meant to add that there are a couple of outstanding pysp bugs, and in my "free" time I'm adding code coverage tests for pyomo.repn.  In particular, those tests have confirmed that the StandardRepn supports logic needed to get other pysp tests working.

I'm working with Gabe to resolve the remaining pysp tests, after which I think we're pretty much ready to merge this PR.

--Bill

[Nicholson, Bethany L.]

Here are the timing results for the gas network problem that I mentioned yesterday. These aren’t super extensive (just a single run) but I’m satisfied that there isn’t a significant performance change for large-scale PDAE-constrained problems. The orange rows are problem sizes in the original table that were too large to run on my VM.

 

Bethany

 

Pyomo Master, Python 3.5, Ipopt 3.12.4 with MA27
Scenarios	Nx	Variables	Constraints	Iterations	Creation (sec)	Solve (sec)	Read/Write + Solve (sec)
1	2	9686	9144	57	0.724	2.097	3.845
1	6	25718	25128	52	1.66	12.05	15.385
1	10	41750	41112	57	2.86	23.003	27.768
1	20	81830	81072	69	5.86	47.892	56.86
1	60	242150	240912	112	15.7	316.796	341.602
1	100	402470	400752	100	32.04	503.605	545.649
2	2	19371	18508	53	1.32	7.717	10.593
2	6	51435	50476	81	4.1	48.246	55.601
2	10	83499	82444	88	6.45	84.136	94.752
2	20	163659	162364	79	12.81	129.964	146.738
2	60	484299	482044	92	33.38	653.566	701.312
2	100	804939	801724
3	2	29056	27872	64	2.37	23.782	27.997
3	6	77152	75824	68	6.76	48.825	57.993
3	10	125248	123776	70	9.81	130.798	144.194
3	20	245488	243656	94	21.46	241.284	266.346
3	60	726448	723176
3	100	1207408	1202696
Expr-dev, Python 3.5, Ipopt 3.12.4 with MA27
Scenarios	Nx	Variables	Constraints	Iterations	Creation (sec)	Solve (sec)	Read/Write + Solve (sec)
1	2	9686	9144	57	1.19	2.371	3.728
1	6	25718	25128	52	1.78	11.251	14.775
1	10	41750	41112	57	4.71	22.421	27.467
1	20	81830	81072	69	5.78	54.791	64.697
1	60	242150	240912	118	17.32	344.29	372.26
1	100	402470	400752	95	28.26	464.604	510.018
2	2	19371	18508	69	1.28	8.336	10.954
2	6	51435	50476	81	4.61	47.819	54.944
2	10	83499	82444	88	7.06	83.702	95.48
2	20	163659	162364	82	10.66	144.382	162.894
2	60	484299	482044	94	35.36	719.137	773.251
2	100	804939	801724
3	2	29056	27872	50	1.96	18.649	22.492
3	6	77152	75824	67	5.99	50.585	61.174
3	10	125248	123776	74	10.27	132.122	147.935
3	20	245488	243656	94	20.09	234.289	236.662
3	60	726448	723176
3	100	1207408	1202696

--
You received this message because you are subscribed to the Google Groups "Pyomo Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-develope...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Gabriel Hackebeil]

Hey Bethany,

Since the solve times seem to vary by as much as 10% (which may or may not be indicative of a problem), can you show the last column as "Read/Write - Solve”?

[Nicholson, Bethany L.]

Sure, here it is:

 

Pyomo Master, Python 3.5, Ipopt 3.12.4 with MA27

Scenarios

Nx

Variables

Constraints

Iterations

Creation (sec)

Solve (sec)

Read/Write + Solve (sec)

Read/Write (sec)

1

2

9686

9144

57

0.724

2.097

3.845

1.748

1

6

25718

25128

52

1.66

12.05

15.385

3.335

1

10

41750

41112

57

2.86

23.003

27.768

4.765

1

20

81830

81072

69

5.86

47.892

56.86

8.968

1

60

242150

240912

112

15.7

316.796

341.602

24.806

1

100

402470

400752

100

32.04

503.605

545.649

42.044

2

2

19371

18508

53

1.32

7.717

10.593

2.876

2

6

51435

50476

81

4.1

48.246

55.601

7.355

2

10

83499

82444

88

6.45

84.136

94.752

10.616

2

20

163659

162364

79

12.81

129.964

146.738

16.774

2

60

484299

482044

92

33.38

653.566

701.312

47.746
2	100	804939	801724

3

2

29056

27872

64

2.37

23.782

27.997

4.215

3

6

77152

75824

68

6.76

48.825

57.993

9.168

3

10

125248

123776

70

9.81

130.798

144.194

13.396

3

20

245488

243656

94

21.46

241.284

266.346

25.062

3	60	726448	723176
3	100	1207408	1202696
Expr-dev, Python 3.5, Ipopt 3.12.4 with MA27

Scenarios

Nx

Variables

Constraints

Iterations

Creation (sec)

Solve (sec)

Read/Write + Solve (sec)

Read/Write (sec)

1

2

9686

9144

57

1.19

2.371

3.728

1.357

1

6

25718

25128

52

1.78

11.251

14.775

3.524

1

10

41750

41112

57

4.71

22.421

27.467

5.046

1

20

81830

81072

69

5.78

54.791

64.697

9.906

1

60

242150

240912

118

17.32

344.29

372.26

27.97

1

100

402470

400752

95

28.26

464.604

510.018

45.414

2

2

19371

18508

69

1.28

8.336

10.954

2.618

2

6

51435

50476

81

4.61

47.819

54.944

7.125

2

10

83499

82444

88

7.06

83.702

95.48

11.778

2

20

163659

162364

82

10.66

144.382

162.894

18.512

2

60

484299

482044

94

35.36

719.137

773.251

54.114
2	100	804939	801724

3

2

29056

27872

50

1.96

18.649

22.492

3.843

3

6

77152

75824

67

5.99

50.585

61.174

10.589

3

10

125248

123776

74

10.27

132.122

147.935

15.813

3

20

245488

243656

94

20.09

234.289

263.662

29.373

[Gabriel Hackebeil]

I’ve push fixes for the PySP-SMPS tools. This clears one major obstacle to merging.

PySP is still very broken, but I believe the issues are contained within a pretty small piece of code related to the advanced preprocessing that we do. This shouldn’t take long to fix, but I’m going to hold off until the Cplex and Gurobi direct / persistent interfaces are fully functional, as these interfaces drive a large portion of the logic in the PySP code that needs to be modified.

Gabe