rAMPL API letter "I" being deleted

[jaberdeen]

Hi, I'm using the rAMPL API to read in, use, and alter the members of a certain set. One of these members is simply the string "I" (uppercase "i" for clarity), as defined in the .mod file itself. However when reading this into R using "ampl$getSet("set_name")$getValues()" I get a list of every member of the set, bar "I", which is replaced by an empty string value. If I try to set this value to "I" in R (using $setValues()) it simply doesn't work. Having "I" as a member of a set simply doesn't pass through rAMPL API.

This seems like it's an obvious bug, so I figured I'd report it here, but if it's not a bug, or is unavoidable I'm curious what the reason is behind it, and what other issues of this nature exist.

Thanks

[Patrick OCallaghan]

Hi Rob and AMPL team,

I work with Josh. I've been enjoying using AMPL for the last few months. Really happy, but we're having issues with the rampl api (in addition to the apparent bug that Josh mentions).

For instance, after loading the dietmodel.R file using source() in R (not Rstudio), and calling the function:

> dietmodel(solver="knitro", modelDirectory="./rampl-examples/models")

I get the following standard output:

...

Knitro 13.0.1: Locally optimal or satisfactory solution.
objective 119.9897589; feasibility error 0
8 iterations; 0 function evaluations
Objective: 119.989759

...

So far so good. But then when want to access the parameter "n_min" via

> amplparent = ampl$getParameter("n_min")

I get:

Error in .External(list(name = "CppMethod__invoke_notvoid", address = <pointer: 0x0>,  :
  NULL value passed as symbol address

The odd thing is that invoking the same command via rstudio was working well for Josh earlier today.

So, in addition to Josh's question I would like to know answers to:

1. is there something wrong with the way I am using getParameter?

2. Is the following a reasonable workflow? (The nice thing about getParameter (followed by getValues)  is that these methods would allow me to access the full structure of the parameter that is in the mod file and set its values without doing much preprocessing in R.)

I would like to truncate the attached mod file just before line 830 "solving the model" and call it model.mod; read this mod file into R; getParameter, getValues, and set real values for parameters such as:

param RAW_CON_FLW {Regions, Sectors} default Uniform(UInf, USup) >= 0;

Finally I would like to run the loop that follows line 830 in a .run file.

The idea is that I can use existing R code for post processing of results and producing latex tables and figures, etc.

3. do we need to issue the command "update data" at any point?

Many thanks,
Patrick

[AMPL Google Group]

Hi Rob and Patrick,

Rob, regarding you question we were not able to reproduce it:

> library(rAMPL)
> ampl <- new(AMPL)
> ampl$eval("set I := 1..10;")
> ampl$getSet("I")$getValues()
I
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8
9 9
10 10

Could you please send us a runnable example?

Patrick, for 1., could you please send us the script you are running as well? Regarding 2., you can do that, and it is common to mix operations with AMPL commands with API actions like retrieving and updating data. Regarding 3, you need to issue that command if you want update the data in AMPL without using the API to do the update.

--
Filipe Brandão
am...@googlegroups.com

{#HS:1984344617-111506#}

[Patrick OCallaghan]

Hi Felipe,

For Josh's bug: try it with the file I attached earlier "maiwar.mod". On line 719, the set "Sectors" is defined via

set Sectors := "A, B, .., H, I,  .."

It is the set Sectors that we can't import to R without "loosing the "I"".

Regarding question 1: take a look at the rampl examples of the rampl api. In there, you will find a dietmodel.R file.

I produced this example with unmodified rampl example files provided by AMPL.

Alternatively, you could try reproducing my error using any of the parameters in my maiwar.mod file.

For instance, after reading in "maiwar.mod" truncated to line 830, try:

amplparent = ampl$getParameter("RAW_INV_SHR")

[Patrick OCallaghan]

So, just to clarify, how to reproduce my problem with the dietmodel.R file:

step 1:

>source("path/to/dietmodel.R")

>dietmodel(solver="knitro", modelDirectory="./rampl-examples/models")

> amplparent = ampl$getParameter("n_min")

Error in .External(list(name = "CppMethod__invoke_notvoid", address = <pointer: 0x0>,  :
  NULL value passed as symbol address

Not sure how else I can help you help me. I assume you have access to the dietmodel.R file. It comes with rampl.

On Saturday, August 20, 2022 at 2:31:18 AM UTC+10 AMPL Google Group wrote:

[Patrick OCallaghan]

Okay, I fixed question 1: it is simply because the entity doesn't exist, because it is only locally defined (within the function dietmodel). Once unwrapped, it works well within rstudio. (Will check whether I can reproduce in R)

[Patrick OCallaghan]

Okay, so I got rid of all default assignments to uniform distribution in maiwar.mod.

All such RAW_*_FLW parameters now default to 50e-2.

Now when I run

```

amplparentity = ampl$getParameter("RAW_INV_FLW")

amplparentity$getValues()

```

I still get nothing. "getInstances" on the other hand works quite well, but it is awkward to deal with:

```

a = amplparentity$getInstances()

 > a[1]
[[1]]
[[1]][[1]]
[1] "SEQ"

[[1]][[2]]
[1] Inf

[[1]][[3]]
[1] 0.5.

```

The whole point of this exercise is to pull in the parameter structure from ampl without many manipulations in R.

[Patrick OCallaghan]

PPS. note that the "Inf" in a[1] is a consequence of Josh's bug. (It should be an "I".)

On Saturday, August 20, 2022 at 12:21:59 PM UTC+10 Patrick OCallaghan wrote:

[Patrick OCallaghan]

PPPS: okay, I think I get the hang of multi-layered lists.

On Saturday, August 20, 2022 at 12:21:59 PM UTC+10 Patrick OCallaghan wrote:

[Patrick OCallaghan]

Hi Felipe and team,

I have your dietmodel example working fine in R (in addition to Rstudio). It seems that was indeed just the fact that the objects were only locally defined (within the dietmodel function).

I am still having trouble with my own model though. The R code I run is the following:

```

> rm(list=ls())

> library(rAMPL)
> ampl = new(AMPL)
> modeldir <- "../ampl/code/"; ampl$read(paste(modeldir, "gladmodel.mod", sep=""))
A [*] :=
;

> amplparentity = ampl$getParameter("RAW_INV_FLW")

> amplparentity$xref()
[1] "INV_FLW_RSUM"   "SHR_INV_CES"    "inv_sec_CD"     "inv_sec_CES"  
[5] "cinv_sec_CES"   "inv_sec"        "kap_transition"

> amplparentity$getValues()
Error in amplparentity$getValues() : No indices and no values retrieved.

>amplparentity$getInstances()

...

[[330]]
[[330]][[1]]
[1] "SEQ"

[[330]][[2]]
[1] "Q"

[[330]][[3]]
[1] "J"

[[330]][[4]]
[1] "Uniform(UInf,\n  USup)"

....

amplparentity$get()

Error in amplparentity$get() : Wrong number of indices used!

```

Note that "gladmodel.mod" is simply a truncated version of "maiwar.mod" the file I shared in an earlier post: upto line 830. In particular, note that "gladmodel.mod" includes line 828: "update data;"

So the entity is well-defined, but I think it has something to do with the fact that the uniform distribution values have not been assigned to the parameter at this stage of mod. I assume that it is only once we start solving?

So then I issue the command

```

ampl$solve()

```

and it works,

but then when I issue:

```

amplparentity$getValues()

```

My rstudio session crashes (see screenshot).

This is repeated when I start a new session and try again.

On Saturday, August 20, 2022 at 9:23:48 AM UTC+10 Patrick OCallaghan wrote:

[AMPL Google Group]

Thanks for reporting the issue with 'I' as member of a set. We found the issue and it will be fixed in the next release (that should come out tomorrow). It was affecting all APIs, not just rAMPL.

Regarding the issue with the dietmodel example, once an ampl object is closed or destroyed (e.g., due to being local to a function), operations on entity objects are not valid anymore since they reference the parent ampl object that was destroyed.

Regarding RAW_INV_FLW, getValues will return the values of the parameter if there were values assigned to it. In this case everything is at default. You can avoid that by setting the values for every entry with "let {r in Regions, s1 in Sectors, s2 in Sectors} RAW_INV_FLW[r, s1, s2] := Uniform(UInf, USup);" at the end of gladmodel.mod. Note, that you will need to replace sector I by something else like II until that issue is fixed (in line 719, you can put something like "set Sectors := A B C D E F G H II J K L M N PbSc P Q R T U;").

--
Filipe Brandão
am...@googlegroups.com

{#HS:1984344617-111506#}

On Sat, Aug 20, 2022 at 1:25 PM UTC, AMPL Modeling Language <am...@googlegroups.com> wrote:

PPS. note that the "Inf" in a[1] is a consequence of Josh's bug. (It should be an "I".)

On Sat, Aug 20, 2022 at 1:25 PM UTC, AMPL Modeling Language <am...@googlegroups.com> wrote:

Okay, so I got rid of all default assignments to uniform distribution in
maiwar.mod.
All such RAW_*_FLW parameters now default to 50e-2.
Now when I run
```

amplparentity = ampl$getParameter("RAW_INV_FLW")

amplparentity$getValues()
```
I still get nothing. "getInstances" on the other hand works quite well, but
it is awkward to deal with:
```
a = amplparentity$getInstances()
> a[1]
[[1]]
[[1]][[1]]
[1] "SEQ"

[[1]][[2]]
[1] Inf

[[1]][[3]]
[1] 0.5.
```
The whole point of this exercise is to pull in the parameter structure from
ampl without many manipulations in R.

On Sat, Aug 20, 2022 at 1:25 PM UTC, AMPL Modeling Language <am...@googlegroups.com> wrote:

PPPS: okay, I think I get the hang of multi-layered lists.

On Sat, Aug 20, 2022 at 2:22 AM UTC, AMPL Modeling Language <am...@googlegroups.com> wrote:

Okay, I fixed question 1: it is simply because the entity doesn't exist, because it is only locally defined (within the function dietmodel). Once unwrapped, it works well within rstudio. (Will check whether I can reproduce in R)

On Sat, Aug 20, 2022 at 2:22 AM UTC, AMPL Modeling Language <am...@googlegroups.com> wrote:

So, just to clarify, how to reproduce my problem with the dietmodel.R file:

step 1:
>source("path/to/dietmodel.R")

>dietmodel(solver="knitro", modelDirectory="./rampl-examples/models")

> amplparent = ampl$getParameter("n_min")

Error in .External(list(name = "CppMethod__invoke_notvoid", address =
<pointer: 0x0>, :
NULL value passed as symbol address

Not sure how else I can help you help me. I assume you have access to the
dietmodel.R file. It comes with rampl.

[AMPL Google Group]

We have just released rAMPL v2.0.11.0-20220823 fixing this issue. You can upgrade with:

> install.packages("https://ampl.com/dl/API/rAMPL.tar.gz", repos=NULL, INSTALL_opts=c("--no-multiarch", "--no-staged-install"))

--
Filipe Brandão
am...@googlegroups.com

{#HS:1984344617-111506#}

On Mon, Aug 22, 2022 at 3:58 PM UTC, AMPL Google Group <am...@googlegroups.com> wrote:

Thanks for reporting the issue with 'I' as member of a set. We found the issue and it will be fixed in the next release (that should come out tomorrow). It was affecting all APIs, not just rAMPL.

Regarding the issue with the dietmodel example, once an ampl object is closed or destroyed (e.g., due to being local to a function), operations on entity objects are not valid anymore since they reference the parent ampl object that was destroyed.

Regarding RAW_INV_FLW, getValues will return the values of the parameter if there were values assigned to it. In this case everything is at default. You can avoid that by setting the values for every entry with "let {r in Regions, s1 in Sectors, s2 in Sectors} RAW_INV_FLW[r, s1, s2] := Uniform(UInf, USup);" at the end of gladmodel.mod. Note, that you will need to replace sector I by something else like II until that issue is fixed (in line 719, you can put something like "set Sectors := A B C D E F G H II J K L M N PbSc P Q R T U;").

--
Filipe Brandão
am...@googlegroups.com

On Sat, Aug 20, 2022 at 1:25 PM UTC, AMPL Modeling Language <am...@googlegroups.com> wrote:

PPS. note that the "Inf" in a[1] is a consequence of Josh's bug. (It should be an "I".)

On Sat, Aug 20, 2022 at 1:25 PM UTC, AMPL Modeling Language <am...@googlegroups.com> wrote:

Okay, so I got rid of all default assignments to uniform distribution in
maiwar.mod.
All such RAW_*_FLW parameters now default to 50e-2.
Now when I run
```

amplparentity = ampl$getParameter("RAW_INV_FLW")

amplparentity$getValues()
```
I still get nothing. "getInstances" on the other hand works quite well, but
it is awkward to deal with:
```
a = amplparentity$getInstances()
> a[1]
[[1]]
[[1]][[1]]
[1] "SEQ"

[[1]][[2]]
[1] Inf

[[1]][[3]]
[1] 0.5.
```
The whole point of this exercise is to pull in the parameter structure from
ampl without many manipulations in R.

On Sat, Aug 20, 2022 at 1:25 PM UTC, AMPL Modeling Language <am...@googlegroups.com> wrote:

PPPS: okay, I think I get the hang of multi-layered lists.

On Sat, Aug 20, 2022 at 2:22 AM UTC, AMPL Modeling Language <am...@googlegroups.com> wrote:

Okay, I fixed question 1: it is simply because the entity doesn't exist, because it is only locally defined (within the function dietmodel). Once unwrapped, it works well within rstudio. (Will check whether I can reproduce in R)

On Sat, Aug 20, 2022 at 2:22 AM UTC, AMPL Modeling Language <am...@googlegroups.com> wrote:

So, just to clarify, how to reproduce my problem with the dietmodel.R file:

step 1:
>source("path/to/dietmodel.R")

>dietmodel(solver="knitro", modelDirectory="./rampl-examples/models")

> amplparent = ampl$getParameter("n_min")

Error in .External(list(name = "CppMethod__invoke_notvoid", address =
<pointer: 0x0>, :
NULL value passed as symbol address

Not sure how else I can help you help me. I assume you have access to the
dietmodel.R file. It comes with rampl.

[Patrick OCallaghan]

Hi Felipe, thanks for the update and fixing the "I" issue.

I don't understand how your response to the RAW_INV_FLW issue address my problem. In any case, I have a work around using get.Instances and I"m happy with my solution: no crashes, just some extra code to unpack multi-layered lists.

I am still not happy with the API though. I guess it is not R-specific. But I find it impractical to use. I don't like the errors I get: I find myself debugging an error such as 0/0 by attempting to create an .nl file. This way I get real ampl feedback with the name of parameter causing the problem not just "error can't evaluate 0/0".

Moreover, I find it a pain to access all the variables and parameters as everything requires two extra steps. (On the other hand, I'd also like to improve the ampl repl so that it the history of commands issued is recorded and searchable like a normal terminal.)

QUESTION: Is there a way to automatically instantiate all parameters in R: or is it a question of pulling them "manually" one-by-one?

QUESTION: I would like to use rampl to  to generate a mod file that contains all the updated parameter values that I have fed in using rampl. When I do so presently, I find that the values are not updated and I just get back an unhelpfully re-ordered version of mod file I read into R in the first place.

Best wishes,
Patrick

[AMPL Google Group]

Hi Patrick,

1. Regarding RAW_INV_FLW, depending on how you declare and initialize the parameter, there are three possibilities:

a) If you declare "param RAW_INV_FLW {Regions, Sectors, Sectors} default Uniform(0.4999, 0.5001) >= 0;", getValues will only retrieve values for entires that receive a value (i.e., if you don't assign any values, it will not return anything), and you are able to update individual entries:

> ampl <- new(AMPL)
> ampl$eval("set Regions;")
> ampl$eval("set Sectors;")
> ampl$getSet("Regions")$setValues(c("SEQ"))
> ampl$getSet("Sectors")$setValues(c("A", "B", "C", "D", "E", "F", "G", "H", "I", "J", "K", "L", "M", "N", "PbSc", "P", "Q", "R", "T", "U"))
>
> ampl$eval("param RAW_INV_FLW {Regions, Sectors, Sectors} default Uniform(0.4999, 0.5001) >= 0;")
> ampl$getParameter("RAW_INV_FLW")$getValues()[1:10,]
Error in ampl$getParameter("RAW_INV_FLW")$getValues() :

No indices and no values retrieved.

> # No values were retrieved here
> ampl$getParameter("RAW_INV_FLW")$set(c("SEQ", "A", "A"), 5.0)
> ampl$getParameter("RAW_INV_FLW")$set(c("SEQ", "A", "I"), 15.2)
> head(ampl$getParameter("RAW_INV_FLW")$getValues(), 10)
index0 index1 index2 RAW_INV_FLW
1 SEQ A A 5.0
2 SEQ A I 15.2
> # The values for entries that were set were retrieve here

b) If you declare "param RAW_INV_FLW {Regions, Sectors, Sectors} := Uniform(0.4999, 0.5001) >= 0;", all entries will be set in the declaration, getValues will return all of them, but you will not be able to update individual entries:

> ampl <- new(AMPL)
> ampl$eval("set Regions;")
> ampl$eval("set Sectors;")
> ampl$getSet("Regions")$setValues(c("SEQ"))
> ampl$getSet("Sectors")$setValues(c("A", "B", "C", "D", "E", "F", "G", "H", "I", "J", "K", "L", "M", "N", "PbSc", "P", "Q", "R", "T", "U"))
>
> ampl$eval("param RAW_INV_FLW {Regions, Sectors, Sectors} := Uniform(0.4999, 0.5001) >= 0;")
> head(ampl$getParameter("RAW_INV_FLW")$getValues(), 10)
index0 index1 index2 RAW_INV_FLW
1 SEQ A A 0.5000218
2 SEQ A B 0.4999380
3 SEQ A C 0.5000844
4 SEQ A D 0.5000914
5 SEQ A E 0.4999211
6 SEQ A F 0.5000428
7 SEQ A G 0.5000103
8 SEQ A H 0.4999526
9 SEQ A I 0.4999699
10 SEQ A J 0.4999814
> # assigning values to RAW_INV_FLW is not allowed if the values are difined in the declaration
> ampl$getParameter("RAW_INV_FLW")$set(c("SEQ", "A", "A"), 5.0)
Error in ampl$getParameter("RAW_INV_FLW")$set(c("SEQ", "A", "A"), 5) :
Error executing "let" command:


RAW_INV_FLW has an = assignment in the model.

c) If you declare "param RAW_INV_FLW {Regions, Sectors, Sectors} >= 0;" and use "let {r in Regions, s1 in Sectors, s2 in Sectors} RAW_INV_FLW[r, s1, s2] := Uniform(0.4999, 0.5001);" to assign the values, then getValues will retrieve all entries, and you will still be able to update individual entries:

> ampl <- new(AMPL)
> ampl$eval("set Regions;")
> ampl$eval("set Sectors;")
> ampl$getSet("Regions")$setValues(c("SEQ"))
> ampl$getSet("Sectors")$setValues(c("A", "B", "C", "D", "E", "F", "G", "H", "I", "J", "K", "L", "M", "N", "PbSc", "P", "Q", "R", "T", "U"))
>
> ampl$eval("param RAW_INV_FLW {Regions, Sectors, Sectors} >= 0;")
> ampl$eval("let {r in Regions, s1 in Sectors, s2 in Sectors} RAW_INV_FLW[r, s1, s2] := Uniform(0.4999, 0.5001);")
> ampl$getParameter("RAW_INV_FLW")$set(c("SEQ", "A", "A"), 3.14)
> head(ampl$getParameter("RAW_INV_FLW")$getValues(), 10)
index0 index1 index2 RAW_INV_FLW
1 SEQ A A 3.1400000
2 SEQ A B 0.4999380
3 SEQ A C 0.5000844
4 SEQ A D 0.5000914
5 SEQ A E 0.4999211
6 SEQ A F 0.5000428
7 SEQ A G 0.5000103
8 SEQ A H 0.4999526
9 SEQ A I 0.4999699
10 SEQ A J 0.4999814



2. Could you please provide an example where you get the "error can't evaluate 0/0" without context? Usually you should get context like the following:

> ampl <- new(AMPL)
> ampl$eval("param d; param p{i in 1..10} := i/d;")
> ampl$getParameter("d")$set(5)
> ampl$getParameter("p")$getValues()
index0 p
1 1 0.2
2 2 0.4
3 3 0.6
4 4 0.8
5 5 1.0
6 6 1.2
7 7 1.4
8 8 1.6
9 9 1.8
10 10 2.0
> ampl$getParameter("d")$set(0)
> ampl$getParameter("p")$getValues()
Error in ampl$getParameter("p")$getValues() :
Error executing "display" command:
error processing param p[1]:

can't compute 1/0


3. > Moreover, I find it a pain to access all the variables and parameters as everything requires two extra steps.

Could you please provide an example? If you mean having to write something like ampl$getParameter("p")$getValues(), there is also the option to write ampl$getData("p"), and you can even retrieve data for multiple variables and parameters simultaneously if they are indexed over the same set:

> ampl <- new(AMPL)
> ampl$eval("param p1{i in 1..10} := i;")
> ampl$eval("param p2{i in 1..10} := 2*i;")
> ampl$eval("param p3{i in 1..10} := 3*i;")
> ampl$getData("p1, p2, p3")
index0 p1 p2 p3
1 1 1 2 3
2 2 2 4 6
3 3 3 6 9
4 4 4 8 12
5 5 5 10 15
6 6 6 12 18
7 7 7 14 21
8 8 8 16 24
9 9 9 18 27
10 10 10 20 30

4. > QUESTION: Is there a way to automatically instantiate all parameters in R: or is it a question of pulling them "manually" one-by-one?

AMPL uses lazy evaluation, the parameters are instantiated when they are needed. If you want to trigger the instantiation in order to see if there are any errors, you can use ampl$eval("solexpand;") or ampl$eval("write gmod;") as these commands will trigger the instantiation of what is used in the model. Anything that is not being used in the model will still not be instantiated.

5. > QUESTION: I would like to use rampl to to generate a mod file that contains all the updated parameter values that I have fed in using rampl. When I do so presently, I find that the values are not updated and I just get back an unhelpfully re-ordered version of mod file I read into R in the first place.

The snapshot feature (https://dev.ampl.com/en/latest/features.html#snapshot-feature-save-and-restore-ampl-sessions) may be what you are looking for. You need to use x-ampl instead of regular ampl and you can do that in R as follows:

> ampl <- new(AMPL, new(Environment, "", "x-ampl"))

You can then us the snapshot command as follows to produce a model file with all the declarations and all the data:

> ampl$eval("snapshot > snapshot.mod;")

--
Filipe Brandão
am...@googlegroups.com

{#HS:1984344617-111506#}

On Thu, Aug 25, 2022 at 3:43 AM UTC, AMPL Modeling Language <am...@googlegroups.com> wrote:

Hi Felipe, thanks for the update and fixing the "I" issue.

I don't understand how your response to the RAW_INV_FLW issue address my problem. In any case, I have a work around using get.Instances and I"m happy with my solution: no crashes, just some extra code to unpack multi-layered lists.

I am still not happy with the API though. I guess it is not R-specific. But I find it impractical to use. I don't like the errors I get: I find myself debugging an error such as 0/0 by attempting to create an .nl file. This way I get real ampl feedback with the name of parameter causing the problem not just "error can't evaluate 0/0".

Moreover, I find it a pain to access all the variables and parameters as everything requires two extra steps. (On the other hand, I'd also like to improve the ampl repl so that it the history of commands issued is recorded and searchable like a normal terminal.)

QUESTION: Is there a way to automatically instantiate all parameters in R: or is it a question of pulling them "manually" one-by-one?

QUESTION: I would like to use rampl to to generate a mod file that contains all the updated parameter values that I have fed in using rampl. When I do so presently, I find that the values are not updated and I just get back an unhelpfully re-ordered version of mod file I read into R in the first place.

Best wishes,
Patrick

On Tue, Aug 23, 2022 at 10:57 PM UTC, AMPL Google Group <am...@googlegroups.com> wrote:

We have just released rAMPL v2.0.11.0-20220823 fixing this issue. You can upgrade with:

> install.packages("https://ampl.com/dl/API/rAMPL.tar.gz", repos=NULL, INSTALL_opts=c("--no-multiarch", "--no-staged-install"))

--
Filipe Brandão
am...@googlegroups.com

[Patrick OCallaghan]

Thanks Felipe, that's very helpful. I'll digest and provide some answers when I have more time.