HTTP OPTIONS method to retrieve signatures of available functions

[Julius Chrobak]

Hello All,

This is a proposal for a new functionality. The idea is to be able to
retrieve signatures of all or a specific function in a structured form
from a running Bandicoot process. These signatures can for example be
valuable for applications which provide a dynamic UI to call Bandicoot
functions.

The proposed solution is to use the HTTP OPTIONS method on either the
"/" URI, the "*" URI or a specific function URI "/MyFunc". Independent
from the Request-URI the response body would provide the signature
details as a relation (i.e. standard CSV relation body) with the
following structure:

rel {
fname: string, # function name
method: string # either GET or POST
pidx: int, # parameter index: -1 - no params; 0 -
return; 1, 2, 3 … input parameters
prel: int # a flag 1/0 whether the parameter is a
part of a relational type or not
pname: string, # parameter name
ptype: string, # parameter type: int, real, long, string
}

Examples:
# function signature with no parameters
fn MyFunc()

# signature response
fname:string,method:string,pidx:int,prel:int,pname:string,ptype:string
MyFunc,GET,-1,0,,,

# function signature with primitive parameters only
fn MyFunc(param1: int, param2: real)

# signature response
fname:string,method:string,pidx:int,prel:int,pname:string,ptype:string
MyFunc,POST,1,0,param1,int
MyFunc,POST,2,0,param2,int

# function signature with mixed parameters
fn MyFunc(param1:int, param2: rel {a: int, b: long})

# signature response
fname:string,method:string,pidx:int,prel:int,pname:string,ptype:string
MyFunc,POST,1,0,param1,int
MyFunc,POST,2,1,a,int
MyFunc,POST,2,1,b,long

# function signature with mixed parameters and result type
fn MyFunc(param1: int, param2: rel {a: int, b: long}): rel {a: real}

# signature response
fname:string,method:string,pidx:int,prel:int,pname:string,ptype:string
MyFunc,POST,0,1,a,real
MyFunc,POST,1,0,param1,int
MyFunc,POST,2,1,a,int
MyFunc,POST,2,1,b,long


If the Request-URI is "/" or "*" the response body would contain
signatures of all available functions. If the Request-URI is a
specific function URI the response would only provide the signature of
the specified function or an empty response would be provided if the
function does not exists.

Feel free to comment on this proposal.

Regards,
Julius

[Julius Chrobak]

I have pushed the implementation of this functionality to a new remote
branch 'origin/signatures'.

Feel free to review and comment on it.

Regards,
Julius

[Ostap Cherkashin]

hello

i've done the code review and now i see what you mean with "too many transformations", it is indeed. for each OPTIONS request:

    * an iteration through all functions assembling Signature objects

    * transforming Signatures into a Rel object (iteration)

    * unpacking the Rel object (iteration)

    * freeing the Rel object (iteration)

    * freeing the Signatures (iteration)

i think there is an easy way to solve this by adding "extern char *pack_func(Func *fn)" into the pack module. this way there is no need to create intermediate relations (which are anyways used only for this purpose).

here are all of my comments and suggestions (in a random order):

1.) avoid unnecessary transformations, e.g. with pack_func()

1.a) error codes can be handled with pack_err()

2.) OPTIONS requests are not that rare in CORS (cross-origin resource sharing) environments. by producing the signatures back we are creating unnecessary workload for both bandicoot and a browser. i was thinking about using the GET instead. we have a reserved keyword "fn" so the URLs could be structured as follows:

http://hostname/fn          - interface description for all functions

http://hostname/fn/Hello    - interface description for the function Hello

3.) env_signatures is called for each OPTIONS request and never freed.

4.) http_opts can perform the write in two steps (headers and body) thus avoiding the allocation of another response body.

5.) patch contains trailing spaces

6.) there is no need for signature_append() to be external

7.) simplifying the result type:

7.a) we could remove the "method" attribute. as it was pointed out on the bandilab.org comments, we inappropriately use GET for those function calls which have side-effects. on the other hand exposing the information whether a function has side effects or not might not be such a good idea either, so the simplest thing could be to switch to POST only. and meanwhile we can use the same logic for determining which method to use as when we use curl :-)

7.b) parameter indexes are unnecessary. http calls are only concerned with the parameter names rather than their order (the same applies to relations: attribute names rather than their order). here is how the result can be structured:

fname - function name

pname - parameter name

pattr - parameter attribute

ptype - parameter type (or attribute type if pattr is not "")

i adjusted your examples accordingly:

# function signature with no parameters

fn MyFunc()

# signature response

fname string,pname string,pattr string,ptype string

MyFunc,,,

# function signature with primitive parameters only

fn MyFunc(param1: int, param2: real)

# signature response

fname string,pname string,pattr string,ptype string

MyFunc,param1,,int

MyFunc,param2,,int

# function signature with mixed parameters

fn MyFunc(param1:int, param2: rel {a: int, b: long})

# signature response

fname string,pname string,pattr string,ptype string

MyFunc,param1,,int

MyFunc,param2,a,int

MyFunc,param2,b,long

# function signature with mixed parameters and result type

fn MyFunc(param1: int, param2: rel {a: int, b: long}): rel {a: real}

# signature response

fname string,pname string,pattr string,ptype string

MyFunc,param1,,int

MyFunc,param2,a,int

MyFunc,param2,b,long

MyFunc,return,a,real

[Julius Chrobak]

I've updated the origin/signatures branch to fix the "too many transformations" problem. The signatures of functions are now returned using a GET method as suggested in the 2nd point below. I have also modified the result type as follows:

fname - function name (string)

pname - parameter name (string)

ppos - parameter position (int)

pattr - parameter attribute (string)

ptype - parameter (or attribute) type (string)

The position of the parameters will be required in the future to be able to implement function calls within Bandicoot language.

I have not change the packing of the error codes. This is still done using the relation structure.

Feel fee anyone to checkout the branch and let me know what you think. 

To test just do:

./ctl pack

./bin/bandicoot start -p 12345 -d bin/volume -s bin/volume/state -c test/test_defs.b

curl http://localhost:12345/fn

or

curl http://localhost:12345/fn/test_sign1

Regards,

   Julius

[Ostap Cherkashin]

it is much better and i can even check on interface definitions right from the browser :-) i am fine with it going to master and leaving (1.a) and even my further comments to (7.b) aside. here is a brief summary of comments from the previous email:

closed (done or not applicable): 1.) 2.) 3.) 4.) 6.) 7.a)

not-done: 1.a) 5.)

7.b) so that everyone understands the language passes parameters via positions (indexes) rather than parameter names, and in order to be able to use the same interface description (provided by /fn* URLs) within the bandicoot we need to extend it with the parameter positions. on the other hand, currently, it is not clear whether /fn/* interface is a good fit for this. therefore i suggest to leave positions out for now and introduce them along with the corresponding bandicoot functionality. i don't see any serious compatibility harm with this approach. also, if this comment is applied point (9) can be avoided all together.

further comments:

8.) http reply for signatures (and bodies) can be simplified:

--- from:

+            status = http_200(io);

+            while (status == 200 &&

+                    (len = pack_fn2csv(fns, cnt, res, MAX_BLOCK, &i)))

+                status = http_chunk(io, res, len);

+

+            if (status == 200)

+                status = http_chunk(io, NULL, 0);

--- to:

status = http_200(io);

while (status == 200) {

    int len = pack_fn2csv(fns, cnt, res, MAX_BLOCK, &i);

    status = http_chunk(io, res, len);

    if (len == 0)

        break;

}

---

9.) keeping relational and primitive parameters in one place, spreads some ugly code all over the place, e.g.:

---

+        Rel *rp = NULL;

+        int rpidx = fn_rpidx(fn);

+        if (rpidx > -1)

+            rp = fn->p.rels[rpidx];

---

+        int pplen = (rp == NULL) ? fn->p.len : fn->p.len - 1;

---

+            if (fn->p.rels[i] != NULL)

+                continue;

---

+extern int fn_rpidx(Func *fn)

+{

+    for (int i = 0; i < fn->p.len; ++i)

+        if (fn->p.rels[i] != NULL)

+            return i;

+

+    return -1;

+}

---

9.a) it can be avoided by putting the parameter position within the parameter declaration and leaving relational and primitive parameters separate.

9.b) parameter indexes depend on how the attributes are reduced by the language, would be nice to see a test case for this (e.g. two functions with parameters in the opposite order).

10.) this can fit on two lines and become more readable

---

+        char *names[] = {"fname",

+                         "pattr",

+                         "pname",

+                         "ppos",

+                         "ptype"};

+        Type types[] = {String,

+                        String,

+                        String,

+                        Int,

+                        String};

---

11.) since the signatures result is a string it makes sense to swap "pattr" with "pname" to make it more readable

12.) for some reason one test case is commented out:

---

-    if (http_opts(bad_io) != -200)

+    /*

+    if (http_opts(bad_io, NULL) != -200)

         fail();

+        */

---

- ostap

[Julius Chrobak]

Another round of clean up is done. I've implemented the following:

7) + 9) I removed the ppos from the the result of the /fn/* call, however I've implemented the point 9) including a language test to make sure the positions of parameters are equal to the order they are written in a source.

8) rewritten a bit

10) + 11) + 12) fixed

Julius

[Ostap Cherkashin]

i think it is ready to go to the master :-)

- ostap