Curious about history/reasoning behind find_exposed_functions mechanism

[Brendan Barnwell]

I recently dug into web2py and discovered how it decides which controller functions are exposed as endpoints.  I found that it does this with the find_exposed_functions function in compileapp, which uses a regex to textually search the source code of the controller file for lines of the form "def somefunc():" (at the beginning of the line).  I'm curious about the history and reasoning behind this decision.

To be frank, I was quite shocked to discover this.  It means that the determination of which functions are exposed is not programmatically manipulable -- that is, there is no way to use Python logic to expose things based on attributes, or to expose other callable objects, etc.

So I assume there must be some reason why things were done this way.  What are the advantages of determining function exposure textually, rather than executing the controller file in a namespace and then inspecting its live Python objects to determine which ones to expose?

[Anthony]

A web2py controller is not like a Python module. It is meant to be executed in a prepared environment to generate the response to an HTTP request. Because the top level of the controller may include code you wouldn't want executed unless proceeding with an HTTP request, it must first be determined whether the requested function actually exists in the file before executing it (in order to allow execution to be aborted in case the function is not present). Also, when web2py compiles applications, it creates a separate bytecode compiled file for each function in each controller -- in order to do this, it must know the function names, again, without executing the file (as the compilation process takes place outside the environment of an HTTP request, and we don't want any side effects).

Anthony

[Brendan Barnwell]

On Wednesday, January 18, 2017 at 2:12:35 PM UTC-8, Anthony wrote:

A web2py controller is not like a Python module. It is meant to be executed in a prepared environment to generate the response to an HTTP request. Because the top level of the controller may include code you wouldn't want executed unless proceeding with an HTTP request, it must first be determined whether the requested function actually exists in the file before executing it (in order to allow execution to be aborted in case the function is not present). Also, when web2py compiles applications, it creates a separate bytecode compiled file for each function in each controller -- in order to do this, it must know the function names, again, without executing the file (as the compilation process takes place outside the environment of an HTTP request, and we don't want any side effects).

I see, thanks.  What kind of things can be usefully done at the top level that would cause problems outside the request environment?  Most of the stuff I see looking through the docs is stuff happening internally to controller functions, except for a small amount of setup stuff (like "service = Service()") that doesn't appear to have side effects.  Are there useful web2py design patterns that involve significant logic in the controller top-level?

[Anthony]

For example, there may be some code that handles authentication for the entire controller or sets some DAL attributes that apply to the entire controller (e.g., defining a _common_filter). This means that to prevent this code from throwing errors, all the model files that would typically run during a request to this controller must be run (and that can be difficult to determine, as it is possible for the model files themselves to dynamically determine which other model files run on any given request).

Anthony

[Anthony]

Note, if we want to support other patterns of exposing actions in controllers (besides "def some_action():"), we could simply change the pattern matcher to accommodate other possibilities. You could also simply define a function to act like more of a router and include any functionality within it that you like.

Anthony