Is it possible to speed up reimporting a module

[André]

In my app (http://reeborg.ca/reeborg.html) I use Brython to enable writing programs in Python to control a virtual robot.  Users can also write programs in Javascript to do the same. 

When using Python, I need to make some definitions available to the users prior to executing their programs; I do this via something like 

def compile_program(src):

    my_globals = {}

    my_globals.update(globals())  # ensure that Python builtins are available

    exec("from my_module import *", my_globals)

    exec(user_program, my_globals)

A while ago, I found out that switching programming mode randomly from programming in Javascript to Python and back, which the users are allowed to do, could result in errors.  A solution to this problem was the following:

def compile_program(src):

    if 'my_module' in sys.modules:

        del sys.modules['my_module']  # forces a fresh import

    my_globals = {}

    my_globals.update(globals())

    exec("from my_module import *", my_globals)

    exec(user_program, my_globals)

The problem is that, this is slow: running my integration test suite is now annoyingly long.  Furthermore, the server is being asked to do a lot more work since there is a fresh request each time.

I have tried to do the following to try to cache the result of the import

MY_MODULE = {}

exec("from my_module import *", MY_MODULE)

def compile_program(src):

    my_globals = {}

    my_globals.update(globals())

    my_globals.update(MY_MODULE)

    exec(user_program, my_globals)

However, this resulted in the same errors as if I let Brython do its own caching.

I did try to also add

    if 'MY_MODULE' in my_globals:

        del my_globals['MY_MODULE']  

prior to doing

    my_globals.update(MY_MODULE)

but that did not work.

Does anyone have any suggestion for getting Brython to simulate the result of doing a fresh import, but without actually doing an Ajax call to import a module each time?

André

P.S. I'm currently running an older version of Brython (__BRYTHON__.__MAGIC__ == "3.2.7"). If a proposed solution requires a more recent version, I will upgrade.

     

[Andre Roberge]

Sorry for the typo...

"src" as the argument for compile_program()  is what I wrote as "user_program" on the last line of that function.

--
You received this message because you are subscribed to a topic in the Google Groups "brython" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/brython/nquHAbgVQEw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to brython+unsubscribe@googlegroups.com.
To post to this group, send email to bry...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/brython/331c0bd8-2154-4119-8490-c4323be0a91a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kiko]

Hi André,

I did not tried but maybe using local storage to import the src of the needed modules from it via exec?

--
You received this message because you are subscribed to the Google Groups "brython" group.
To unsubscribe from this group and stop receiving emails from it, send an email to brython+unsubscribe@googlegroups.com.

To post to this group, send email to bry...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/brython/CAGMu_%3DqyGgBF2Df4%3D9QX_KpGbAskq%3DYdb_g82WAe4_mAm84ihg%40mail.gmail.com.

[André]

Hi Kiko,

Thanks for the suggestion ... it would have been faster, but it does not work in my case. 

I think I have figured out why putting the module in local storage OR in a module dict (as I tried before) does not work in my case.  

My program mixes Javascript objects and Python code. I think the cause is that I make some objects (Python custom Exceptions in this case) known globally via window["MyException"].  In some cases, I import MyModule twice (in two different modules written by a user). Doing a regular import, Brython recognizes that the module has already been imported and does not redefine anything.  Trying to simulate an import via exec("MyModule") reassigns each time window["MyException"] to a different object which is not seen as belonging to a common module (and thus be the same object).  

I think I might be able to get around this particular quirk ... but it will require a bit of fiddling with my code.

André

To unsubscribe from this group and all its topics, send an email to brython+u...@googlegroups.com.

To post to this group, send email to bry...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/brython/331c0bd8-2154-4119-8490-c4323be0a91a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "brython" group.

To unsubscribe from this group and stop receiving emails from it, send an email to brython+u...@googlegroups.com.

To post to this group, send email to bry...@googlegroups.com.

[André]

Once again, thanks for the suggestion: it did prompt me to look at things more closely.  Using localStorage could work, and would be faster than reimporting the module each time.  However, the fastest method (once I figured out the source of the problem) is to import the module once and use exec() once to populate a dict.  [Using local storage to save the source would require to use exec() each time.]

Then, when I actually need to use the "from my_module import *" into another module, all I need to do is to update the globals() of that other module; this saves the translation py->js that Brython needs to do when doing exec().

Running my integration tests now takes 1/3 of the time it required previously.

André

On Wednesday, 26 July 2017 03:05:20 UTC-3, kiko (on pybonacci) wrote:

To unsubscribe from this group and all its topics, send an email to brython+u...@googlegroups.com.

To post to this group, send email to bry...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/brython/331c0bd8-2154-4119-8490-c4323be0a91a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "brython" group.

To unsubscribe from this group and stop receiving emails from it, send an email to brython+u...@googlegroups.com.

To post to this group, send email to bry...@googlegroups.com.