Does Python have AUTOLOAD?
Gareth Lee
advice on the following.
Perl5 has a mechanism in the form of AUTOLOAD for detecting when a
function is called which is not defined within the scope. When this occurs
the AUTOLOAD function is called as a proxy and can report an error, import
the function from a new module or define the function on the fly. For
instance, I think the code would look something like:
package AutoLoader;
fred(33); # fred is not defined
AUTOLOAD {
# $AUTOLOAD is "fred", and
# @_ are fred's parameters, so
# we can create fred
eval "sub $AUTOLOAD" . '{ print "@_\n"}';
# execute the newly created fred
goto &$AUTOLOAD
}
Presumably if fred is called in Python before it is defined a NameError
results and this can be caught. Can one get to the function name and
parameters. I figure the function name would be the parameter passed
with the exception, but how would I get to the parameters to allow me
to write a handler of the form shown above in Perl?
Thanks in advance.
regards,
Gareth Lee.
--
Centre for Intelligent Information Processing Systems,
Dept. of Electrical & Electronic Engineering, phone: +61 9 380 1768
University of Western Australia, fax: +61 9 380 1168
Perth, WA 6907, Australia. PGP key: finger gar...@rama.ee.uwa.edu.au
Aaron Watters
>
> I'm fairly new to Python and wondered if someone could offer some
> advice on the following.
>
> Perl5 has a mechanism in the form of AUTOLOAD for detecting when a
> function is called which is not defined within the scope. When this occurs
> the AUTOLOAD function is called as a proxy and can report an error, import
I dunno... maybe using the restricted execution hooks you could do
something like this, but even if you could, I've a feeling it might
not be advisable... There may be a more natural way to reach
your higher level goal, in fact I'd be surprised if there wasn't...
Could you pop up a few frames? -- Aaron Watters.
===
Restore the Hapsburgs (sp?). Any monarchy that can keep the
Balkans peaceful for hundreds of years is doing something right.
Guido van Rossum
block but there is no way to "repair" an exception.
You can override import by defining your own __import__ intrinsic
function, and you can have an object with magic attributes using a
__getattr__ discipline, but there's no way to autoload on NameError
exceptions.
--Guido van Rossum (home page: http://www.python.org/~guido/)
Bertil Reinhammar (Signum)
Gareth Lee <gar...@ee.uwa.edu.au>:
> package AutoLoader;
>
> fred(33); # fred is not defined
>
> AUTOLOAD {
Guido van Rossum <gu...@monty.cnri.reston.va.us>:
> You can't do that in Python.
[cut]
What would the expected semantics be in case this feature did exist in Python ?
From what I can see, assume there are more than one module with fred(), which
one do we get ? Just searching the PYTHONPATH for a module with a fred() is not
really what I would like as I my control is, eh, lost.
I find it most appropiate to get a NameError as the lack of the proper import
seems to be a programming error.
---BeRe
Guido van Rossum
>From what I can see, assume there are more than one module with fred(), which
>one do we get ? Just searching the PYTHONPATH for a module with a fred() is not
>really what I would like as I my control is, eh, lost.
>I find it most appropiate to get a NameError as the lack of the proper import
>seems to be a programming error.
That's easy to answer. Most systems supporting autoload that I've seen
have some table that says "this function is loaded from that file/module".
(E.g. Emacs.)
However in Python, which is mostly used in scripts, not interactively,
I don't see much value in autoload except for lazy programmers.
Note that you can already do the following:
def spam(*args, **kws):
"substitute to load the real spam when first called"
global spam # affects the import on the next line
from sketch import spam # overwrites this spam definition
return apply(spam, args, kws) # call the real spam
This acts as an on-demand-import. But is it worth it? You'd have to
synthesize large numbers of such functions in each namespace (module)
that would want to use this feature for it to be useful.
I rather stick "import sketch" in the top of my module, or in the
function where I need to use spam.
--Guido van Rossum
Skip Montanaro
What would the expected semantics be in case this feature did exist in Python ?
From what I can see, assume there are more than one module with fred(), which
one do we get ? Just searching the PYTHONPATH for a module with a fred() is not
really what I would like as I my control is, eh, lost.
I find it most appropiate to get a NameError as the lack of the proper import
seems to be a programming error.
I agree if the item generating NameError is a non-module identifier. If, on
the other hand you have
foo.fred(33)
it might be reasonable to expect the system to try and find the foo module
using the normal module search mechanism. At the point in the interpreter
where the NameError would be generated is there enough context to silently
perform
import foo
and place foo in the current scope when the unimported module name is found?
The distinction between types of undefined identifiers generating NameErrors
makes it difficult (or impossible) to recover in a generic NameError
handler. I think you'd have to avoid the generation of a NameError
exception altogether (maybe predicated on the value of a yet-to-be-defined
sys.implicit_import variable's value).
I'm not too keen on having automatic module lookups for some reason I can't
yet identify (just gut feeling). In theory, if
x = 1
does not require a declaration of x as a variable, why should
foo.fred(33)
require declaration of foo as a module?
(Think, think, think, ...)
Ah, I see the possible problem. Suppose I have a class:
class foo:
def split(self, s, sep = ' '):
pass
and I write:
string.split(s)
where the identifier string does not exist in the current scope. What if
instead of forgetting to import string, the author forgot to instantiate
foo:
string = foo()
An implicit import would definitely be the wrong thing...
Skip Montanaro | Today's Acronym: YAUWWWI (yow'-ee) Yet Another Useless
sk...@calendar.com | World-Wide Web Idea
(518)372-5583 | --- Check out Musi-Cal: http://concerts.calendar.com/
Donald Beaudry
> I rather stick "import sketch" in the top of my module, or in the
> function where I need to use spam.
I'd have to agree, but I think that there are really two aspect to
"autoload" that should be considered before completly tossing the idea
aside. One is programmer convenience: just mention the function name
and like magic it appears. The other is efficiency. Why load a
module if you're not going to use it? Personally, I see no value in
the first aspect. It ranks right up there with 'from x import *' as
far as I am concerned.
However, there *might* be some efficiency to be gained in certain
applications by introducing the concept of a "lazy import". The idea
would be to "import" a module but not really load it until the first
time its getattr method is called. This would let you import all of
your modules at the top of your program without regard for where or
how often you are going to need it. I don't really care for putting
import statments inside of conditionals or functions. So, this idea
would appeal to me (but not enough for me to bother implementing it
though ;).
--
Donald Beaudry Silicon Graphics
Digital Publishing/DMS 1 Cabot Road
do...@sgi.com Hudson, MA 01749
...So much code, so little time...
Daniel Larsson
Being a C++ programmer I'm fed up with implicit stuff happening
at times I'm not expecting it. The benefit gained is far less
than the confusion such a feature could result in.
As Skip pointed out, the '.' notation is used not only in
<module>.<feature>, but in <class>.<feature> and <object>.<feature>.
The parser cannot really now in general what the programmer
intended, so it shouldn't even try, IMHO.
--
Daniel Larsson, ABB Industrial Systems AB
Guido van Rossum
> applications by introducing the concept of a "lazy import". The idea
> would be to "import" a module but not really load it until the first
> time its getattr method is called. This would let you import all of
> your modules at the top of your program without regard for where or
> how often you are going to need it. I don't really care for putting
> import statments inside of conditionals or functions. So, this idea
> would appeal to me (but not enough for me to bother implementing it
> though ;).
One can do *this* by defining an __import__ discipline (which
redefines the semantics of the import statement) that searches for the
module but doesn't load it, and instead returns a fake object with a
__getattr__ method that loads the module. The problem I see with this
is that someone doing
import foo
and using
foo.bar
a lot will be stuck with the fake object for foo, and thus incur the
overhead of the foo.bar lookup on every use. [Think...] Ah no, if
the __getattr__ sticks all of the module's objects on the fake object
once it is loaded, lookup will have the same speed (__getatr__ is only
invoked for objects not in the module).
Any takers?
Tom Christiansen
In comp.lang.python, Guido van Rossum <gu...@CNRI.Reston.VA.US> writes:
:I don't see much value in autoload except for lazy programmers.
Well, there's the efficiency issue. Why does one do demand-paging of
object code? Same reason -- you don't load it in until you need it.
With an interpreter or pcode-interpreter, loading in a huge module
at start-up time may not be worth it, especially if you can get a
proper interface definition without having to get at the implementation.
Imagine a module like POSIX that defines bazillions of functions
and constants. If you only need a few of them, do you really
want to take the time to load them all in? Agreed, the user
should probably learn to ask for just a few things, or the module author
should design a better module, but it still may be merit considering
whether it's worth delaying some of that cost.
--tom
--
Tom Christiansen Perl Consultant, Gamer, Hiker tch...@mox.perl.com
"It's okay to be wrong temporarily." --Larry Wall
Guido van Rossum
> :I don't see much value in autoload except for lazy programmers.
>
> Well, there's the efficiency issue. Why does one do demand-paging of
> object code? Same reason -- you don't load it in until you need it.
> With an interpreter or pcode-interpreter, loading in a huge module
> at start-up time may not be worth it, especially if you can get a
> proper interface definition without having to get at the implementation.
>
> Imagine a module like POSIX that defines bazillions of functions
> and constants. If you only need a few of them, do you really
> want to take the time to load them all in? Agreed, the user
> should probably learn to ask for just a few things, or the module author
> should design a better module, but it still may be merit considering
> whether it's worth delaying some of that cost.
Well, once you start loading a file you might as well finish it (at
least this is what all autoload systems I'm aware of do -- doesn't
Perl's?).
So the issue is simply whether it makes sense to delay loading of the
file. If you are going to use even a single object from that huge
module, you will have to load it (all of it!) sooner or later, so the
only gain you make is startup time (which often counts, admittedly).
This (avoiding the slow startup by delayed import) can already be done
in Python by placing the import statement inside the function that
uses the module -- after all, import is an executable statement, not a
declaration (technically, 'global' is the only declaration statement
in Python). And a second import is free (since it doesn't need to do
the load, it's essentially one dictionary lookup).
Daniel Larsson
>
> ...
>
> Any takers?
Ok, couldn't resist ;-). I noticed that the ModuleLoader will
always import the .py file even if the .pyc file exists because
in the list imp.get_suffixes() returns, .pyc occurrs after .py.
I guess this is intentional (otherwise you would have to compare
modification dates).
Guido van Rossum
> always import the .py file even if the .pyc file exists because
> in the list imp.get_suffixes() returns, .pyc occurrs after .py.
Actually, it's supposed to check out the .pyc file and see if it
matches the .py file in a later stage. A timestamp recorded in the
.pyc file must match the mtime of the .py file. Not sure whether
you've broken this behavior.
Daniel Larsson
I guess that is done by imp.load_source, right? If so, I've not
broken it.
Guido van Rossum
> > matches the .py file in a later stage. A timestamp recorded in the
> > .pyc file must match the mtime of the .py file. Not sure whether
> > you've broken this behavior.
>
> broken it.
Probably not. Try "python -v", it should print suitable comments.
Gareth Lee
Daniel Larsson wrote:
>
> [...] The benefit gained is far less
> than the confusion such a feature could result in.
Not if it is added in such a way as not to intefere with existing code.
Then people have the option of using the feature or ignoring it.
> As Skip pointed out, the '.' notation is used not only in
> <module>.<feature>, but in <class>.<feature> and <object>.<feature>.
> The parser cannot really now in general what the programmer
> intended, so it shouldn't even try, IMHO.
I agree!
[This turned out to be rather long -- you have been warned!!]
Having had a chance to follow the thread that resulted from my original post
I thought it worthwhile to post a follow up. Firstly, several people
commented, "why bother?". Let me describe the application that I am planning
and then, perhaps, my request will make more sense.
I am envisaging a multimedia hypertext system where pages are drawn and
managed by a large C graphics library (which already exists and is currently
linked into a Perl interpreter). Each of the screens within the package
would be controlled by a Python "applet" embodied in a module or class (yet
to be decided).
As the number of screens within a specific package grows, so the number of
modules would increase 1:1. Since there could be several hundred screens
within the final package it would be very inconvenient (and time consuming)
to explicitly import each at the top of the central program. Moreover, only
one of the modules would be active at any time and when a module is exited,
its memory/resources should be automatically reclaimed. Thus a hyperlink to
jump to a item 34 within screen 56 would take the form:
del screen55 # get rid of the previous screen resources
import screen56 # import the next screen applet code
screen56.start() # realise the screen
screen56.item34() # activate the desired item within the screen
Ideally the autoloader would allow me to write the code
screen56.start()
screen56.item34()
and would take care of the searching, loading and unloading.
Of source, I always have the option of writing a function to do this management
in a non-transparent way, ie:
GotoScreen("screen56", "item34")
This is rather inellegent, but could be made to work perfectly (I think??)
If autoload capabilities are added to Pyhthon I would advocate that they not
interfere with the existing operation of the system in any way (unlike some of
the previous proposals which would change the behaviour of import). The
existing perl model fits well with this philosophy. I think a good
strategy would therefore be:
1. User calls an undefined function, ie. fred(33)
2. Python searches for a method entitled __autoload__ in the current scope.
If found self.__autoload__(self, undefined_function_name, tuple_of_parameters)
is called to handle the error, otherwise,
3. Pyhton searches for __main__.__autoload__ and calls it with the same
parameters. If missing,
4. Python generates a NameError exception as before.
If __autoload__ is called, no exception results and the value returned is
returned by fred(). Execution then continues from the next statement.
The side effect of this implementation is limited to a slight invasion of the
existing namespace.
However, to put this whole thing into perspective I'm the first to admit that
I'm trying to create a rather esoteric system and, since it is possible to
implement this using existing language features it should not be considered a
high priority.
That said, one of the strengths of languages such as Python and Perl is that
they offer a large and diverse set of operations (even though any specific
project will only use a small subset). __autoload__ is one area that is
uniquely suited to interpreted languages and would therefore be a useful
addition to the existing capabilities.
Fredrik Lundh
> 2. Python searches for a method entitled __autoload__ in the
> current scope. If found self.__autoload__(self,
> undefined_function_name, tuple_of_parameters) is called to handle
> the error, otherwise,
Wouldn't something like getattr/setattr on the module level be better
idea? With such a mechanism, you could solve the SWIG global variable
problem as well.
(IMO, turning Python into Perl doesn't sound like a good idea...)
> Of cource, I always have the option of writing a function to do this
> management in a non-transparent way, ie:
> GotoScreen("screen56", "item34")
> This is rather inellegent, but could be made to work perfectly (I think??)
Why would something like the above, or goto(screen=56, item=34), or
even goto(56, 34) be considered non-transparent?
As for the implementation, it is just a matter of using "compile/exec"
in an appropriate fashion. I mean, you don't have to import code to
run it...
/F
David Ascher
> Having had a chance to follow the thread that resulted from my original post
> I thought it worthwhile to post a follow up. Firstly, several people
> commented, "why bother?". Let me describe the application that I am planning
> and then, perhaps, my request will make more sense.
> Of source, I always have the option of writing a function to do this management
> in a non-transparent way, ie:
>
> GotoScreen("screen56", "item34")
>
> This is rather inellegent, but could be made to work perfectly (I think??)
What's inelegant about it? It works today, doesn't require an obviously
controversial language change, etc. etc.
You could even fake the autoloading yourself with your own 'autoload'
class by overriding the __getattr__ calls, so that:
autoload.screen56.item34.show()
would work, if you prefer that look.
--david ascher
Aaron Watters
Gareth Lee wrote:
> As the number of screens within a specific package grows, so the number of
> modules would increase 1:1. Since there could be several hundred screens
> within the final package it would be very inconvenient (and time consuming)
> to explicitly import each at the top of the central program. Moreover, only
> one of the modules would be active at any time and when a module is exited,
> its memory/resources should be automatically reclaimed. Thus a hyperlink to
> jump to a item 34 within screen 56 would take the form:
>
> del screen55 # get rid of the previous screen resources
> import screen56 # import the next screen applet code
> screen56.start() # realise the screen
> screen56.item34() # activate the desired item within the screen
I may well be missing something, but I think you should use instances instead
of modules to implement your screens... The initializers for instances could
be put in an array for easy configuration; you can use inheritance for code
sharing; cleanup of local data space is automagic, the initializer need only
import other modules as needed (inside a the initialization operation...)
thescreen = screen[55](parameters).start()
thescreen.item(34)
thescreen = screen[56](parameters).start() # old screen is cleaned up...
thescreen.item(9)
300 screens may not require as much code/memory as you think (especially
if inheritance pays off), so I wouldn't worry too much about unloading
modules unless it turns out to be a problem later... -- Aaron Watters
====
He's a humble man, and with good reason.
-- Winston Churchill, on a colleague.
David M. Beazley
In article <960701111...@arnold.image.ivab.se> Fredrik Lundh <Fredri...@ivab.se> writes:
> > 2. Python searches for a method entitled __autoload__ in the
> > current scope. If found self.__autoload__(self,
> > undefined_function_name, tuple_of_parameters) is called to handle
> > the error, otherwise,
>
> idea? With such a mechanism, you could solve the SWIG global variable
> problem as well.
As a follow-up to this message, I'd like to mention that I have been
working on the SWIG global variable problem recently and have switched
to an approach using getattr/setattr. So far it seems to work
nicely (well, much better than before). If anyone is interested in playing
with some super-experimental versions of the SWIG Python module that
do this, I've been putting up a new version periodically on the
SWIG homepage.
http://www.cs.utah.edu/~beazley/SWIG
Of course, you should use at your own risk :-).
Cheers,
Dave Beazley