Import error in settings.py

[Julien]

Hi there,

I'm trying import a class definition in settings.py with:
from myapp.models import MyModel

But the compilation fails and I get the following error in the
console:
"Error: Can't find the file 'settings.py' in the directory containing
'E:\\workspace\\myproject\\manage.py'. It appears you've customized
things.
You'll have to run django-admin.py, passing it your settings module.
(If the file settings.py does indeed exist, it's causing an
ImportError somehow.)"

Apparently the problem is only when trying to import classes under
modules (apps). In fact, I've also got a file "local_settings.py" at
the project root, and the following import works (still in
settings.py):
from local_settings import *

Is that a pythonpath problem or something else? Unfortunately this
happens at compilation time so I can't debug.

Thank you!

Julien

[Malcolm Tredinnick]

On Tue, 2008-02-12 at 05:35 -0800, Julien wrote:
> Hi there,
>
> I'm trying import a class definition in settings.py with:
> from myapp.models import MyModel

Oh, don't do that. Very bad. :-(

Importing models means they start to get registered with the app_cache
(to track reverse relations, amongst other things), which means we need
to know about other applications, which is specified in a setting, which
is in the settings file. Spot the circular dependency.

In settings.py, you can't really expect to use anything from Django
itself. You can write arbitrary Python code that talks to other parts of
your system or whatever, but large portions of the core of Django
requires settings to have been fully imported before they can be used.
So you cannot use them inside settings itself.

Perhaps if you explain what you're trying to achieve we could suggest an
alternate route.

Regards,
Malcolm

--
I don't have a solution, but I admire your problem.
http://www.pointy-stick.com/blog/

[Julien]

Oh, I see!! Thanks for the explanation.

So, what I'm trying to achieve is to specify a model class as a
setting that can be used in some views.

As you've said, doing "from myapp.forms import MyModelForm" is not
possible, so I tried:

MY_MODEL_FORM = 'myapp.forms.MyModelForm'

And then, in some view I do:

from django.conf import settings
splitted = settings.MY_MODEL_FORM.split('.')
module = __import__(splitted[0])
for i in range(len(splitted)-1):
module = getattr(module, splitted[i+1])
my_model_form_class = module
my_model_form = my_model_form_class()


Ok, I know, the above code looks like overkill to instantiate the
object specified in the setting. But I could not find anyway to do
that properly in Python. The problem is that the above code actually
works, but only if the module 'forms.py' containing the object
(MyModelForm) has been compiled to forms.pyc

If that 'forms' module is never referenced anywhere with an import
statement, then it is not compiled, and therefore the above code
generates an AttributeError: 'module' object has no attribute
'MyModelForm'

I suspect there might be a way in Python to instantiate a class using
its full name (including parent modules) and to compile the .py module
if it hasn't already been compiled.

Any idea?

Thanks a lot!

Julien

On Feb 13, 12:48 am, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:

[Malcolm Tredinnick]

On Tue, 2008-02-12 at 06:08 -0800, Julien wrote:
> Oh, I see!! Thanks for the explanation.
>
> So, what I'm trying to achieve is to specify a model class as a
> setting that can be used in some views.
>
> As you've said, doing "from myapp.forms import MyModelForm" is not
> possible, so I tried:
>
> MY_MODEL_FORM = 'myapp.forms.MyModelForm'

Now you see why all the Django settings like this are strings. :-)

> And then, in some view I do:
>
> from django.conf import settings
> splitted = settings.MY_MODEL_FORM.split('.')
> module = __import__(splitted[0])
> for i in range(len(splitted)-1):
> module = getattr(module, splitted[i+1])
> my_model_form_class = module
> my_model_form = my_model_form_class()

You're certainly on the right track. However, you're not using the full
power of __import__(). Have a read of the Python docs (in the Library
Reference, have a look at builtin functions. It's the first entry in
§2.1). Basically, the trick is that you can pass a dotted string form as
the first argument to __import__. However, you also need to watch out
for what __import__ returns (explained in the docs). So, typically,
you'll be doing this:

klass = __import__(settings.MY_MODEL_FORM, {}, {}, '')

where I've used "klass" instead of "my_model_form_class". Passing '' as
the last argument means that klass will now be a reference to what you
expect (try leaving it off and look at what klass is, if you're
interested).

Not surprisingly, Django does this sort of thing all over the place.
See, for example, django/template/loader.py or
django/db/models/loader.py (just search for __import__). It's the
standard way to import something dynamically on those rare occasions
when you need to.

Regards,
Malcolm

--
Plan to be spontaneous - tomorrow.
http://www.pointy-stick.com/blog/

[Julien]

Hi,

Thanks for the tip. However I still get an ImportError: "No module
name MyModel".

I tried the following:

klass = __import__(settings.MY_MODEL_FORM, {}, {}, '')
klass = __import__(settings.MY_MODEL_FORM, {}, {}, [''])

klass = __import__(settings.MY_MODEL_FORM, globals(), locals(), [''])

I even tried to hard code it:
klass = __import__("myapp.forms.MyModelForm", {}, {}, [''])

But I still get that ImportError. What is strange is that if I do:
from myapp.forms import MyModelForm

... it works... so the module should obviously be visible and found.

What am I missing? I looked in the Django code, and indeed, __import__
is used in a lot of places, and exactly in the way that you have
suggested. I just don't understand why it does not work for me :(

Am I missing something?

Many thanks,

Julien

On Feb 13, 1:17 am, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:

[Malcolm Tredinnick]

On Tue, 2008-02-12 at 14:12 -0800, Julien wrote:
> Hi,
>
> Thanks for the tip. However I still get an ImportError: "No module
> name MyModel".
>
> I tried the following:
> klass = __import__(settings.MY_MODEL_FORM, {}, {}, '')
> klass = __import__(settings.MY_MODEL_FORM, {}, {}, [''])
> klass = __import__(settings.MY_MODEL_FORM, globals(), locals(), [''])
>
> I even tried to hard code it:
> klass = __import__("myapp.forms.MyModelForm", {}, {}, [''])
>
> But I still get that ImportError. What is strange is that if I do:
> from myapp.forms import MyModelForm

Hmm. Are you perhaps trying to execute this whilst myapp.forms is still
being imported? For example, something that myapp.forms imports if
running this line of code? That won't work (until a module is fully
imported, its name exists in sys.modules to prevent circular imports,
but the value (module) attached to that name is empty, so you cannot
access the internals of a module until it's been fully imported).

If that's not the case, can you import 'myapp.forms' using the above
technique (you're probably right that a list is needed at the end, btw;
I posted without actually checking too carefully)? What I'm trying to
work out is where in the dotted import path it all goes wrong.

My first thought was that you didn't have "myapp" on your Python path
(the parent directory of myapp has to be on the Python path), but that
would probably generate a message saying it couldn't import "myapp",
rather than something about "MyModelForm" (guessing; I didn't test it).

So, how far can you get down the import chain here?

(a) klass = __import__("myapp", {}, {}, [''])
(b) klass = __import__("myapp.forms", {}, {}, [''])
(c) klass = __import__("myapp.forms", {}, {}, ['MyModelForm'])

None of these are what you want, but it might give some idea as to where
the problem really lies.

Malcolm

--
The cost of feathers has risen; even down is up!
http://www.pointy-stick.com/blog/

[Julien]

Hi!

I can't try this out at the moment as I don't have access to the code.
But I just thought I'd add the fact that forms.py is never compiled
(no trace of a forms.pyc). Should the __import__ function force the
compilation of a module when it's imported, or does it require that
module to be compiled prior (and if so, how can you force that
compilation)?

On Feb 13, 12:23 pm, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:

[Malcolm Tredinnick]

On Tue, 2008-02-12 at 17:41 -0800, Julien wrote:
> Hi!
>
> I can't try this out at the moment as I don't have access to the code.
> But I just thought I'd add the fact that forms.py is never compiled
> (no trace of a forms.pyc). Should the __import__ function force the
> compilation of a module when it's imported, or does it require that
> module to be compiled prior (and if so, how can you force that
> compilation)?

The .pyc file thing is a red herring here. Python will byte compile
files it imports and save them to disk, if it can (i.e. if it has write
permission). However, if it cannot do so, no problems, it's just a bit
slower the next time (because it has to recompile).

So if you're in a position where a .pyc file loads and the
corresponding .py file does not, something is seriously messed up. I
doubt that's going on.

That would lend evidence to my wondering if you are running the
__import__ statement from something that is being imported by forms.py
(so forms.py hasn't finished importing at the time you execute that line
and, thus, forsm.MyModelForm wouldn't exist).

Still, a quick test here shows that __import__ does save the .pyc file
when the calling process had write-permission to the directory I was
importing from.

Regards,
Malcolm

--
Quantum mechanics: the dreams stuff is made of.
http://www.pointy-stick.com/blog/

[Julien]

Hello,

The module was not compiled, because it was the __import__ function
itself that raised an exception and so didn't have the chance to do
the compilation.

As you've suggested, I tried:

klass = __import__("myapp", {}, {}, [''])

-> Works, returns <module myapp>

klass = __import__("myapp.forms", {}, {}, [''])

-> Works, returns <module myapp.forms>, and compiles "forms.pyc"!!

klass = __import__("myapp.forms", {}, {}, ['MyModelForm'])

-> Works, and returns the same thing as above: <module myapp.forms>

But, although the module is now compiled, the following still doesn't
work:

klass = __import__("myapp.forms.MyModelForm", {}, {}, [''])

For info, MyModelForm is an instance of ModelFormMetaclass. I also
tried importing another model, still in vain:
klass = __import__("myapp.models.MyOtherModel", {}, {}, [''])


Looks like we're getting closer. It doesn't want to import the class,
although it can import all of the modules... Strange!!


On Feb 13, 12:45 pm, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:

[Malcolm Tredinnick]

On Tue, 2008-02-12 at 18:30 -0800, Julien wrote:
> Hello,
>
> The module was not compiled, because it was the __import__ function
> itself that raised an exception and so didn't have the chance to do
> the compilation.
>
> As you've suggested, I tried:
>
> klass = __import__("myapp", {}, {}, [''])
> -> Works, returns <module myapp>
>
> klass = __import__("myapp.forms", {}, {}, [''])
> -> Works, returns <module myapp.forms>, and compiles "forms.pyc"!!
>
> klass = __import__("myapp.forms", {}, {}, ['MyModelForm'])
> -> Works, and returns the same thing as above: <module myapp.forms>
>
> But, although the module is now compiled, the following still doesn't
> work:
> klass = __import__("myapp.forms.MyModelForm", {}, {}, [''])
>
> For info, MyModelForm is an instance of ModelFormMetaclass. I also
> tried importing another model, still in vain:
> klass = __import__("myapp.models.MyOtherModel", {}, {}, [''])

Oh, doh! I'm an idiot. The answer was there all along.

You can't do "import myapp.models.MyOtherModel", because MyOtherModel
isn't a *module*. It's something inside a module. That's just normal
Python behaviour.

So you have to do

module = __import__("myapp.forms", {}, {}, ['MyModelForm'])

(using either 'MyModelForm' or '' in the last component). Then

klass = module.MyModelForm

Of course, in your case, that means splitting off the last dotted piece
of the string to work out the form class. This is exactly what we do in
django.template.loader.find_template_source(), for example, to separate
the template loader function from the model it's contained in.

I'm so sorry for misleading you for a little while there. Complete brain
failure on my part. But it all makes perfect sense now.

Regards,
Malcolm

--
Depression is merely anger without enthusiasm.
http://www.pointy-stick.com/blog/

[Julien]

Great! Thanks a lot, it worked!

Here's a little function that I made and that is quite helpful:

def get_class(class_path):
i = class_path.rfind('.')
module_path, class_name = class_path[:i], class_path[i+1:]
module = __import__(module_path, globals(), locals(),
[class_name])
return getattr(module, class_name)

Thanks again for your help. That wasn't easy, but that made me visit
some parts of Python and Django that I didn't know about. And it does
demystify a lot of things!

On Feb 13, 1:40 pm, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:

[Felix Unchained]

Just thought you might get a kick out of the fact that I just found this very useful – 4 years later. Thanks.

[Collin Anderson]

You can also do this:

import importlib
module_name, name = path.rsplit('.', 1)
return getattr(importlib.import_module(module_name), name)