UUIDs eventually choke as primary key

M Mackey

unread,

Feb 12, 2018, 4:00:53 PM2/12/18

to Django users

I've run into a strange issue with using a UUID as primary key, and I'm hoping we can either verify this is a bug, or figure out what I've done wrong.

I've got a core model object with a UUID for it's primary key. (Generated external to this system, thus using that for when additional information comes in.) And there are other model objects referencing it. Normally everything runs fine. But after a while, web pages that have been running fine start telling me that the UUID isn't valid (even though it is). (Sorry I don't have a copy of the error at this moment, I'll post here when it happens again.) I know it's not a simple coding error, because I can simply restart Apache and everything is fine again. And these aren't POST responses or anything complex like that. It occurs on a simple listing page.

When I dig into the exceptions, I see something that seems contradictory. It looks like the UUID constructor has been called with an instance of a UUID (it says type UUID doesn't have method 'replace'). But it doesn't seem like that should be possible because the code that calls that constructor first checks the type. (See django/db/models/fields/__init__.py - UUIDField.to_python)

So it's a double head scratcher. Once this problem crops up, I have to restart the web server (but then it's okay for a while). And the errors it gives me don't make much sense.

I don't know what kind of caching problem it could be - I haven't enabled any of the available caching systems. Also doesn't seem to crop up (at least not as often) in the built in dev test server.

Anyone seen this, or have any ideas?

Thanks,

M

Ɐha Hah

unread,

Feb 12, 2018, 5:52:24 PM2/12/18

to django...@googlegroups.com

Use a storage/caching layer of CUIDs instead? http://usecuid.org/

Aha

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/d283cde8-76ff-4218-a504-4a92508ae2a2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

M Mackey

unread,

Feb 22, 2018, 8:30:12 PM2/22/18

to Django users

Got it to happen inside the admin interface. It looks like, at least in this case, it's some kind of cache. I don't see the cache stuff on the user's listing page. I'll see if I can put a condensed version of the error here:

ValidationError at /admin/<app>/log/1418765/change/

["'f45fecf0-2cad-4c66-a67f-41665732086c' is not a valid UUID."]

Error during template rendering

In template.../python3.6/site-packages/django/contrib/admin/templates/admin/base.html, error at line 0

'%(value)s' is not a valid UUID.

1 {% load i18n static %}<!DOCTYPE html>

2 {% get_current_language as LANGUAGE_CODE %}{% get_current_language_bidi as LANGUAGE_BIDI %}

3 <html lang="{{ LANGUAGE_CODE|default:"en-us" }}" {% if LANGUAGE_BIDI %}dir="rtl"{% endif %}>

4 <head>

5 <title>{% block title %}{% endblock %}</title>

6 <link rel="stylesheet" type="text/css" href="{% block stylesheet %}{% static "admin/css/base.css" %}{% endblock %}" />

7 {% block extrastyle %}{% endblock %}

8 {% if LANGUAGE_BIDI %}<link rel="stylesheet" type="text/css" href="{% block stylesheet_rtl %}{% static "admin/css/rtl.css" %}{% endblock %}" />{% endif %}

9 {% block extrahead %}{% endblock %}

10 {% block responsive %}

Traceback

.../django/db/models/fields/related_descriptors.py in __get__

rel_obj = self.field.get_cached_value(instance) ...

.../django/db/models/fields/mixins.py in get_cached_value

return instance._state.fields_cache[cache_name] ...

During handling of the above exception ('package'), another exception occurred:

.../django/db/models/fields/__init__.py in to_python

return uuid.UUID(value) ...

/usr/local/lib/python3.6/uuid.py in __init__

hex = hex.replace('urn:', '').replace('uuid:', '') ...

During handling of the above exception ('UUID' object has no attribute 'replace'), another exception occurred:

.../django/core/handlers/exception.py in inner....

M Mackey

unread,

Feb 23, 2018, 11:20:15 AM2/23/18

to Django users

The thing that gets me is this part:

.../django/db/models/fields/__init__.py in to_python
return uuid.UUID(value) ...
/usr/local/lib/python3.6/uuid.py in __init__
hex = hex.replace('urn:', '').replace('uuid:', '') ...

During handling of the above exception ('UUID' object has no attribute 'replace'), another exception occurred:

If you dig into the code, it looks like (based on the exception) a UUID is being passed into the UUID constructor. But the lines before that constructor call happens check to see if it's a UUID. I'm wondering if I'm somehow getting two different UUID classes. I have noticed in the python path that there are two paths to .../lib/python3.6. One from my virtualenv, and one at /usr/local/. Not sure where to clear that up, since I don't believe I've got my apache env set up to pull from both places.

Tom Evans

unread,

Feb 23, 2018, 12:12:33 PM2/23/18

to django...@googlegroups.com

On Fri, Feb 23, 2018 at 4:20 PM, M Mackey <mm...@mac.com> wrote:
> I have noticed in the python
> path that there are two paths to .../lib/python3.6. One from my virtualenv,
> and one at /usr/local/. Not sure where to clear that up, since I don't
> believe I've got my apache env set up to pull from both places.

That would happen if you created your virtualenv with
--use-site-packages, it, er, puts the path to the site packages in the
pythonpath.

Cheers

Tom

M Mackey

unread,

Feb 23, 2018, 3:48:14 PM2/23/18

to Django users

My notes don't say I did that, and I tried a new env and sys.path still shows /usr/local/lib/python3.6.

Maybe that's just a red herring. But I don't have any other good explanation for the UUID getting passed to the UUID __init__.

M Mackey

unread,

Feb 23, 2018, 3:59:28 PM2/23/18

to Django users

Adding details I seem to have left out. The id is defined like this:

id = models.UUIDField(primary_key=True, default=uuid.uuid1, editable=False)

and I'm running on PostgreSQL.

I'm pretty sure I never create a row without already having the key.

On Monday, February 12, 2018 at 1:00:53 PM UTC-8, M Mackey wrote:

M Mackey

unread,

Feb 23, 2018, 4:17:34 PM2/23/18

to Django users

Not to belabor the point, but this is the part that I find weird. This is down in the "During handling of the above exception ('UUID' object has no attribute 'replace'), another exception occurred" section.

/usr/local/venvs/attrack/lib/python3.6/site-packages/django/db/models/fields/__init__.py in to_python

if value is not None and not isinstance(value, uuid.UUID): <<-------- 1) CHECKS FOR NOT UUID

try:

return uuid.UUID(value) <<----- 2) Pretty sure first exception is in here

except (AttributeError, ValueError):

raise exceptions.ValidationError(

self.error_messages['invalid'],

code='invalid',

params={'value': value}, <<------ Traceback says new exception here

)

return value

Local vars

Variable Value

self <django.db.models.fields.UUIDField: id>

value UUID('338fa43f-3e07-4e83-8525-d195d2fc64d4') <<-------- 3) REPORTS AS UUID

So, if 3) is right and 'value' is a UUID, 1) should keep us from getting to 2). But we get there.

M Mackey

unread,

Mar 1, 2018, 11:53:13 AM3/1/18

to Django users

I added some debug support, and there are only two classes called UUID when this fails:

uuid UUID: <module 'uuid' from '/usr/local/lib/python3.6/uuid.py'>

psycopg2.extensions UUID: <module 'psycopg2.extensions' from '.../python3.6/site-packages/psycopg2/extensions.py'>

The psycopg extension is a converter, and it seems highly unlikely that it was put into place as the value, and I doubt it would report it's __str__ as the UUID string, so it does not appear to be a case of mistaken identity. Which makes the failure of "not instance" to catch the UUID after a while all the more weird...

I'm pretty baffled on this one. Am I really the only one running into this?

M

M Mackey

unread,

Mar 15, 2018, 5:20:17 PM3/15/18

to Django users

Well, I don't seem to be able to get around this. I'm going to have to switch away from using a UUID as a primary key, which fortunately doesn't seem like too much work.

M. Page-Lieberman

unread,

Mar 16, 2018, 7:24:03 AM3/16/18

to django...@googlegroups.com

Hi, I've only been using Django for a few weeks really, so I'm not sure how much I can help, but this is concerning for me, and I'd like to help resolve it.

Are you able to just include your full models.py file here? I'll run it on my computer and try to debug too.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/038daba7-fa72-4c17-8944-3c15ffb6a8f6%40googlegroups.com.

Ahmed Cheikh

unread,

Mar 16, 2018, 7:24:15 AM3/16/18

to Django users

Hi, I think the problem comes from mod_wsgi. I had the same problem when virtualhost listen on multiple ports.

Please look at this GH issue https://github.com/GrahamDumpleton/mod_wsgi/issues/206

and try this by https://github.com/GrahamDumpleton/mod_wsgi/issues/206#issuecomment-305019825 by Dumpleton.

The real issue : isinstance(value, uuid.UUID) is failling because of the use of sub interpreters.

Details in the mod_wsgi documentation : http://modwsgi.readthedocs.io/en/develop/user-guides/processes-and-threading.html#python-sub-interpreters and http://modwsgi.readthedocs.io/en/develop/configuration-directives/WSGIApplicationGroup.html.

That's what you should be looking for.

Reply all

Reply to author

Forward