[Django] #16484: Duplicate entry sessions error
Django
-----------------------+----------------------------------
Reporter: hash.3g@… | Owner: nobody
Type: Bug | Status: new
Milestone: | Component: contrib.sessions
Version: 1.3 | Severity: Normal
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------+----------------------------------
Hi, Sometimes we receive "duplicate entry" error from django sessions. See
Traceback.
{{{
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-
packages/django/core/handlers/base.py", line 178, in get_response
response = middleware_method(request, response)
File "/usr/local/lib/python2.6/dist-
packages/django/contrib/sessions/middleware.py", line 36, in
process_response
request.session.save()
File "/usr/local/lib/python2.6/dist-
packages/django/contrib/sessions/backends/db.py", line 63, in save
obj.save(force_insert=must_create, using=using)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/base.py",
line 460, in save
self.save_base(using=using, force_insert=force_insert,
force_update=force_update)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/base.py",
line 553, in save_base
result = manager._insert(values, return_id=update_pk, using=using)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/manager.py",
line 195, in _insert
return insert_query(self.model, values, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py",
line 1436, in insert_query
return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/local/lib/python2.6/dist-
packages/django/db/models/sql/compiler.py", line 791, in execute_sql
cursor = super(SQLInsertCompiler, self).execute_sql(None)
File "/usr/local/lib/python2.6/dist-
packages/django/db/models/sql/compiler.py", line 735, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.6/dist-
packages/django/db/backends/mysql/base.py", line 86, in execute
return self.cursor.execute(query, args)
File "/usr/lib/pymodules/python2.6/MySQLdb/cursors.py", line 166, in
execute
self.errorhandler(self, exc, value)
File "/usr/lib/pymodules/python2.6/MySQLdb/connections.py", line 35, in
defaulterrorhandler
raise errorclass, errorvalue
IntegrityError: (1062, “Duplicate entry ‘e110c3788ccb21c76865285446102338’
for key 'PRIMARY'”)
}}}
Periodically we do cleaning of out-of-date sessions, but it does not help
from current error.
--
Ticket URL: <https://code.djangoproject.com/ticket/16484>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: hash.3g@… | Owner: nobody
Type: Bug | Status: closed
Version: 1.3 | Severity: Normal
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
--------------------------------------+----------------------------------
Changes (by aaugustin):
* status: new => closed
* needs_better_patch: => 0
* resolution: => needsinfo
* needs_tests: => 0
* needs_docs: => 0
Comment:
From code inspection, I don't see how this can happen: it the call to
`obj.save(force_insert=must_create, using=using)`, `must_create` is
`False`, so the session should just be updated if it already exists. Here,
you're hitting a naked `raise`, which means that the original author
thought this should never happen.
I can only imagine one cause for a race condition: if two requests running
in two separate processes open a transaction, then commit: under some
isolation levels, this may fail.
Could you provide more information about your database setup so we can try
to reproduce this?
--
Ticket URL: <https://code.djangoproject.com/ticket/16484#comment:1>
Django
Reporter: hash.3g@… | Owner: nobody
Type: Bug | Status: reopened
Version: 1.3 | Severity: Normal
UI/UX: 0 |
--------------------------------------+----------------------------------
Changes (by hash.3g@…):
* status: closed => reopened
* resolution: needsinfo =>
Comment:
Here's my settings for project, and I attached my.cnf file if needed
{{{
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql', # Add 'postgresql_psycopg2',
'postgresql', 'mysql', 'sqlite3' or 'oracle'.
'NAME': 'dou', # Or path to database file if
using sqlite3.
'USER': 'hash', # Not used with sqlite3.
'PASSWORD': '*******', # Not used with sqlite3.
'HOST': '', # Set to empty string for
localhost. Not used with sqlite3.
'PORT': '', # Set to empty string for
default. Not used with sqlite3.
'OPTIONS': {"init_command": "SET storage_engine=INNODB;"}
}
}
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/16484#comment:2>
Django
Reporter: hash.3g@… | Owner: nobody
Type: Bug | Status: reopened
Version: 1.3 | Severity: Normal
UI/UX: 0 |
--------------------------------------+----------------------------------
Comment (by aaugustin):
Which version of MySQL are you using?
--
Ticket URL: <https://code.djangoproject.com/ticket/16484#comment:3>
Django
Reporter: hash.3g@… | Owner: nobody
Type: Bug | Status: reopened
Version: 1.3 | Severity: Normal
UI/UX: 0 |
--------------------------------------+----------------------------------
Comment (by anonymous):
I am using 5.1.49
--
Ticket URL: <https://code.djangoproject.com/ticket/16484#comment:4>
Django
Reporter: hash.3g@… | Owner: nobody
Type: Bug | Status: reopened
Version: 1.3 | Severity: Normal
Triage Stage: Accepted | Has patch: 0
UI/UX: 0 |
-------------------------------------+----------------------------------
Changes (by aaugustin):
* stage: Unreviewed => Accepted
Comment:
It's really hard to tell what happens here :/
I will accept the ticket because the traceback proves the error.
--
Ticket URL: <https://code.djangoproject.com/ticket/16484#comment:5>
Django
Reporter: hash.3g@… | Owner: nobody
Type: Bug | Status: reopened
Component: contrib.sessions | Version: 1.3
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Comment (by sidmitra.del@…):
I think i can duplicate the error. I have the same Mysql version(with
INNODB).
{{{
File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-
packages/django/core/handlers/base.py", line 178, in get_response
response = middleware_method(request, response)
File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-
packages/django/contrib/sessions/middleware.py", line 36, in
process_response
request.session.save()
File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-
packages/django/contrib/sessions/backends/cached_db.py", line 31, in save
super(SessionStore, self).save(must_create)
File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-
packages/django/contrib/sessions/backends/db.py", line 63, in save
obj.save(force_insert=must_create, using=using)
File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-
packages/django/db/models/base.py", line 460, in save
self.save_base(using=using, force_insert=force_insert,
force_update=force_update)
File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-
packages/django/db/models/base.py", line 553, in save_base
result = manager._insert(values, return_id=update_pk, using=using)
File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-
packages/django/db/models/manager.py", line 195, in _insert
return insert_query(self.model, values, **kwargs)
File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-
packages/django/db/models/query.py", line 1436, in insert_query
return query.get_compiler(using=using).execute_sql(return_id)
File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-
packages/johnny/cache.py", line 344, in newfun
ret = original(cls, *args, **kwargs)
File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-
packages/django/db/models/sql/compiler.py", line 791, in execute_sql
cursor = super(SQLInsertCompiler, self).execute_sql(None)
File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-
packages/johnny/cache.py", line 293, in newfun
return original(cls, *args, **kwargs)
File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-
packages/django/db/models/sql/compiler.py", line 735, in execute_sql
cursor.execute(sql, params)
File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-
packages/django/db/backends/mysql/base.py", line 86, in execute
return self.cursor.execute(query, args)
File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-
packages/MySQLdb/cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-
packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
IntegrityError: (1062, “Duplicate entry
‘e4d75e04ee5995bd2e03d25a697a00d1’ for key 'PRIMARY'”)
}}}
Here're my session settings:
{{{
SESSION_ENGINE =
'django.contrib.sessions.backends.cached_db'
SESSION_COOKIE_AGE = 2 * 60 * 60
SESSION_EXPIRE_AT_BROWSER_CLOSE = True
}}}
Note: I use johnnny-cache alongside
--
Ticket URL: <https://code.djangoproject.com/ticket/16484#comment:6>
Django
Reporter: hash.3g@… | Owner: nobody
Type: Bug | Status: reopened
Component: contrib.sessions | Version: 1.3
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Comment (by aaugustin):
#18344 may describe the same problem.
--
Ticket URL: <https://code.djangoproject.com/ticket/16484#comment:7>
Django
Reporter: hash.3g@… | Owner: nobody
Type: Bug | Status: closed
Component: contrib.sessions | Version: 1.3
Severity: Normal | Resolution: duplicate
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by aaugustin):
* status: reopened => closed
* resolution: => duplicate
Comment:
This is suspiciously similar to #18557.
The conclusion [https://groups.google.com/d/topic/django-developers
/wvpysb-fXtc/discussion on the mailing list] was to use the READ COMMITTED
isolation level (recommended), or database-level autocommit of every query
(not recommended).
MySQL defaults to REPEATABLE READ, which appears to make such situations
intractable.
--
Ticket URL: <https://code.djangoproject.com/ticket/16484#comment:8>
Django
Reporter: hash.3g@… | Owner: nobody
Type: Bug | Status: closed
Component: contrib.sessions | Version: 1.3
Severity: Normal | Resolution: duplicate
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by jokerejoker):
This is still a problem for Django 1.5.1
{{{
File "/usr/local/lib/python2.7/dist-
packages/Django-1.5.1-py2.7.egg/django/core/handlers/base.py", line 187,
in get_response
response = middleware_method(request, response)
File "/usr/local/lib/python2.7/dist-
packages/Django-1.5.1-py2.7.egg/django/contrib/sessions/middleware.py",
line 38, in process_response
request.session.save()
File "/usr/local/lib/python2.7/dist-
packages/Django-1.5.1-py2.7.egg/django/contrib/sessions/backends/db.py",
line 57, in save
obj.save(force_insert=must_create, using=using)
File "/usr/local/lib/python2.7/dist-
packages/Django-1.5.1-py2.7.egg/django/db/models/base.py", line 546, in
save
force_update=force_update, update_fields=update_fields)
File "/usr/local/lib/python2.7/dist-
packages/Django-1.5.1-py2.7.egg/django/db/models/base.py", line 650, in
save_base
result = manager._insert([self], fields=fields, return_id=update_pk,
using=using, raw=raw)
File "/usr/local/lib/python2.7/dist-
packages/Django-1.5.1-py2.7.egg/django/db/models/manager.py", line 215, in
_insert
return insert_query(self.model, objs, fields, **kwargs)
File "/usr/local/lib/python2.7/dist-
packages/Django-1.5.1-py2.7.egg/django/db/models/query.py", line 1661, in
insert_query
return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/local/lib/python2.7/dist-
packages/Django-1.5.1-py2.7.egg/django/db/models/sql/compiler.py", line
937, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-
packages/Django-1.5.1-py2.7.egg/django/db/backends/mysql/base.py", line
122, in execute
six.reraise(utils.IntegrityError,
utils.IntegrityError(*tuple(e.args)), sys.exc_info()[2])
File "/usr/local/lib/python2.7/dist-
packages/Django-1.5.1-py2.7.egg/django/db/backends/mysql/base.py", line
120, in execute
return self.cursor.execute(query, args)
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in
execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36,
in defaulterrorhandler
raise errorclass, errorvalue
IntegrityError: (1062, "Duplicate entry '2v4qsr4c75qbdkn27u3dypfig46fca2z'
for key 'PRIMARY'")
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/16484#comment:9>
Django
Reporter: hash.3g@… | Owner: nobody
Type: Bug | Status: closed
Component: contrib.sessions | Version: 1.3
Severity: Normal | Resolution: duplicate
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by anonymous):
Replying to [comment:9 jokerejoker]:
> This is still a problem for Django 1.5.1
There was no claim above that this would be "fixed" by Django 1.5.1, since
it's suspected the underlying problem is MySQL's default "reapeatable
read" transaction isolation level. There is no way (aside from including a
commit within `get_and_create` itself, which is completely unacceptable)
to "fix" this inside of Django. The fix is to change your transaction
isolation level to "read committed".
--
Ticket URL: <https://code.djangoproject.com/ticket/16484#comment:10>
Django
Reporter: hash.3g@… | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Dylan Young):
* status: closed => new
* version: 1.3 => 1.8
* resolution: duplicate =>
Comment:
I'm not sure where to put this, as this bug has been marked duplicate of
another bug, marked duplicate of another bug...(most of which seem to have
been closed despite not being fixed), so I'll place this here.
The common conclusion seems to be that this behaviour comes from MySQL,
but this surfaces using PostgreSQL as well:
{{{
File "/home/www/env/local/lib/python2.7/site-
packages/django/core/handlers/base.py", line 223, in get_response
response = middleware_method(request, response)
File "/home/www/env/local/lib/python2.7/site-
packages/django/contrib/sessions/middleware.py", line 50, in
process_response
request.session.save()
File "/home/www/env/local/lib/python2.7/site-
packages/django/contrib/sessions/backends/cached_db.py", line 64, in save
super(SessionStore, self).save(must_create)
File "/home/www/env/local/lib/python2.7/site-
packages/django/contrib/sessions/backends/db.py", line 65, in save
obj.save(force_insert=must_create, using=using)
File "/home/www/env/local/lib/python2.7/site-
packages/django/db/models/base.py", line 734, in save
force_update=force_update, update_fields=update_fields)
File "/home/www/env/local/lib/python2.7/site-
packages/django/db/models/base.py", line 762, in save_base
updated = self._save_table(raw, cls, force_insert, force_update, using,
update_fields)
File "/home/www/env/local/lib/python2.7/site-
packages/django/db/models/base.py", line 846, in _save_table
result = self._do_insert(cls._base_manager, using, fields, update_pk,
raw)
File "/home/www/env/local/lib/python2.7/site-
packages/django/db/models/base.py", line 885, in _do_insert
using=using, raw=raw)
File "/home/www/env/local/lib/python2.7/site-
packages/django/db/models/manager.py", line 127, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/home/www/env/local/lib/python2.7/site-
packages/django/db/models/query.py", line 920, in _insert
return query.get_compiler(using=using).execute_sql(return_id)
File "/home/www/env/local/lib/python2.7/site-
packages/django/db/models/sql/compiler.py", line 974, in execute_sql
cursor.execute(sql, params)
File "/home/www/env/local/lib/python2.7/site-
packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/home/www/env/local/lib/python2.7/site-packages/django/db/utils.py",
line 98, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/home/www/env/local/lib/python2.7/site-
packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
IntegrityError: duplicate key value violates unique constraint
“django_session_pkey” DETAIL: Key
(session_key)=(34j86lxxh4alwelayjq5i77e8lppwdhp) already exists.
}}}
This is with the cached_db session storage and no special psycopg options.
Thoughts?
--
Ticket URL: <https://code.djangoproject.com/ticket/16484#comment:11>
Django
Reporter: hash.3g@… | Owner: nobody
Type: Bug | Status: closed
Component: contrib.sessions | Version: 1.8
Severity: Normal | Resolution: duplicate
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* status: new => closed
* resolution: => duplicate
Comment:
Providing only a traceback does not help much (also none of the previous
discussion references PostgreSQL, so reopening doesn't seem appropriate).
Please open a new ticket with complete details to reproduce the issue and
make sure you can reproduce it with the latest version of Django rather
than 1.8.
--
Ticket URL: <https://code.djangoproject.com/ticket/16484#comment:12>
Django
Reporter: hash.3g@… | Owner: nobody
Type: Bug | Status: closed
Component: contrib.sessions | Version: 1.8
Severity: Normal | Resolution: duplicate
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Dylan Young):
What information do you need? The stack trace is entirely within django,
so I don't know what else I can provide you. A quick check of Stack
Overflow says that this is a common, if sporadic, issue.
I mentioned all relevant configurations that came to mind; please request
more info if you need it: postgresql version? psycopg version? Not sure
what you need. Please elaborate before I open another issue to have it
closed immediately,
In any case, I'll see if I can pinpoint the problem myself if I get time.
If you can get me the information I need to get you the information you
need, I'll open a new issue.
It is a race condition, so I cannot spend the time reproducing it on 1.9
or later (that would be crazy).
If you can point me to a commit that you think may have solved the issue
in 1.9, or later, please do.
--
Ticket URL: <https://code.djangoproject.com/ticket/16484#comment:13>