[Django] #23541: RunSQL Bug Django 1.7
Django
-------------------------------+------------------------------------------
Reporter: abhillman | Owner: nobody
Type: Uncategorized | Status: new
Component: Migrations | Version: 1.7
Severity: Normal | Keywords: migrations runsql postgresql
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+------------------------------------------
Django migrations are wonderful, but I have found a bug. Here's how to
reproduce (there is probably a more minimal case):
1. Install this function at a shell for PostgreSQL
```CREATE FUNCTION exec(text) returns text language plpgsql volatile
AS $f$
BEGIN
EXECUTE $1;
RETURN $1;
END;
$f$;```
source: http://wiki.postgresql.org/wiki/Dynamic_DDL
2. Write a RunSQL operation to help rename some tables
```class Migration(migrations.Migration):
...
operations = [
migrations.RunPython(
code=x
),
migrations.RunSQL(
sql="""select exec(format('alter table %I rename to %I',
tablename, regexp_replace(tablename, '^hello, 'goodbye'))) from pg_tables
where tablename like 'hello%';"""
),
]```
3. Run the migration. I got this error:
```Traceback (most recent call last):
File "./manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/Users/abhillman/stable_env/lib/python2.7/site-
packages/django/core/management/__init__.py", line 385, in
execute_from_command_line
utility.execute()
File "/Users/abhillman/stable_env/lib/python2.7/site-
packages/django/core/management/__init__.py", line 377, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/abhillman/stable_env/lib/python2.7/site-
packages/django/core/management/base.py", line 288, in run_from_argv
self.execute(*args, **options.__dict__)
File "/Users/abhillman/stable_env/lib/python2.7/site-
packages/django/core/management/base.py", line 338, in execute
output = self.handle(*args, **options)
File "/Users/abhillman/stable_env/lib/python2.7/site-
packages/django/core/management/commands/migrate.py", line 160, in handle
executor.migrate(targets, plan, fake=options.get("fake", False))
File "/Users/abhillman/stable_env/lib/python2.7/site-
packages/django/db/migrations/executor.py", line 63, in migrate
self.apply_migration(migration, fake=fake)
File "/Users/abhillman/stable_env/lib/python2.7/site-
packages/django/db/migrations/executor.py", line 97, in apply_migration
migration.apply(project_state, schema_editor)
File "/Users/abhillman/stable_env/lib/python2.7/site-
packages/django/db/migrations/migration.py", line 107, in apply
operation.database_forwards(self.app_label, schema_editor,
project_state, new_state)
File "/Users/abhillman/stable_env/lib/python2.7/site-
packages/django/db/migrations/operations/special.py", line 69, in
database_forwards
schema_editor.execute(statement)
File "/Users/abhillman/stable_env/lib/python2.7/site-
packages/django/db/backends/schema.py", line 98, in execute
cursor.execute(sql, params)
File "/Users/abhillman/stable_env/lib/python2.7/site-
packages/django/db/backends/utils.py", line 81, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/Users/abhillman/stable_env/lib/python2.7/site-
packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
IndexError: list index out of range```
Not sure what's going on here. Could it be the custom function?
--
Ticket URL: <https://code.djangoproject.com/ticket/23541>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: abhillman | Owner: nobody
Type: Uncategorized | Status: new
Component: Migrations | Version: 1.7
Keywords: migrations runsql | Triage Stage:
postgresql | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by abhillman):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Old description:
New description:
Django migrations are wonderful, but I have found a bug. Here's how to
reproduce (there is probably a more minimal case):
1. Install this function at a shell for PostgreSQL
{{{CREATE FUNCTION exec(text) returns text language plpgsql volatile
AS $f$
BEGIN
EXECUTE $1;
RETURN $1;
END;
$f$;}}}
source: http://wiki.postgresql.org/wiki/Dynamic_DDL
2. Write a RunSQL operation to help rename some tables
{{{class Migration(migrations.Migration):
...
operations = [
migrations.RunPython(
code=x
),
migrations.RunSQL(
sql="""select exec(format('alter table %I rename to %I',
tablename, regexp_replace(tablename, '^hello, 'goodbye'))) from pg_tables
where tablename like 'hello%';"""
),
]
...}}}
Not sure what's going on here. Could it be the custom PGSQL function?
--
--
Ticket URL: <https://code.djangoproject.com/ticket/23541#comment:1>
Django
Reporter: abhillman | Owner: nobody
Type: Uncategorized | Status: new
Component: Migrations | Version: 1.7
Keywords: migrations runsql | Triage Stage:
postgresql | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Description changed by abhillman:
Old description:
> Django migrations are wonderful, but I have found a bug. Here's how to
> reproduce (there is probably a more minimal case):
>
> 1. Install this function at a shell for PostgreSQL
>
> {{{CREATE FUNCTION exec(text) returns text language plpgsql volatile
> AS $f$
> BEGIN
> EXECUTE $1;
> RETURN $1;
> END;
> $f$;}}}
>
> source: http://wiki.postgresql.org/wiki/Dynamic_DDL
>
> 2. Write a RunSQL operation to help rename some tables
>
> {{{class Migration(migrations.Migration):
> ...
> operations = [
> migrations.RunPython(
> code=x
> ),
> migrations.RunSQL(
> sql="""select exec(format('alter table %I rename to %I',
> tablename, regexp_replace(tablename, '^hello, 'goodbye'))) from pg_tables
> where tablename like 'hello%';"""
> ),
> ]
> ...}}}
> Not sure what's going on here. Could it be the custom PGSQL function?
New description:
Django migrations are wonderful, but I have found a bug. Here's how to
reproduce (there is probably a more minimal case):
1. Install this function at a shell for PostgreSQL
{{{
CREATE FUNCTION exec(text) returns text language plpgsql volatile
AS $f$
BEGIN
EXECUTE $1;
RETURN $1;
END;
$f$;
}}}
source: http://wiki.postgresql.org/wiki/Dynamic_DDL
2. Write a RunSQL operation to help rename some tables
{{{
class Migration(migrations.Migration):
...
operations = [
migrations.RunPython(
code=x
),
migrations.RunSQL(
sql="""select exec(format('alter table %I rename to %I',
tablename, regexp_replace(tablename, '^hello, 'goodbye'))) from pg_tables
where tablename like 'hello%';"""
),
]
...
}}}
Not sure what's going on here. Could it be the custom PGSQL function?
--
--
Ticket URL: <https://code.djangoproject.com/ticket/23541#comment:2>
Django
Reporter: abhillman | Owner: nobody
Type: Uncategorized | Status: new
Component: Migrations | Version: 1.7
Keywords: migrations runsql | Triage Stage:
postgresql | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Description changed by abhillman:
Old description:
> Django migrations are wonderful, but I have found a bug. Here's how to
> reproduce (there is probably a more minimal case):
>
> 1. Install this function at a shell for PostgreSQL
>
> {{{
> CREATE FUNCTION exec(text) returns text language plpgsql volatile
> AS $f$
> BEGIN
> EXECUTE $1;
> RETURN $1;
> END;
> $f$;
> }}}
>
> source: http://wiki.postgresql.org/wiki/Dynamic_DDL
>
> 2. Write a RunSQL operation to help rename some tables
>
> {{{
> class Migration(migrations.Migration):
> ...
> operations = [
> migrations.RunPython(
> code=x
> ),
> migrations.RunSQL(
> sql="""select exec(format('alter table %I rename to %I',
> tablename, regexp_replace(tablename, '^hello, 'goodbye'))) from pg_tables
> where tablename like 'hello%';"""
> ),
> ]
> ...
> Not sure what's going on here. Could it be the custom PGSQL function?
New description:
Django migrations are wonderful, but I believe I have found a bug. The
essence of what I am trying to do here: rename all tables starting with
the name "hello" (i.e. hello_1, hello2_, hello_3, ...) to start with the
name "goodbye (goodbye_1, goodbye_2, goodbye_3, ...), which could be very
helpful when renaming an application.
Here's how to reproduce (there is probably a more minimal case):
1. Install this function at a shell for PostgreSQL (very handy for
{{{
CREATE FUNCTION exec(text) returns text language plpgsql volatile
AS $f$
BEGIN
EXECUTE $1;
RETURN $1;
END;
$f$;
}}}
source: http://wiki.postgresql.org/wiki/Dynamic_DDL
2. Write a RunSQL operation to help rename some tables
{{{
class Migration(migrations.Migration):
...
operations = [
migrations.RunPython(
code=x
),
migrations.RunSQL(
sql="""select exec(format('alter table %I rename to %I',
tablename, regexp_replace(tablename, '^hello, 'goodbye'))) from pg_tables
where tablename like 'hello%';"""
),
]
...
}}}
Not sure what's going on here. Could it be the custom PGSQL function?
--
--
Ticket URL: <https://code.djangoproject.com/ticket/23541#comment:3>
Django
Reporter: abhillman | Owner: nobody
Type: Uncategorized | Status: new
Component: Migrations | Version: 1.7
Keywords: migrations runsql | Triage Stage:
postgresql | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Description changed by abhillman:
Old description:
> Django migrations are wonderful, but I believe I have found a bug. The
> essence of what I am trying to do here: rename all tables starting with
> the name "hello" (i.e. hello_1, hello2_, hello_3, ...) to start with the
> name "goodbye (goodbye_1, goodbye_2, goodbye_3, ...), which could be very
> helpful when renaming an application.
>
> Here's how to reproduce (there is probably a more minimal case):
>
> 1. Install this function at a shell for PostgreSQL (very handy for
>
> {{{
> CREATE FUNCTION exec(text) returns text language plpgsql volatile
> AS $f$
> BEGIN
> EXECUTE $1;
> RETURN $1;
> END;
> $f$;
> }}}
>
> source: http://wiki.postgresql.org/wiki/Dynamic_DDL
>
> 2. Write a RunSQL operation to help rename some tables
>
> {{{
> class Migration(migrations.Migration):
> ...
> operations = [
> migrations.RunPython(
> code=x
> ),
> migrations.RunSQL(
> sql="""select exec(format('alter table %I rename to %I',
> tablename, regexp_replace(tablename, '^hello, 'goodbye'))) from pg_tables
> where tablename like 'hello%';"""
> ),
> ]
> ...
> Not sure what's going on here. Could it be the custom PGSQL function?
New description:
Django migrations are wonderful, but I believe I have found a bug. The
essence of what I am trying to do here: rename all tables starting with
the name "hello" (i.e. hello_1, hello2_, hello_3, ...) to start with the
name "goodbye (goodbye_1, goodbye_2, goodbye_3, ...), which could be very
helpful when renaming an application.
Here's how to reproduce (there is probably a more minimal case):
1. Install this function at a shell for PostgreSQL (very handy for
{{{
CREATE FUNCTION exec(text) returns text language plpgsql volatile
AS $f$
BEGIN
EXECUTE $1;
RETURN $1;
END;
$f$;
}}}
source: http://wiki.postgresql.org/wiki/Dynamic_DDL
2. Write a RunSQL operation to help rename some tables
{{{
class Migration(migrations.Migration):
...
operations = [
migrations.RunSQL(
sql="""select exec(format('alter table %I rename to %I',
tablename, regexp_replace(tablename, '^hello, 'goodbye'))) from pg_tables
where tablename like 'hello%';"""
),
]
...
}}}
Not sure what's going on here. Could it be the custom PGSQL function?
--
--
Ticket URL: <https://code.djangoproject.com/ticket/23541#comment:4>
Django
Reporter: abhillman | Owner: nobody
Type: Uncategorized | Status: new
Component: Migrations | Version: 1.7
Keywords: migrations runsql | Triage Stage:
postgresql | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by abhillman):
Okay. I looked into this a bit more. I seemed to be able to resolve my
issue by escaping the '%' character (i.e. convert all instances of '%%' to
'%'). Inspecting the stack trace, I am not entirely clear on why this is
happening. Perhaps this is the correct behavior on django's behalf. Any
enlightenment on this issue would be much appreciated. In addition, I
wonder if some warning message or other error message might be appropriate
or helpful in this sort of situation.
--
Ticket URL: <https://code.djangoproject.com/ticket/23541#comment:5>
Django
Reporter: abhillman | Owner: nobody
Type: Uncategorized | Status: new
Component: Migrations | Version: 1.7
Keywords: migrations runsql | Triage Stage:
postgresql | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by claudep):
I think I encountered something similar while working on an unrelated
issue.
Could you try applying the following diff and tell us if this changes
anything for you?
{{{
diff --git a/django/db/backends/schema.py b/django/db/backends/schema.py
index b7e285d..e2d5b54 100644
--- a/django/db/backends/schema.py
+++ b/django/db/backends/schema.py
@@ -95,7 +95,7 @@ class BaseDatabaseSchemaEditor(object):
# Log the command we're running, then run it
logger.debug("%s; (params %r)" % (sql, params))
if self.collect_sql:
- self.collected_sql.append((sql % tuple(map(self.quote_value,
params))) + ";")
+ self.collected_sql.append((sql % tuple(map(self.quote_value,
params or []))) + ";")
else:
with self.connection.cursor() as cursor:
cursor.execute(sql, params)
@@ -268,7 +268,7 @@ class BaseDatabaseSchemaEditor(object):
"table": self.quote_name(model._meta.db_table),
"definition": ", ".join(column_sqls)
}
- self.execute(sql, params)
+ self.execute(sql, params or None)
# Add any index_togethers
for field_names in model._meta.index_together:
fields = [model._meta.get_field_by_name(field)[0] for field
in field_names]
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/23541#comment:6>
Django
Reporter: abhillman | Owner: nobody
Type: Uncategorized | Status: new
Component: Migrations | Version: 1.7
Keywords: migrations runsql | Triage Stage:
postgresql | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by timgraham):
I think #23426 is related.
--
Ticket URL: <https://code.djangoproject.com/ticket/23541#comment:7>
Django
Reporter: abhillman | Owner: nobody
Type: Uncategorized | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: duplicate
Keywords: migrations runsql | Triage Stage:
postgresql | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by claudep):
* status: new => closed
* resolution: => duplicate
Comment:
I'd even say it's a duplicate.
--
Ticket URL: <https://code.djangoproject.com/ticket/23541#comment:8>