Bug#774792: neutron-common fails upgrade, neutron-manage-db

[Benedikt Trefzer]

Package: neutron-common
Version: 2014.1.3-8
Severity: normal

Hi
Upgrade of neutron-common without DB fails with error:

root@jessie:/# apt-get upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
neutron-common python-neutron
2 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/1193 kB of archives.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n]
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
Preconfiguring packages ...
PKG-Openstack now calling: dbc_go neutron-common configure 2014.1.3-7
(Reading database ... 35990 files and directories currently installed.)
Preparing to unpack .../neutron-common_2014.1.3-8_all.deb ...
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
Unpacking neutron-common (2014.1.3-8) over (2014.1.3-7) ...
Preparing to unpack .../python-neutron_2014.1.3-8_all.deb ...
Unpacking python-neutron (2014.1.3-8) over (2014.1.3-7) ...
Setting up python-neutron (2014.1.3-8) ...
Setting up neutron-common (2014.1.3-8) ...
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
PKG-Openstack now calling: dbc_go neutron-common configure 2014.1.3-7
dbconfig-common: writing config to /etc/dbconfig-common/neutron-common.conf
No handlers could be found for logger "neutron.common.legacy"
Traceback (most recent call last):
File "/usr/bin/neutron-db-manage", line 10, in <module>
sys.exit(main())
File "/usr/lib/python2.7/dist-packages/neutron/db/migration/cli.py",
line 169, in main
CONF.command.func(config, CONF.command.name)
File "/usr/lib/python2.7/dist-packages/neutron/db/migration/cli.py",
line 82, in do_upgrade_downgrade
do_alembic_command(config, cmd, revision, sql=CONF.command.sql)
File "/usr/lib/python2.7/dist-packages/neutron/db/migration/cli.py",
line 60, in do_alembic_command
getattr(alembic_command, cmd)(config, *args, **kwargs)
File "/usr/lib/python2.7/dist-packages/alembic/command.py", line 125,
in upgrade
script.run_env()
File "/usr/lib/python2.7/dist-packages/alembic/script.py", line 203,
in run_env
util.load_python_file(self.dir, 'env.py')
File "/usr/lib/python2.7/dist-packages/alembic/util.py", line 212, in
load_python_file
module = load_module_py(module_id, path)
File "/usr/lib/python2.7/dist-packages/alembic/compat.py", line 58, in
load_module_py
mod = imp.load_source(module_id, path, fp)
File
"/usr/lib/python2.7/dist-packages/neutron/db/migration/alembic_migrations/env.py",
line 103, in <module>
run_migrations_online()
File
"/usr/lib/python2.7/dist-packages/neutron/db/migration/alembic_migrations/env.py",
line 76, in run_migrations_online
poolclass=pool.NullPool)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/__init__.py",
line 362, in create_engine
return strategy.create(*args, **kwargs)
File
"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line
49, in create
u = url.make_url(name_or_url)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/url.py", line
176, in make_url
return _parse_rfc1738_args(name_or_url)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/url.py", line
225, in _parse_rfc1738_args
"Could not parse rfc1738 URL from string '%s'" % name)
sqlalchemy.exc.ArgumentError: Could not parse rfc1738 URL from string ''
dpkg: error processing package neutron-common (--configure):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
neutron-common
E: Sub-process /usr/bin/dpkg returned an error code (1)


Steps to reproduce on a fresh jessie system:

add
deb http://snapshot.debian.org/archive/debian/20150101T040509Z/
jessie main
to /etc/apt/sources.list

run:
# export DEBIAN_FRONTEND=noninteractive; # Important !!
# apt-get install neutron-common=2014.1.3-7 python-neutron=2014.1.3-7

remove in /etc/neutron/neutron.conf section [database] the line startin
with connection.
remark: The database configuration is only necessary for package
neutron-server, all
other neutron agents/services do not need a DB connection.

run:
#apt-get upgrade

error message is above.




I did some additional investigation on this:

1.) The problem occurs in neutron-common line 743:
neutron-db-manage --config-file /etc/neutron/neutron.conf
upgrade head
this fails if the Database connection is not known.

2.) If you set the debconf value of neutron/configure_db to false,
then neutron-db-manage is not run, the error does not occur.

3.) There could be a similiar issue in nova-common. But there
nova/configure_db has a default value of false.
neutron/configure_db default value is true.

4.) if you set the connection string to
connection =
postgresql://blah_neutron:XXXX...@myDBhost.somewhere.ch/neutron_db
it gets overwritten with:
connection = sqlite:////var/lib/neutron/neutrondb
this violates debian policy overwritting a config file. (upgrade
then does not fail, since the sqlite file is still available from
the initial setup !
But the used neutron_db is not updated.)

5.) there is bug #770944 which seems to fix some override of the
connection string.
But see #4 ?

6.) Probably the start of neutron-db-manage should not depend on wheter
postinst script installed a db or not
(debconf neutron/configure_db).


Cheers

Benedikt Trefzer


--
To UNSUBSCRIBE, email to debian-bugs-...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org

[Thomas Goirand]

On 01/07/2015 06:37 PM, Benedikt Trefzer wrote:
> run:
> # export DEBIAN_FRONTEND=noninteractive; # Important !!
> # apt-get install neutron-common=2014.1.3-7 python-neutron=2014.1.3-7
>
> remove in /etc/neutron/neutron.conf section [database] the line startin
> with connection.

Why would you do this?

> remark: The database configuration is only necessary for package
> neutron-server, all
> other neutron agents/services do not need a DB connection.

Well, this may be somehow related, but read on...

In the neutron-common.postinst, I can read (re-indented, so that it read
better through a mail client):

db_get neutron/configure_db
if [ "$RET" = "true" ] ; then
# Set the sql_connection directive
pkgos_dbc_postinst --suite neutron ${N_CONF} database \
connection neutron $@
# Migrating to latest release of the db

neutron-db-manage --config-file /etc/neutron/neutron.conf \
upgrade head

fi

By default, neutron/configure_db is set to false. So if you use the
non-interactive mode, and don't preseed the neutron/configure_db value,
nothing will happen. But that's not what happened in your installation.

Here, you've asked though debconf for handling of the database bits of
Neutron. Then you removed the database directive from the config file,
then performed an upgrade. It then obviously fails to connect to the db
when attempting to handle the db and its upgrade.

To sum-up: do stupid things, then shit happens. Probably this would be a
better title for this bug? :)

Your thoughts?

Cheers,

Thomas Goirand (zigo)

[Benedikt Trefzer]

Hi Thomas

See neutron/debian/neutron-common.templates
Line 46 to 48:

46 Template: neutron/configure_db
47 Type: boolean
48 Default: true

So no I did not asked debconf to handle my database setting.

And by the way: I do not think it's stupid to delete unused wrong
settings in a config file (the default install creates a stupid setting
by using sqlite on localhost, which is definitly wrong on a distributed
neutron installation).

Cheers

Benedikt

[Thomas Goirand]

Yeah, I've understood that on the other bug report. I've uploaded
version -10 and asked the release team for an unblock.

> So no I did not asked debconf to handle my database setting.

Yup, my bad...

> And by the way: I do not think it's stupid to delete unused wrong
> settings in a config file

It would have been if the default was false, and if you had willingly
reverted it. Though here the issue is the default value that was wrong.
Now, probably you'd be better off altering your current debconf db to
avoid any further issue on upgrades.

Cheers,

Thomas