Strange overlaps in independent Pyramid projects

[Laurent DAVERIO]

Hello list,

I don't know exactly how to report 2 related "overlap" problems I've
recently encountered, and maybe it's not even the right list to report
them. But even then, it's relevant to Pyramid.

These problems started to appear when I created a second Pyramid project
under an existing virtualenv. The two Pyramid projects are called "foo"
and "bar". Each or them uses SQLAlchemy table inheritance, on two
distinct PostgreSQL databases, i.e. :

- A common e-commerce module provides a base class "Article", attached
to a table. This represents a generic article in the catalogue.

- The "foo" module provides a class "FooArticle", which inherits from
"Article". This creates to 2 linked tables, "article" and
"foo_article", in Pg database "foo".

- Similarly, the "bar" module provides a class "BarArticle" which also
inherits from "Article". This creates 2 tables, "article" and
"bar_article", in Pg database "bar".

- To make code more legible, the two classes "FooArticle" and
"BarArticle" are always used under the alias "Article" in their
respective projects (the parent class "Article" is never used directly),
i.e.:

from foo.models import FooArticle as Article

or:

from bar.models import BarArticle as Article

=====

Overlap #1: SQLAlchemy maps the wrong class to the returned data :

I assume this is because the classes bear the same names (or aliases) in
the two distinct projects :

$ cd foo
$ pshell development.ini

In [1]: from foo.models import *

In [2]: Article
Out[2]: foo.models.article.FooArticle

In [3]: print Session.query(Article)
SELECT [...]
FROM article JOIN foo_article ON article.id = foo_article.id

In [4]: a = Session.query(Article).first()

In [5]: a
Out[5]: <BarArticle id=19844>


In that example, "Article" is an alias of "FooArticle", which is
confirmed by [2]. The generated SQL query interrogates the right tables,
as shown by [3]. But the object returned by [4] should be an instance of
FooArticle, not BarArticle [5]!


=====

Overlap #2: Supervisord picks the wrong gunicorn_paster script

It happens even when the two projects are moved to separate virtualenvs.
I've been obliged to replace it with daemontools, temporarily or
permanently.

For instance, with the line below:

command = /projects/venv2/bin/gunicorn -w 4 --paste
/projects/venv2/bar/bar/production.ini

supervisord incorrectly launches /projects/venv1/bin/gunicorn...

This is totally illogical... :(


Laurent.

[Wichert Akkerman]

> On 30 Oct 2014, at 11:30, Laurent DAVERIO <ldav...@gmail.com> wrote:
> Overlap #1: SQLAlchemy maps the wrong class to the returned data :
>
> I assume this is because the classes bear the same names (or aliases) in the two distinct projects :
>
> $ cd foo
> $ pshell development.ini
>
> In [1]: from foo.models import *
>
> In [2]: Article
> Out[2]: foo.models.article.FooArticle
>
> In [3]: print Session.query(Article)
> SELECT [...]
> FROM article JOIN foo_article ON article.id = foo_article.id
>
> In [4]: a = Session.query(Article).first()
>
> In [5]: a
> Out[5]: <BarArticle id=19844>
>
>
> In that example, "Article" is an alias of "FooArticle", which is confirmed by [2]. The generated SQL query interrogates the right tables, as shown by [3]. But the object returned by [4] should be an instance of FooArticle, not BarArticle [5]!

You may want to ask about that one on the SQLAlchemy list, it might be a bug. The fact that you alias the classes to Article is not related (and SQLAlchemy can’t even see that). I suspect this has more to do with your use of row ids and inheritance setup.

> =====
>
> Overlap #2: Supervisord picks the wrong gunicorn_paster script
>
> It happens even when the two projects are moved to separate virtualenvs. I've been obliged to replace it with daemontools, temporarily or permanently.
>
> For instance, with the line below:
>
> command = /projects/venv2/bin/gunicorn -w 4 --paste /projects/venv2/bar/bar/production.ini
>
> supervisord incorrectly launches /projects/venv1/bin/gunicorn...
>
> This is totally illogical... :(

Don’t know about that one.

Wichert.

[Jonathan Vanasco]

Re: SqlAlchemy

The first thing I do with problems like this, is to remove every single .pyc and .pyo file in my projects' directory tree.

9/10 times, the problem is that I changed the some imports and old .pyc files are being picked up.  one way this can affect you is in this example:  if you rename foo.py to bar.py; and do not update the imports, python will use foo.pyc even though foo.py doesn't exist.  i think you can sometimes also run into permission issue, where the interpreter can't write the .pyc or .pyo files.  

there's a lot of really weird things that can happen if this is the case -- but a solid first step in solving issues like this is to remove every .pyc and .pyo file.

The second thing I would do, is to find a minimal way to recreate the problem.  If it is a bug, no one will be able to fix it if they can't recreate it.  If its not a bug, you won't be able to reproduce it , and probably find the real bug as a typo or other error.

Re: gunicorn

that looks like it's using the wrong gunicorn, not the wrong script.

are you sure that the supervisord file is correct?  i know this sound stupid, but i've had issues where I missed copy/pasting a section header