Testing problems

80 views
Skip to first unread message

Daniele Procida

unread,
Dec 16, 2014, 4:34:24 PM12/16/14
to Django Cms-Developers
Hi everyone.

Can you spend five minutes checking something please - we're trying to get to the bottom of a problem that mysteriously affects some users but not others.

Can you do the following (it's from <http://docs.django-cms.org/en/support-3.0.x/contributing/testing.html#running-tests> by the way, and you will need to have Firefox installed):


# create a virtual environment
virtualenv test-django-cms

# activate it
cd test-django-cms/
source bin/activate

# get django CMS from GitHub
git clone g...@github.com:divio/django-cms.git

# install the dependencies for testing
# note that requirements files for other Django versions are also provided
pip install -r django-cms/test_requirements/django-1.6.txt

# run the test suite
python django-cms/develop.py test


Do you get any errors? What operating system and Python version are you running?

Thanks, your feedback will be very helpful.

Daniele

Angelo Dini

unread,
Dec 17, 2014, 3:45:35 AM12/17/14
to django-cms...@googlegroups.com
Tests are running using the procedure described above on:

OSX 10.10.1 | MacBook Pro (Retina, 15-inch, Late 2013)

getting the following terminal feedback on "master" branch:

Stefan Foulis

unread,
Dec 17, 2014, 3:47:03 AM12/17/14
to django-cms...@googlegroups.com
The testsuite failed for me.

mkvirtualenv test-django-cms
hub clone -p divio/django-cms
cd django-cms
pip install -r test_requirements/django-1.6.txt
python develop.py test


Firefox opens a window while running the testsuite. But nothing happens and it closes after a few seconds.

I have:
OSX Yosemite 10.10.1 (14B25)
python 2.7.8 (installed with homebrew)
Firefox 30.0
Chrome 39.0.2171.95 (64-bit)

Stefan Foulis

unread,
Dec 17, 2014, 4:05:29 AM12/17/14
to django-cms...@googlegroups.com
@angelo looking at the dpaste you probably mean the tests are failing (like for me).

Christopher Glass

unread,
Dec 17, 2014, 4:39:16 AM12/17/14
to django-cms...@googlegroups.com
So, I think the instructions are wrong :)

instead of:
python django-cms/develop.py test

try doing:
cd django-cms
python develop.py test

That should solve the templates not being found.

I still get some errors however, because I have no display (even
though firefox is installed and I use selenium successfully in other
projects).

OS: Ubuntu 14.04 (up to date, in an LXC however hence the absence of display).
Python 2.7.6
Firefox 34.0

I'll try on a machine with a display later on.

- Chris
> --
> You received this message because you are subscribed to the Google Groups "django CMS developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to django-cms-devel...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

Benjamin Wohlwend

unread,
Dec 17, 2014, 5:02:07 AM12/17/14
to Django Cms Developers
Upgrading selenium to 2.44.0 and running the tests inside the django-cms folder as advised by Chris took care of both types of failures, the test suite passes for me now.

OS: Ubuntu 14.10
Python: 2.7.8
Firefox: 34.0

Cheers,
Beni

Daniele Procida

unread,
Dec 17, 2014, 5:07:29 AM12/17/14
to Django Cms-Developers
On Wed, Dec 17, 2014, Christopher Glass <tri...@gmail.com> wrote:

>So, I think the instructions are wrong :)
>
>instead of:
>python django-cms/develop.py test
>
>try doing:
>cd django-cms
>python develop.py test
>
>That should solve the templates not being found.

Excellent, that explains why sometimes I was getting 39 and sometimes 9 errors!

But still, 9 errors: <http://dpaste.com/1K02QJ1>

(not test failures, by the way - actual errors).

Daniele

Jonas Obrist

unread,
Dec 17, 2014, 7:19:39 AM12/17/14
to django-cms...@googlegroups.com
Just a sidenote to Chris:

If you have some sort of X installed, you can pip install xvfbwrapper in the venv and run the tests with --xvfb and they won't need a physical display.

Christopher Glass

unread,
Dec 17, 2014, 8:55:14 AM12/17/14
to django-cms...@googlegroups.com
Ah good to know. I'll give it a try when I'm done fighting my daily dragons.

- Chris

Jonas Obrist

unread,
Dec 17, 2014, 8:26:03 PM12/17/14
to django-cms...@googlegroups.com
Turns out the issue was missing gettext on OSX/outdated selenium.


On Wednesday, December 17, 2014 10:55:14 PM UTC+9, ChrisGlass wrote:
Ah good to know. I'll give it a try when I'm done fighting my daily dragons.

- Chris

On Wed, Dec 17, 2014 at 3:19 PM, Jonas Obrist <ojii...@gmail.com> wrote:
> Just a sidenote to Chris:
>
> If you have some sort of X installed, you can pip install xvfbwrapper in the
> venv and run the tests with --xvfb and they won't need a physical display.
>
> --
> You received this message because you are subscribed to the Google Groups
> "django CMS developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an

Tim Graham

unread,
Dec 20, 2014, 7:29:39 AM12/20/14
to django-cms...@googlegroups.com
I also confirm that updating to selenium 2.44.0 from 2.42.1 fixes the issue on Ubuntu 14.04 with Firefox 34.0.

I have problems when running the selenium tests with Django 1.7 though. Is it a known issue?

Traceback (the last failure seems to a broken test, unrelated to selenium)
http://dpaste.com/16AR8SY

Not sure how often the selenium tests break without a unit test also showing a failure. In Django's own test suite, it's very rare and we don't run them by default as you may now. They are only run when executing the tests with --selenium or if an environment variable is set. You might consider a similar strategy to help make the test suite a tiny bit less painful for testing changes where the selenium tests aren't relevant.

Tim Graham

unread,
Jan 23, 2015, 9:53:35 PM1/23/15
to django-cms...@googlegroups.com
Still having problems with selenium on develop.

Minimal command to reproduce:

$ python develop.py test --failfast cms.ContextTests cms.PlaceholderBasicTests

test_context_current_page (cms.tests.views.ContextTests) ... ok
test_copy_from_language (cms.tests.frontend.PlaceholderBasicTests) ... fail

In `test_context_current_page`, the problematic line appears to be `self.client.get("/en/plain_view/")`. I guess there may be some cached state leaking?

With Django 1.7, this line in the failing test

self.driver.get('%s/it/?%s' % (self.live_server_url, get_cms_setting('CMS_TOOLBAR_URL__EDIT_ON')))

redirects to

http://localhost:8000/en/it/?edit  (which is a 404)

When the test passes on 1.6, the redirect is to

http://localhost:8000/en/?edit

Any advice on places to look?

Iacopo Spalletti

unread,
Jan 24, 2015, 2:00:26 AM1/24/15
to django-cms...@googlegroups.com
On 24/01/2015 03:53, Tim Graham wrote:
> Still having problems with selenium on develop.
>
> Minimal command to reproduce:
>
> $ python develop.py test --failfast cms.ContextTests
> cms.PlaceholderBasicTests
>
> test_context_current_page (cms.tests.views.ContextTests) ... ok
> test_copy_from_language (cms.tests.frontend.PlaceholderBasicTests) ... fail
>
> In `test_context_current_page`, the problematic line appears to be
> `self.client.get("/en/plain_view/")`. I guess there may be some cached
> state leaking?
>
> With Django 1.7, this line in the failing test
>
> self.driver.get('%s/it/?%s' % (self.live_server_url,
> get_cms_setting('CMS_TOOLBAR_URL__EDIT_ON')))
>
> redirects to
>
> http://localhost:8000/en/it/?edit (which is a 404)
>
> When the test passes on 1.6, the redirect is to
>
> http://localhost:8000/en/?edit
>
> Any advice on places to look?


Cannot replicate the failure on Linux (running both the complete test
suite and the above testcases) using selenium 2.44.0 and firefox 35.0.

Are you running on mac?
> > email to django-cms-devel...@googlegroups.com.
> > For more options, visit
> https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "django CMS developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-cms-devel...@googlegroups.com
> <mailto:django-cms-devel...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.


--
Saluti

Iacopo Spalletti

Tim Graham

unread,
Jan 24, 2015, 8:31:16 AM1/24/15
to django-cms...@googlegroups.com
Running Ubuntu 14.04, selenium 2.44.0, and firefox 35.0.

On 1.7, the failing test always wants to prefix "/en/" on whatever URLs I try to access. Can you point me to the code that does that?
>              > email to django-cms-developers+unsub...@googlegroups.com.
>              > For more options, visit
>             https://groups.google.com/d/optout
>             <https://groups.google.com/d/optout>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "django CMS developers" group.
> To unsubscribe from this group and stop receiving emails from it, send

Iacopo Spalletti

unread,
Jan 25, 2015, 1:48:20 PM1/25/15
to django-cms...@googlegroups.com
On 24/01/2015 14:31, Tim Graham wrote:
> Running Ubuntu 14.04, selenium 2.44.0, and firefox 35.0.
>
> On 1.7, the failing test always wants to prefix "/en/" on whatever URLs
> I try to access. Can you point me to the code that does that?

Is #3797 the fix for these issues?
> django-cms-devel...@googlegroups.com <javascript:>.
> > > For more options, visit
> > https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>
> > <https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>>.
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "django CMS developers" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send
> > an email to django-cms-devel...@googlegroups.com
> <javascript:>
> > <mailto:django-cms-devel...@googlegroups.com
> <javascript:>>.
> Saluti
>
> Iacopo Spalletti
>
> --
> You received this message because you are subscribed to the Google
> Groups "django CMS developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-cms-devel...@googlegroups.com
> <mailto:django-cms-devel...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.


--
Iacopo Spalletti

Nephila s.a.s. - Firenze
Telefono: +39 055 5357189
Assistenza Tecnica: +39 055 3985730
http://nephila.it

Tim Graham

unread,
Jan 25, 2015, 1:57:45 PM1/25/15
to django-cms...@googlegroups.com
Unfortunately, no, just a related thing I noticed while trying to debug the issue.
>      >              > For more options, visit
>      > https://groups.google.com/d/optout
>     <https://groups.google.com/d/optout>
>      >             <https://groups.google.com/d/optout
>     <https://groups.google.com/d/optout>>.
>      >
>      > --
>      > You received this message because you are subscribed to the Google
>      > Groups "django CMS developers" group.
>      > To unsubscribe from this group and stop receiving emails from it,
>     send
>      > an email to django-cms-developers+unsub...@googlegroups.com
>     <javascript:>
>      > <mailto:django-cms-developers+unsub...@googlegroups.com
>     <javascript:>>.
>      > For more options, visit https://groups.google.com/d/optout
>     <https://groups.google.com/d/optout>.
>
>
>     --
>     Saluti
>
>     Iacopo Spalletti
>
> --
> You received this message because you are subscribed to the Google
> Groups "django CMS developers" group.
> To unsubscribe from this group and stop receiving emails from it, send

Tim Graham

unread,
Jan 25, 2015, 10:05:24 PM1/25/15
to django-cms...@googlegroups.com
The root cause is some new caching in 1.7 introduced here: https://github.com/django/django/commit/2bab9d6d#diff-5682903b7aed1649cdb7095331e90d0dR408
and the fact that django-cms uses it's own settings override classes instead of Django's. The latter connects an appropriate receiver to clear the cache on settings change: https://github.com/django/django/blob/ee23e03637aa8b82311f93b0a660574a0512891a/django/utils/translation/trans_real.py#L49-L58

Does cms.test_utils.testcases.SettingsOverrideTestCase predate django.test.utils.override_settings()? Anyone know if there's a reason we can't eliminate the former and start using the latter?

Iacopo Spalletti

unread,
Jan 26, 2015, 2:58:37 AM1/26/15
to django-cms...@googlegroups.com
Il 26/01/2015 04:05, Tim Graham ha scritto:
> The root cause is some new caching in 1.7 introduced here:
> https://github.com/django/django/commit/2bab9d6d#diff-5682903b7aed1649cdb7095331e90d0dR408
> and the fact that django-cms uses it's own settings override classes
> instead of Django's. The latter connects an appropriate receiver to
> clear the cache on settings change:
> https://github.com/django/django/blob/ee23e03637aa8b82311f93b0a660574a0512891a/django/utils/translation/trans_real.py#L49-L58
>
> Does cms.test_utils.testcases.SettingsOverrideTestCase predate
> django.test.utils.override_settings()? Anyone know if there's a reason
> we can't eliminate the former and start using the latter?

If I remember correctly custom-built settings override predates even
Django 1.3; I see no reason to stick with it.
> > django-cms-devel...@googlegroups.com
> <javascript:>.
> > > > For more options, visit
> > > https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>
> > <https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>>
> > > <https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>
> > <https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>>>.
> > >
> > > --
> > > You received this message because you are subscribed
> to the Google
> > > Groups "django CMS developers" group.
> > > To unsubscribe from this group and stop receiving
> emails from it,
> > send
> > > an email to
> django-cms-devel...@googlegroups.com
> > <javascript:>
> > >
> <mailto:django-cms-devel...@googlegroups.com
> > Saluti
> >
> > Iacopo Spalletti
> >
> > --
> > You received this message because you are subscribed to the
> Google
> > Groups "django CMS developers" group.
> > To unsubscribe from this group and stop receiving emails from
> it, send
> > an email to django-cms-devel...@googlegroups.com
> > <mailto:django-cms-devel...@googlegroups.com>.
> Iacopo Spalletti
>
> Nephila s.a.s. - Firenze
> Telefono: +39 055 5357189
> Assistenza Tecnica: +39 055 3985730
> http://nephila.it
>
> --
> You received this message because you are subscribed to the Google
> Groups "django CMS developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-cms-devel...@googlegroups.com
> <mailto:django-cms-devel...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.


--
Saluti

Iacopo Spalletti

Tim Graham

unread,
Jan 26, 2015, 5:19:36 PM1/26/15
to django-cms...@googlegroups.com
I added a commit to https://github.com/divio/django-cms/pull/3797 and now I have a passing test suite on 1.7 so that could be merged to remedy the issue for now.

In follow up to that, I am working to replace SettingsOverrideTestCase and SettingsOverride with Django's utilities. The one usage I am not sure about is in non-test code (cms.views): https://github.com/divio/django-cms/blob/0dbb961f8427a74845dc9f226987b8f4a525bed8/cms/views.py#L114 It seems unclean to be importing from test_utils (or django.test). The situation on modifying settings at runtime is... unclear: https://code.djangoproject.com/ticket/14628 Possibly we should just move SettingsOverride to cms.utils and keep using it for now.
>         <javascript:>.
>          >      >              > For more options, visit
>          >      > https://groups.google.com/d/optout
>         <https://groups.google.com/d/optout>
>          >     <https://groups.google.com/d/optout
>         <https://groups.google.com/d/optout>>
>          >      >             <https://groups.google.com/d/optout
>         <https://groups.google.com/d/optout>
>          >     <https://groups.google.com/d/optout
>         <https://groups.google.com/d/optout>>>.
>          >      >
>          >      > --
>          >      > You received this message because you are subscribed
>         to the Google
>          >      > Groups "django CMS developers" group.
>          >      > To unsubscribe from this group and stop receiving
>         emails from it,
>          >     send
>          >      > an email to
>         django-cms-developers+unsub...@googlegroups.com
>          >     <javascript:>
>          >      >
>         <mailto:django-cms-developers+unsub...@googlegroups.com
>          >     <javascript:>>.
>          >      > For more options, visit
>         https://groups.google.com/d/optout
>         <https://groups.google.com/d/optout>
>          >     <https://groups.google.com/d/optout
>         <https://groups.google.com/d/optout>>.
>          >
>          >
>          >     --
>          >     Saluti
>          >
>          >     Iacopo Spalletti
>          >
>          > --
>          > You received this message because you are subscribed to the
>         Google
>          > Groups "django CMS developers" group.
>          > To unsubscribe from this group and stop receiving emails from
>         it, send
>          > For more options, visit https://groups.google.com/d/optout
>         <https://groups.google.com/d/optout>.
>
>
>         --
>         Iacopo Spalletti
>
>         Nephila s.a.s. - Firenze
>         Telefono: +39 055 5357189
>         Assistenza Tecnica: +39 055 3985730
>         http://nephila.it
>
> --
> You received this message because you are subscribed to the Google
> Groups "django CMS developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
Reply all
Reply to author
Forward
0 new messages