Randomly clearing database tables

[msbuck]

Twice now, I have had most of my django tables cleared from my
database including my app tables. Obviously I need to track this down.
In addition to the two tables created by my app, the AUTH_USER,
AUTH_USER_USER_PERMISSIONS and DJANGO_ADMIN_LOG are cleared. The
AUTH_PERMISSION, DJANGO_CONTENT_TYPE, and DJANGO_SESSION tables were
not cleared. Just to be clear, the tables still exist, there just
isn't any data in them.

I am using Eclipse with the PyDev plug-in to start the development
server. I also run my tests from Eclipse. I am using the development
server and connecting to an Oracle database. Immediately before this
happened, I was killing and restarting the development server while
working on an implementation problem (I was trying to get pdf
generation working). My app is fairly simple ... two tables, one has a
foreignKey to the other. I have written Django tests which use a
fixture to feed in data.

The only theory I have so far is that somehow the test framework's
clean-up code gets executed. I've looked a little at the testing
framework code but haven't gotten very far to see if this is a
plausible theory.

This is a rare occurrance (the other time this happened was a month
ago). Our group is new to Django so this is a rather upsetting
situation and makes people more reluctant to try it.

Any help or thoughts are appreciated.

[Ilian Iliev]

Hi,

this sounds like a bad configuration of the tests database.
Is it possible that your test are clearing your database?
Have you made some special test configuration?

--
eng. Ilian Iliev
Web Software Developer

Mobile: +359 88 66 08 400
Website: http://ilian.i-n-i.org

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django...@googlegroups.com.
To unsubscribe from this group, send email to django-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

[msbuck]

I'll look at any theory right now :-). I'm using the standard django
TestCase to write tests for my models, views, and forms. These are in
separate modules in the app test directory where I have imported them
into the __init__.py. My view tests do use a fixture to provide data
for the app's two tables. I think all of this is pretty standard.

I should note that I was not running tests when the tables where
cleared.

My current theory is that somehow manage.py flush was called. I did
not do it from the command line and I don't have this set up as a
command in Eclipse, so somehow it got called within the code. Only a
theory though ... I wish I could replicate the problem.

Thanks for the suggestion.

[John Handelaar]

On 4 October 2011 15:11, msbuck <mbuc...@usgs.gov> wrote:
> Twice now, I have had most of my django tables cleared from my
> database including my app tables. Obviously I need to track this down.
> In addition to the two tables created by my app, the AUTH_USER,
> AUTH_USER_USER_PERMISSIONS and DJANGO_ADMIN_LOG are cleared. The
> AUTH_PERMISSION, DJANGO_CONTENT_TYPE, and DJANGO_SESSION tables were
> not cleared. Just to be clear, the tables still exist, there just
> isn't any data in them.

Theory: you have a fixtures data file and you ran syncdb ?

jh

[msbuck]

I don't have an initial_data.* fixtures file although I do use one in
my tests. This data was not loaded. I didn't run syncdb or flush (very
certain of this). My current theory is that flush was somehow called
from my code. I just wish I could figure out how.

Thanks for the suggestion.


On Oct 4, 10:49 am, John Handelaar <j...@userfrenzy.com> wrote:

[msbuck]

Good news! Yesterday, I was repeatedly starting the development server
from Eclipse which is on the Run As -> menu which cascades to Run
Server or Run Python Unit Test. I must have accidently clicked that we
I really wanted to start my server. That command runs the PyDev Test
runner and that's what clears out my tables. Now I just have to figure
out how to disable the option in Eclipse or get it to do nothing. Any
suggestions in this area would be appreciated and thanks to all for
the suggestions.

[Ian]

On Oct 4, 10:30 am, msbuck <mbuckn...@usgs.gov> wrote:
> Good news! Yesterday, I was repeatedly starting the development server
> from Eclipse which is on the Run As  -> menu which cascades to Run
> Server or Run Python Unit Test. I must have accidently clicked that we
> I really wanted to start my server. That command runs the PyDev Test
> runner and that's what clears out my tables. Now I just have to figure
> out how to disable the option in Eclipse or get it to do nothing. Any
> suggestions in this area would be appreciated and thanks to all for
> the suggestions.

What does your DATABASES setting look like? Everything test-related
is supposed to be run under a separate schema and tablespace to avoid
modifying actual data. It sounds like something is running in the
wrong place.

[msbuck]

Below is the settings. I am fairly new to python unit testing so my
understanding of things could be better. But right now, if I select
Run As -> Python Unit Test will my project is selected, it runs all of
my Django tests and clobbers my working db as a result. The test
runner also appears to be ignoring my @skipUnless decorators that
should prevent db tests from running if settings.IS_TEST is set to
False. I'm getting frustrated again!



# This checks to see if django tests are running (i.e. manage.py test)
if argv and 1 < len(argv):
IS_TEST = 'test' == argv[1]
else:
IS_TEST = False

if IS_TEST:
DB_ENGINE = 'django.db.backends.sqlite3'
else:
DB_ENGINE = 'django.db.backends.oracle'

DATABASES = {
'default': {
'ENGINE': DB_ENGINE, # Add 'postgresql_psycopg2',
'postgresql', 'mysql', 'sqlite3' or 'oracle'.
'NAME': 'db_name', # Or path to database
file if using sqlite3.
'USER': 'spin_log_user', # Not used with
sqlite3.
'PASSWORD': 'XXXXX', # Not used with sqlite3.
'HOST' : '',
'PORT' : '',

[Ian Kelly]

On Tue, Oct 4, 2011 at 2:28 PM, msbuck <mbuc...@usgs.gov> wrote:
> Below is the settings. I am fairly new to python unit testing so my
> understanding of things could be better. But right now, if I select
> Run As -> Python Unit Test will my project is selected, it runs all of
> my Django tests and clobbers my working db as a result. The test
> runner also appears to be ignoring my @skipUnless decorators that
> should prevent db tests from running if settings.IS_TEST is set to
> False. I'm getting frustrated again!

I haven't used PyDev, but I would imagine it's probably importing the
management commands and running them programmatically, rather than
simply trying to run manage.py. That would explain why your IS_TEST
is False. According to the docs, PyDev has a DJANGO_SETTINGS_MODULE
variable. I suggest setting that to an alternate settings.py that
contains your test settings.

I still can't see any reason why the tests would be flushing your
tables, though. Does it happen when you issue manage.py test (using
the oracle backend) as well, or only from PyDev? Could there be
something in your app tests that might be causing the flush?

[msbuck]

The behavior I was seeing was as if the tests where run without using
manage.py. My app tests at this point are very simple. I use a fixture
to load data to do the view tests but don't explictly delete anything
from the tables.

I did basically what you suggested. You can set up the environment
variables that PyDev uses for unit tests and it can be different from
the regular running environment so I'm setting it to an explicit
settings.py module where sqlite3 is used.

I need to learn more about how PyDev runs it's unit tests. That will
be my next step.

Thanks for the advice.


On Oct 5, 12:09 pm, Ian Kelly <ian.g.ke...@gmail.com> wrote: