Randomly Deleted Orders and OrderItems.

[Brian Tol]

We recently ran into an odd situation with some missing data. We're
using 0.9.1.pre.

Here's what we're seeing:

a. An orderitem got deleted from the order. There were originally 3
items for the order (as verified by the email receipt, and the
transaction record in PayPal), but there are now only 2 items in the
database

b. One item got deleted all together. The customer received an email
confirmation, and we have a record of the card being processed in
PayPal. However there is no order in the database.

We didn't receive any error messages, and nothing else seems to be out
of place. The order associated with the missing orderitem had no admin
history associated with it: if a user deleted the product, they did so
via the shell or the dbshell.

Any thoughts as to what happened, or where we could be looking for
problems? Are there certain conditions where Satchmo decides to delete
an order?

It seems fairly random at this point: the store gets dozens of orders
a week.

Thanks for your help in advance!

-Brian

[John-Scott]

I have also experienced some untraceable strangeness. One of my
clients had a couple of orders that ended up in an odd state. They had
been marked as 'shipped' which was supposed to trigger payment
capture, but this never happened. Nothing was recorded in the Satchmo
log, no "Order payment failures", etc. to indicate what went wrong.

Core devs:
Any reason not to use database transactions to make sure we don't end
up with incomplete/corrupted data? I don't have any evidence that
would have saved my bacon in this particular case. But in the process
of investigating I noticed how many complex views there are creating
several different types of objects in one go but are not wrapped in a
transaction.

The only bit of code I can find that uses transactions currently is
product.forms.ProductImportForm.

In another thread [1] a user identified an issue with the way certain
livesettings are obtained (but this appears to only affect 0.8?).

I'm by no means a db transaction expert, so curious if there are other
known issues that make liberal use of transactions more complicated
than sprinkling commit_on_success decorators throughout the views.

If not, shall I make a ticket and get started on this in my bitbucket
fork?

Cheers,
John-Scott

[Chris Moffitt]

Core devs:
Any reason not to use database transactions to make sure we don't end
up with incomplete/corrupted data? I don't have any evidence that
would have saved my bacon in this particular case. But in the process
of investigating I noticed how many complex views there are creating
several different types of objects in one go but are not wrapped in a
transaction.

 

Losing data or having data in an inconsistent state is certainly very bad. In general, transactions would make sense but as you mention it will be hard to verify that more explicit transaction management would fix the issue.

So, to answer your question, forking and starting a branch with improved transaction management would be fine. I would just want to make sure we're smart about where it's used and not just using the shotgun approach to trying to solve the problem.

-Chris

[Mrainess]

Chris,

What about this, from Django middleware docs:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Transaction middleware

class django.middleware.transaction.TransactionMiddleware

Binds commit and rollback to the request/response phase. If a view
function runs successfully, a commit is done. If it fails with an
exception, a rollback is done.

The order of this middleware in the stack is important: middleware
modules running outside of it run with commit-on-save - the default
Django behavior. Middleware modules running inside it (coming later in
the stack) will be under the same transaction control as the view
functions.


Tying transactions to HTTP requests

The recommended way to handle transactions in Web requests is to tie
them to the request and response phases via Django’s
TransactionMiddleware.

It works like this: When a request starts, Django starts a
transaction. If the response is produced without problems, Django
commits any pending transactions. If the view function produces an
exception, Django rolls back any pending transactions.

To activate this feature, just add the TransactionMiddleware
middleware to your MIDDLEWARE_CLASSES setting:

MIDDLEWARE_CLASSES = (
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.cache.CacheMiddleware',
'django.middleware.transaction.TransactionMiddleware',
)

The order is quite important. The transaction middleware applies not
only to view functions, but also for all middleware modules that come
after it. So if you use the session middleware after the transaction
middleware, session creation will be part of the transaction.

An exception is CacheMiddleware, which is never affected. The cache
middleware uses its own database cursor (which is mapped to its own
database connection internally).
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

-Mark

[lzantal]

Hi,

John-Scott are you also using PayPal like Brian Tol?
If so it's more of a paypal payment module issue then satchmo.
Just a thought

lzantal

[Brian Tol]

Hi Izantal,

> If so it's more of a paypal payment module issue then satchmo.

Are you suggesting that the payment module may be deleting orders and
orderitems somehow? If so, any thoughts how that might be happening?

-Brian

[John-Scott Atlakson]

On Aug 4, 2010, at 9:19 AM, Brian Tol wrote:

> Hi Izantal,
>
>> If so it's more of a paypal payment module issue then satchmo.
>
> Are you suggesting that the payment module may be deleting orders and
> orderitems somehow? If so, any thoughts how that might be happening?
>
> -Brian
>
> On Aug 3, 2:40 pm, lzantal <lzan...@gmail.com> wrote:
>> Hi,
>>
>> John-Scott are you also using PayPal like Brian Tol?
>> If so it's more of a paypal payment module issue then satchmo.
>> Just a thought
>>
>> lzantal

I am not using PayPal.

Without knowing the nitty gritty, I do not see how the payment module could have been responsible for missing data in Brian's case. All the order information was intact when the payment was captured by PayPal. At some point after this, OrderItems or entire Orders went missing.

In satchmo_store.shop.models there is OrderManager.remove_partial_order() which will "Delete cart from request if it exists and is incomplete (has no status)". There is also Order.remove_all_items() which will "Delete all items belonging to this order." These were the only two explicit references to deleting Order and OrderItem objects, so it may be worth checking into when these are called.

John-Scott

[hynekcer]

Very useful is database statement logging at first.
Can you reproduce the problem at least once on a server where you have
permissions to change database server configuration?
Which database do you use? I explain the simplest form for Postgresql
and MySql.

For Postgresql
in the file /etc/postgresql/*/main/postgresql.conf uncomment the line
#log_statement = 'none'
and change it to
log_statement = mod
# This controls which SQL statements are logged. With value "mod" logs
all ddl statements, plus data-modifying statements such as INSERT,
UPDATE, DELETE...
read http://www.postgresql.org/docs/current/static/runtime-config-logging.html
for more details
Restart Postgresql server.
Log file is usually /var/log/postgresql/postgresql-*-main.log*

If you use MySQL:
-- file /etc/my.cnf --
[mysqld]
# usual options
# ...
# binary log for satchmo
log-bin=/var/lib/mysqlyour-mysql-binary-log-file-prefix
binlog-do-db=name_of_your_satchmo_database
# for more details http://dev.mysql.com/doc/refman/5.0/en/binary-log.html
---
Restart MySQL server.
Logged commands can be later converted from binary form to text form
by this command:
$ mysqlbinlog /var/lib/mysqlyour-mysql-binary-log-file-prefix.*
>output.sql

In both cases make a backup of original database at the same time.
Also regularly save the http log to be not overwritten old logs by a
log rotate.
This is sufficient for later analyzing and for reconstructing any
previous state of database.

(I am also interested in this error. If you possibly can not analyze
logs by yourself and very probably can not give out of your hands
contained private data, it would be time consuming for both but still
possible to analyze it interactively and some contract would by
necessary.)