when memcached goes down, Authorize.net stops working

54 views
Skip to first unread message

Kevin Harvey

unread,
Apr 28, 2012, 11:17:52 AM4/28/12
to satchm...@googlegroups.com
Hello all, I just finished debugging an error that stopped Authorize.net transactions through my Satchmo store. At the confirmation screen, after trying to submit the payment, the page refreshed and I got a 'Credit card number is required' message. It turns out that memcached had stopped running on my server, and without a cache Satchmo won't talk to Authorize.net.

That's a pretty serious dependency that I never thought to monitor before. Has anyone else ever seen that? I probably need to start monitoring that memcached process directly, but what else can I do to avoid this situation in the future?

Mike Hostetler

unread,
Apr 28, 2012, 11:21:15 AM4/28/12
to satchm...@googlegroups.com

I've seen the same thing and even wrote about it on the list (though I didn't know Memcache had problems at the time). For payments to not process because cache is down is very weird for me to get a handle on, too.

On Apr 28, 2012 10:17 AM, "Kevin Harvey" <kcha...@gmail.com> wrote:
>
> Hello all, I just finished debugging an error that stopped Authorize.net transactions through my Satchmo store. At the confirmation screen, after trying to submit the payment, the page refreshed and I got a 'Credit card number is required' message. It turns out that memcached had stopped running on my server, and without a cache Satchmo won't talk to Authorize.net.
>
> That's a pretty serious dependency that I never thought to monitor before. Has anyone else ever seen that? I probably need to start monitoring that memcached process directly, but what else can I do to avoid this situation in the future?
>

> --
> You received this message because you are subscribed to the Google Groups "Satchmo users" group.
> To view this discussion on the web visit https://groups.google.com/d/msg/satchmo-users/-/WakEZt5AVKAJ.
> To post to this group, send email to satchm...@googlegroups.com.
> To unsubscribe from this group, send email to satchmo-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/satchmo-users?hl=en.

Chris Moffitt

unread,
Apr 28, 2012, 11:25:49 AM4/28/12
to satchm...@googlegroups.com
Yes, I've heard of it. I'm open to ideas on how to handle it more gracefully. It's part of trying not to permanently store credit card numbers, so we encrypt them in the cache. Obviously, if the cache goes down, things get wonky.

I'm open to other ideas that would be more robust but would be secure.

-Chris

Kevin Harvey

unread,
Apr 28, 2012, 11:46:24 AM4/28/12
to satchm...@googlegroups.com
I'm using 0.9.1, and the problem didn't seem to be logged very well. I don't have a CACHE_PREFIX in my settings.py, and when I restart the server I get this:

Sat, 28 Apr 2012 10:03:33 root         INFO     Satchmo Started
Sat, 28 Apr 2012 10:03:33 keyedcache   WARNING  No CACHE_PREFIX found in settings, using SITE_ID.  Please update your settings to add a CACHE_PREFIX

Perhaps a start would be for Satchmo to throw an error, log similarly to the above, or refuse to start altogether if the cache is inaccessible.

Thanks!


On Saturday, April 28, 2012 11:25:49 AM UTC-4, Chris Moffitt wrote:
Yes, I've heard of it. I'm open to ideas on how to handle it more gracefully. It's part of trying not to permanently store credit card numbers, so we encrypt them in the cache. Obviously, if the cache goes down, things get wonky.

I'm open to other ideas that would be more robust but would be secure.

-Chris

On Sat, Apr 28, 2012 at 10:21 AM, Mike Hostetler <> wrote:

I've seen the same thing and even wrote about it on the list (though I didn't know Memcache had problems at the time). For payments to not process because cache is down is very weird for me to get a handle on, too.

On Apr 28, 2012 10:17 AM, "Kevin Harvey" <> wrote:
>
> Hello all, I just finished debugging an error that stopped Authorize.net transactions through my Satchmo store. At the confirmation screen, after trying to submit the payment, the page refreshed and I got a 'Credit card number is required' message. It turns out that memcached had stopped running on my server, and without a cache Satchmo won't talk to Authorize.net.
>
> That's a pretty serious dependency that I never thought to monitor before. Has anyone else ever seen that? I probably need to start monitoring that memcached process directly, but what else can I do to avoid this situation in the future?
>
> --
> You received this message because you are subscribed to the Google Groups "Satchmo users" group.
> To view this discussion on the web visit https://groups.google.com/d/msg/satchmo-users/-/WakEZt5AVKAJ.
> To post to this group, send email to satchm...@googlegroups.com.

> To unsubscribe from this group, send email to satchmo-users+unsubscribe@googlegroups.com.


> For more options, visit this group at http://groups.google.com/group/satchmo-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "Satchmo users" group.
To post to this group, send email to satchm...@googlegroups.com.
To unsubscribe from this group, send email to satchmo-users+unsubscribe@googlegroups.com.

lacry...@gmail.com

unread,
Apr 29, 2012, 9:08:21 AM4/29/12
to satchm...@googlegroups.com

What about saving the number in session data, encrypted, and the key in the cookie, or the other way around? And removing it when the transaction is done.

-----Mensaje original-----
De: Chris Moffitt
Enviados: 28/04/2012 12:25:49
Asunto: Re: when memcached goes down, Authorize.net stops working

Yes, I've heard of it. I'm open to ideas on how to handle it more
gracefully. It's part of trying not to permanently store credit card
numbers, so we encrypt them in the cache. Obviously, if the cache goes
down, things get wonky.

I'm open to other ideas that would be more robust but would be secure.

-Chris

On Sat, Apr 28, 2012 at 10:21 AM, Mike Hostetler
<mi...@squarepegsystems.com>wrote:

> I've seen the same thing and even wrote about it on the list (though I
> didn't know Memcache had problems at the time). For payments to not process
> because cache is down is very weird for me to get a handle on, too.
>
> On Apr 28, 2012 10:17 AM, "Kevin Harvey" <kcha...@gmail.com> wrote:
> >
> > Hello all, I just finished debugging an error that stopped Authorize.net
> transactions through my Satchmo store. At the confirmation screen, after
> trying to submit the payment, the page refreshed and I got a 'Credit card
> number is required' message. It turns out that memcached had stopped
> running on my server, and without a cache Satchmo won't talk to
> Authorize.net.
> >
> > That's a pretty serious dependency that I never thought to monitor
> before. Has anyone else ever seen that? I probably need to start monitoring
> that memcached process directly, but what else can I do to avoid this
> situation in the future?
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "Satchmo users" group.
> > To view this discussion on the web visit
> https://groups.google.com/d/msg/satchmo-users/-/WakEZt5AVKAJ.
> > To post to this group, send email to satchm...@googlegroups.com.
> > To unsubscribe from this group, send email to
> satchmo-user...@googlegroups.com.
> > For more options, visit this group at
> http://groups.google.com/group/satchmo-users?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Satchmo users" group.
> To post to this group, send email to satchm...@googlegroups.com.
> To unsubscribe from this group, send email to
> satchmo-user...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/satchmo-users?hl=en.
>

--
You received this message because you are subscribed to the Google Groups "Satchmo users" group.
To post to this group, send email to satchm...@googlegroups.com.
To unsubscribe from this group, send email to satchmo-user...@googlegroups.com.

hynekcer

unread,
Apr 30, 2012, 3:24:01 PM4/30/12
to Satchmo users
I am trying to summarize what is well known. (I am not interested in
these payment methods and I have not studied the implementation):

disadvantage of database storage of secret temporary data:
- data can stay saved after server restart or after database backup
etc. (althought they will be usually smart removed some way)

disadvantage of memcached storage:
- data are sometimes more volatile than is desiderable
- it is more complicated to protect memcached data than database data.
(Database access can be fine selective protected by username and it is
a general practise. Memcached access can be restricted by SASL in new
versions, but all users/apps has access to the same data.

I think a solution: Use a temporary cookie named e.g.
"encryption_secret_2" with a short time of live approx 30 min and a
data "encryption_secret_1" saved in the session. Booth secrets will be
used to create an enccyption key. The owner of the server can not
decrypt the secret data without cooperation with the user and never
can decrypt it after 30 minutes. All these data will be removed if
they are not necassary for better transparency. The cookie should be
never logged. Is it a good idea? Does anybody want to implement it?

I think refusing server starting for missing cache is not a good idea
and it would cause more issues. (What if cache would be tested
sometimes earlier than memcached is completely started? etc.) On the
other hand, I agree that missing cache can cause refusing some payment
method to be selected. It is better to acceppt an order and say
eventually that no payment is currently possible than a dead web site
or problems with ccn written above.

On 29 dub, 15:08, "lacrymol...@gmail.com" <lacrymol...@gmail.com>
wrote:
> What about saving the number in session data, encrypted, and the key in the cookie, or the other way around? And removing it when the transaction is done.
>
> -----Mensaje original-----
> De: Chris Moffitt
> Enviados:  28/04/2012 12:25:49
> Asunto:  Re: when memcached goes down, Authorize.net stops working
>
> Yes, I've heard of it. I'm open to ideas on how to handle it more
> gracefully. It's part of trying not to permanently store credit card
> numbers, so we encrypt them in the cache. Obviously, if the cache goes
> down, things get wonky.
>
> I'm open to other ideas that would be more robust but would be secure.
>
> -Chris
>
> On Sat, Apr 28, 2012 at 10:21 AM, Mike Hostetler
> <m...@squarepegsystems.com>wrote:
>
>
>
>
>
>
>
>
>
> > I've seen the same thing and even wrote about it on the list (though I
> > didn't know Memcache had problems at the time). For payments to not process
> > because cache is down is very weird for me to get a handle on, too.
>
Reply all
Reply to author
Forward
0 new messages