new release today, switched redis provider

33 views

Skip to first unread message

Jeff Lindsay

unread,

Apr 7, 2012, 3:34:53 AM4/7/12

to reque...@googlegroups.com

Today our current Redis account became full, causing bins to not be created. In fact, nothing would happen when you click Create Bin. I tried to figure out how to get Redis to act as an LRU, but our redis provider Redis To Go doesn't let you change configuration. It's also expensive. For only 500MB, it costs over $100/mo. Our current plan is for 20MB. However, usage has blown up since I put up the redirect on postbin.org.

What I did was move to a new Redis provider: openredis. They're still expensive but cheaper than Redis To Go. I set up a 200MB plan, 10x increase from before. Keep in mind, storage is mostly about concurrent usage and spikes in usage since all data eventually expires.

Another thing I did, I changed how we serialize bin and request data into Redis. Instead of using pickle, we now use msgpack, which compresses the data a significant amount compared to even JSON. With the previous code, the average Bin was 45KB. We'll see what the average is with msgpack in a day or so.

Here are some graphs of the usage spike:

http://webnumbr.com/requestbin-count

http://webnumbr.com/requestbin-requests

Here is the change if anybody wants to review:

https://github.com/progrium/requestbin/commit/42f424b6e6f79325f42302b140ffac5bd1e1411a

Lastly, I did not bother to migrate bins. Since they are ephemeral, it wasn't worth the delay in releasing this fix to migrate them.