OpenKeyval CLI

47 views
Skip to first unread message

Joseph McCullough

unread,
May 10, 2012, 9:44:59 PM5/10/12
to hack...@googlegroups.com
Hi all!

I put together a little CLI for OpenKeyval.org. OpenKeyval, as it sounds, is an open API for storing and retrieving key => value pairs. It's like a nosql server that anyone has access to without registration.


Why would you ever use this? One use case is simulating state when a service doesn't offer an API for such. For example,
I will soon be adding a section to the Vert blog that has a window of an embedded codestream whenever I'm working on open source projects. Unfortunately, codestre.am doesn't have an API that lets me query if I am streaming or not. Thus I can create a wrapper that sets "josephmccullough_isStreaming" to the embed codestream URL, runs the codestre.am client, and upon exit, sets "josephmccullough_isStreaming" to "False". All the while, my blog queries the url for my key/value pair (which would just be api.openkeyval.org/josephmccullough_isStreaming), and if the result isn't false, generates an embed tag (or param, whatever it'll be) and places the stream on my blog so people can check out what I'm working on.

I don't feel like setting up a database for this, and as long as I create a nice, long key prefix, I shouldn't expect my data to be overwritten (and if it is, I won't care. Of course validation will need to occur every time you plan on using the data)

Feedback would be great. An alternative would actually be much better, because I feel this is sloppy.

--
Joseph McCullough
Vert Studios - Lead Developer
903 330 5057
http://www.vertstudios.com/

Justin Reese

unread,
May 11, 2012, 10:10:49 AM5/11/12
to hack...@googlegroups.com
On May 10, 2012, at 8:44 PM, Joseph McCullough wrote:

> I don't feel like setting up a database for this, and as long as I create a nice, long key prefix, I shouldn't expect my data to be overwritten (and if it is, I won't care. Of course validation will need to occur every time you plan on using the data)

I like the idea of a dead-simple k/v store, but this seems impossible to use with any degree of reliability.

"Only use it for times you don't care about getting an unexpected or false result."

Huh? The single most important function of a data store is giving me back what I put in. If you have to bake in an expectation that the value returned will be wrong, and you never know if it'll get stomped on again between this write and the next read, what's the point of using it at all? Even non-critical applications deserve some level of expectability.

I don't mean to poo-poo an impressive and altruistic idea, I just don't really understand it beyond a tech demo.

Having said that, I love how much you contribute to open source, so carry on soldier.

Joseph McCullough

unread,
May 11, 2012, 10:28:40 AM5/11/12
to hack...@googlegroups.com
Well, I suppose the probability of a key being brute forced is the same as a password being brute forced. You can have keys up to 128 chars long, so your prefix can be extremely large. Keeping your key safe will be important at that point.

I was just extremely reluctant to say "OpenKeyval is 100% secure and no one will ever write over your data" because I'd hate to be the reason why someone emptied your bank account. I was honestly covering my own ass more than providing a technical opinion. 

Also, something I didn't mention, upon a successful POST for setting a value to a key, you get a JSON respons with a read-only key that you can provide whenever you don't want the data to be overwritten to agent A but still want agent A to use the data.

Justin Reese

unread,
May 11, 2012, 10:36:08 AM5/11/12
to hack...@googlegroups.com
Yeah, I read up on the site about read-only keys. I guess I'm less worried about people targeting you specifically for malfeasance, and more worried about accidental key duplication and brute-force malfeasance. It'd be incredibly trivial to write a script that just iterated through all possible keys, writing "poop" to each one. You could even run it from an EC to make it unlikely they'd block your IP because of the collateral damage.

I'm not saying it's a useless or unimpressive tech demo, just that it seems too fragile for anything BUT a tech demo.

But I thought Facebook was overvalued at $1B and that HAML was a great idea so I'm not too reliable on these matters.

Joseph McCullough

unread,
May 11, 2012, 11:48:25 AM5/11/12
to hack...@googlegroups.com
Well brute forcing a password is an equally trivial process. It's about time, not about script cleverness. 

Check out this gist: https://gist.github.com/2660518

To keep the math easy, we assume all keys have to be the same length (which certainly isn't the case and adds a whole new magnitude). 

So for a billion keys checked per second between 10 clustered machines (assuming no redundancy when it comes to checking keys of course), it would take 3.014 * 10^140 years to exhaust all keys of length 100. A billion keys per second is an extremely generous amount.

Justin Reese

unread,
May 11, 2012, 11:58:29 AM5/11/12
to hack...@googlegroups.com
Well right, but remember that I'm not talking about finding the needle in the haystack. I'm talking about how fun it is to burn the haystack down.

Given a sufficiently complex key, your data is pretty safe from being intentionally targeted, but you still can't rely on the data in there being what you put in, and that's the single most important function of a data store, right?

I'm not saying it's terribly likely someone will set "674d87asjghgahaskheue23739=poop", just that you have no reason to store data unless you want to reliably retrieve it. If you have to take a "we might get junk back" approach to your data, then you're either talking about super-trivial use cases (which, to be fair, you were) or tech demos.

will killian

unread,
May 11, 2012, 12:00:18 PM5/11/12
to hack...@googlegroups.com
If you don't use a datasource because you're afraid that terrorists will
set all the values to "poop", then the terrorists have already won. :(

* Justin Reese <jus...@justinreese.com>:
--
will killian
http://tsuasai.com

IgrowBeards

unread,
May 11, 2012, 12:01:44 PM5/11/12
to hack...@googlegroups.com
I lold :)

-- 
IgrowBeards
Sent with Sparrow

Joseph McCullough

unread,
May 11, 2012, 12:04:13 PM5/11/12
to hack...@googlegroups.com
But "burning the haystack down" isn't trivial when it comes to time. The math is based on someone saying "Set key AAAAAAAAAAAAAAAAAA to poop. Now set AAAAAAAAAAAAAAB to poop", and so on. Of course it won't take exactly 3.014 * 10^140 years for your key to be the one "selected" to be assigned to poop, but it will be sufficiently close to that magnitude. 

Justin Reese

unread,
May 11, 2012, 12:09:58 PM5/11/12
to hack...@googlegroups.com
Ohhhhkay, I'll stop fighting this. But let me know what production system you use this in so I can be sure not to rely on it. ;)

will killian

unread,
May 11, 2012, 12:16:01 PM5/11/12
to hack...@googlegroups.com

Joseph McCullough

unread,
May 11, 2012, 12:20:49 PM5/11/12
to hack...@googlegroups.com
I didn't mean to come off as abrasive, I think it's an interesting subject and I really appreciate you pitching in. It's probably one of the most high level discussions to occur on the hacktyler mailing list, and that's got me pumped.

OpenKeyval seems so much more vulnerable than standard authenticated systems due to how we interact with it. The question I'm posing, which I think is an awesome one for us to discuss, is essentially

"Is there any difference between someone bruteforcing keys in an unauthenticated environment and bruteforcing an authenticated environment?"

And there ARE some differences:

* Someone is guaranteed to get hurt in bruteforce attacks on OKV if they don't have a sufficiently long key. 
* Brute force attacks on authenticated systems have to deal with user and password combinations, whereas OKV doesn't. 

Justin Reese

unread,
May 11, 2012, 12:24:57 PM5/11/12
to hack...@googlegroups.com
On May 11, 2012, at 11:16 AM, will killian wrote:

> http://api.openkeyval.org/Qdt9WMpaF8vTmYgb8SRE

That would have been a more effective challenge had you used the read-only key. :D

Justin Reese

unread,
May 11, 2012, 12:36:47 PM5/11/12
to hack...@googlegroups.com
Heh, don't worry, you don't come across as abrasive. It just sounds like you can imagine more reasons to use it than I can. I can't think of anytime I'd rely on it outside of a tech demo.

will killian

unread,
May 11, 2012, 12:40:32 PM5/11/12
to hack...@googlegroups.com
I do agree w/ Justin's points, even though I'm being a smart ass about
it.

As far as brute-force attacks, OKV is vulnerable to the type of attack
that Justin described, where every key is overwritten. It doesn't seem
like targeting specific individuals would be particularly easy, though.
So I guess it boils down to what types of attacks you want to be
protected from. I wouldn't want to rely on something like OKV for
anything important, but, as in all security considerations, it'll boil
down to a debate on convenience versus security.

* Joseph McCullough <jos...@vertstudios.com>:
--
will killian

█▀▀▀▀▀█ █ ▄▄▄▀▀█ █▀▀▀▀▀█
█ ███ █ ██▄ █ ▀ ▄ █ ███ █
█ ▀▀▀ █ ▄██ ▀ ▀▀█ █ ▀▀▀ █
▀▀▀▀▀▀▀ ▀ █▄▀▄▀ █ ▀▀▀▀▀▀▀
▀█ ▄▀█▀▄▄ ▀ ▀█▄▄▄▄ ▀▄█▀█▀
▄▀ ▄▄▀▀▀█▀ ▄ ▄▀▀ ▀▀▀█▄
▄█▀█▄ ▀█ ▄▀▄▀▀█ ▀▀ ▄▀▀█▀
▀█▀█▀ ▀█▀█▀█▄▀ ▄ ██▄
▀▀▀ ▀ ▀▀▄▄▄ ▀█▄▀█▀▀▀█▀█
█▀▀▀▀▀█ ▄ ▄ ▄ ▄ █ ▀ ██▄▄▄
█ ███ █ ▀▀▄▀▄▀▀▄▀██▀▀▄█▄█
█ ▀▀▀ █ ▄ ██▀█▀█▀▄▀█▄▄▄█
▀▀▀▀▀▀▀ ▀▀ ▀▀ ▀ ▀ ▀▀▀

Joseph McCullough

unread,
May 11, 2012, 12:54:39 PM5/11/12
to hack...@googlegroups.com
The question I have is how likely is it that a brute force attack will "get to" your key before anyone cares about that data anymore. I mean the same thing applies to RSA: Eventually someone can factor that prime number RSA spits out. Can't anything be brute forced given enough time? So where do we draw the line, and why? 

Chris Abraham

unread,
May 14, 2012, 1:49:24 PM5/14/12
to hack...@googlegroups.com
Gentlemen,
Forgive me for hijacking your thread, we are having a film festival meeting today @ 4:00 pm in the main street gallery. If any of you can make it, I'll buy the first Stella, Sorry for the short notice, hope to see you there,
Chris 

Jacob Lindsey

unread,
May 14, 2012, 2:31:39 PM5/14/12
to hack...@googlegroups.com
I surely won't be able to make that.  If any HT people are though shoot me an email, I'd like to send my ideas with you.  : )

IgrowBeards

unread,
May 14, 2012, 2:33:10 PM5/14/12
to hack...@googlegroups.com

I won't either and Chris is out of town. Reefdog?
-- 
IgrowBeards
Sent with Sparrow

Justin Reese

unread,
May 14, 2012, 2:39:12 PM5/14/12
to hack...@googlegroups.com
Very much wish I could, but we leave at dawn's crack tomorrow, so today is heavy with packing/planning/securing. Sorry Chris! I'll pass world along to Kenny though, in case he's available.

- Justin
Reply all
Reply to author
Forward
0 new messages