Question about ast.literal_eval

[Frank Millman]

Hi all

I am trying to emulate a SQL check constraint in Python. Quoting from
the PostgreSQL docs, "A check constraint is the most generic constraint
type. It allows you to specify that the value in a certain column must
satisfy a Boolean (truth-value) expression."

The problem is that I want to store the constraint as a string, and I
was hoping to use ast.literal_eval to evaluate it, but it does not work.

>>> x = 'abc'
>>> x in ('abc', xyz')
True
>>> b = "x in ('abc', 'xyz')"
>>> eval(b)
True
>>> from ast import literal_eval
>>> literal_eval(b)
ValueError: malformed node or string: <_ast.Compare object at ...>

Is there a safe way to do what I want? I am using python 3.3.

Thanks

Frank Millman

[Carlos Nepomuceno]

It seems to me you can't use ast.literal_eval()[1] to evaluate that kind of expression because it's just for literals[2].

Why don't you use eval()?

[1] http://docs.python.org/2/library/ast.html#ast-helpers

[2] http://docs.python.org/2/reference/lexical_analysis.html#literals

----------------------------------------
> To: pytho...@python.org
> From: fr...@chagford.com
> Subject: Question about ast.literal_eval
> Date: Mon, 20 May 2013 09:05:48 +0200

> --
> http://mail.python.org/mailman/listinfo/python-list

[Chris Angelico]

On Mon, May 20, 2013 at 5:05 PM, Frank Millman <fr...@chagford.com> wrote:
> Hi all
>
> I am trying to emulate a SQL check constraint in Python. Quoting from the
> PostgreSQL docs, "A check constraint is the most generic constraint type. It
> allows you to specify that the value in a certain column must satisfy a
> Boolean (truth-value) expression."
>
> The problem is that I want to store the constraint as a string, and I was
> hoping to use ast.literal_eval to evaluate it, but it does not work.
>
>>>> x = 'abc'
>>>> x in ('abc', xyz')
> True
>>>> b = "x in ('abc', 'xyz')"
>>>> eval(b)
> True
>>>> from ast import literal_eval
>>>> literal_eval(b)
> ValueError: malformed node or string: <_ast.Compare object at ...>
>
> Is there a safe way to do what I want? I am using python 3.3.

An SQL constraint has a whole lot that Python can't do conveniently,
and vice versa, so I think you do yourself a disservice by trying to
tie yourself to that.

The problem here is that 'in' is an operator, so this is not a
literal. One possible solution would be to separate out your operator
(and have just a handful permitted), and then use ast.literal_eval for
just the second operand - something like this:

import ast
import operator
ops = {
'in':lambda x,y: x in y, # operator.contains has the args backwards
'==':operator.eq, # or use '=' for more SQL-like syntax
'<':operator.lt,
'>':operator.gt,
}

op, value = 'in', "('abc', 'xyz')"
x = 'abc'

if ops[op](x,ast.literal_eval(value)):
print("Constraint passed")
else:
print("Ignore this one")

ChrisA

[Frank Millman]

[Corrected top-posting]

>> To: pytho...@python.org
>> From: fr...@chagford.com
>> Subject: Question about ast.literal_eval
>> Date: Mon, 20 May 2013 09:05:48 +0200
>>

>> Hi all
>>
>> I am trying to emulate a SQL check constraint in Python. Quoting from
>> the PostgreSQL docs, "A check constraint is the most generic constraint
>> type. It allows you to specify that the value in a certain column must
>> satisfy a Boolean (truth-value) expression."
>>
>> The problem is that I want to store the constraint as a string, and I
>> was hoping to use ast.literal_eval to evaluate it, but it does not work.
>>

On 20/05/2013 09:34, Carlos Nepomuceno wrote:

> It seems to me you can't use ast.literal_eval()[1] to evaluate that kind of expression
> because it's just for literals[2].
>
> Why don't you use eval()?
>

Because users can create their own columns, with their own constraints.
Therefore the string is user-modifiable, so it cannot be trusted.

Frank

[Chris Angelico]

On Mon, May 20, 2013 at 5:50 PM, Frank Millman <fr...@chagford.com> wrote:
> On 20/05/2013 09:34, Carlos Nepomuceno wrote:
>> Why don't you use eval()?
>>
>
> Because users can create their own columns, with their own constraints.
> Therefore the string is user-modifiable, so it cannot be trusted.

Plenty of reason right there :)

Is it a requirement that they be able to key in a constraint as a
single string? We have a similar situation in one of the systems at
work, so we divided the input into three(ish) parts: pick a field,
pick an operator (legal operators vary according to field type -
integers can't be compared against regular expressions, timestamps can
use >= and < only), then enter the other operand. Sure, that cuts out
a few possibilities, but you get 99.9%+ of all usage and it's easy to
sanitize.

ChrisA

[Carlos Nepomuceno]

----------------------------------------
> To: pytho...@python.org
> From: fr...@chagford.com
> Subject: Re: Question about ast.literal_eval
> Date: Mon, 20 May 2013 09:50:02 +0200

>
> [Corrected top-posting]
>
>>> To: pytho...@python.org
>>> From: fr...@chagford.com
>>> Subject: Question about ast.literal_eval
>>> Date: Mon, 20 May 2013 09:05:48 +0200
>>>
>>> Hi all
>>>
>>> I am trying to emulate a SQL check constraint in Python. Quoting from
>>> the PostgreSQL docs, "A check constraint is the most generic constraint
>>> type. It allows you to specify that the value in a certain column must
>>> satisfy a Boolean (truth-value) expression."
>>>
>>> The problem is that I want to store the constraint as a string, and I
>>> was hoping to use ast.literal_eval to evaluate it, but it does not work.
>>>
>

> On 20/05/2013 09:34, Carlos Nepomuceno wrote:
>

>> It seems to me you can't use ast.literal_eval()[1] to evaluate that kind of expression
>> because it's just for literals[2].
>>

>> Why don't you use eval()?
>>
>
> Because users can create their own columns, with their own constraints.
> Therefore the string is user-modifiable, so it cannot be trusted.

I understand your motivation but I don't know what protection ast.literal_eval() is offering that eval() doesn't.

> Frank
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list

[Frank Millman]

On 20/05/2013 09:55, Chris Angelico wrote:
> On Mon, May 20, 2013 at 5:50 PM, Frank Millman <fr...@chagford.com> wrote:

>> On 20/05/2013 09:34, Carlos Nepomuceno wrote:
>>> Why don't you use eval()?
>>>
>>
>> Because users can create their own columns, with their own constraints.
>> Therefore the string is user-modifiable, so it cannot be trusted.
>

> Plenty of reason right there :)
>
> Is it a requirement that they be able to key in a constraint as a
> single string? We have a similar situation in one of the systems at
> work, so we divided the input into three(ish) parts: pick a field,
> pick an operator (legal operators vary according to field type -
> integers can't be compared against regular expressions, timestamps can
> use >= and < only), then enter the other operand. Sure, that cuts out
> a few possibilities, but you get 99.9%+ of all usage and it's easy to
> sanitize.
>
> ChrisA
>

It is not a requirement, no. I just thought it would be a convenient
short-cut.

I had in mind something similar to your scheme above, so I guess I will
have to bite the bullet and implement it.

Thanks

Frank

[Chris Angelico]

On Mon, May 20, 2013 at 5:55 PM, Carlos Nepomuceno
<carlosne...@outlook.com> wrote:
> I understand your motivation but I don't know what protection ast.literal_eval() is offering that eval() doesn't.

eval will *execute code*, while literal_eval will not. That's the
protection. With ast.literal_eval, all that can happen is that it
produces a single result value. In this case, unfortunately, that's
insufficient; a comparison needs to be done, ergo it's not an entire
literal. But something else in the ast module may be able to serve, or
maybe literal_eval can do the bulk of the work.

ChrisA

[Frank Millman]

On 20/05/2013 09:55, Carlos Nepomuceno wrote:
> ----------------------------------------

>>>
>>> Why don't you use eval()?
>>>
>>
>> Because users can create their own columns, with their own constraints.
>> Therefore the string is user-modifiable, so it cannot be trusted.
>

> I understand your motivation but I don't know what protection ast.literal_eval() is offering that eval() doesn't.
>

Quoting from the manual -

"Safely evaluate an expression node or a string containing a Python
expression. The string or node provided may only consist of the
following Python literal structures: strings, bytes, numbers, tuples,
lists, dicts, sets, booleans, and None."

The operative word is 'safely'. I don't know the details, but it
prevents the kinds of exploits that can be carried out by malicious code
using eval().

I believe it is the same problem as SQL injection, which is solved by
using parameterised queries.

Frank

[Steven D'Aprano]

On Mon, 20 May 2013 10:55:35 +0300, Carlos Nepomuceno wrote:

> I understand your motivation but I don't know what protection
> ast.literal_eval() is offering that eval() doesn't.

eval will evaluate any legal Python expression:


py> eval("__import__('os').system('echo Mwahaha! Now you are pwned!') or 42")
Mwahaha! And now you are pwned!
42


ast.literal_eval() does exactly what the name says: it will evaluate any
legal Python LITERAL, including ints, floats, lists, dicts and strings,
but not arbitrary expressions.


py> ast.literal_eval('123')
123
py> ast.literal_eval('[123, None, "spam"]')
[123, None, 'spam']



--
Steven

[Fábio Santos]

On 20 May 2013 09:19, "Frank Millman" <fr...@chagford.com> wrote:
> Quoting from the manual -
>
> "Safely evaluate an expression node or a string containing a Python expression. The string or node provided may only consist of the following Python literal structures: strings, bytes, numbers, tuples, lists, dicts, sets, booleans, and None."
>
> The operative word is 'safely'. I don't know the details, but it prevents the kinds of exploits that can be carried out by malicious code using eval().

Literals are only a subset of expressions. The documentation is a bit misleading, by stating it accepts a "python expression".

This individual is rightfully confused.

[Frank Millman]

Can anyone see anything wrong with the following approach. I have not
definitely decided to do it this way, but I have been experimenting and
it seems to work.

I store the boolean test as a json'd list of 6-part tuples. Each element
of the tuple is a string, defined as follows -

0 - for the first entry in the list, the word 'check' (a placeholder -
it is discarded at evaluation time), for any subsequent entries the word
'and' or 'or'.

1 - left bracket - either '(' or ''.

2 - column name to check - it will be validated on entry.

3 - operator - must be one of '=', '!=', '<', '>', '<=', '>=', 'in',
'is', 'is not'. At evaluation time, '=' is changed to '=='.

4 - value to compare - at evaluation time I call
str(literal_eval(value)) to ensure that it is safe.

5 - right bracket - either ')' or ''.

At evaluation time I loop through the list, construct the boolean test
as a string, and call eval() on it.

Here are some examples -

check = []
check.append(('check', '', 'name', 'in', "('abc', 'xyz')", ''))

check = []
check.append(('check', '', 'value', '>=', '0', ''))

check = []
check.append(('check', '(', 'descr', 'is not', 'None', ''))
check.append(('and', '', 'alt', 'is', 'None', ')'))
check.append(('or', '(', 'descr', 'is', 'None', ''))
check.append(('and', '', 'alt', 'is not', 'None', ')'))

I don't plan to check the logic - I will just display the exception if
it does not evaluate.

It seems safe to me. Can anyone see a problem with it?

Frank

[Steven D'Aprano]

On Mon, 20 May 2013 15:26:02 +0200, Frank Millman wrote:

> Can anyone see anything wrong with the following approach. I have not
> definitely decided to do it this way, but I have been experimenting and
> it seems to work.
>
> I store the boolean test as a json'd list of 6-part tuples. Each element
> of the tuple is a string, defined as follows -
>
> 0 - for the first entry in the list, the word 'check' (a placeholder -
> it is discarded at evaluation time), for any subsequent entries the word
> 'and' or 'or'.
>
> 1 - left bracket - either '(' or ''.
>
> 2 - column name to check - it will be validated on entry.
>
> 3 - operator - must be one of '=', '!=', '<', '>', '<=', '>=', 'in',
> 'is', 'is not'. At evaluation time, '=' is changed to '=='.
>
> 4 - value to compare - at evaluation time I call
> str(literal_eval(value)) to ensure that it is safe.
>
> 5 - right bracket - either ')' or ''.
>
> At evaluation time I loop through the list, construct the boolean test
> as a string, and call eval() on it.

[...]

> It seems safe to me. Can anyone see a problem with it?

It seems safe to me too, but then any fool can come up with a system
which they themselves cannot break :-)

I think the real worry is validating the column name. That will be
critical. Personally, I would strongly suggest writing your own mini-
evaluator that walks the list and evaluates it by hand. It isn't as
convenient as just calling eval, but *definitely* safer.

If you do call eval, make sure you supply the globals and locals
arguments. The usual way is:

eval(expression, {'__builtins__': None}, {})

which gives you an empty locals() and a minimal, (mostly) safe globals.

Finally, as a "belt-and-braces" approach, I wouldn't even call eval
directly, but call a thin wrapper that raises an exception if the
expression contains an underscore. Underscores are usually the key to
breaking eval, so refusing to evaluate anything with an underscore raises
the barrier very high.

And even with all those defences, I wouldn't allow untrusted data from
the Internet anywhere near this. Just because I can't break it, doesn't
mean it's safe.


--
Steven

[Chris Angelico]

On Mon, May 20, 2013 at 11:26 PM, Frank Millman <fr...@chagford.com> wrote:
> 0 - for the first entry in the list, the word 'check' (a placeholder - it is
> discarded at evaluation time), for any subsequent entries the word 'and' or
> 'or'.
>
> 1 - left bracket - either '(' or ''.
>

> 5 - right bracket - either ')' or ''.

I think what you have is safe, but extremely complicated to work with.
Six separate pieces, and things have to be in the right slots... I
think you've spent too many "complexity points" on the above three
components, and you're getting too little return for them. What
happens if the nesting is mucked up? Could get verrry messy to check.

Combining multiple conditions with a mixture of ands and ors is a
nightmare to try to explain (unless you just point to the Python docs,
which IMO costs you even more complexity points); the only safe option
is to parenthesize everything. The system I pushed for at work (which
was finally approved and implemented a few months ago) is more or less
this: conditions are grouped together into blocks; for each group, you
can choose whether it's "all" or "any" (aka and/or), and you choose
whether the overall result is all-groups or any-group. That still
costs a few complexity points (and, btw, our *actual* implementation
is a bit more complicated than that, but I think we could cut it down
to what I just described here without loss of functionality), but it
gives the bulk of what people will actually want without the
complexities of point-and-click code.

The downside of that sort of system is that it requires a two-level
tree. On the flip side, that's often how people will be thinking about
their conditions anyway (eg using a pair of conditions ">" and "<" to
implement a range check - conceptually it's a single check), so that
won't cost too much.

ChrisA

[Chris Angelico]

On Tue, May 21, 2013 at 2:12 AM, Steven D'Aprano
<steve+comp....@pearwood.info> wrote:
> Personally, I would strongly suggest writing your own mini-
> evaluator that walks the list and evaluates it by hand. It isn't as
> convenient as just calling eval, but *definitely* safer.

Probably faster, too, for what it's worth - eval is pretty expensive.

ChrisA

[matt.n...@gmail.com]

You might find the asteval module (https://pypi.python.org/pypi/asteval) useful. It provides a relatively safe "eval", for example:

>>> import asteval
>>> a = asteval.Interpreter()
>>> a.eval('x = "abc"')
>>> a.eval('x in ("abc", "xyz")')
True
>>> a.eval('import os')
NotImplementedError
import os
'Import' not supported
>>> a.eval('__import__("os")')
NameError
__import__("os")
name '__import__' is not defined

This works by maintaining an internal namespace (a flat dictionary), and walking the AST generated for the expression. It supports most Python syntax,
including if, for, while, and try/except blocks, and function definitions, and with the notable exceptions of eval, exec, class, lambda, yield, and import. This requires Python2.6 and higher, and does work with Python3.3.

Of course, it is not guaranteed to be completely safe, but it does disallow imports, which seems like the biggest vulnerability concern listed here. Currently, there is no explicit protection against long-running calculations for denial of service attacks. If you're exposing an SQL database to user-generated code, that may be worth considering.

Cheers,

--Matt Newville

[Frank Millman]

On 21/05/2013 04:39, matt.n...@gmail.com wrote:
>
> You might find the asteval module (https://pypi.python.org/pypi/asteval) useful. It provides a relatively safe "eval", for example:
>
> >>> import asteval
> >>> a = asteval.Interpreter()
> >>> a.eval('x = "abc"')
> >>> a.eval('x in ("abc", "xyz")')
> True
> >>> a.eval('import os')
> NotImplementedError
> import os
> 'Import' not supported
> >>> a.eval('__import__("os")')
> NameError
> __import__("os")
> name '__import__' is not defined
>
> This works by maintaining an internal namespace (a flat dictionary), and walking the AST generated for the expression. It supports most Python syntax,
> including if, for, while, and try/except blocks, and function definitions, and with the notable exceptions of eval, exec, class, lambda, yield, and import. This requires Python2.6 and higher, and does work with Python3.3.
>
> Of course, it is not guaranteed to be completely safe, but it does disallow imports, which seems like the biggest vulnerability concern listed here. Currently, there is no explicit protection against long-running calculations for denial of service attacks. If you're exposing an SQL database to user-generated code, that may be worth considering.

Thanks for this, Matt. I will definitely look into it.

Frank

[Frank Millman]

On 20/05/2013 18:12, Steven D'Aprano wrote:
> On Mon, 20 May 2013 15:26:02 +0200, Frank Millman wrote:
>
>> Can anyone see anything wrong with the following approach. I have not
>> definitely decided to do it this way, but I have been experimenting and
>> it seems to work.
>>

[...]
>

> It seems safe to me too, but then any fool can come up with a system
> which they themselves cannot break :-)
>

Thanks for the detailed response.

> I think the real worry is validating the column name. That will be
> critical.

I would not pass the actual column name to eval(), I would use it to
retrieve a value from a data object and pass that to eval(). However,
then your point becomes 'validating the value retrieved'. I had not
thought about that. I will investigate further.

> Personally, I would strongly suggest writing your own mini-
> evaluator that walks the list and evaluates it by hand. It isn't as
> convenient as just calling eval, but *definitely* safer.
>

I am not sure I can wrap my mind around mixed 'and's, 'or's, and brackets.

[Thinking aloud]
Maybe I can manually reduce each internal test to a True or False,
substitute them in the list, and then call eval() on the result.

eval('(True and False) or (False or True)')

I will experiment with that.

> If you do call eval, make sure you supply the globals and locals
> arguments. The usual way is:
>
> eval(expression, {'__builtins__': None}, {})
>
> which gives you an empty locals() and a minimal, (mostly) safe globals.
>

Thanks - I did not know about that.

> Finally, as a "belt-and-braces" approach, I wouldn't even call eval
> directly, but call a thin wrapper that raises an exception if the
> expression contains an underscore. Underscores are usually the key to
> breaking eval, so refusing to evaluate anything with an underscore raises
> the barrier very high.
>
> And even with all those defences, I wouldn't allow untrusted data from
> the Internet anywhere near this. Just because I can't break it, doesn't
> mean it's safe.
>

All good advice - much appreciated.

Frank

[Frank Millman]

You may be right, Chris, but I don't think my approach is all that bad.

The vast majority of tests will be simple - either a single line, or two
lines for a range check, with no brackets at all.

If the requirement is more complicated than that, well, I don't think
the complication can be avoided, and at least this approach gives full
control.

FWIW, I use the same approach to allow users to construct their own
WHERE clauses in custom views. Again, the vast majority are simple, but
there are times when it can get complicated.

The proof of the pudding will be when I try to get ordinary users to get
their own hands dirty - I am not there yet. If I ever get this released,
the business model will be free software, but support will be charged
for. So if a user gets out of his/her depth, there will be assistance
available.

Time will tell who is right ... ;-)

Frank

[Steven D'Aprano]

On Tue, 21 May 2013 08:30:03 +0200, Frank Millman wrote:

> On 20/05/2013 18:12, Steven D'Aprano wrote:

>> Personally, I would strongly suggest writing your own mini- evaluator
>> that walks the list and evaluates it by hand. It isn't as convenient as
>> just calling eval, but *definitely* safer.
>>
>>
> I am not sure I can wrap my mind around mixed 'and's, 'or's, and
> brackets.

Parsers are a solved problem in computer science, he says as if he had a
clue what he was talking about *wink*

Here's a sketch of a solution... suppose you have a sequence of records,
looking like this:

(bool_op, column_name, comparison_op, literal)

with appropriate validation on each field. The very first record has
bool_op set to "or". Then, you do something like this:


import operator
OPERATORS = {
'=': operator.eq,
'is': operator.is_,
'<': operator.lt,
# etc.
}


def eval_op(column_name, op, literal):
value = lookup(column_name) # whatever...
return OPERATORS[op](value, literal)

result = False

for (bool_op, column_name, comparison_op, literal) in sequence:
flag = eval_op(column_name, comparison_op, literal)
if bool_op == 'and':
result = result and flag
else:
assert bool_op == 'or'
result = result or flag
# Lazy processing?
if result:
break


and in theory it should all Just Work.




--
Steven

[Chris Angelico]

On Tue, May 21, 2013 at 4:46 PM, Frank Millman <fr...@chagford.com> wrote:
> You may be right, Chris, but I don't think my approach is all that bad.

Frankly, I'm not altogether convinced that our approach is right
either :) But like the Oracle in the Matrix, I'm not here to push you
to one decision or another so much as to just put the options in front
of you and let you make up your own <bowdlerized> mind. Except in a
few cases where I'm really certain of my ground (like "don't put any
untrusted data through eval()"...).

> The vast majority of tests will be simple - either a single line, or two
> lines for a range check, with no brackets at all.
>
> If the requirement is more complicated than that, well, I don't think the
> complication can be avoided, and at least this approach gives full control.

Yeah, and this is where the issue of complexity points comes in.
You're spending a lot of them on functionality that most users won't
even use, and those who do will use only occasionally. You're forcing
them to match their brackets (not just have the same number of each
type, but also to get the nesting correct), and according to your
current spec, there can be no more than one open/close bracket at each
condition, so they'll have to arbitrarily add dummy conditions to make
certain forms of nesting work. You're exposing a lot of the underlying
interpreter, while forcing the user to dance wearing a body cast.
Sure, it can work, but it's unnecessarily hard.

> FWIW, I use the same approach to allow users to construct their own WHERE
> clauses in custom views. Again, the vast majority are simple, but there are
> times when it can get complicated.

Our alpha system is actually online, and we have exactly that system -
a query builder that renders down to a WHERE clause. If you're
curious, message me offline and I'll give you the URL.

> The proof of the pudding will be when I try to get ordinary users to get
> their own hands dirty - I am not there yet. If I ever get this released, the
> business model will be free software, but support will be charged for. So if
> a user gets out of his/her depth, there will be assistance available.
>
> Time will tell who is right ... ;-)

Who is right, and who is dead. Hey, are you aware that both Steven and
I come from Australia, and that we are used to having people not trust
us? Truly, you have a dizzying intellect!

ChrisA
... couldn't resist...

[Frank Millman]

That's very clever - thanks, Steven.

It doesn't address the issue of brackets. I imagine that the answer is
something like -

maintain a stack of results
for each left bracket, push a level
for each right bracket, pop the result

or something ...

I am sure that with enough trial and error I can get it working, but I
might cheat for now and use the trick I mentioned earlier of calling
eval() on a sequence of manually derived True/False values. I really
can't see anything going wrong with that.

BTW, thanks to ChrisA for the following tip -

import operator
ops = {
'in':lambda x,y: x in y, # operator.contains has the args backwards

I would have battled with that one.

Frank

[Fábio Santos]

On 21 May 2013 09:10, "Frank Millman" <fr...@chagford.com> wrote:
> It doesn't address the issue of brackets. I imagine that the answer is something like -
>
>   maintain a stack of results
>   for each left bracket, push a level
>   for each right bracket, pop the result
>
> or something ...
>

Time for me to suggest pyparsing or PLY. You're better off creating your own AST and walking it to produce python or SQL than reinventing the wheel, I think.

[Mark Lawrence]

On 21/05/2013 09:23, F�bio Santos wrote:
>
> On 21 May 2013 09:10, "Frank Millman" <fr...@chagford.com

Or pick one from this lot http://nedbatchelder.com/text/python-parsers.html

--
If you're using GoogleCrap� please read this
http://wiki.python.org/moin/GoogleGroupsPython.

Mark Lawrence