[Python-Dev] SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register
Wes Turner
# Forwarded message
From: Wes Turner <wes.t...@gmail.com>
Date: Wednesday, September 12, 2018
Subject: SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register
To: distutils-sig <distut...@python.org>
On Wednesday, September 12, 2018, Wes Turner <wes.t...@gmail.com> wrote:On Wednesday, September 12, 2018, Joni Orponen <j.or...@4teamwork.ch> wrote:On Wed, Sep 12, 2018 at 8:48 PM Wes Turner <wes.t...@gmail.com> wrote:Should C extensions that compile all add`-mindirect-branch=thunk -mindirect-branch-register` [1] to mitigate the risk of Spectre variant 2 (which does indeed affect user space applications as well as kernels)?Are those available on GCC <= 4.2.0 as per PEP 513?AFAIU, onlyGCC 7.3 and 8 have the retpoline (indirect-branch=thunk) support enabled by the `-mindirect-branch=thunk -mindirect-branch-register` CFLAGS.
"What is a retpoline and how does it work?"
Wes Turner
Wes Turner
On Wed, Sep 12, 2018, 12:29 Joni Orponen <j.or...@4teamwork.ch> wrote:On Wed, Sep 12, 2018 at 8:48 PM Wes Turner <wes.t...@gmail.com> wrote:Should C extensions that compile all add`-mindirect-branch=thunk -mindirect-branch-register` [1] to mitigate the risk of Spectre variant 2 (which does indeed affect user space applications as well as kernels)?Are those available on GCC <= 4.2.0 as per PEP 513?
Pretty sure no manylinux1 compiler is ever going to get these mitigations.For manylinux2010 on x86-64, we can easily use a much newer compiler: RH maintains a recent compiler, currently gcc 7.3, or if that doesn't work for some reason then the conda folks have be apparently figured out how to build the equivalent from gcc upstream releases.
Unfortunately, the manylinux2010 infrastructure is not quite ready... I'm pretty sure it needs some volunteers to push it to the finish line, though unfortunately I haven't had enough time to keep track.
Antoine Pitrou
Hi,
Please don't cross-post so heavily. python-dev is sufficient for this.
If you want to push this forward, I suggest you measure performance of
Python compiled with and without the Spectre mitigation options, and
report the results here. That will help vendors and packagers decide
whether they want to pursue the route of enabling those options.
Note there are plenty of data-driven conditional jumps in Python. It
will not be easy to determine which ones are vulnerable to exploiting
through speculative execution of a mispredicted branch. The bytecode
evaluation loop sounds like a potential attack target, but it's also
performance-sensitive.
Regards
Antoine.
Python-Dev mailing list
Pytho...@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/dev-python%2Bgarchive-30976%40googlegroups.com
Steve Dower
available) and determined (with the help of some of the original
Spectre/Meltdown investigation team) that there is no significant value
in enabling these flags for Python.
It boiled down to:
* Python allows arbitrary code execution by design
* Pure Python code in CPython has very long per-instruction opcode
sequences that cannot easily be abused or timed
* Injected pure Python code cannot be coerced into generating native
code that is able to abuse Spectre/Meltdown but not able to abuse other
attacks more easily
* Code injection itself is outside of this particular threat model
By comparison with JavaScript, most JS JITs can be easily coerced into
generating specific native code that can break sandbox guarantees (e.g.
browser tabs). Python offers none of these guarantees.
Distributors are of course free to enable these flags for their own
builds, but I recommend against it for the official binaries, and would
suggest that it's worth more PR than actual security and nobody else
needs to enable it either.
(Extension authors with significant scriptable C code need to perform
their own analysis. I'm only talking about CPython here.)
Cheers,
Steve
On 16Sep2018 0707, Wes Turner wrote:
> Should Python builds add `-mindirect-branch=thunk
> -mindirect-branch-register` to CFLAGS?
>
> Where would this be to be added in the build scripts with which
> architectures?
>
> /QSpectre is the MSVC build flag for Spectre Variant 1:
>
> > The /Qspectre option is available in Visual Studio 2017 version 15.7
> and later.
>
> https://docs.microsoft.com/en-us/cpp/build/reference/qspectre?view=vs-2017
>
> security@ directed me to the issue tracker / lists,
> so I'm forwarding this to python-dev and python-ideas, as well.
>
> # Forwarded message
> From: *Wes Turner* <wes.t...@gmail.com <mailto:wes.t...@gmail.com>>
> Date: Wednesday, September 12, 2018
> Subject: SEC: Spectre variant 2: GCC: -mindirect-branch=thunk
> -mindirect-branch-register
> To: distutils-sig <distut...@python.org
> <mailto:distut...@python.org>>
>
>
> Should C extensions that compile all add
> `-mindirect-branch=thunk -mindirect-branch-register` [1] to mitigate the
> risk of Spectre variant 2 (which does indeed affect user space
> applications as well as kernels)?
>
> [1]
> https://github.com/speed47/spectre-meltdown-checker/issues/119#issuecomment-361432244
> <https://github.com/speed47/spectre-meltdown-checker/issues/119#issuecomment-361432244>
> [2] https://en.wikipedia.org/wiki/Spectre_(security_vulnerability)
> <https://en.wikipedia.org/wiki/Spectre_%28security_vulnerability%29>
> [3]
> https://en.wikipedia.org/wiki/Speculative_Store_Bypass#Speculative_execution_exploit_variants
> <https://en.wikipedia.org/wiki/Speculative_Store_Bypass#Speculative_execution_exploit_variants>
>
> On Wednesday, September 12, 2018, Wes Turner <wes.t...@gmail.com
> <mailto:wes.t...@gmail.com>> wrote:
>
> On Wednesday, September 12, 2018, Joni Orponen
> <j.or...@4teamwork.ch <mailto:j.or...@4teamwork.ch>> wrote:
>
> On Wed, Sep 12, 2018 at 8:48 PM Wes Turner
> <wes.t...@gmail.com <mailto:wes.t...@gmail.com>> wrote:
>
> Should C extensions that compile all add
> `-mindirect-branch=thunk -mindirect-branch-register` [1]
> to mitigate the risk of Spectre variant 2 (which does
> indeed affect user space applications as well as kernels)?
>
>
> Are those available on GCC <= 4.2.0 as per PEP 513?
>
>
> AFAIU, only
> GCC 7.3 and 8 have the retpoline (indirect-branch=thunk) support
> enabled by the `-mindirect-branch=thunk
> -mindirect-branch-register` CFLAGS.
>
>
> On Wednesday, September 12, 2018, Wes Turner <wes.t...@gmail.com
> <mailto:wes.t...@gmail.com>> wrote:
>
> "What is a retpoline and how does it work?"
> https://stackoverflow.com/questions/48089426/what-is-a-retpoline-and-how-does-it-work
> <https://stackoverflow.com/questions/48089426/what-is-a-retpoline-and-how-does-it-work>
>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Pytho...@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org
Wes Turner
On Monday, September 17, 2018, Steve Dower <steve...@python.org> wrote:
I investigated this thoroughly some time ago (when the MSVC flags became available) and determined (with the help of some of the original Spectre/Meltdown investigation team) that there is no significant value in enabling these flags for Python.
It boiled down to:
* Python allows arbitrary code execution by design
* Pure Python code in CPython has very long per-instruction opcode sequences that cannot easily be abused or timed
* Injected pure Python code cannot be coerced into generating native code that is able to abuse Spectre/Meltdown but not able to abuse other attacks more easily
* Code injection itself is outside of this particular threat model
By comparison with JavaScript, most JS JITs can be easily coerced into generating specific native code that can break sandbox guarantees (e.g. browser tabs). Python offers none of these guarantees.
Distributors are of course free to enable these flags for their own builds, but I recommend against it for the official binaries, and would suggest that it's worth more PR than actual security and nobody else needs to enable it either.
(Extension authors with significant scriptable C code need to perform their own analysis. I'm only talking about CPython here.)
Cheers,
Steve
On 16Sep2018 0707, Wes Turner wrote:
Should Python builds add `-mindirect-branch=thunk -mindirect-branch-register` to CFLAGS?
Where would this be to be added in the build scripts with which architectures?
/QSpectre is the MSVC build flag for Spectre Variant 1:
> The /Qspectre option is available in Visual Studio 2017 version 15.7 and later.
https://docs.microsoft.com/en-us/cpp/build/reference/qspectre?view=vs-2017
security@ directed me to the issue tracker / lists,
so I'm forwarding this to python-dev and python-ideas, as well.
# Forwarded message
From: *Wes Turner* <wes.t...@gmail.com <mailto:wes.t...@gmail.com>>
Date: Wednesday, September 12, 2018
Subject: SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register
To: distutils-sig <distut...@python.org <mailto:distutils-sig@python.org>>
Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
Wes Turner
I thought I read that RH has a kernel flag for userspace?
If you want to push this forward, I suggest you measure performance of
Python compiled with and without the Spectre mitigation options, and
report the results here. That will help vendors and packagers decide
whether they want to pursue the route of enabling those options.
Steve Dower
> On Monday, September 17, 2018, Steve Dower <steve...@python.org
>
> I investigated this thoroughly some time ago (when the MSVC flags
> became available) and determined (with the help of some of the
> original Spectre/Meltdown investigation team) that there is no
> significant value in enabling these flags for Python.
>
> What did you fuzz with?
> Does that assume that e.g. Fortify has identified all bugs in CPython C?
> There have been a number of variants that have been disclosed; which did
> who test for?
Don't change the subject.
> It boiled down to:
> * Python allows arbitrary code execution by design
>
> Yet binaries built with GCC do have NX? Unless nested functions in C
> extensions?
I don't know anything about GCC settings. Binaries for Windows have been
built with this option for over a decade. It's unrelated to
Spectre/Meltdown.
> * Pure Python code in CPython has very long per-instruction opcode
> sequences that cannot easily be abused or timed
>
> A demonstration of this would be helpful.
That's not how proof-of-concepts work. You can't assume that the lack of
a demonstration proves it is possible - at best you have to assume that
it proves it is *not* possible, but really it just proves that nobody
has a demonstration yet.
What I could demonstrate (again) if I thought it would be worthwhile is
that the changes enabled by the flag do not affect the normal
interpreter loop, and do not affect any code that can be called fast
enough to potentially leak information. Feel free to go ahead and build
with/without the flags and compare the disassembly (and if you do this
and find that compilers are detecting new cases since I looked, *that*
would be very helpful to share directly with the security team).
> * Injected pure Python code cannot be coerced into generating native
> code that is able to abuse Spectre/Meltdown but not able to abuse
> other attacks more easily
>
> So, not impossible.
Of course it's not impossible. But why would you
> * Code injection itself is outside of this particular threat model
>
> [Jupyter] Notebook servers are as wide open to arbitrary code execution
> as browser JS JITs; often with VMs and/or containers as a 'sandbox'
> `pip install requirements.txt` installs and executes unsigned code:
> Python, C extensions
>
> What can a container do to contain a speculative execution exploit
> intending to escape said container?
Python's threat model does not treat the Python process as a sandbox. To
say it another way, if you assume the Python process is a sandbox,
you're on your own.
Arbitrary code, Python or otherwise, can totally escape the process, and
then it's up to the OS to protect against escaping the machine. We do
what we can to reduce unnecessary arbitrary code, but unless you've
properly protected your environment then you have a lot more to worry
about besides speculative execution vulnerabilities.
> By comparison with JavaScript, most JS JITs can be easily coerced
> into generating specific native code that can break sandbox
> guarantees (e.g. browser tabs). Python offers none of these guarantees.
>
>
> This is faulty logic. Because Python does not have a JIT sandbox,
> speculative execution is not a factor for Python?
Because Python does not have a (native) JIT at all, speculative
execution relies on identifying vulnerable and reusable code patterns
within the C code and being able to invoke those directly. Because pure
Python code does not allow this (without relying on other bugs), there
is no way to do this within the threat model we use.
Once you allow arbitrary or unvalidated native code, you are outside the
threat model and hence on your own. And if you find a bug that lets pure
Python code move the instruction pointer to arbitrary native code, that
should be reported to the security team.
> Distributors are of course free to enable these flags for their own
> builds, but I recommend against it for the official binaries, and
> would suggest that it's worth more PR than actual security and
> nobody else needs to enable it either.
>
> (Extension authors with significant scriptable C code need to
> perform their own analysis. I'm only talking about CPython here.)
>
>
> Extension installers (and authors) are not likely to perform any such
> analysis.
Then it is their fault if they are compromised. Open source software
relies on users validating the software themselves, as there is no legal
recourse against developers who do not do it.
> Extensions are composed of arbitrary C, which certainly can both
> directly exploit and indirectly enable remote exploitation of Spectre
> and Meltdown vulnerabilities.
If arbitrary C is running, we can't help you anymore.
> Most users of python are installing arbitrary packages (without hashes
> or signatures).
If they are concerned about Spectre/Meltdown, they should stop doing
this. They should also stop if they are concerned about 1000 other
issues that are much more likely than Spectre/Meltdown.
Cheers,
Steve
_______________________________________________
Python-Dev mailing list
Pytho...@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/dev-python%2Bgarchive-30976%40googlegroups.com
Nathaniel Smith
_______________________________________________
Python-Dev mailing list
Pytho...@python.org
https://mail.python.org/mailman/listinfo/python-dev
Wes Turner
Franklin? Lee
https://www.thomas-krenn.com/en/wiki/Safety_instructions_for_Meltdown_and_Spectre
On Mon, Sep 17, 2018 at 6:10 PM Wes Turner <wes.t...@gmail.com> wrote:
>
> To summarize:
>
> - CPython may be vulnerable to speculative execution vulnerabilities, but none are known.
> - In general, CPython is currently too slow for speculative execution exploitation to be practical.
> - Sandboxed, JIT'ed JS is not too slow for speculative execution exploitation to be practical
> - (Not otherwise discussed here: PyPy's sandboxed JIT may not be too slow for speculative execution exploitation to be practical.)
Spectre/Meltdown, but I think the idea is that, if Python takes about
2 milliseconds to run your code, then a difference of +- 10
microseconds is indistinguishable from noise. Try to write software
that can learn to distinguish two similar computers using the running
time of certain functions.
Javascript can be crafted to get close enough to some C programs.
ASM.js and WebAssembly might help.
PyPy's need for mitigation is independent of CPython's.
> - Because there is no exploit provided (or currently thought possible with just CPython), this security-related dialogue is regarded as a nuisance.
wasn't really anything to worry about. He believes that any exploit of
it will also imply an easier exploit is possible. He also says that
this particular fix won't really help.
Nathaniel is annoyed because Spectre is tricky to understand, and he
assumes you don't understand it as well as you think because you
haven't shown him that you have the expertise to understand it.
> Here's a good write-up:
> Safety_instructions_for_Meltdown_and_Spectre
makes the interpreter vulnerable to the attack? Under what conditions
would CPython be vulnerable to this attack, but not an easier attack
of at least the same severity?
The article I linked at the top of this email does not give advice for
interpreter writers at all. It only says what end users and chip
manufacturers ought to do. It is not relevant.
INADA Naoki
>
> To summarize:
>
> - CPython may be vulnerable to speculative execution vulnerabilities, but none are known.
> - In general, CPython is currently too slow for speculative execution exploitation to be practical.
> - Sandboxed, JIT'ed JS is not too slow for speculative execution exploitation to be practical
> - (Not otherwise discussed here: PyPy's sandboxed JIT may not be too slow for speculative execution exploitation to be practical.)
>
In case of JavaScript, browser may load attacking code and run it while
user watching websites.
Browsers provides sandbox for JS, but attacker code may be able to
bypass the sandbox by Spectre or Meltdown. So browsers disabled
high precision timer until OSes are updated.
This topic is totally unrelated to compiler options: these compiler options
doesn't prohibit running attacking code, it just guard branches from
branch target injection.
Does my understanding collect? Why should we discuss about execution speed?
I think this topic should split to two topics: (1) Guard Python
process from Spectre/Meltdown
attack from other process, (2) Prohibit Python code attack other
processes by using
Spectre/Meltdown.
Regards,
--
INADA Naoki <songof...@gmail.com>
Franklin? Lee
>
> On Tue, Sep 18, 2018 at 7:08 AM Wes Turner <wes.t...@gmail.com> wrote:
> >
> > To summarize:
> >
> > - CPython may be vulnerable to speculative execution vulnerabilities, but none are known.
> > - In general, CPython is currently too slow for speculative execution exploitation to be practical.
> > - Sandboxed, JIT'ed JS is not too slow for speculative execution exploitation to be practical
> > - (Not otherwise discussed here: PyPy's sandboxed JIT may not be too slow for speculative execution exploitation to be practical.)
> >
>
> As far as I know, execution speed is important for attacker, not victim.
> In case of JavaScript, browser may load attacking code and run it while
> user watching websites.
> Browsers provides sandbox for JS, but attacker code may be able to
> bypass the sandbox by Spectre or Meltdown. So browsers disabled
> high precision timer until OSes are updated.
>
> This topic is totally unrelated to compiler options: these compiler options
> doesn't prohibit running attacking code, it just guard branches from
> branch target injection.
>
> Does my understanding collect? Why should we discuss about execution speed?
amount of time it takes for the CPU to load a value from memory and
invalidate a branch prediction:
https://hackernoon.com/timing-is-everything-understanding-the-meltdown-and-spectre-attacks-5e1946e44f9f
Stefan Ring
> I think this topic should split to two topics: (1) Guard Python
> process from Spectre/Meltdown
> attack from other process, (2) Prohibit Python code attack other
> processes by using
> Spectre/Meltdown.
Spectre "mitigation".
Wes Turner
On Thursday, September 20, 2018, Stefan Ring <stef...@gmail.com> wrote:
On Tue, Sep 18, 2018 at 8:38 AM INADA Naoki <songof...@gmail.com> wrote:
> I think this topic should split to two topics: (1) Guard Python
> process from Spectre/Meltdown
> attack from other process, (2) Prohibit Python code attack other
> processes by using
> Spectre/Meltdown.
(3) Guard Python from performance degradation by overly aggressive
Spectre "mitigation".
Franklin? Lee
>
> On Thursday, September 20, 2018, Stefan Ring <stef...@gmail.com> wrote:
>>
>> On Tue, Sep 18, 2018 at 8:38 AM INADA Naoki <songof...@gmail.com> wrote:
>>
>> > I think this topic should split to two topics: (1) Guard Python
>> > process from Spectre/Meltdown
>> > attack from other process, (2) Prohibit Python code attack other
>> > processes by using
>> > Spectre/Meltdown.
>>
>> (3) Guard Python from performance degradation by overly aggressive
>> Spectre "mitigation".
>
>
> > Spectre has the potential of having a greater impact on cloud providers than Meltdown. Whereas Meltdown allows unauthorized applications to read from privileged memory to obtain sensitive data from processes running on the same cloud server, Spectre can allow malicious programs to induce a hypervisor to transmit the data to a guest system running on top of it.
> - Private SSL certs
> - Cached keys and passwords in non-zeroed RAM
> - [...]
>
> https://en.wikipedia.org/wiki/Spectre_(security_vulnerability)
mean that companies like Amazon, Microsoft (Steve), and Docker should
have done analyses on CPython's vulnerability to these exploits?
Has/should/can anyone officially representing Python contact the
companies and ask them?
When I followed your quote to find the context, I found it uses, as
its source, a Forbes article. The source cited by THAT article is
Daniel Gruss, who was one of the researchers. Should someone from the
PSF contact the researchers? Steve says he spoke to some of them to
judge whether the proposed compiler flags would help, and decided
against it.
Absent of expert input, here's my non-expert take: That quote requires
an OS-level fix. A Python program without the proper permissions can't
do such things unless there is a vulnerability with the OS, and it is
extremely unlikely for anyone to update Python for Spectre but not
update the OS (and they'd be screwed in any case). And even if there
is a vulnerability in the OS, maybe the way to exploit it is by using
arbitrary Python execution (which you need before you can TRY to use
Spectre) on this Python interpreter. You can then write a new binary
file and run THAT, and it will be fast enough. That's not something
you can fix about CPython.
Also, (again with my understanding) the problem of Spectre and
Meltdown are that you can escape sandboxes and the like, such as the
user/kernel divide, or a software sandbox like that provided by a
JavaScript VM. For CPython to be "vulnerable" to these attacks, it
needs to have some kind of sandbox or protection to break out of.
Instead, we sometimes have sandboxes AROUND CPython (like Jupyter) or
WITHIN CPython. I don't see how it makes sense to talk about a sandbox
escape FOR CPython (yet).
Your original post linked to a discussion about Linux using those
build flags. Linux is a kernel, and has such protections that can be
bypassed, so it has something to worry about. Malicious code can be
native code, which (to my understanding) will be fast enough to
exploit the cache miss time. Here's Google's article about the
retpoline and why it helps:
https://support.google.com/faqs/answer/7625886
As of yet, you have quoted passages that have little relevance to
interpreter devs, especially non-JIT interpreters, and you have linked
to entire articles for non-experts with little relevance to
interpreter devs. This doesn't show that you have any better of an
understanding than I have, which is less than the understanding that
some of the Python devs have, and much less than what Steve has. In
short, it LOOKS like you don't know what you're talking about. If you
have a different and deeper understanding of the problem, then you
need to show it, and say why there is a problem for CPython
specifically. Or find someone who can do that for you.
> Here's one:
> https://github.com/Eugnis/spectre-attack/blob/master/Source.c
>
> Is this too slow in CPython with:
> - Coroutines (asyncio (tulip))
> - PyPy JIT *
> - Numba JIT *
> - C Extensions *
> - Cython *
>
> * Not anyone here's problem.
starred examples are fast enough, but it's probably more subtle than I
think and requires further analysis by their devs. I also think
there's something important I'm still missing about what's required
and what it can do.
I don't see what coroutines have to do with it. Coroutines are still
Python code, and they're subject to the GIL.
Wes Turner
Franklin? Lee
Mark Lawrence
> I feel like you are actively undermining attempts to prevent
> exploitation of known vulnerabilities because the software in question
> is currently too slow.
>
--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.
Mark Lawrence
Wes Turner
On Friday, September 21, 2018, Mark Lawrence <bream...@gmail.com> wrote:
On 21/09/18 08:11, Wes Turner wrote:
I feel like you are actively undermining attempts to prevent exploitation of known vulnerabilities because the software in question is currently too slow.
Did you write "How to Win Friends And Influence People"?
--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.
Mark Lawrence
_______________________________________________
Python-Dev mailing list
Pytho...@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com