[llvm-dev] retpoline mitigation and 6.0

274 views
Skip to first unread message

Hans Wennborg via llvm-dev

unread,
Feb 1, 2018, 4:11:11 AM2/1/18
to Chandler Carruth, Rui Ueyama, Reid Kleckner, llvm-dev
Hi all,

I saw the retpoline mitigation landed in r323155. Are we ready to merge this to 6.0, or are there any open issues that we're waiting for? Also, were there any followups I should know about? Also, release notes please :-)

Thanks,
Hans

Chandler Carruth via llvm-dev

unread,
Feb 1, 2018, 4:17:03 AM2/1/18
to Hans Wennborg, llvm-dev
There was one open issue that i landed the fix for today.

I was letting bots clear out before i ping you with the patch series to merge.

Hans Wennborg via llvm-dev

unread,
Feb 1, 2018, 4:27:46 AM2/1/18
to Chandler Carruth, llvm-dev
Sounds great, thanks!

Tom Stellard via llvm-dev

unread,
Feb 1, 2018, 1:08:39 PM2/1/18
to Hans Wennborg, Chandler Carruth, Rui Ueyama, Reid Kleckner, llvm-dev
On 02/01/2018 01:10 AM, Hans Wennborg via llvm-dev wrote:
> Hi all,
>
> I saw the retpoline mitigation landed in r323155. Are we ready to merge this to 6.0, or are there any open issues that we're waiting for? Also, were there any followups I should know about? Also, release notes please :-)
>

There was a follow up commit for lld: r323288

-Tom

> Thanks,
> Hans
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

David Woodhouse via llvm-dev

unread,
Feb 2, 2018, 7:04:11 PM2/2/18
to Hans Wennborg, Chandler Carruth, Rui Ueyama, Reid Kleckner, llvm-dev, Thomas Gleixner, Guenter Roeck
Eep, please can we keep the command line option for clang and the thunk
ABI matching what GCC, the Linux kernel and Xen are all doing?

To say that I am not stunningly keen on
https://lkml.org/lkml/2018/2/2/975 would be a bit of an
understatement...

Chandler Carruth via llvm-dev

unread,
Feb 2, 2018, 7:24:07 PM2/2/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
Two aspects to this...

One, we're somewhat reluctant to guarantee an ABI here. At least I am. While we don't *expect* rampant divergence here, I don't want this to become something we cannot change if there are good reasons to do so. We've already changed the thunks once based on feedback (putting LFENCE after the PAUSE).

Given that we don't want this to be a guaranteed part of the ABI, I really want the thunk names to be specific about which implementation is being used. For example, if we come up with a new, better thunk interface, we would want to give it a new name (even if it were just a suffix of "2") so that we don't get runtime failures. Since LLVM and GCC can't possible release in sync, IMO they *should* use different names. I asked the GCC developers to include 'gcc' in the name, but at least the person I asked was not at all receptive.


Two, I actually agree with you about the command line flags. I asked for it to be '-mretpoline'. I think a short, clear flag is really best, and we've very publicly documented this technique as 'retpoline'. But the GCC community has a fairly different design AFAICT... The only embedded thunks the offer are inline (ours are always out-of-line, even if they aren't external). Also, they support thunking indirect branches that we don't: ret. Our feature really is oriented around exposing the retpoline mitigation, and I don't think there is any demand for the complexity that would be entailed by generalizing it further. But I'm sure the GCC folks have a very reasonably different perspective here that make them prefer a quite different feature (they can thunk rets) and naming scheme.


So I don't really know how to fix this. I think there are reasonable arguments in each direction. IMO, the easiest improvement would be for GCC to add command line aliases spelled using 'retpoline' for the specific use case, but keep the more general commandline interface for which they have reasonable use cases.

The naming thing I think is essentially by-design so that if we (or you) need to split apart the implementation, there are distinct names available. Until then, aliases seem very cheap to maintain?

Chandler Carruth via llvm-dev

unread,
Feb 2, 2018, 7:27:30 PM2/2/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
On Fri, Feb 2, 2018 at 4:23 PM Chandler Carruth <chan...@google.com> wrote:
On Fri, Feb 2, 2018 at 4:03 PM David Woodhouse <dw...@infradead.org> wrote:
On Thu, 2018-02-01 at 10:10 +0100, Hans Wennborg via llvm-dev wrote:
>
> I saw the retpoline mitigation landed in r323155. Are we ready to
> merge this to 6.0, or are there any open issues that we're waiting
> for? Also, were there any followups I should know about? Also,
> release notes please :-)

Eep, please can we keep the command line option for clang and the thunk
ABI matching what GCC, the Linux kernel and Xen are all doing?

To say that I am not stunningly keen on
https://lkml.org/lkml/2018/2/2/975 would be a bit of an
understatement...

A minor note on this specific patch:

You don't need '-mretpoline -mretpoline-external-thunk'. The second flag implies the first. (Or should, if not, its a bug.) Our goal was that you needed exactly one flag to enable this in whatever form.

David Woodhouse via llvm-dev

unread,
Feb 2, 2018, 7:36:45 PM2/2/18
to Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck
On Sat, 2018-02-03 at 00:23 +0000, Chandler Carruth wrote:
>
> Two aspects to this...
>
> One, we're somewhat reluctant to guarantee an ABI here. At least I
> am. While we don't *expect* rampant divergence here, I don't want
> this to become something we cannot change if there are good reasons
> to do so. We've already changed the thunks once based on feedback
> (putting LFENCE after the PAUSE).

Surely adding the lfence was changing your implementation, not the ABI?

And if we really are talking about the *ABI* not the implementation,
I'm not sure I understand your concern.

The ABI for each thunk is that it is identical in all respects, apart
from speculation, to 'jump *\reg'. There's not a lot of scope for a
'v2' of that, surely?

I could live with the command-line divergence, although it's
suboptimal. But *please* since these things also need to be symbols
exported to loadable modules, can't we keep the thunk names identical?

You can even use __x86_indirect_thunk_\reg for *now* and reserve the
right to use a different name if you ever do revise the ABI.

Chandler Carruth via llvm-dev

unread,
Feb 2, 2018, 7:52:18 PM2/2/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
On Fri, Feb 2, 2018 at 4:36 PM David Woodhouse <dw...@infradead.org> wrote:
On Sat, 2018-02-03 at 00:23 +0000, Chandler Carruth wrote:
>
> Two aspects to this...
>
> One, we're somewhat reluctant to guarantee an ABI here. At least I
> am. While we don't *expect* rampant divergence here, I don't want
> this to become something we cannot change if there are good reasons
> to do so. We've already changed the thunks once based on feedback
> (putting LFENCE after the PAUSE).

Surely adding the lfence was changing your implementation, not the ABI?

And if we really are talking about the *ABI* not the implementation,
I'm not sure I understand your concern.

The ABI for each thunk is that it is identical in all respects, apart
from speculation, to 'jump *\reg'. There's not a lot of scope for a
'v2' of that, surely?

While I hope we have converged and never need to chaneg them, when we started, there was actually a substantially different ABI proposed, and multiple different ones. So some changes already were needed.

We already have at least one change that has been proposed, but we haven't decided to pursue yet: a thunk which includes the *load* of the address, and specifically a collection to handle common repeated patterns of loads that before retpoline would have folded into addressing modes. And these would definitely have a different ABI.
 

I could live with the command-line divergence, although it's
suboptimal. But *please* since these things also need to be symbols
exported to loadable modules, can't we keep the thunk names identical?

At least for LLVM when not using *external* thunks, we specifically *do not export* the symbol. As a consequence, DSOs built with two versions of LLVM (or LLVM and GCC) with different thunk behavior and each will find the correct thunk. That was a specific part of the design.

While you *can* export your external thunk, that's a choice of the code defining the thunk.
 

You can even use __x86_indirect_thunk_\reg for *now* and reserve the
right to use a different name if you ever do revise the ABI.

Maybe I don't understand, but I'm surprised that there is a significant burden to having aliases for now instead? 

David Woodhouse via llvm-dev

unread,
Feb 2, 2018, 7:59:54 PM2/2/18
to Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck


On Sat, 2018-02-03 at 00:51 +0000, Chandler Carruth wrote:
While you *can* export your external thunk, that's a choice of the code defining the thunk.

The driving force in the kernel is to be able to runtime patch the thunks away, when running on a CPU or in a mode that doesn't need them. We really want to have central implementations and have everything use them.


You can even use __x86_indirect_thunk_\reg for *now* and reserve the
right to use a different name if you ever do revise the ABI.

Maybe I don't understand, but I'm surprised that there is a significant burden to having aliases for now instead? 

You can make that work within the kernel image itself, and export only the existing names. It gets somewhat harder to support loadable modules which attempt to use the thunks by different names to the function that's exported. I'm not sure how we'd hack up the unresolved symbols in the module objects to match the exported symbol names.

Chandler Carruth via llvm-dev

unread,
Feb 2, 2018, 8:08:41 PM2/2/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
On Fri, Feb 2, 2018 at 4:59 PM David Woodhouse <dw...@infradead.org> wrote:


On Sat, 2018-02-03 at 00:51 +0000, Chandler Carruth wrote:
While you *can* export your external thunk, that's a choice of the code defining the thunk.

The driving force in the kernel is to be able to runtime patch the thunks away, when running on a CPU or in a mode that doesn't need them. We really want to have central implementations and have everything use them.

Sure, I can see why for the kernel, this is important. It also gets to essentially define its ABI however it wants.

For our other users, they specifically don't want this in the ABI (because they actually have DSOs and other junk being linked together). And yes, we actually have a decent number of users of this in userspace and outside the kernel. =/
 


You can even use __x86_indirect_thunk_\reg for *now* and reserve the
right to use a different name if you ever do revise the ABI.

Maybe I don't understand, but I'm surprised that there is a significant burden to having aliases for now instead? 

You can make that work within the kernel image itself, and export only the existing names. It gets somewhat harder to support loadable modules which attempt to use the thunks by different names to the function that's exported. I'm not sure how we'd hack up the unresolved symbols in the module objects to match the exported symbol names.

I had actually wanted to originally have the '-mretpoline-external-thunk' flag take a completely custom name for the thunks on the command line. Unfortunately, that proved remarkably annoying to implement in LLVM (for pretty silly reasons, but it is what it is). I didn't do that in large part because it seemed easy for users of external thunks to alias things themselves.

If the way loadable modules in the kernel work actually makes that quite hard, we can look again at allowing the external thunks to have a user controlled name. This seems strictly superior to any kind of agreed-to naming scheme as it gives you much more control. But it'll be really messy in LLVM to do[1], so I'd appreciate understanding how important this is for the kernel.

-Chandler

[1]: Yeah, it really shouldn't be. We are aware that this is a nasty aspect to how various components of LLVM are wired together and want to have better infrastructure here. But today, we don't. And especially since all of this needs to be backported to old versions of Clang and LLVM, we can't fix that and instead need to work around the existing infrastructure.

Chandler Carruth via llvm-dev

unread,
Feb 2, 2018, 9:04:51 PM2/2/18
to Guenter Roeck, llvm-dev, Thomas Gleixner
On Fri, Feb 2, 2018 at 5:56 PM Guenter Roeck <li...@roeck-us.net> wrote:
On 02/02/2018 04:27 PM, Chandler Carruth wrote:
> On Fri, Feb 2, 2018 at 4:23 PM Chandler Carruth <chan...@google.com <mailto:chan...@google.com>> wrote:

>
>     On Fri, Feb 2, 2018 at 4:03 PM David Woodhouse <dw...@infradead.org <mailto:dw...@infradead.org>> wrote:
>
>         On Thu, 2018-02-01 at 10:10 +0100, Hans Wennborg via llvm-dev wrote:
>          >
>          > I saw the retpoline mitigation landed in r323155. Are we ready to
>          > merge this to 6.0, or are there any open issues that we're waiting
>          > for? Also, were there any followups I should know about? Also,
>          > release notes please :-)
>
>         Eep, please can we keep the command line option for clang and the thunk
>         ABI matching what GCC, the Linux kernel and Xen are all doing?
>
>         To say that I am not stunningly keen on
>         https://lkml.org/lkml/2018/2/2/975 would be a bit of an
>         understatement...
>
>
> A minor note on this specific patch:
>
> You don't need '-mretpoline -mretpoline-external-thunk'. The second flag implies the first. (Or should, if not, its a bug.) Our goal was that you needed exactly one flag to enable this in whatever form.

The llvm commit log says:

"... They can write this custom thunk and use `-mretpoline-external-thunk`
*in addition* to `-mretpoline`. In this case, on x86-64 thu thunk names
must be: ... "

which I understand to mean that _both_ are needed. I'll be happy to stand
corrected if that is guaranteed not to be the case.

Gah, sorry. I thought I had updated that before landing, but I must have missed it.

The intent is that you *do not* need both. That changed during review, but that was the intent. The test seems to confirm that this should work. If you see anything that makes you think it isn't working, please give a shout (or file a bug) and we'll fix.

 

Thanks,
Guenter

Guenter Roeck via llvm-dev

unread,
Feb 3, 2018, 3:31:54 AM2/3/18
to Chandler Carruth, David Woodhouse, Thomas Gleixner, llvm-dev
On 02/02/2018 04:27 PM, Chandler Carruth wrote:
> On Fri, Feb 2, 2018 at 4:23 PM Chandler Carruth <chan...@google.com <mailto:chan...@google.com>> wrote:

>
> On Fri, Feb 2, 2018 at 4:03 PM David Woodhouse <dw...@infradead.org <mailto:dw...@infradead.org>> wrote:
>
> On Thu, 2018-02-01 at 10:10 +0100, Hans Wennborg via llvm-dev wrote:
> >
> > I saw the retpoline mitigation landed in r323155. Are we ready to
> > merge this to 6.0, or are there any open issues that we're waiting
> > for? Also, were there any followups I should know about? Also,
> > release notes please :-)
>
> Eep, please can we keep the command line option for clang and the thunk
> ABI matching what GCC, the Linux kernel and Xen are all doing?
>
> To say that I am not stunningly keen on
> https://lkml.org/lkml/2018/2/2/975 would be a bit of an
> understatement...
>
>
> A minor note on this specific patch:
>
> You don't need '-mretpoline -mretpoline-external-thunk'. The second flag implies the first. (Or should, if not, its a bug.) Our goal was that you needed exactly one flag to enable this in whatever form.

The llvm commit log says:

"... They can write this custom thunk and use `-mretpoline-external-thunk`
*in addition* to `-mretpoline`. In this case, on x86-64 thu thunk names
must be: ... "

which I understand to mean that _both_ are needed. I'll be happy to stand
corrected if that is guaranteed not to be the case.

Thanks,
Guenter

David Woodhouse via llvm-dev

unread,
Feb 3, 2018, 9:37:11 AM2/3/18
to Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck
On Sat, 2018-02-03 at 01:08 +0000, Chandler Carruth wrote:
>
> Sure, I can see why for the kernel, this is important. It also gets
> to essentially define its ABI however it wants.
>
> For our other users, they specifically don't want this in the ABI
> (because they actually have DSOs and other junk being linked
> together). And yes, we actually have a decent number of users of this
> in userspace and outside the kernel. =/

Using the *external* thunk in order to provide their own? While
claiming it isn't an ABI? That seems... odd.

Nobody cares what you call it when you're doing it inline or in a
COMDAT section. It's *only* the external thunk where it absolutely
*does* become an ABI, where we care about consistency.

Chandler Carruth via llvm-dev

unread,
Feb 6, 2018, 5:09:00 PM2/6/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
So, I was waiting to hear a definitive response on whether using aliases is hard, and didn't see one here, which is why I haven't responded further.

However, a colleauge pointed me at an LKML thread where it seems there *is* a definitive response?

I'm really looking for clear direction: we can try to implement custom naming, but it will add undesirable complexity to the compiler. Do we need it for the kernel? I have to ask because I genuinely don't know what is or isn't reasonable in the kernel.

On Sat, Feb 3, 2018 at 6:36 AM David Woodhouse <dw...@infradead.org> wrote:
It's *only* the external thunk where it absolutely
*does* become an ABI, where we care about consistency.

In the future, if there is such a powerful need for consistency, it would be good to actually engage with more than one compiler community. =/ As I said, I tried to talk to the GCC developers and made no progress but also heard no strong arguments that this kind of consistency was actually necessary.

I really do want to produce a feature that addresses the kernel's needs,  but we need to know what they are and have some chance to figure out how to find a solution that also doesn't cause problems for the compiler. This is just a note for the future though, the retpoline stuff is above.

David Woodhouse via llvm-dev

unread,
Feb 6, 2018, 5:56:45 PM2/6/18
to Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck


On Tue, 2018-02-06 at 22:08 +0000, Chandler Carruth wrote:
So, I was waiting to hear a definitive response on whether using aliases is hard, and didn't see one here, which is why I haven't responded further.

However, a colleauge pointed me at an LKML thread where it seems there *is* a definitive response?

Aliases are hard when the compiler is directly emitting calls to a function which is exported to modules, yet the compiler and the kernel disagree on what the symbol is actually called.

I spent a happy Sunday evening in a hotel room a few weeks ago, trying to make them work before telling the GCC folk "screw it this is too horrible please keep the symbol name as it is".

If I was insufficiently definitive, that's because I was inviting someone to prove me wrong — which I was still doing in the LKML thread you saw.

I'm really looking for clear direction: we can try to implement custom naming, but it will add undesirable complexity to the compiler. Do we need it for the kernel? I have to ask because I genuinely don't know what is or isn't reasonable in the kernel.

On Sat, Feb 3, 2018 at 6:36 AM David Woodhouse <dw...@infradead.org> wrote:
It's *only* the external thunk where it absolutely
*does* become an ABI, where we care about consistency.

In the future, if there is such a powerful need for consistency, it would be good to actually engage with more than one compiler community. =/ As I said, I tried to talk to the GCC developers and made no progress but also heard no strong arguments that this kind of consistency was actually necessary.

Yes, it would be good to engage with more than one compiler community. In the grand scheme of things that's probably one of the *smaller* things that didn't get handled right during this whole fiasco.

You'll note that when I added 16-bit support to LLVM/clang, I *did* make sure we did things consistently, and now the kernel boot code builds with -m16 with both clang and GCC.
On this occasion, I didn't get that right. I could have turned up here sooner after the embargo was broken. I take responsibility for that delay; I'm sorry.

I really do want to produce a feature that addresses the kernel's needs,  but we need to know what they are and have some chance to figure out how to find a solution that also doesn't cause problems for the compiler. This is just a note for the future though, the retpoline stuff is above.

At this point, what we really want is for identical thunks to have identical names — just like we do for builtins and other stuff, to avoid having differences between clang and GCC which just end up seeming capricious and being hard to work around. Having matching command line options would be a bonus, but isn't imperative.

If you reserve the right to invent new thunk ABIs later and thus have different names for those, that's absolutely fine. You'd break our build if you added those unconditionally anyway; we can choose to support them or not. And hopefully if GCC did the same, then again the thunk symbols would match.

Chandler Carruth via llvm-dev

unread,
Feb 6, 2018, 7:16:39 PM2/6/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
On Tue, Feb 6, 2018 at 2:56 PM David Woodhouse <dw...@infradead.org> wrote:
On Tue, 2018-02-06 at 22:08 +0000, Chandler Carruth wrote:
So, I was waiting to hear a definitive response on whether using aliases is hard, and didn't see one here, which is why I haven't responded further.

However, a colleauge pointed me at an LKML thread where it seems there *is* a definitive response?

Aliases are hard when the compiler is directly emitting calls to a function which is exported to modules, yet the compiler and the kernel disagree on what the symbol is actually called.

I spent a happy Sunday evening in a hotel room a few weeks ago, trying to make them work before telling the GCC folk "screw it this is too horrible please keep the symbol name as it is".

Nah, this is plenty definitive for me. ;]

At this point, what we really want is for identical thunks to have identical names — just like we do for builtins and other stuff, to avoid having differences between clang and GCC which just end up seeming capricious and being hard to work around. Having matching command line options would be a bonus, but isn't imperative.

After talking to several others (to make sure we don't have to do this whole thing yet again) we'll change the external thunk names to match what GCC is using. Hopefully this doesn't come back to bite us. =]

We'll also make sure those patches get backported too so that no released versions have the old behavior.

David Woodhouse via llvm-dev

unread,
Feb 6, 2018, 7:21:53 PM2/6/18
to Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck


On Wed, 2018-02-07 at 00:16 +0000, Chandler Carruth wrote:
At this point, what we really want is for identical thunks to have identical names — just like we do for builtins and other stuff, to avoid having differences between clang and GCC which just end up seeming capricious and being hard to work around. Having matching command line options would be a bonus, but isn't imperative.


After talking to several others (to make sure we don't have to do this whole thing yet again) we'll change the external thunk names to match what GCC is using. Hopefully this doesn't come back to bite us. =]

We'll also make sure those patches get backported too so that no released versions have the old behavior.

Thank you.

For reference, is there a way to turn *off* the retpoline which has been enabled on the command line? 

For init functions which run only at startup before any attacker can be in the system, we currently mark the function __attribute__((indirect_thunk("keep"))). Is there a clang equivalent?

It's not particularly important; a minor optimisation we can live without if we have to. 

Chandler Carruth via llvm-dev

unread,
Feb 6, 2018, 7:24:41 PM2/6/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
Can you file a bug to track either explaining how to do that or implementing it?

My guess is that it doesn't work yet, will be made to work by a future change that has already been proposed but not implemented...

Also, I'm going to hypothesize this will introduce yet another divergence between GCC and Clang. Not sure there is anyway to avoid that at this point. (We still have the significant divergence in that GCC supports thunking in ways that LLVM doesn't, and that in turn changes several aspects of the feature design.)

Chandler Carruth via llvm-dev

unread,
Feb 6, 2018, 7:26:32 PM2/6/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
What name do we use for when the target address is pushed onto the stack? What are the semantics? Is there a spec anywheere?

LLVM only needs this on 32-bit x86, but we do kind of need an answer before we update all of our branches with new names.....

David Woodhouse via llvm-dev

unread,
Feb 6, 2018, 7:33:05 PM2/6/18
to Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck
That would be __x86_indirect_thunk but the kernel doesn't use it. We use -mindirect-branch-register and only ever expect the compiler to use the register versions which are CET-compatible.

However, in at least one case in the 32-bit kernel we do emit the old ret-equivalent retpoline inline, because there literally wasn't a single register we could use (yay x86).

I would definitely consider ditching our use of -mindirect-thunk-register with GCC for 32-bit and exporting the __x86_indirect_thunk, to be consistent if that's what clang wants to do.


Chandler Carruth via llvm-dev

unread,
Feb 6, 2018, 7:36:45 PM2/6/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
:: sigh :: is there no way to change the name?

We use a "push" suffix to reduce ambiguity about what convention is expected here.... But I guess we can just use the base name if that's already shipped.
 

David Woodhouse via llvm-dev

unread,
Feb 6, 2018, 7:47:07 PM2/6/18
to Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck
On Wed, 2018-02-07 at 00:36 +0000, Chandler Carruth wrote:

> >
> > That would be __x86_indirect_thunk but the kernel doesn't use it.
> > We use -mindirect-branch-register and only ever expect the compiler
> > to use the register versions which are CET-compatible.
> >
> > However, in at least one case in the 32-bit kernel we do emit the
> > old ret-equivalent retpoline inline, because there literally wasn't
> > a single register we could use (yay x86).
> >
> > I would definitely consider ditching our use of -mindirect-thunk-
> > register with GCC for 32-bit and exporting the
> > __x86_indirect_thunk, to be consistent if that's what clang wants
> > to do.
> >
> :: sigh :: is there no way to change the name?
>
> We use a "push" suffix to reduce ambiguity about what convention is
> expected here.... But I guess we can just use the base name if that's
> already shipped.

It has indeed already shipped in GCC 7.3; sorry. It had no
disambiguation in its name because it was the original retpoline,
before we realised that CET would break things.

The other thing to keep an eye on is the *return* thunk, which might
end up being needed on Skylake-era CPUs. See the thread at
https://lkml.org/lkml/2018/2/4/147

Chandler Carruth via llvm-dev

unread,
Feb 6, 2018, 7:57:08 PM2/6/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
I'm strongly of the opinion that I think Arjan expressed:

- retpoline alone is probably fine with sufficient RSB stuffing patches in the kernel
- if some folks are worried about the security risk here and running on SKX, they should use IBRS.

Given the speed of IBRS on SKX and the complexity & runtime hit of thunking ret, I really don't see a good motivation for us teaching the compiler how to do this.

Chandler Carruth via llvm-dev

unread,
Feb 6, 2018, 7:58:46 PM2/6/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
Also, could you patch and test Clang with the Linux kernel after I make this change? I'd like to know that we actually successfully call the correct thunks and that they behave correctly. I'm not super worried, but good to  actually get this right. I'm am slightly more worried about the stack-based retpoline than the register ones just due to the overall lower amount of testing we've had there.

Chandler Carruth via llvm-dev

unread,
Feb 6, 2018, 9:00:18 PM2/6/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
The patch is up for review here: https://reviews.llvm.org/D42998

Chandler Carruth via llvm-dev

unread,
Feb 7, 2018, 1:21:04 AM2/7/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
I've landed the patch in r324449.

Before we merge this into two different Clang release branches and almost immediately release one of them, I would really like someone to confirm that this patch works well with the Linux kernel. David, if you're up for that, it would be great. Alternatively, Guenter or someone else here can help.

David Woodhouse via llvm-dev

unread,
Feb 7, 2018, 5:49:53 AM2/7/18
to Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck
On Wed, 2018-02-07 at 06:20 +0000, Chandler Carruth wrote:
> I've landed the patch in r324449.
>
> Before we merge this into two different Clang release branches and
> almost immediately release one of them, I would really like someone
> to confirm that this patch works well with the Linux kernel. David,
> if you're up for that, it would be great. Alternatively, Guenter or
> someone else here can help.

Hm, please could we also have the %V asm constraint modifier? That
allows us to emit calls to the thunks from inline asm using the
register that the compiler chose for us:

asm volatile ("call __x86_indirect_thunk_%V[thunk_target]" : :
[thunk_target] "r" (the_function));

Other than that, I get the following errors with LLVM+Clang master, and
my tree at
http://git.infradead.org/users/dwmw2/linux-retpoline.git/shortlog/refs/heads/ibpb

I haven't paid much attention to llvmlinux for a while; are these
expected? I'll try turning off config options until I can get something
to build...

Script started on 2018-02-07 10:36:07+0000
]0;dwoodhou@i7:~/git/linux-2.6 [dwoodhou@i7 linux-2.6]$ make CC=~/git/llvm-build/bin/clang vmlinux  -k
  CHK     include/config/kernel.release
  CHK     include/generated/uapi/linux/version.h
  CHK     include/generated/utsrelease.h
  CHK     include/generated/bounds.h
  CHK     include/generated/timeconst.h
  CHK     include/generated/asm-offsets.h
  CALL    scripts/checksyscalls.sh
  DESCEND  objtool
  CHK     scripts/mod/devicetable-offsets.h
  CHK     include/generated/compile.h
  CC      arch/x86/events/core.o
clang-7.0: /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:9092: bool llvm::ScalarEvolution::isLoopEntryGuardedByCond(const llvm::Loop*, llvm::CmpInst::Predicate, const llvm::SCEV*, const llvm::SCEV*): Assertion `(!ProvedNonStrictComparison || !ProvedNonEquality) && "Why we were unable to prove (Pred, LHS, RHS) via constant ranges?"' failed.
#0 0x0000000003cdcc0e llvm::sys::PrintStackTrace(llvm::raw_ostream&) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:398:0
#1 0x0000000003cdcca1 PrintStackTraceSignalHandler(void*) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:462:0
#2 0x0000000003cdb146 llvm::sys::RunSignalHandlers() /home/dwmw2/git/llvm/lib/Support/Signals.cpp:49:0
#3 0x0000000003cdc583 SignalHandler(int) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:252:0
#4 0x00007f9b52e75a80 __restore_rt (/lib64/libpthread.so.0+0x12a80)
#5 0x00007f9b5197666b __GI_raise (/lib64/libc.so.6+0x3766b)
#6 0x00007f9b51978381 __GI_abort (/lib64/libc.so.6+0x39381)
#7 0x00007f9b5196e8fa __assert_fail_base (/lib64/libc.so.6+0x2f8fa)
#8 0x00007f9b5196e972 (/lib64/libc.so.6+0x2f972)
#9 0x0000000002f60e6b llvm::ScalarEvolution::isLoopEntryGuardedByCond(llvm::Loop const*, llvm::CmpInst::Predicate, llvm::SCEV const*, llvm::SCEV const*) /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:9110:0
#10 0x0000000002f3a1f8 llvm::ScalarEvolution::getSignExtendExpr(llvm::SCEV const*, llvm::Type*, unsigned int) /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:1991:0
#11 0x0000000003dd8df8 (anonymous namespace)::SimplifyIndvar::strengthenOverflowingOperation(llvm::BinaryOperator*, llvm::Value*) /home/dwmw2/git/llvm/lib/Transforms/Utils/SimplifyIndVar.cpp:652:0
#12 0x0000000003dd9578 (anonymous namespace)::SimplifyIndvar::simplifyUsers(llvm::PHINode*, llvm::IVVisitor*) /home/dwmw2/git/llvm/lib/Transforms/Utils/SimplifyIndVar.cpp:800:0
#13 0x0000000003dd9757 llvm::simplifyUsersOfIV(llvm::PHINode*, llvm::ScalarEvolution*, llvm::DominatorTree*, llvm::LoopInfo*, llvm::SmallVectorImpl<llvm::WeakTrackingVH>&, llvm::SCEVExpander&, llvm::IVVisitor*) /home/dwmw2/git/llvm/lib/Transforms/Utils/SimplifyIndVar.cpp:832:0
#14 0x0000000003a4235a (anonymous namespace)::IndVarSimplify::simplifyAndExtend(llvm::Loop*, llvm::SCEVExpander&, llvm::LoopInfo*) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:1812:0
#15 0x0000000003a447db (anonymous namespace)::IndVarSimplify::run(llvm::Loop*) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:2461:0
#16 0x0000000003a44f64 (anonymous namespace)::IndVarSimplifyLegacyPass::runOnLoop(llvm::Loop*, llvm::LPPassManager&) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:2576:0
#17 0x0000000002ed44ff llvm::LPPassManager::runOnFunction(llvm::Function&) /home/dwmw2/git/llvm/lib/Analysis/LoopPass.cpp:204:0
#18 0x00000000035a1af1 llvm::FPPassManager::runOnFunction(llvm::Function&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1520:0
#19 0x0000000002e0311c (anonymous namespace)::CGPassManager::RunPassOnSCC(llvm::Pass*, llvm::CallGraphSCC&, llvm::CallGraph&, bool&, bool&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:156:0
#20 0x0000000002e040a4 (anonymous namespace)::CGPassManager::RunAllPassesOnSCC(llvm::CallGraphSCC&, llvm::CallGraph&, bool&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:423:0
#21 0x0000000002e043a5 (anonymous namespace)::CGPassManager::runOnModule(llvm::Module&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:479:0
#22 0x00000000035a1fc3 (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1597:0
#23 0x00000000035a2693 llvm::legacy::PassManagerImpl::run(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1700:0
#24 0x00000000035a288b llvm::legacy::PassManager::run(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1732:0
#25 0x0000000003f8b5d1 (anonymous namespace)::EmitAssemblyHelper::EmitAssembly(clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/BackendUtil.cpp:801:0
#26 0x0000000003f8df15 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::DataLayout const&, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/BackendUtil.cpp:1180:0
#27 0x0000000004aec157 clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/CodeGenAction.cpp:292:0
#28 0x00000000057fec75 clang::ParseAST(clang::Sema&, bool, bool) /home/dwmw2/git/llvm/tools/clang/lib/Parse/ParseAST.cpp:161:0
#29 0x00000000045f2b13 clang::ASTFrontendAction::ExecuteAction() /home/dwmw2/git/llvm/tools/clang/lib/Frontend/FrontendAction.cpp:1005:0
#30 0x0000000004aea206 clang::CodeGenAction::ExecuteAction() /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/CodeGenAction.cpp:1032:0
#31 0x00000000045f256d clang::FrontendAction::Execute() /home/dwmw2/git/llvm/tools/clang/lib/Frontend/FrontendAction.cpp:908:0
#32 0x000000000459028f clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) /home/dwmw2/git/llvm/tools/clang/lib/Frontend/CompilerInstance.cpp:992:0
#33 0x0000000004734158 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) /home/dwmw2/git/llvm/tools/clang/lib/FrontendTool/ExecuteCompilerInvocation.cpp:252:0
#34 0x0000000001c119a2 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) /home/dwmw2/git/llvm/tools/clang/tools/driver/cc1_main.cpp:221:0
#35 0x0000000001c06f5d ExecuteCC1Tool(llvm::ArrayRef<char const*>, llvm::StringRef) /home/dwmw2/git/llvm/tools/clang/tools/driver/driver.cpp:309:0
#36 0x0000000001c07b94 main /home/dwmw2/git/llvm/tools/clang/tools/driver/driver.cpp:389:0
#37 0x00007f9b5196000a __libc_start_main (/lib64/libc.so.6+0x2100a)
#38 0x0000000001c0480a _start (/home/dwmw2/git/llvm-build/bin/clang-7.0+0x1c0480a)
Stack dump:
0. Program arguments: /home/dwmw2/git/llvm-build/bin/clang-7.0 -cc1 -triple x86_64-unknown-linux-gnu -S -disable-free -main-file-name core.c -mrelocation-model static -mthread-model posix -mllvm -warn-stack-size=2048 -mdisable-fp-elim -relaxed-aliasing -fmath-errno -masm-verbose -no-integrated-as -mconstructor-aliases -fuse-init-array -mcode-model kernel -target-cpu x86-64 -target-feature -sse -target-feature -mmx -target-feature -sse2 -target-feature -3dnow -target-feature -avx -target-feature -x87 -target-feature +retpoline-external-thunk -disable-red-zone -dwarf-column-info -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -coverage-notes-file /home/dwmw2/git/linux-2.6/arch/x86/events/core.gcno -nostdsysteminc -nobuiltininc -resource-dir /home/dwmw2/git/llvm-build/lib/clang/7.0.0 -dependency-file arch/x86/events/.core.o.d -MT arch/x86/events/core.o -sys-header-deps -isystem /home/dwmw2/git/llvm-build/lib/clang/7.0.0/include -include ./include/linux/kconfig.h -I ./arch/x86/include -I ./arch/x86/include/generated -I ./include -I ./arch/x86/include/uapi -I ./arch/x86/include/generated/uapi -I ./include/uapi -I ./include/generated/uapi -D __KERNEL__ -D CONFIG_AS_CFI=1 -D CONFIG_AS_CFI_SIGNAL_FRAME=1 -D CONFIG_AS_CFI_SECTIONS=1 -D CONFIG_AS_FXSAVEQ=1 -D CONFIG_AS_SSSE3=1 -D CONFIG_AS_CRC32=1 -D CONFIG_AS_AVX=1 -D CONFIG_AS_AVX2=1 -D CONFIG_AS_AVX512=1 -D CONFIG_AS_SHA1_NI=1 -D CONFIG_AS_SHA256_NI=1 -D RETPOLINE -D CC_USING_FENTRY -D KBUILD_BASENAME="core" -D KBUILD_MODNAME="core" -O2 -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -Werror-implicit-function-declaration -Wno-format-security -Wno-sign-compare -Wno-unused-variable -Wno-format-invalid-specifier -Wno-gnu -Wno-address-of-packed-member -Wno-tautological-compare -Wdeclaration-after-statement -Wno-pointer-sign -Werror=implicit-int -Werror=strict-prototypes -Werror=date-time -Werror=incompatible-pointer-types -Wno-initializer-overrides -Wno-unused-value -Wno-format -Wno-sign-compare -Wno-format-zero-length -Wno-uninitialized -std=gnu89 -fno-dwarf-directory-asm -fdebug-compilation-dir /home/dwmw2/git/linux-2.6 -ferror-limit 19 -fmessage-length 80 -pg -mfentry -fwrapv -stack-protector 2 -mstack-alignment=8 -fwchar-type=short -fno-signed-wchar -fobjc-runtime=gcc -fno-common -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -o /tmp/core-26897d.s -x c arch/x86/events/core.c 
1. <eof> parser at end of file
2. Per-module optimization passes
3. Running pass 'CallGraph Pass Manager' on module 'arch/x86/events/core.c'.
4. Running pass 'Loop Pass Manager' on function '@x86_reserve_hardware'
5. Running pass 'Induction Variable Simplification' on basic block '%for.body15.i'
clang-7.0: [0;1;31merror: [0munable to execute command: Aborted (core dumped) [0m
clang-7.0: [0;1;31merror: [0mclang frontend command failed due to signal (use -v to see invocation) [0m
clang version 7.0.0  (llvm/trunk 324455)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/dwmw2/git/llvm-build/bin
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace, preprocessed source, and associated run script. [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: 
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at: [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: /tmp/core-4220fe.c [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: /tmp/core-4220fe.sh [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: 

******************** [0m
make[2]: *** [scripts/Makefile.build:317: arch/x86/events/core.o] Error 254
make[2]: Target '__build' not remade because of errors.
make[1]: *** [scripts/Makefile.build:575: arch/x86/events] Error 2
  CC      arch/x86/hyperv/mmu.o
In file included from arch/x86/hyperv/mmu.c:9:
[1m./arch/x86/include/asm/mshyperv.h:189:23: [0m [0;1;31merror: [0m [1minvalid operand in inline asm:
      'mov $4, %r8999: .pushsection .discard.nospec .long 999b - . .popsection
      661: call *$5662:.skip -(((6651f-6641f)-(662b-661b)) > 0) *
      ((6651f-6641f)-(662b-661b)),0x90663:.pushsection .altinstructions,"a"
      .long 661b - . .long 6641f - . .word ( 7*32+12) .byte 663b-661b .byte
      6651f-6641f .byte 663b-662b.popsection.pushsection .altinstr_replacement,
      "ax"6641: call __x86_indirect_thunk_${5:V}6651: .popsection' [0m
        __asm__ __volatile__("mov %4, %%r8\n"
[0;1;32m                             ^
[0m [1m./arch/x86/include/asm/mshyperv.h:189:23: [0m [0;1;31merror: [0m [1minvalid operand in inline asm:
      'mov $4, %r8999: .pushsection .discard.nospec .long 999b - . .popsection
      661: call *$5662:.skip -(((6651f-6641f)-(662b-661b)) > 0) *
      ((6651f-6641f)-(662b-661b)),0x90663:.pushsection .altinstructions,"a"
      .long 661b - . .long 6641f - . .word ( 7*32+12) .byte 663b-661b .byte
      6651f-6641f .byte 663b-662b.popsection.pushsection .altinstr_replacement,
      "ax"6641: call __x86_indirect_thunk_${5:V}6651: .popsection' [0m
[1m./arch/x86/include/asm/mshyperv.h:189:23: [0m [0;1;31merror: [0m [1minvalid operand in inline asm:
      'mov $4, %r8999: .pushsection .discard.nospec .long 999b - . .popsection
      661: call *$5662:.skip -(((6651f-6641f)-(662b-661b)) > 0) *
      ((6651f-6641f)-(662b-661b)),0x90663:.pushsection .altinstructions,"a"
      .long 661b - . .long 6641f - . .word ( 7*32+12) .byte 663b-661b .byte
      6651f-6641f .byte 663b-662b.popsection.pushsection .altinstr_replacement,
      "ax"6641: call __x86_indirect_thunk_${5:V}6651: .popsection' [0m
[1m./arch/x86/include/asm/mshyperv.h:189:23: [0m [0;1;31merror: [0m [1minvalid operand in inline asm:
      'mov $4, %r8999: .pushsection .discard.nospec .long 999b - . .popsection
      661: call *$5662:.skip -(((6651f-6641f)-(662b-661b)) > 0) *
      ((6651f-6641f)-(662b-661b)),0x90663:.pushsection .altinstructions,"a"
      .long 661b - . .long 6641f - . .word ( 7*32+12) .byte 663b-661b .byte
      6651f-6641f .byte 663b-662b.popsection.pushsection .altinstr_replacement,
      "ax"6641: call __x86_indirect_thunk_${5:V}6651: .popsection' [0m
[1m./arch/x86/include/asm/mshyperv.h:189:23: [0m [0;1;31merror: [0m [1minvalid operand in inline asm:
      'mov $4, %r8999: .pushsection .discard.nospec .long 999b - . .popsection
      661: call *$5662:.skip -(((6651f-6641f)-(662b-661b)) > 0) *
      ((6651f-6641f)-(662b-661b)),0x90663:.pushsection .altinstructions,"a"
      .long 661b - . .long 6641f - . .word ( 7*32+12) .byte 663b-661b .byte
      6651f-6641f .byte 663b-662b.popsection.pushsection .altinstr_replacement,
      "ax"6641: call __x86_indirect_thunk_${5:V}6651: .popsection' [0m
[1m./arch/x86/include/asm/mshyperv.h:189:23: [0m [0;1;31merror: [0m [1minvalid operand in inline asm:
      'mov $4, %r8999: .pushsection .discard.nospec .long 999b - . .popsection
      661: call *$5662:.skip -(((6651f-6641f)-(662b-661b)) > 0) *
      ((6651f-6641f)-(662b-661b)),0x90663:.pushsection .altinstructions,"a"
      .long 661b - . .long 6641f - . .word ( 7*32+12) .byte 663b-661b .byte
      6651f-6641f .byte 663b-662b.popsection.pushsection .altinstr_replacement,
      "ax"6641: call __x86_indirect_thunk_${5:V}6651: .popsection' [0m
6 errors generated.
make[2]: *** [scripts/Makefile.build:317: arch/x86/hyperv/mmu.o] Error 1
make[2]: Target '__build' not remade because of errors.
make[1]: *** [scripts/Makefile.build:575: arch/x86/hyperv] Error 2
  CC      arch/x86/kernel/cpu/microcode/intel.o
clang-7.0: /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:9092: bool llvm::ScalarEvolution::isLoopEntryGuardedByCond(const llvm::Loop*, llvm::CmpInst::Predicate, const llvm::SCEV*, const llvm::SCEV*): Assertion `(!ProvedNonStrictComparison || !ProvedNonEquality) && "Why we were unable to prove (Pred, LHS, RHS) via constant ranges?"' failed.
#0 0x0000000003cdcc0e llvm::sys::PrintStackTrace(llvm::raw_ostream&) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:398:0
#1 0x0000000003cdcca1 PrintStackTraceSignalHandler(void*) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:462:0
#2 0x0000000003cdb146 llvm::sys::RunSignalHandlers() /home/dwmw2/git/llvm/lib/Support/Signals.cpp:49:0
#3 0x0000000003cdc583 SignalHandler(int) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:252:0
#4 0x00007fc240864a80 __restore_rt (/lib64/libpthread.so.0+0x12a80)
#5 0x00007fc23f36566b __GI_raise (/lib64/libc.so.6+0x3766b)
#6 0x00007fc23f367381 __GI_abort (/lib64/libc.so.6+0x39381)
#7 0x00007fc23f35d8fa __assert_fail_base (/lib64/libc.so.6+0x2f8fa)
#8 0x00007fc23f35d972 (/lib64/libc.so.6+0x2f972)
#9 0x0000000002f60e6b llvm::ScalarEvolution::isLoopEntryGuardedByCond(llvm::Loop const*, llvm::CmpInst::Predicate, llvm::SCEV const*, llvm::SCEV const*) /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:9110:0
#10 0x0000000002f3a1f8 llvm::ScalarEvolution::getSignExtendExpr(llvm::SCEV const*, llvm::Type*, unsigned int) /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:1991:0
#11 0x0000000003dd8d75 (anonymous namespace)::SimplifyIndvar::strengthenOverflowingOperation(llvm::BinaryOperator*, llvm::Value*) /home/dwmw2/git/llvm/lib/Transforms/Utils/SimplifyIndVar.cpp:649:0
#12 0x0000000003dd9578 (anonymous namespace)::SimplifyIndvar::simplifyUsers(llvm::PHINode*, llvm::IVVisitor*) /home/dwmw2/git/llvm/lib/Transforms/Utils/SimplifyIndVar.cpp:800:0
#13 0x0000000003dd9757 llvm::simplifyUsersOfIV(llvm::PHINode*, llvm::ScalarEvolution*, llvm::DominatorTree*, llvm::LoopInfo*, llvm::SmallVectorImpl<llvm::WeakTrackingVH>&, llvm::SCEVExpander&, llvm::IVVisitor*) /home/dwmw2/git/llvm/lib/Transforms/Utils/SimplifyIndVar.cpp:832:0
#14 0x0000000003a4235a (anonymous namespace)::IndVarSimplify::simplifyAndExtend(llvm::Loop*, llvm::SCEVExpander&, llvm::LoopInfo*) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:1812:0
#15 0x0000000003a447db (anonymous namespace)::IndVarSimplify::run(llvm::Loop*) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:2461:0
#16 0x0000000003a44f64 (anonymous namespace)::IndVarSimplifyLegacyPass::runOnLoop(llvm::Loop*, llvm::LPPassManager&) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:2576:0
#17 0x0000000002ed44ff llvm::LPPassManager::runOnFunction(llvm::Function&) /home/dwmw2/git/llvm/lib/Analysis/LoopPass.cpp:204:0
#18 0x00000000035a1af1 llvm::FPPassManager::runOnFunction(llvm::Function&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1520:0
#19 0x0000000002e0311c (anonymous namespace)::CGPassManager::RunPassOnSCC(llvm::Pass*, llvm::CallGraphSCC&, llvm::CallGraph&, bool&, bool&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:156:0
#20 0x0000000002e040a4 (anonymous namespace)::CGPassManager::RunAllPassesOnSCC(llvm::CallGraphSCC&, llvm::CallGraph&, bool&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:423:0
#21 0x0000000002e043a5 (anonymous namespace)::CGPassManager::runOnModule(llvm::Module&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:479:0
#22 0x00000000035a1fc3 (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1597:0
#23 0x00000000035a2693 llvm::legacy::PassManagerImpl::run(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1700:0
#24 0x00000000035a288b llvm::legacy::PassManager::run(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1732:0
#25 0x0000000003f8b5d1 (anonymous namespace)::EmitAssemblyHelper::EmitAssembly(clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/BackendUtil.cpp:801:0
#26 0x0000000003f8df15 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::DataLayout const&, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/BackendUtil.cpp:1180:0
#27 0x0000000004aec157 clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/CodeGenAction.cpp:292:0
#28 0x00000000057fec75 clang::ParseAST(clang::Sema&, bool, bool) /home/dwmw2/git/llvm/tools/clang/lib/Parse/ParseAST.cpp:161:0
#29 0x00000000045f2b13 clang::ASTFrontendAction::ExecuteAction() /home/dwmw2/git/llvm/tools/clang/lib/Frontend/FrontendAction.cpp:1005:0
#30 0x0000000004aea206 clang::CodeGenAction::ExecuteAction() /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/CodeGenAction.cpp:1032:0
#31 0x00000000045f256d clang::FrontendAction::Execute() /home/dwmw2/git/llvm/tools/clang/lib/Frontend/FrontendAction.cpp:908:0
#32 0x000000000459028f clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) /home/dwmw2/git/llvm/tools/clang/lib/Frontend/CompilerInstance.cpp:992:0
#33 0x0000000004734158 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) /home/dwmw2/git/llvm/tools/clang/lib/FrontendTool/ExecuteCompilerInvocation.cpp:252:0
#34 0x0000000001c119a2 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) /home/dwmw2/git/llvm/tools/clang/tools/driver/cc1_main.cpp:221:0
#35 0x0000000001c06f5d ExecuteCC1Tool(llvm::ArrayRef<char const*>, llvm::StringRef) /home/dwmw2/git/llvm/tools/clang/tools/driver/driver.cpp:309:0
#36 0x0000000001c07b94 main /home/dwmw2/git/llvm/tools/clang/tools/driver/driver.cpp:389:0
#37 0x00007fc23f34f00a __libc_start_main (/lib64/libc.so.6+0x2100a)
#38 0x0000000001c0480a _start (/home/dwmw2/git/llvm-build/bin/clang-7.0+0x1c0480a)
Stack dump:
0. Program arguments: /home/dwmw2/git/llvm-build/bin/clang-7.0 -cc1 -triple x86_64-unknown-linux-gnu -S -disable-free -main-file-name intel.c -mrelocation-model static -mthread-model posix -mllvm -warn-stack-size=2048 -mdisable-fp-elim -relaxed-aliasing -fmath-errno -masm-verbose -no-integrated-as -mconstructor-aliases -fuse-init-array -mcode-model kernel -target-cpu x86-64 -target-feature -sse -target-feature -mmx -target-feature -sse2 -target-feature -3dnow -target-feature -avx -target-feature -x87 -target-feature +retpoline-external-thunk -disable-red-zone -dwarf-column-info -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -coverage-notes-file /home/dwmw2/git/linux-2.6/arch/x86/kernel/cpu/microcode/intel.gcno -nostdsysteminc -nobuiltininc -resource-dir /home/dwmw2/git/llvm-build/lib/clang/7.0.0 -dependency-file arch/x86/kernel/cpu/microcode/.intel.o.d -MT arch/x86/kernel/cpu/microcode/intel.o -sys-header-deps -isystem /home/dwmw2/git/llvm-build/lib/clang/7.0.0/include -include ./include/linux/kconfig.h -I ./arch/x86/include -I ./arch/x86/include/generated -I ./include -I ./arch/x86/include/uapi -I ./arch/x86/include/generated/uapi -I ./include/uapi -I ./include/generated/uapi -D __KERNEL__ -D CONFIG_AS_CFI=1 -D CONFIG_AS_CFI_SIGNAL_FRAME=1 -D CONFIG_AS_CFI_SECTIONS=1 -D CONFIG_AS_FXSAVEQ=1 -D CONFIG_AS_SSSE3=1 -D CONFIG_AS_CRC32=1 -D CONFIG_AS_AVX=1 -D CONFIG_AS_AVX2=1 -D CONFIG_AS_AVX512=1 -D CONFIG_AS_SHA1_NI=1 -D CONFIG_AS_SHA256_NI=1 -D RETPOLINE -D CC_USING_FENTRY -D KBUILD_BASENAME="intel" -D KBUILD_MODNAME="microcode" -O2 -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -Werror-implicit-function-declaration -Wno-format-security -Wno-sign-compare -Wno-unused-variable -Wno-format-invalid-specifier -Wno-gnu -Wno-address-of-packed-member -Wno-tautological-compare -Wdeclaration-after-statement -Wno-pointer-sign -Werror=implicit-int -Werror=strict-prototypes -Werror=date-time -Werror=incompatible-pointer-types -Wno-initializer-overrides -Wno-unused-value -Wno-format -Wno-sign-compare -Wno-format-zero-length -Wno-uninitialized -std=gnu89 -fno-dwarf-directory-asm -fdebug-compilation-dir /home/dwmw2/git/linux-2.6 -ferror-limit 19 -fmessage-length 80 -pg -mfentry -fwrapv -stack-protector 2 -mstack-alignment=8 -fwchar-type=short -fno-signed-wchar -fobjc-runtime=gcc -fno-common -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -o /tmp/intel-e7ba31.s -x c arch/x86/kernel/cpu/microcode/intel.c 
1. <eof> parser at end of file
2. Per-module optimization passes
3. Running pass 'CallGraph Pass Manager' on module 'arch/x86/kernel/cpu/microcode/intel.c'.
4. Running pass 'Loop Pass Manager' on function '@microcode_sanity_check'
5. Running pass 'Induction Variable Simplification' on basic block '%while.body70'
clang-7.0: [0;1;31merror: [0munable to execute command: Aborted (core dumped) [0m
clang-7.0: [0;1;31merror: [0mclang frontend command failed due to signal (use -v to see invocation) [0m
clang version 7.0.0  (llvm/trunk 324455)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/dwmw2/git/llvm-build/bin
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace, preprocessed source, and associated run script. [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: 
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at: [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: /tmp/intel-c10d76.c [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: /tmp/intel-c10d76.sh [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: 

******************** [0m
make[4]: *** [scripts/Makefile.build:317: arch/x86/kernel/cpu/microcode/intel.o] Error 254
make[4]: Target '__build' not remade because of errors.
make[3]: *** [scripts/Makefile.build:575: arch/x86/kernel/cpu/microcode] Error 2
make[3]: Target '__build' not remade because of errors.
make[2]: *** [scripts/Makefile.build:575: arch/x86/kernel/cpu] Error 2
make[2]: Target '__build' not remade because of errors.
make[1]: *** [scripts/Makefile.build:575: arch/x86/kernel] Error 2
  CC      arch/x86/xen/multicalls.o
In file included from arch/x86/xen/multicalls.c:27:
[1m./arch/x86/include/asm/xen/hypercall.h:221:15: [0m [0;1;31merror: [0m [1minvalid operand in inline
      asm: '999: .pushsection .discard.nospec .long 999b - . .popsection 661:
      call *$7662:.skip -(((6651f-6641f)-(662b-661b)) > 0) *
      ((6651f-6641f)-(662b-661b)),0x90663:.pushsection .altinstructions,"a"
      .long 661b - . .long 6641f - . .word ( 7*32+12) .byte 663b-661b .byte
      6651f-6641f .byte 663b-662b.popsection.pushsection .altinstr_replacement,
      "ax"6641: call __x86_indirect_thunk_${7:V}6651: .popsection' [0m
        asm volatile(CALL_NOSPEC
[0;1;32m                     ^
[0m [1m./arch/x86/include/asm/nospec-branch.h:102:2: [0m [0;1;30mnote: [0mexpanded from macro
      'CALL_NOSPEC' [0m
        ANNOTATE_NOSPEC_ALTERNATIVE                             \
[0;1;32m        ^
[0m [1m./arch/x86/include/asm/nospec-branch.h:90:2: [0m [0;1;30mnote: [0mexpanded from macro
      'ANNOTATE_NOSPEC_ALTERNATIVE' [0m
        "999:\n\t"                                              \
[0;1;32m        ^
[0m1 error generated.
make[2]: *** [scripts/Makefile.build:317: arch/x86/xen/multicalls.o] Error 1
make[2]: Target '__build' not remade because of errors.
make[1]: *** [scripts/Makefile.build:575: arch/x86/xen] Error 2
make[1]: Target '__build' not remade because of errors.
make: *** [Makefile:1018: arch/x86] Error 2
  CC      drivers/cpuidle/governors/menu.o
clang-7.0: /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:9092: bool llvm::ScalarEvolution::isLoopEntryGuardedByCond(const llvm::Loop*, llvm::CmpInst::Predicate, const llvm::SCEV*, const llvm::SCEV*): Assertion `(!ProvedNonStrictComparison || !ProvedNonEquality) && "Why we were unable to prove (Pred, LHS, RHS) via constant ranges?"' failed.
#0 0x0000000003cdcc0e llvm::sys::PrintStackTrace(llvm::raw_ostream&) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:398:0
#1 0x0000000003cdcca1 PrintStackTraceSignalHandler(void*) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:462:0
#2 0x0000000003cdb146 llvm::sys::RunSignalHandlers() /home/dwmw2/git/llvm/lib/Support/Signals.cpp:49:0
#3 0x0000000003cdc583 SignalHandler(int) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:252:0
#4 0x00007f8d6a5baa80 __restore_rt (/lib64/libpthread.so.0+0x12a80)
#5 0x00007f8d690bb66b __GI_raise (/lib64/libc.so.6+0x3766b)
#6 0x00007f8d690bd381 __GI_abort (/lib64/libc.so.6+0x39381)
#7 0x00007f8d690b38fa __assert_fail_base (/lib64/libc.so.6+0x2f8fa)
#8 0x00007f8d690b3972 (/lib64/libc.so.6+0x2f972)
#9 0x0000000002f60e6b llvm::ScalarEvolution::isLoopEntryGuardedByCond(llvm::Loop const*, llvm::CmpInst::Predicate, llvm::SCEV const*, llvm::SCEV const*) /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:9110:0
#10 0x0000000002f5f6d1 llvm::ScalarEvolution::isKnownPredicate(llvm::CmpInst::Predicate, llvm::SCEV const*, llvm::SCEV const*) /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:8672:0
#11 0x0000000003a40edf (anonymous namespace)::WidenIV::pushNarrowIVUsers(llvm::Instruction*, llvm::Instruction*) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:1510:0
#12 0x0000000003a415f4 (anonymous namespace)::WidenIV::createWideIV(llvm::SCEVExpander&) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:1621:0
#13 0x0000000003a42464 (anonymous namespace)::IndVarSimplify::simplifyAndExtend(llvm::Loop*, llvm::SCEVExpander&, llvm::LoopInfo*) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:1821:0
#14 0x0000000003a447db (anonymous namespace)::IndVarSimplify::run(llvm::Loop*) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:2461:0
#15 0x0000000003a44f64 (anonymous namespace)::IndVarSimplifyLegacyPass::runOnLoop(llvm::Loop*, llvm::LPPassManager&) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:2576:0
#16 0x0000000002ed44ff llvm::LPPassManager::runOnFunction(llvm::Function&) /home/dwmw2/git/llvm/lib/Analysis/LoopPass.cpp:204:0
#17 0x00000000035a1af1 llvm::FPPassManager::runOnFunction(llvm::Function&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1520:0
#18 0x0000000002e0311c (anonymous namespace)::CGPassManager::RunPassOnSCC(llvm::Pass*, llvm::CallGraphSCC&, llvm::CallGraph&, bool&, bool&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:156:0
#19 0x0000000002e040a4 (anonymous namespace)::CGPassManager::RunAllPassesOnSCC(llvm::CallGraphSCC&, llvm::CallGraph&, bool&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:423:0
#20 0x0000000002e043a5 (anonymous namespace)::CGPassManager::runOnModule(llvm::Module&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:479:0
#21 0x00000000035a1fc3 (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1597:0
#22 0x00000000035a2693 llvm::legacy::PassManagerImpl::run(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1700:0
#23 0x00000000035a288b llvm::legacy::PassManager::run(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1732:0
#24 0x0000000003f8b5d1 (anonymous namespace)::EmitAssemblyHelper::EmitAssembly(clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/BackendUtil.cpp:801:0
#25 0x0000000003f8df15 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::DataLayout const&, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/BackendUtil.cpp:1180:0
#26 0x0000000004aec157 clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/CodeGenAction.cpp:292:0
#27 0x00000000057fec75 clang::ParseAST(clang::Sema&, bool, bool) /home/dwmw2/git/llvm/tools/clang/lib/Parse/ParseAST.cpp:161:0
#28 0x00000000045f2b13 clang::ASTFrontendAction::ExecuteAction() /home/dwmw2/git/llvm/tools/clang/lib/Frontend/FrontendAction.cpp:1005:0
#29 0x0000000004aea206 clang::CodeGenAction::ExecuteAction() /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/CodeGenAction.cpp:1032:0
#30 0x00000000045f256d clang::FrontendAction::Execute() /home/dwmw2/git/llvm/tools/clang/lib/Frontend/FrontendAction.cpp:908:0
#31 0x000000000459028f clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) /home/dwmw2/git/llvm/tools/clang/lib/Frontend/CompilerInstance.cpp:992:0
#32 0x0000000004734158 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) /home/dwmw2/git/llvm/tools/clang/lib/FrontendTool/ExecuteCompilerInvocation.cpp:252:0
#33 0x0000000001c119a2 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) /home/dwmw2/git/llvm/tools/clang/tools/driver/cc1_main.cpp:221:0
#34 0x0000000001c06f5d ExecuteCC1Tool(llvm::ArrayRef<char const*>, llvm::StringRef) /home/dwmw2/git/llvm/tools/clang/tools/driver/driver.cpp:309:0
#35 0x0000000001c07b94 main /home/dwmw2/git/llvm/tools/clang/tools/driver/driver.cpp:389:0
#36 0x00007f8d690a500a __libc_start_main (/lib64/libc.so.6+0x2100a)
#37 0x0000000001c0480a _start (/home/dwmw2/git/llvm-build/bin/clang-7.0+0x1c0480a)
Stack dump:
0. Program arguments: /home/dwmw2/git/llvm-build/bin/clang-7.0 -cc1 -triple x86_64-unknown-linux-gnu -S -disable-free -main-file-name menu.c -mrelocation-model static -mthread-model posix -mllvm -warn-stack-size=2048 -mdisable-fp-elim -relaxed-aliasing -fmath-errno -masm-verbose -no-integrated-as -mconstructor-aliases -fuse-init-array -mcode-model kernel -target-cpu x86-64 -target-feature -sse -target-feature -mmx -target-feature -sse2 -target-feature -3dnow -target-feature -avx -target-feature -x87 -target-feature +retpoline-external-thunk -disable-red-zone -dwarf-column-info -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -coverage-notes-file /home/dwmw2/git/linux-2.6/drivers/cpuidle/governors/menu.gcno -nostdsysteminc -nobuiltininc -resource-dir /home/dwmw2/git/llvm-build/lib/clang/7.0.0 -dependency-file drivers/cpuidle/governors/.menu.o.d -MT drivers/cpuidle/governors/menu.o -sys-header-deps -isystem /home/dwmw2/git/llvm-build/lib/clang/7.0.0/include -include ./include/linux/kconfig.h -I ./arch/x86/include -I ./arch/x86/include/generated -I ./include -I ./arch/x86/include/uapi -I ./arch/x86/include/generated/uapi -I ./include/uapi -I ./include/generated/uapi -D __KERNEL__ -D CONFIG_AS_CFI=1 -D CONFIG_AS_CFI_SIGNAL_FRAME=1 -D CONFIG_AS_CFI_SECTIONS=1 -D CONFIG_AS_FXSAVEQ=1 -D CONFIG_AS_SSSE3=1 -D CONFIG_AS_CRC32=1 -D CONFIG_AS_AVX=1 -D CONFIG_AS_AVX2=1 -D CONFIG_AS_AVX512=1 -D CONFIG_AS_SHA1_NI=1 -D CONFIG_AS_SHA256_NI=1 -D RETPOLINE -D CC_USING_FENTRY -D KBUILD_BASENAME="menu" -D KBUILD_MODNAME="menu" -O2 -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -Werror-implicit-function-declaration -Wno-format-security -Wno-sign-compare -Wno-unused-variable -Wno-format-invalid-specifier -Wno-gnu -Wno-address-of-packed-member -Wno-tautological-compare -Wdeclaration-after-statement -Wno-pointer-sign -Werror=implicit-int -Werror=strict-prototypes -Werror=date-time -Werror=incompatible-pointer-types -Wno-initializer-overrides -Wno-unused-value -Wno-format -Wno-sign-compare -Wno-format-zero-length -Wno-uninitialized -std=gnu89 -fno-dwarf-directory-asm -fdebug-compilation-dir /home/dwmw2/git/linux-2.6 -ferror-limit 19 -fmessage-length 80 -pg -mfentry -fwrapv -stack-protector 2 -mstack-alignment=8 -fwchar-type=short -fno-signed-wchar -fobjc-runtime=gcc -fno-common -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -o /tmp/menu-07e9d7.s -x c drivers/cpuidle/governors/menu.c 
1. <eof> parser at end of file
2. Per-module optimization passes
3. Running pass 'CallGraph Pass Manager' on module 'drivers/cpuidle/governors/menu.c'.
4. Running pass 'Loop Pass Manager' on function '@menu_select'
5. Running pass 'Induction Variable Simplification' on basic block '%for.body'
clang-7.0: [0;1;31merror: [0munable to execute command: Aborted (core dumped) [0m
clang-7.0: [0;1;31merror: [0mclang frontend command failed due to signal (use -v to see invocation) [0m
clang version 7.0.0  (llvm/trunk 324455)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/dwmw2/git/llvm-build/bin
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace, preprocessed source, and associated run script. [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: 
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at: [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: /tmp/menu-af3103.c [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: /tmp/menu-af3103.sh [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: 

******************** [0m
make[3]: *** [scripts/Makefile.build:317: drivers/cpuidle/governors/menu.o] Error 254
make[3]: Target '__build' not remade because of errors.
make[2]: *** [scripts/Makefile.build:575: drivers/cpuidle/governors] Error 2
make[2]: Target '__build' not remade because of errors.
make[1]: *** [scripts/Makefile.build:575: drivers/cpuidle] Error 2
  CC      drivers/pci/iov.o
clang-7.0: /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:9092: bool llvm::ScalarEvolution::isLoopEntryGuardedByCond(const llvm::Loop*, llvm::CmpInst::Predicate, const llvm::SCEV*, const llvm::SCEV*): Assertion `(!ProvedNonStrictComparison || !ProvedNonEquality) && "Why we were unable to prove (Pred, LHS, RHS) via constant ranges?"' failed.
#0 0x0000000003cdcc0e llvm::sys::PrintStackTrace(llvm::raw_ostream&) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:398:0
#1 0x0000000003cdcca1 PrintStackTraceSignalHandler(void*) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:462:0
#2 0x0000000003cdb146 llvm::sys::RunSignalHandlers() /home/dwmw2/git/llvm/lib/Support/Signals.cpp:49:0
#3 0x0000000003cdc583 SignalHandler(int) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:252:0
#4 0x00007f05e113aa80 __restore_rt (/lib64/libpthread.so.0+0x12a80)
#5 0x00007f05dfc3b66b __GI_raise (/lib64/libc.so.6+0x3766b)
#6 0x00007f05dfc3d381 __GI_abort (/lib64/libc.so.6+0x39381)
#7 0x00007f05dfc338fa __assert_fail_base (/lib64/libc.so.6+0x2f8fa)
#8 0x00007f05dfc33972 (/lib64/libc.so.6+0x2f972)
#9 0x0000000002f60e6b llvm::ScalarEvolution::isLoopEntryGuardedByCond(llvm::Loop const*, llvm::CmpInst::Predicate, llvm::SCEV const*, llvm::SCEV const*) /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:9110:0
#10 0x0000000002f3a1f8 llvm::ScalarEvolution::getSignExtendExpr(llvm::SCEV const*, llvm::Type*, unsigned int) /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:1991:0
#11 0x0000000003dd8d75 (anonymous namespace)::SimplifyIndvar::strengthenOverflowingOperation(llvm::BinaryOperator*, llvm::Value*) /home/dwmw2/git/llvm/lib/Transforms/Utils/SimplifyIndVar.cpp:649:0
#12 0x0000000003dd9578 (anonymous namespace)::SimplifyIndvar::simplifyUsers(llvm::PHINode*, llvm::IVVisitor*) /home/dwmw2/git/llvm/lib/Transforms/Utils/SimplifyIndVar.cpp:800:0
#13 0x0000000003dd9757 llvm::simplifyUsersOfIV(llvm::PHINode*, llvm::ScalarEvolution*, llvm::DominatorTree*, llvm::LoopInfo*, llvm::SmallVectorImpl<llvm::WeakTrackingVH>&, llvm::SCEVExpander&, llvm::IVVisitor*) /home/dwmw2/git/llvm/lib/Transforms/Utils/SimplifyIndVar.cpp:832:0
#14 0x0000000003a4235a (anonymous namespace)::IndVarSimplify::simplifyAndExtend(llvm::Loop*, llvm::SCEVExpander&, llvm::LoopInfo*) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:1812:0
#15 0x0000000003a447db (anonymous namespace)::IndVarSimplify::run(llvm::Loop*) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:2461:0
#16 0x0000000003a44f64 (anonymous namespace)::IndVarSimplifyLegacyPass::runOnLoop(llvm::Loop*, llvm::LPPassManager&) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:2576:0
#17 0x0000000002ed44ff llvm::LPPassManager::runOnFunction(llvm::Function&) /home/dwmw2/git/llvm/lib/Analysis/LoopPass.cpp:204:0
#18 0x00000000035a1af1 llvm::FPPassManager::runOnFunction(llvm::Function&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1520:0
#19 0x0000000002e0311c (anonymous namespace)::CGPassManager::RunPassOnSCC(llvm::Pass*, llvm::CallGraphSCC&, llvm::CallGraph&, bool&, bool&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:156:0
#20 0x0000000002e040a4 (anonymous namespace)::CGPassManager::RunAllPassesOnSCC(llvm::CallGraphSCC&, llvm::CallGraph&, bool&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:423:0
#21 0x0000000002e043a5 (anonymous namespace)::CGPassManager::runOnModule(llvm::Module&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:479:0
#22 0x00000000035a1fc3 (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1597:0
#23 0x00000000035a2693 llvm::legacy::PassManagerImpl::run(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1700:0
#24 0x00000000035a288b llvm::legacy::PassManager::run(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1732:0
#25 0x0000000003f8b5d1 (anonymous namespace)::EmitAssemblyHelper::EmitAssembly(clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/BackendUtil.cpp:801:0
#26 0x0000000003f8df15 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::DataLayout const&, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/BackendUtil.cpp:1180:0
#27 0x0000000004aec157 clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/CodeGenAction.cpp:292:0
#28 0x00000000057fec75 clang::ParseAST(clang::Sema&, bool, bool) /home/dwmw2/git/llvm/tools/clang/lib/Parse/ParseAST.cpp:161:0
#29 0x00000000045f2b13 clang::ASTFrontendAction::ExecuteAction() /home/dwmw2/git/llvm/tools/clang/lib/Frontend/FrontendAction.cpp:1005:0
#30 0x0000000004aea206 clang::CodeGenAction::ExecuteAction() /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/CodeGenAction.cpp:1032:0
#31 0x00000000045f256d clang::FrontendAction::Execute() /home/dwmw2/git/llvm/tools/clang/lib/Frontend/FrontendAction.cpp:908:0
#32 0x000000000459028f clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) /home/dwmw2/git/llvm/tools/clang/lib/Frontend/CompilerInstance.cpp:992:0
#33 0x0000000004734158 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) /home/dwmw2/git/llvm/tools/clang/lib/FrontendTool/ExecuteCompilerInvocation.cpp:252:0
#34 0x0000000001c119a2 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) /home/dwmw2/git/llvm/tools/clang/tools/driver/cc1_main.cpp:221:0
#35 0x0000000001c06f5d ExecuteCC1Tool(llvm::ArrayRef<char const*>, llvm::StringRef) /home/dwmw2/git/llvm/tools/clang/tools/driver/driver.cpp:309:0
#36 0x0000000001c07b94 main /home/dwmw2/git/llvm/tools/clang/tools/driver/driver.cpp:389:0
#37 0x00007f05dfc2500a __libc_start_main (/lib64/libc.so.6+0x2100a)
#38 0x0000000001c0480a _start (/home/dwmw2/git/llvm-build/bin/clang-7.0+0x1c0480a)
Stack dump:
0. Program arguments: /home/dwmw2/git/llvm-build/bin/clang-7.0 -cc1 -triple x86_64-unknown-linux-gnu -S -disable-free -main-file-name iov.c -mrelocation-model static -mthread-model posix -mllvm -warn-stack-size=2048 -mdisable-fp-elim -relaxed-aliasing -fmath-errno -masm-verbose -no-integrated-as -mconstructor-aliases -fuse-init-array -mcode-model kernel -target-cpu x86-64 -target-feature -sse -target-feature -mmx -target-feature -sse2 -target-feature -3dnow -target-feature -avx -target-feature -x87 -target-feature +retpoline-external-thunk -disable-red-zone -dwarf-column-info -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -coverage-notes-file /home/dwmw2/git/linux-2.6/drivers/pci/iov.gcno -nostdsysteminc -nobuiltininc -resource-dir /home/dwmw2/git/llvm-build/lib/clang/7.0.0 -dependency-file drivers/pci/.iov.o.d -MT drivers/pci/iov.o -sys-header-deps -isystem /home/dwmw2/git/llvm-build/lib/clang/7.0.0/include -include ./include/linux/kconfig.h -I ./arch/x86/include -I ./arch/x86/include/generated -I ./include -I ./arch/x86/include/uapi -I ./arch/x86/include/generated/uapi -I ./include/uapi -I ./include/generated/uapi -D __KERNEL__ -D CONFIG_AS_CFI=1 -D CONFIG_AS_CFI_SIGNAL_FRAME=1 -D CONFIG_AS_CFI_SECTIONS=1 -D CONFIG_AS_FXSAVEQ=1 -D CONFIG_AS_SSSE3=1 -D CONFIG_AS_CRC32=1 -D CONFIG_AS_AVX=1 -D CONFIG_AS_AVX2=1 -D CONFIG_AS_AVX512=1 -D CONFIG_AS_SHA1_NI=1 -D CONFIG_AS_SHA256_NI=1 -D RETPOLINE -D CC_USING_FENTRY -D KBUILD_BASENAME="iov" -D KBUILD_MODNAME="iov" -O2 -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -Werror-implicit-function-declaration -Wno-format-security -Wno-sign-compare -Wno-unused-variable -Wno-format-invalid-specifier -Wno-gnu -Wno-address-of-packed-member -Wno-tautological-compare -Wdeclaration-after-statement -Wno-pointer-sign -Werror=implicit-int -Werror=strict-prototypes -Werror=date-time -Werror=incompatible-pointer-types -Wno-initializer-overrides -Wno-unused-value -Wno-format -Wno-sign-compare -Wno-format-zero-length -Wno-uninitialized -std=gnu89 -fno-dwarf-directory-asm -fdebug-compilation-dir /home/dwmw2/git/linux-2.6 -ferror-limit 19 -fmessage-length 80 -pg -mfentry -fwrapv -stack-protector 2 -mstack-alignment=8 -fwchar-type=short -fno-signed-wchar -fobjc-runtime=gcc -fno-common -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -o /tmp/iov-eee364.s -x c drivers/pci/iov.c 
1. <eof> parser at end of file
2. Per-module optimization passes
3. Running pass 'CallGraph Pass Manager' on module 'drivers/pci/iov.c'.
4. Running pass 'Loop Pass Manager' on function '@pci_enable_sriov'
5. Running pass 'Induction Variable Simplification' on basic block '%while.body.i'
clang-7.0: [0;1;31merror: [0munable to execute command: Aborted (core dumped) [0m
clang-7.0: [0;1;31merror: [0mclang frontend command failed due to signal (use -v to see invocation) [0m
clang version 7.0.0  (llvm/trunk 324455)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/dwmw2/git/llvm-build/bin
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace, preprocessed source, and associated run script. [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: 
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at: [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: /tmp/iov-d7800f.c [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: /tmp/iov-d7800f.sh [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: 

******************** [0m
make[2]: *** [scripts/Makefile.build:317: drivers/pci/iov.o] Error 254
make[2]: Target '__build' not remade because of errors.
make[1]: *** [scripts/Makefile.build:575: drivers/pci] Error 2
  CC      drivers/usb/host/xhci.o
clang-7.0: /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:9092: bool llvm::ScalarEvolution::isLoopEntryGuardedByCond(const llvm::Loop*, llvm::CmpInst::Predicate, const llvm::SCEV*, const llvm::SCEV*): Assertion `(!ProvedNonStrictComparison || !ProvedNonEquality) && "Why we were unable to prove (Pred, LHS, RHS) via constant ranges?"' failed.
#0 0x0000000003cdcc0e llvm::sys::PrintStackTrace(llvm::raw_ostream&) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:398:0
#1 0x0000000003cdcca1 PrintStackTraceSignalHandler(void*) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:462:0
#2 0x0000000003cdb146 llvm::sys::RunSignalHandlers() /home/dwmw2/git/llvm/lib/Support/Signals.cpp:49:0
#3 0x0000000003cdc583 SignalHandler(int) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:252:0
#4 0x00007f7b05b99a80 __restore_rt (/lib64/libpthread.so.0+0x12a80)
#5 0x00007f7b0469a66b __GI_raise (/lib64/libc.so.6+0x3766b)
#6 0x00007f7b0469c381 __GI_abort (/lib64/libc.so.6+0x39381)
#7 0x00007f7b046928fa __assert_fail_base (/lib64/libc.so.6+0x2f8fa)
#8 0x00007f7b04692972 (/lib64/libc.so.6+0x2f972)
#9 0x0000000002f60e6b llvm::ScalarEvolution::isLoopEntryGuardedByCond(llvm::Loop const*, llvm::CmpInst::Predicate, llvm::SCEV const*, llvm::SCEV const*) /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:9110:0
#10 0x0000000002f3a1f8 llvm::ScalarEvolution::getSignExtendExpr(llvm::SCEV const*, llvm::Type*, unsigned int) /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:1991:0
#11 0x0000000003dd8df8 (anonymous namespace)::SimplifyIndvar::strengthenOverflowingOperation(llvm::BinaryOperator*, llvm::Value*) /home/dwmw2/git/llvm/lib/Transforms/Utils/SimplifyIndVar.cpp:652:0
#12 0x0000000003dd9578 (anonymous namespace)::SimplifyIndvar::simplifyUsers(llvm::PHINode*, llvm::IVVisitor*) /home/dwmw2/git/llvm/lib/Transforms/Utils/SimplifyIndVar.cpp:800:0
#13 0x0000000003dd9757 llvm::simplifyUsersOfIV(llvm::PHINode*, llvm::ScalarEvolution*, llvm::DominatorTree*, llvm::LoopInfo*, llvm::SmallVectorImpl<llvm::WeakTrackingVH>&, llvm::SCEVExpander&, llvm::IVVisitor*) /home/dwmw2/git/llvm/lib/Transforms/Utils/SimplifyIndVar.cpp:832:0
#14 0x0000000003a4235a (anonymous namespace)::IndVarSimplify::simplifyAndExtend(llvm::Loop*, llvm::SCEVExpander&, llvm::LoopInfo*) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:1812:0
#15 0x0000000003a447db (anonymous namespace)::IndVarSimplify::run(llvm::Loop*) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:2461:0
#16 0x0000000003a44f64 (anonymous namespace)::IndVarSimplifyLegacyPass::runOnLoop(llvm::Loop*, llvm::LPPassManager&) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:2576:0
#17 0x0000000002ed44ff llvm::LPPassManager::runOnFunction(llvm::Function&) /home/dwmw2/git/llvm/lib/Analysis/LoopPass.cpp:204:0
#18 0x00000000035a1af1 llvm::FPPassManager::runOnFunction(llvm::Function&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1520:0
#19 0x0000000002e0311c (anonymous namespace)::CGPassManager::RunPassOnSCC(llvm::Pass*, llvm::CallGraphSCC&, llvm::CallGraph&, bool&, bool&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:156:0
#20 0x0000000002e040a4 (anonymous namespace)::CGPassManager::RunAllPassesOnSCC(llvm::CallGraphSCC&, llvm::CallGraph&, bool&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:423:0
#21 0x0000000002e043a5 (anonymous namespace)::CGPassManager::runOnModule(llvm::Module&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:479:0
#22 0x00000000035a1fc3 (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1597:0
#23 0x00000000035a2693 llvm::legacy::PassManagerImpl::run(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1700:0
#24 0x00000000035a288b llvm::legacy::PassManager::run(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1732:0
#25 0x0000000003f8b5d1 (anonymous namespace)::EmitAssemblyHelper::EmitAssembly(clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/BackendUtil.cpp:801:0
#26 0x0000000003f8df15 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::DataLayout const&, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/BackendUtil.cpp:1180:0
#27 0x0000000004aec157 clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/CodeGenAction.cpp:292:0
#28 0x00000000057fec75 clang::ParseAST(clang::Sema&, bool, bool) /home/dwmw2/git/llvm/tools/clang/lib/Parse/ParseAST.cpp:161:0
#29 0x00000000045f2b13 clang::ASTFrontendAction::ExecuteAction() /home/dwmw2/git/llvm/tools/clang/lib/Frontend/FrontendAction.cpp:1005:0
#30 0x0000000004aea206 clang::CodeGenAction::ExecuteAction() /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/CodeGenAction.cpp:1032:0
#31 0x00000000045f256d clang::FrontendAction::Execute() /home/dwmw2/git/llvm/tools/clang/lib/Frontend/FrontendAction.cpp:908:0
#32 0x000000000459028f clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) /home/dwmw2/git/llvm/tools/clang/lib/Frontend/CompilerInstance.cpp:992:0
#33 0x0000000004734158 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) /home/dwmw2/git/llvm/tools/clang/lib/FrontendTool/ExecuteCompilerInvocation.cpp:252:0
#34 0x0000000001c119a2 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) /home/dwmw2/git/llvm/tools/clang/tools/driver/cc1_main.cpp:221:0
#35 0x0000000001c06f5d ExecuteCC1Tool(llvm::ArrayRef<char const*>, llvm::StringRef) /home/dwmw2/git/llvm/tools/clang/tools/driver/driver.cpp:309:0
#36 0x0000000001c07b94 main /home/dwmw2/git/llvm/tools/clang/tools/driver/driver.cpp:389:0
#37 0x00007f7b0468400a __libc_start_main (/lib64/libc.so.6+0x2100a)
#38 0x0000000001c0480a _start (/home/dwmw2/git/llvm-build/bin/clang-7.0+0x1c0480a)
Stack dump:
0. Program arguments: /home/dwmw2/git/llvm-build/bin/clang-7.0 -cc1 -triple x86_64-unknown-linux-gnu -S -disable-free -main-file-name xhci.c -mrelocation-model static -mthread-model posix -mllvm -warn-stack-size=2048 -mdisable-fp-elim -relaxed-aliasing -fmath-errno -masm-verbose -no-integrated-as -mconstructor-aliases -fuse-init-array -mcode-model kernel -target-cpu x86-64 -target-feature -sse -target-feature -mmx -target-feature -sse2 -target-feature -3dnow -target-feature -avx -target-feature -x87 -target-feature +retpoline-external-thunk -disable-red-zone -dwarf-column-info -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -coverage-notes-file /home/dwmw2/git/linux-2.6/drivers/usb/host/xhci.gcno -nostdsysteminc -nobuiltininc -resource-dir /home/dwmw2/git/llvm-build/lib/clang/7.0.0 -dependency-file drivers/usb/host/.xhci.o.d -MT drivers/usb/host/xhci.o -sys-header-deps -isystem /home/dwmw2/git/llvm-build/lib/clang/7.0.0/include -include ./include/linux/kconfig.h -I ./arch/x86/include -I ./arch/x86/include/generated -I ./include -I ./arch/x86/include/uapi -I ./arch/x86/include/generated/uapi -I ./include/uapi -I ./include/generated/uapi -D __KERNEL__ -D CONFIG_AS_CFI=1 -D CONFIG_AS_CFI_SIGNAL_FRAME=1 -D CONFIG_AS_CFI_SECTIONS=1 -D CONFIG_AS_FXSAVEQ=1 -D CONFIG_AS_SSSE3=1 -D CONFIG_AS_CRC32=1 -D CONFIG_AS_AVX=1 -D CONFIG_AS_AVX2=1 -D CONFIG_AS_AVX512=1 -D CONFIG_AS_SHA1_NI=1 -D CONFIG_AS_SHA256_NI=1 -D RETPOLINE -D CC_USING_FENTRY -D KBUILD_BASENAME="xhci" -D KBUILD_MODNAME="xhci_hcd" -O2 -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -Werror-implicit-function-declaration -Wno-format-security -Wno-sign-compare -Wno-unused-variable -Wno-format-invalid-specifier -Wno-gnu -Wno-address-of-packed-member -Wno-tautological-compare -Wdeclaration-after-statement -Wno-pointer-sign -Werror=implicit-int -Werror=strict-prototypes -Werror=date-time -Werror=incompatible-pointer-types -Wno-initializer-overrides -Wno-unused-value -Wno-format -Wno-sign-compare -Wno-format-zero-length -Wno-uninitialized -std=gnu89 -fno-dwarf-directory-asm -fdebug-compilation-dir /home/dwmw2/git/linux-2.6 -ferror-limit 19 -fmessage-length 80 -pg -mfentry -fwrapv -stack-protector 2 -mstack-alignment=8 -fwchar-type=short -fno-signed-wchar -fobjc-runtime=gcc -fno-common -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -o /tmp/xhci-514a71.s -x c drivers/usb/host/xhci.c 
1. <eof> parser at end of file
2. Per-module optimization passes
3. Running pass 'CallGraph Pass Manager' on module 'drivers/usb/host/xhci.c'.
4. Running pass 'Loop Pass Manager' on function '@xhci_try_enable_msi'
5. Running pass 'Induction Variable Simplification' on basic block '%while.body.i'
clang-7.0: [0;1;31merror: [0munable to execute command: Aborted (core dumped) [0m
clang-7.0: [0;1;31merror: [0mclang frontend command failed due to signal (use -v to see invocation) [0m
clang version 7.0.0  (llvm/trunk 324455)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/dwmw2/git/llvm-build/bin
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace, preprocessed source, and associated run script. [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: 
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at: [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: /tmp/xhci-4189be.c [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: /tmp/xhci-4189be.sh [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: 

******************** [0m
make[3]: *** [scripts/Makefile.build:317: drivers/usb/host/xhci.o] Error 254
make[3]: Target '__build' not remade because of errors.
make[2]: *** [scripts/Makefile.build:575: drivers/usb/host] Error 2
  CC      drivers/usb/mon/mon_bin.o
clang-7.0: /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:9092: bool llvm::ScalarEvolution::isLoopEntryGuardedByCond(const llvm::Loop*, llvm::CmpInst::Predicate, const llvm::SCEV*, const llvm::SCEV*): Assertion `(!ProvedNonStrictComparison || !ProvedNonEquality) && "Why we were unable to prove (Pred, LHS, RHS) via constant ranges?"' failed.
#0 0x0000000003cdcc0e llvm::sys::PrintStackTrace(llvm::raw_ostream&) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:398:0
#1 0x0000000003cdcca1 PrintStackTraceSignalHandler(void*) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:462:0
#2 0x0000000003cdb146 llvm::sys::RunSignalHandlers() /home/dwmw2/git/llvm/lib/Support/Signals.cpp:49:0
#3 0x0000000003cdc583 SignalHandler(int) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:252:0
#4 0x00007f1117b0fa80 __restore_rt (/lib64/libpthread.so.0+0x12a80)
#5 0x00007f111661066b __GI_raise (/lib64/libc.so.6+0x3766b)
#6 0x00007f1116612381 __GI_abort (/lib64/libc.so.6+0x39381)
#7 0x00007f11166088fa __assert_fail_base (/lib64/libc.so.6+0x2f8fa)
#8 0x00007f1116608972 (/lib64/libc.so.6+0x2f972)
#9 0x0000000002f60e6b llvm::ScalarEvolution::isLoopEntryGuardedByCond(llvm::Loop const*, llvm::CmpInst::Predicate, llvm::SCEV const*, llvm::SCEV const*) /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:9110:0
#10 0x0000000002f3a1f8 llvm::ScalarEvolution::getSignExtendExpr(llvm::SCEV const*, llvm::Type*, unsigned int) /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:1991:0
#11 0x0000000003dd8d75 (anonymous namespace)::SimplifyIndvar::strengthenOverflowingOperation(llvm::BinaryOperator*, llvm::Value*) /home/dwmw2/git/llvm/lib/Transforms/Utils/SimplifyIndVar.cpp:649:0
#12 0x0000000003dd9578 (anonymous namespace)::SimplifyIndvar::simplifyUsers(llvm::PHINode*, llvm::IVVisitor*) /home/dwmw2/git/llvm/lib/Transforms/Utils/SimplifyIndVar.cpp:800:0
#13 0x0000000003dd9757 llvm::simplifyUsersOfIV(llvm::PHINode*, llvm::ScalarEvolution*, llvm::DominatorTree*, llvm::LoopInfo*, llvm::SmallVectorImpl<llvm::WeakTrackingVH>&, llvm::SCEVExpander&, llvm::IVVisitor*) /home/dwmw2/git/llvm/lib/Transforms/Utils/SimplifyIndVar.cpp:832:0
#14 0x0000000003a4235a (anonymous namespace)::IndVarSimplify::simplifyAndExtend(llvm::Loop*, llvm::SCEVExpander&, llvm::LoopInfo*) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:1812:0
#15 0x0000000003a447db (anonymous namespace)::IndVarSimplify::run(llvm::Loop*) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:2461:0
#16 0x0000000003a44f64 (anonymous namespace)::IndVarSimplifyLegacyPass::runOnLoop(llvm::Loop*, llvm::LPPassManager&) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:2576:0
#17 0x0000000002ed44ff llvm::LPPassManager::runOnFunction(llvm::Function&) /home/dwmw2/git/llvm/lib/Analysis/LoopPass.cpp:204:0
#18 0x00000000035a1af1 llvm::FPPassManager::runOnFunction(llvm::Function&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1520:0
#19 0x0000000002e0311c (anonymous namespace)::CGPassManager::RunPassOnSCC(llvm::Pass*, llvm::CallGraphSCC&, llvm::CallGraph&, bool&, bool&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:156:0
#20 0x0000000002e040a4 (anonymous namespace)::CGPassManager::RunAllPassesOnSCC(llvm::CallGraphSCC&, llvm::CallGraph&, bool&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:423:0
#21 0x0000000002e043a5 (anonymous namespace)::CGPassManager::runOnModule(llvm::Module&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:479:0
#22 0x00000000035a1fc3 (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1597:0
#23 0x00000000035a2693 llvm::legacy::PassManagerImpl::run(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1700:0
#24 0x00000000035a288b llvm::legacy::PassManager::run(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1732:0
#25 0x0000000003f8b5d1 (anonymous namespace)::EmitAssemblyHelper::EmitAssembly(clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/BackendUtil.cpp:801:0
#26 0x0000000003f8df15 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::DataLayout const&, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/BackendUtil.cpp:1180:0
#27 0x0000000004aec157 clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/CodeGenAction.cpp:292:0
#28 0x00000000057fec75 clang::ParseAST(clang::Sema&, bool, bool) /home/dwmw2/git/llvm/tools/clang/lib/Parse/ParseAST.cpp:161:0
#29 0x00000000045f2b13 clang::ASTFrontendAction::ExecuteAction() /home/dwmw2/git/llvm/tools/clang/lib/Frontend/FrontendAction.cpp:1005:0
#30 0x0000000004aea206 clang::CodeGenAction::ExecuteAction() /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/CodeGenAction.cpp:1032:0
#31 0x00000000045f256d clang::FrontendAction::Execute() /home/dwmw2/git/llvm/tools/clang/lib/Frontend/FrontendAction.cpp:908:0
#32 0x000000000459028f clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) /home/dwmw2/git/llvm/tools/clang/lib/Frontend/CompilerInstance.cpp:992:0
#33 0x0000000004734158 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) /home/dwmw2/git/llvm/tools/clang/lib/FrontendTool/ExecuteCompilerInvocation.cpp:252:0
#34 0x0000000001c119a2 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) /home/dwmw2/git/llvm/tools/clang/tools/driver/cc1_main.cpp:221:0
#35 0x0000000001c06f5d ExecuteCC1Tool(llvm::ArrayRef<char const*>, llvm::StringRef) /home/dwmw2/git/llvm/tools/clang/tools/driver/driver.cpp:309:0
#36 0x0000000001c07b94 main /home/dwmw2/git/llvm/tools/clang/tools/driver/driver.cpp:389:0
#37 0x00007f11165fa00a __libc_start_main (/lib64/libc.so.6+0x2100a)
#38 0x0000000001c0480a _start (/home/dwmw2/git/llvm-build/bin/clang-7.0+0x1c0480a)
Stack dump:
0. Program arguments: /home/dwmw2/git/llvm-build/bin/clang-7.0 -cc1 -triple x86_64-unknown-linux-gnu -S -disable-free -main-file-name mon_bin.c -mrelocation-model static -mthread-model posix -mllvm -warn-stack-size=2048 -mdisable-fp-elim -relaxed-aliasing -fmath-errno -masm-verbose -no-integrated-as -mconstructor-aliases -fuse-init-array -mcode-model kernel -target-cpu x86-64 -target-feature -sse -target-feature -mmx -target-feature -sse2 -target-feature -3dnow -target-feature -avx -target-feature -x87 -target-feature +retpoline-external-thunk -disable-red-zone -dwarf-column-info -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -coverage-notes-file /home/dwmw2/git/linux-2.6/drivers/usb/mon/mon_bin.gcno -nostdsysteminc -nobuiltininc -resource-dir /home/dwmw2/git/llvm-build/lib/clang/7.0.0 -dependency-file drivers/usb/mon/.mon_bin.o.d -MT drivers/usb/mon/mon_bin.o -sys-header-deps -isystem /home/dwmw2/git/llvm-build/lib/clang/7.0.0/include -include ./include/linux/kconfig.h -I ./arch/x86/include -I ./arch/x86/include/generated -I ./include -I ./arch/x86/include/uapi -I ./arch/x86/include/generated/uapi -I ./include/uapi -I ./include/generated/uapi -D __KERNEL__ -D CONFIG_AS_CFI=1 -D CONFIG_AS_CFI_SIGNAL_FRAME=1 -D CONFIG_AS_CFI_SECTIONS=1 -D CONFIG_AS_FXSAVEQ=1 -D CONFIG_AS_SSSE3=1 -D CONFIG_AS_CRC32=1 -D CONFIG_AS_AVX=1 -D CONFIG_AS_AVX2=1 -D CONFIG_AS_AVX512=1 -D CONFIG_AS_SHA1_NI=1 -D CONFIG_AS_SHA256_NI=1 -D RETPOLINE -D CC_USING_FENTRY -D KBUILD_BASENAME="mon_bin" -D KBUILD_MODNAME="usbmon" -O2 -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -Werror-implicit-function-declaration -Wno-format-security -Wno-sign-compare -Wno-unused-variable -Wno-format-invalid-specifier -Wno-gnu -Wno-address-of-packed-member -Wno-tautological-compare -Wdeclaration-after-statement -Wno-pointer-sign -Werror=implicit-int -Werror=strict-prototypes -Werror=date-time -Werror=incompatible-pointer-types -Wno-initializer-overrides -Wno-unused-value -Wno-format -Wno-sign-compare -Wno-format-zero-length -Wno-uninitialized -std=gnu89 -fno-dwarf-directory-asm -fdebug-compilation-dir /home/dwmw2/git/linux-2.6 -ferror-limit 19 -fmessage-length 80 -pg -mfentry -fwrapv -stack-protector 2 -mstack-alignment=8 -fwchar-type=short -fno-signed-wchar -fobjc-runtime=gcc -fno-common -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -o /tmp/mon_bin-8f9212.s -x c drivers/usb/mon/mon_bin.c 
1. <eof> parser at end of file
2. Per-module optimization passes
3. Running pass 'CallGraph Pass Manager' on module 'drivers/usb/mon/mon_bin.c'.
4. Running pass 'Loop Pass Manager' on function '@mon_bin_event'
5. Running pass 'Induction Variable Simplification' on basic block '%while.body.i'
clang-7.0: [0;1;31merror: [0munable to execute command: Aborted (core dumped) [0m
clang-7.0: [0;1;31merror: [0mclang frontend command failed due to signal (use -v to see invocation) [0m
clang version 7.0.0  (llvm/trunk 324455)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/dwmw2/git/llvm-build/bin
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace, preprocessed source, and associated run script. [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: 
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at: [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: /tmp/mon_bin-d51116.c [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: /tmp/mon_bin-d51116.sh [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: 

******************** [0m
make[3]: *** [scripts/Makefile.build:317: drivers/usb/mon/mon_bin.o] Error 254
make[3]: Target '__build' not remade because of errors.
make[2]: *** [scripts/Makefile.build:575: drivers/usb/mon] Error 2
make[2]: Target '__build' not remade because of errors.
make[1]: *** [scripts/Makefile.build:575: drivers/usb] Error 2
make[1]: Target '__build' not remade because of errors.
make: *** [Makefile:1018: drivers] Error 2
  CC      net/unix/af_unix.o
clang-7.0: /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:9092: bool llvm::ScalarEvolution::isLoopEntryGuardedByCond(const llvm::Loop*, llvm::CmpInst::Predicate, const llvm::SCEV*, const llvm::SCEV*): Assertion `(!ProvedNonStrictComparison || !ProvedNonEquality) && "Why we were unable to prove (Pred, LHS, RHS) via constant ranges?"' failed.
#0 0x0000000003cdcc0e llvm::sys::PrintStackTrace(llvm::raw_ostream&) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:398:0
#1 0x0000000003cdcca1 PrintStackTraceSignalHandler(void*) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:462:0
#2 0x0000000003cdb146 llvm::sys::RunSignalHandlers() /home/dwmw2/git/llvm/lib/Support/Signals.cpp:49:0
#3 0x0000000003cdc583 SignalHandler(int) /home/dwmw2/git/llvm/lib/Support/Unix/Signals.inc:252:0
#4 0x00007fc4845a7a80 __restore_rt (/lib64/libpthread.so.0+0x12a80)
#5 0x00007fc4830a866b __GI_raise (/lib64/libc.so.6+0x3766b)
#6 0x00007fc4830aa381 __GI_abort (/lib64/libc.so.6+0x39381)
#7 0x00007fc4830a08fa __assert_fail_base (/lib64/libc.so.6+0x2f8fa)
#8 0x00007fc4830a0972 (/lib64/libc.so.6+0x2f972)
#9 0x0000000002f60e6b llvm::ScalarEvolution::isLoopEntryGuardedByCond(llvm::Loop const*, llvm::CmpInst::Predicate, llvm::SCEV const*, llvm::SCEV const*) /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:9110:0
#10 0x0000000002f5f6d1 llvm::ScalarEvolution::isKnownPredicate(llvm::CmpInst::Predicate, llvm::SCEV const*, llvm::SCEV const*) /home/dwmw2/git/llvm/lib/Analysis/ScalarEvolution.cpp:8672:0
#11 0x0000000003a40edf (anonymous namespace)::WidenIV::pushNarrowIVUsers(llvm::Instruction*, llvm::Instruction*) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:1510:0
#12 0x0000000003a415f4 (anonymous namespace)::WidenIV::createWideIV(llvm::SCEVExpander&) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:1621:0
#13 0x0000000003a42464 (anonymous namespace)::IndVarSimplify::simplifyAndExtend(llvm::Loop*, llvm::SCEVExpander&, llvm::LoopInfo*) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:1821:0
#14 0x0000000003a447db (anonymous namespace)::IndVarSimplify::run(llvm::Loop*) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:2461:0
#15 0x0000000003a44f64 (anonymous namespace)::IndVarSimplifyLegacyPass::runOnLoop(llvm::Loop*, llvm::LPPassManager&) /home/dwmw2/git/llvm/lib/Transforms/Scalar/IndVarSimplify.cpp:2576:0
#16 0x0000000002ed44ff llvm::LPPassManager::runOnFunction(llvm::Function&) /home/dwmw2/git/llvm/lib/Analysis/LoopPass.cpp:204:0
#17 0x00000000035a1af1 llvm::FPPassManager::runOnFunction(llvm::Function&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1520:0
#18 0x0000000002e0311c (anonymous namespace)::CGPassManager::RunPassOnSCC(llvm::Pass*, llvm::CallGraphSCC&, llvm::CallGraph&, bool&, bool&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:156:0
#19 0x0000000002e040a4 (anonymous namespace)::CGPassManager::RunAllPassesOnSCC(llvm::CallGraphSCC&, llvm::CallGraph&, bool&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:423:0
#20 0x0000000002e043a5 (anonymous namespace)::CGPassManager::runOnModule(llvm::Module&) /home/dwmw2/git/llvm/lib/Analysis/CallGraphSCCPass.cpp:479:0
#21 0x00000000035a1fc3 (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1597:0
#22 0x00000000035a2693 llvm::legacy::PassManagerImpl::run(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1700:0
#23 0x00000000035a288b llvm::legacy::PassManager::run(llvm::Module&) /home/dwmw2/git/llvm/lib/IR/LegacyPassManager.cpp:1732:0
#24 0x0000000003f8b5d1 (anonymous namespace)::EmitAssemblyHelper::EmitAssembly(clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/BackendUtil.cpp:801:0
#25 0x0000000003f8df15 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::DataLayout const&, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/BackendUtil.cpp:1180:0
#26 0x0000000004aec157 clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/CodeGenAction.cpp:292:0
#27 0x00000000057fec75 clang::ParseAST(clang::Sema&, bool, bool) /home/dwmw2/git/llvm/tools/clang/lib/Parse/ParseAST.cpp:161:0
#28 0x00000000045f2b13 clang::ASTFrontendAction::ExecuteAction() /home/dwmw2/git/llvm/tools/clang/lib/Frontend/FrontendAction.cpp:1005:0
#29 0x0000000004aea206 clang::CodeGenAction::ExecuteAction() /home/dwmw2/git/llvm/tools/clang/lib/CodeGen/CodeGenAction.cpp:1032:0
#30 0x00000000045f256d clang::FrontendAction::Execute() /home/dwmw2/git/llvm/tools/clang/lib/Frontend/FrontendAction.cpp:908:0
#31 0x000000000459028f clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) /home/dwmw2/git/llvm/tools/clang/lib/Frontend/CompilerInstance.cpp:992:0
#32 0x0000000004734158 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) /home/dwmw2/git/llvm/tools/clang/lib/FrontendTool/ExecuteCompilerInvocation.cpp:252:0
#33 0x0000000001c119a2 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) /home/dwmw2/git/llvm/tools/clang/tools/driver/cc1_main.cpp:221:0
#34 0x0000000001c06f5d ExecuteCC1Tool(llvm::ArrayRef<char const*>, llvm::StringRef) /home/dwmw2/git/llvm/tools/clang/tools/driver/driver.cpp:309:0
#35 0x0000000001c07b94 main /home/dwmw2/git/llvm/tools/clang/tools/driver/driver.cpp:389:0
#36 0x00007fc48309200a __libc_start_main (/lib64/libc.so.6+0x2100a)
#37 0x0000000001c0480a _start (/home/dwmw2/git/llvm-build/bin/clang-7.0+0x1c0480a)
Stack dump:
0. Program arguments: /home/dwmw2/git/llvm-build/bin/clang-7.0 -cc1 -triple x86_64-unknown-linux-gnu -S -disable-free -main-file-name af_unix.c -mrelocation-model static -mthread-model posix -mllvm -warn-stack-size=2048 -mdisable-fp-elim -relaxed-aliasing -fmath-errno -masm-verbose -no-integrated-as -mconstructor-aliases -fuse-init-array -mcode-model kernel -target-cpu x86-64 -target-feature -sse -target-feature -mmx -target-feature -sse2 -target-feature -3dnow -target-feature -avx -target-feature -x87 -target-feature +retpoline-external-thunk -disable-red-zone -dwarf-column-info -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -coverage-notes-file /home/dwmw2/git/linux-2.6/net/unix/af_unix.gcno -nostdsysteminc -nobuiltininc -resource-dir /home/dwmw2/git/llvm-build/lib/clang/7.0.0 -dependency-file net/unix/.af_unix.o.d -MT net/unix/af_unix.o -sys-header-deps -isystem /home/dwmw2/git/llvm-build/lib/clang/7.0.0/include -include ./include/linux/kconfig.h -I ./arch/x86/include -I ./arch/x86/include/generated -I ./include -I ./arch/x86/include/uapi -I ./arch/x86/include/generated/uapi -I ./include/uapi -I ./include/generated/uapi -D __KERNEL__ -D CONFIG_AS_CFI=1 -D CONFIG_AS_CFI_SIGNAL_FRAME=1 -D CONFIG_AS_CFI_SECTIONS=1 -D CONFIG_AS_FXSAVEQ=1 -D CONFIG_AS_SSSE3=1 -D CONFIG_AS_CRC32=1 -D CONFIG_AS_AVX=1 -D CONFIG_AS_AVX2=1 -D CONFIG_AS_AVX512=1 -D CONFIG_AS_SHA1_NI=1 -D CONFIG_AS_SHA256_NI=1 -D RETPOLINE -D CC_USING_FENTRY -D KBUILD_BASENAME="af_unix" -D KBUILD_MODNAME="unix" -O2 -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -Werror-implicit-function-declaration -Wno-format-security -Wno-sign-compare -Wno-unused-variable -Wno-format-invalid-specifier -Wno-gnu -Wno-address-of-packed-member -Wno-tautological-compare -Wdeclaration-after-statement -Wno-pointer-sign -Werror=implicit-int -Werror=strict-prototypes -Werror=date-time -Werror=incompatible-pointer-types -Wno-initializer-overrides -Wno-unused-value -Wno-format -Wno-sign-compare -Wno-format-zero-length -Wno-uninitialized -std=gnu89 -fno-dwarf-directory-asm -fdebug-compilation-dir /home/dwmw2/git/linux-2.6 -ferror-limit 19 -fmessage-length 80 -pg -mfentry -fwrapv -stack-protector 2 -mstack-alignment=8 -fwchar-type=short -fno-signed-wchar -fobjc-runtime=gcc -fno-common -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -o /tmp/af_unix-5c03bf.s -x c net/unix/af_unix.c 
1. <eof> parser at end of file
2. Per-module optimization passes
3. Running pass 'CallGraph Pass Manager' on module 'net/unix/af_unix.c'.
4. Running pass 'Loop Pass Manager' on function '@unix_seq_show'
5. Running pass 'Induction Variable Simplification' on basic block '%for.body'
clang-7.0: [0;1;31merror: [0munable to execute command: Aborted (core dumped) [0m
clang-7.0: [0;1;31merror: [0mclang frontend command failed due to signal (use -v to see invocation) [0m
clang version 7.0.0  (llvm/trunk 324455)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/dwmw2/git/llvm-build/bin
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace, preprocessed source, and associated run script. [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: 
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at: [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: /tmp/af_unix-307390.c [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: /tmp/af_unix-307390.sh [0m
clang-7.0: [0;1;30mnote: [0mdiagnostic msg: 

******************** [0m
make[2]: *** [scripts/Makefile.build:317: net/unix/af_unix.o] Error 254
make[2]: Target '__build' not remade because of errors.
make[1]: *** [scripts/Makefile.build:575: net/unix] Error 2
make[1]: Target '__build' not remade because of errors.
make: *** [Makefile:1018: net] Error 2
make: Target 'vmlinux' not remade because of errors.
]0;dwoodhou@i7:~/git/linux-2.6 [dwoodhou@i7 linux-2.6]$ exit

Script done on 2018-02-07 10:40:16+0000

Ed Maste via llvm-dev

unread,
Feb 7, 2018, 1:15:57 PM2/7/18
to Chandler Carruth, llvm-dev
On 6 February 2018 at 19:56, Chandler Carruth via llvm-dev

<llvm...@lists.llvm.org> wrote:
>
> I'm strongly of the opinion that I think Arjan expressed:
>
> - retpoline alone is probably fine with sufficient RSB stuffing patches in
> the kernel
> - if some folks are worried about the security risk here and running on SKX,
> they should use IBRS.
>
> Given the speed of IBRS on SKX and the complexity & runtime hit of thunking
> ret, I really don't see a good motivation for us teaching the compiler how
> to do this.

As far as Clang-compiled kernels are concerned I suspect FreeBSD has
the biggest footprint, so this is very important to us. I'm still
waiting on feedback about this point (retpoline vs IBRS on SKX), but I
expect we will go down the path that you describe above.

We have yet to implement retpoline changes in kernel asm but will do
so soon, so very much want to have a final implementation.

It also seems that we still have a lone register-indirect call
remaining (coming from a C source file) when everything is compiled
with -mretpoline; I'm looking into it.

Guenter Roeck via llvm-dev

unread,
Feb 7, 2018, 2:48:10 PM2/7/18
to Chandler Carruth, David Woodhouse, Thomas Gleixner, llvm-dev
On 02/06/2018 10:20 PM, Chandler Carruth wrote:
> I've landed the patch in r324449.
>
> Before we merge this into two different Clang release branches and almost immediately release one of them, I would really like someone to confirm that this patch works well with the Linux kernel. David, if you're up for that, it would be great. Alternatively, Guenter or someone else here can help.
>
I'll try to test it today.

Guenter

> On Tue, Feb 6, 2018 at 5:59 PM Chandler Carruth <chan...@google.com <mailto:chan...@google.com>> wrote:
>
> The patch is up for review here: https://reviews.llvm.org/D42998
>

> On Tue, Feb 6, 2018 at 4:58 PM Chandler Carruth <chan...@google.com <mailto:chan...@google.com>> wrote:
>
> Also, could you patch and test Clang with the Linux kernel after I make this change? I'd like to know that we actually successfully call the correct thunks and that they behave correctly. I'm not super worried, but good to  actually get this right. I'm am slightly more worried about the stack-based retpoline than the register ones just due to the overall lower amount of testing we've had there.
>

> On Tue, Feb 6, 2018 at 4:56 PM Chandler Carruth <chan...@google.com <mailto:chan...@google.com>> wrote:

_______________________________________________

Guenter Roeck via llvm-dev

unread,
Feb 7, 2018, 2:48:44 PM2/7/18
to David Woodhouse, llvm-dev, Thomas Gleixner
On Wed, Feb 07, 2018 at 10:49:25AM +0000, David Woodhouse wrote:
> On Wed, 2018-02-07 at 06:20 +0000, Chandler Carruth wrote:
> > I've landed the patch in r324449.
> >
> > Before we merge this into two different Clang release branches and
> > almost immediately release one of them, I would really like someone
> > to confirm that this patch works well with the Linux kernel. David,
> > if you're up for that, it would be great. Alternatively, Guenter or
> > someone else here can help.
>
> Hm, please could we also have the %V asm constraint modifier? That
> allows us to emit calls to the thunks from inline asm using the
> register that the compiler chose for us:
>
> asm volatile ("call __x86_indirect_thunk_%V[thunk_target]" : :
> [thunk_target] "r" (the_function));
>
> Other than that, I get the following errors with LLVM+Clang master, and
> my tree at
> http://git.infradead.org/users/dwmw2/linux-retpoline.git/shortlog/refs/heads/ibpb
>

I tried ToT clang with Linux upstream as well as chromeos-4.14,
with 'defconfig'. I don't see any errors when building x86_64.
Lots and lots of warnings, though.

When trying to build for ARCH=i386, I get

arch/x86/events/intel/../perf_event.h:767:21: error:
invalid output size for constraint '=q'

and a clang crash when compiling drivers/gpu/drm/i915/i915_perf.c.

#0 0x0000000001ee982a llvm::sys::PrintStackTrace(llvm::raw_ostream&)
(/usr/local/google/home/groeck/src/clang/build/bin/clang-7.0+0x1ee982a)
#1 0x0000000001ee7a7e llvm::sys::RunSignalHandlers()
(/usr/local/google/home/groeck/src/clang/build/bin/clang-7.0+0x1ee7a7e)
#2 0x0000000001ee7bba SignalHandler(int)
(/usr/local/google/home/groeck/src/clang/build/bin/clang-7.0+0x1ee7bba)
#3 0x00002b7bcb485330 __restore_rt
(/lib/x86_64-linux-gnu/libpthread.so.0+0x10330)
#4 0x0000000002727b04
llvm::DAGTypeLegalizer::PromoteIntOp_ATOMIC_STORE(llvm::AtomicSDNode*)
(/usr/local/google/home/groeck/src/clang/build/bin/clang-7.0+0x2727b04)

Linux: v4.15-11704-ga2e5790d8416
clang/llvm:
clang version 7.0.0 (https://git.llvm.org/git/clang.git/ 848874aed95a913fb45f363120500cebfe54e2ef) (https://git.llvm.org/git/llvm.git/3afd566557f3616881505db0d69f5d19bf55ae14)

I can disable the i915 driver, but the "invalid output size for
constraint '=q'" happens all over the place. Ultimately this means
that I can not really test a 32-bit build, though it would not build
anyway because it requires the following symbols

U __x86_indirect_thunk_esp
U __x86_indirect_thunk

which are both not available in the upstream kernel.

For x86_64, code generation appears to be as expected.
Next, I'll try to apply the patch on top of the Google toolchain
(based on llvm 6.0) and run it on a real system.

Guenter

David Woodhouse via llvm-dev

unread,
Feb 7, 2018, 3:44:59 PM2/7/18
to Guenter Roeck, llvm-dev, Thomas Gleixner
On Wed, 2018-02-07 at 10:11 -0800, Guenter Roeck wrote:

> On Wed, Feb 07, 2018 at 10:49:25AM +0000, David Woodhouse wrote:
> > Hm, please could we also have the %V asm constraint modifier? That
> > allows us to emit calls to the thunks from inline asm using the
> > register that the compiler chose for us:
> >
> >  asm volatile ("call __x86_indirect_thunk_%V[thunk_target]" : :
> > [thunk_target] "r" (the_function)); 
> >
> > Other than that, I get the following errors with LLVM+Clang master, and
> > my tree at
> > http://git.infradead.org/users/dwmw2/linux-retpoline.git/shortlog/refs/heads/ibpb
> >
> I tried ToT clang with Linux upstream as well as chromeos-4.14,
> with 'defconfig'. I don't see any errors when building x86_64.
> Lots and lots of warnings, though.

The defconfig doesn't build here either; it still includes the cpuidle
bits which don't build. I'll update LLVM/clang and try again, in case
it was fixed between your attempt and mine...


> I can disable the i915 driver, but the "invalid output size for
> constraint '=q'" happens all over the place. Ultimately this means
> that I can not really test a 32-bit build, though it would not build
> anyway because it requires the following symbols
>
> U __x86_indirect_thunk_esp
> U __x86_indirect_thunk

The latter I can live with, as discussed, for 32-bit only. We don't
care about CET compatibility there, so I'm OK to implement the bare
ret-equivalent __x86_indirect_thunk.

The former... wtf? Can you show me the code that actually *calls* that.
I am having difficulty imagining any situation in which that's sane.

David Woodhouse via llvm-dev

unread,
Feb 7, 2018, 4:56:04 PM2/7/18
to Guenter Roeck, llvm-dev, Thomas Gleixner
On Wed, 2018-02-07 at 13:16 -0800, Guenter Roeck wrote:

> Here are my exact versions:
>     llvm: 3afd566557f3 ("AMDGPU: Add 32-bit constant address space")
>     clang: 848874aed95a ("[clang-format] Fix ObjC message arguments formatting.")

OK, mine are slightly newer than that now, but I now get a working 64-
bit defconfig build. It'll still break with any PV guest support
enabled because we need %V asm constraint modifiers.

> PATH=${HOME}/src/clang/build/bin:${PATH}
> make CC=clang defconfig
> make CC=clang -j80
>
> The build is a bit noisy right now due to 66f793099a63 ("x86/retpoline: Avoid
> retpolines for built-in __init functions"), since clang doesn't understand
> "__attribute__((indirect_branch("keep")))".

I fixed that (well, worked around it) in my tree.

> > > I can disable the i915 driver, but the "invalid output size for
> > > constraint '=q'" happens all over the place. Ultimately this means
> > > that I can not really test a 32-bit build, though it would not build
> > > anyway because it requires the following symbols
> > >
> > > U __x86_indirect_thunk_esp
> > > U __x86_indirect_thunk
> >
> > The latter I can live with, as discussed, for 32-bit only. We don't
> > care about CET compatibility there, so I'm OK to implement the bare
> > ret-equivalent __x86_indirect_thunk.
> >
> > The former... wtf? Can you show me the code that actually *calls* that.
> > I am having difficulty imagining any situation in which that's sane.
>
> Turns out this was PBKAC. That was with chromeos-4.14, which does not yet
> include the upstream patch removing __x86_indirect_thunk_esp. I don't
> see it with ToT. The "U" was from ./arch/x86/lib/lib-ksyms.o, meaning it
> was internally generated. Sorry for the noise.

OK... which __x86_indirect_thunk* symbols *are* being used by Clang in
32-bit mode? I've added __x86_indirect_thunk for 32-bit now, and if
that's *all* the Clang is using then I'll possibly switch GCC into that
mode too.

Can you take care of filing the tickets for %V0 and "=q"
and attribute__((indirect_branch("keep"))) please? With those fixed, I
think we should be OK again.

Chandler Carruth via llvm-dev

unread,
Feb 7, 2018, 5:00:20 PM2/7/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
Quick response to a detail, I'll respond to more of this when i have more time.

On Wed, Feb 7, 2018 at 1:55 PM David Woodhouse <dw...@infradead.org> wrote:
OK... which __x86_indirect_thunk* symbols *are* being used by Clang in
32-bit mode?

__x86_indirect_thunk
__x86_indirect_thunk_eax
__x86_indirect_thunk_ecx
__x86_indirect_thunk_edx

David Woodhouse via llvm-dev

unread,
Feb 7, 2018, 5:11:55 PM2/7/18
to Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck
On Wed, 2018-02-07 at 22:00 +0000, Chandler Carruth wrote:
>
> On Wed, Feb 7, 2018 at 1:55 PM David Woodhouse <dw...@infradead.org>
> wrote:
> > OK... which __x86_indirect_thunk* symbols *are* being used by Clang
> > in
> > 32-bit mode?
> __x86_indirect_thunk
> __x86_indirect_thunk_eax
> __x86_indirect_thunk_ecx
> __x86_indirect_thunk_edx

Thanks. And you'll *never* do __x86_indirect_thunk (the ret-equivalent
one) on 64-bit, right?

Chandler Carruth via llvm-dev

unread,
Feb 7, 2018, 5:18:03 PM2/7/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
Correct. 

David Woodhouse via llvm-dev

unread,
Feb 7, 2018, 6:27:02 PM2/7/18
to Guenter Roeck, llvm-dev, Thomas Gleixner
On Wed, 2018-02-07 at 21:55 +0000, David Woodhouse via llvm-dev wrote:
> Can you take care of filing the tickets for %V0 and "=q"
> and attribute__((indirect_branch("keep"))) please? With those fixed, I
> think we should be OK again.

Here's %V0 support, which makes the hypervisor guest support build.

diff --git a/lib/Target/X86/X86AsmPrinter.cpp b/lib/Target/X86/X86AsmPrinter.cpp
index 4da7d59df46..f498c098288 100644
--- a/lib/Target/X86/X86AsmPrinter.cpp
+++ b/lib/Target/X86/X86AsmPrinter.cpp
@@ -370,6 +370,8 @@ static void printIntelMemReference(X86AsmPrinter &P, const MachineInstr *MI,
static bool printAsmMRegister(X86AsmPrinter &P, const MachineOperand &MO,
char Mode, raw_ostream &O) {
unsigned Reg = MO.getReg();
+ bool emit_pct = true;
+
switch (Mode) {
default: return true; // Unknown mode.
case 'b': // Print QImode register
@@ -384,6 +386,9 @@ static bool printAsmMRegister(X86AsmPrinter &P, const MachineOperand &MO,
case 'k': // Print SImode register
Reg = getX86SubSuperRegister(Reg, 32);
break;
+ case 'V':
+ emit_pct = false;
+ /* fall through */
case 'q':
// Print 64-bit register names if 64-bit integer registers are available.
// Otherwise, print 32-bit register names.
@@ -391,7 +396,10 @@ static bool printAsmMRegister(X86AsmPrinter &P, const MachineOperand &MO,
break;
}

- O << '%' << X86ATTInstPrinter::getRegisterName(Reg);
+ if (emit_pct)
+ O << '%';
+
+ O << X86ATTInstPrinter::getRegisterName(Reg);
return false;
}

@@ -464,6 +472,7 @@ bool X86AsmPrinter::PrintAsmOperand(const MachineInstr *MI, unsigned OpNo,
case 'w': // Print HImode register
case 'k': // Print SImode register
case 'q': // Print DImode register
+ case 'V': // Print native register without '%'
if (MO.isReg())
return printAsmMRegister(*this, MO, ExtraCode[0], O);
printOperand(*this, MI, OpNo, O);

--
dwmw2

Chandler Carruth via llvm-dev

unread,
Feb 7, 2018, 6:30:29 PM2/7/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
This should go to llvm-commits as a proper review. Do you want to do that David? Want someone on our end to pick it up?

David Woodhouse via llvm-dev

unread,
Feb 7, 2018, 6:41:06 PM2/7/18
to Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck
On Wed, 2018-02-07 at 23:30 +0000, Chandler Carruth wrote:
> This should go to llvm-commits as a proper review. Do you want to do
> that David? Want someone on our end to pick it up?

I'll attempt to add some test cases...

Craig Topper via llvm-dev

unread,
Feb 7, 2018, 6:48:02 PM2/7/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
I'll save you a review iteration. This is updated to match llvm coding style. Capital camel case variable names and uses LLVM_FALLTHROUGH instead of the comment.

 static bool printAsmMRegister(X86AsmPrinter &P, const MachineOperand &MO,
                               char Mode, raw_ostream &O) {
   unsigned Reg = MO.getReg();
+  bool EmitPercent = true;
+
   switch (Mode) {
   default: return true;  // Unknown mode.
   case 'b': // Print QImode register
@@ -384,6 +386,9 @@ static bool printAsmMRegister(X86AsmPrinter &P, const MachineOperand &MO,
   case 'k': // Print SImode register
     Reg = getX86SubSuperRegister(Reg, 32);
     break;
+  case 'V':
+    EmitPercent = false;
+    LLVM_FALLTHROUGH;
   case 'q':
     // Print 64-bit register names if 64-bit integer registers are available.
     // Otherwise, print 32-bit register names.
@@ -391,7 +396,10 @@ static bool printAsmMRegister(X86AsmPrinter &P, const MachineOperand &MO,
     break;
   }

-  O << '%' << X86ATTInstPrinter::getRegisterName(Reg);
+  if (EmitPercent)
+    O << '%';
+
+  O << X86ATTInstPrinter::getRegisterName(Reg);
   return false;
 }


~Craig

David Woodhouse via llvm-dev

unread,
Feb 7, 2018, 7:07:40 PM2/7/18
to Craig Topper, llvm-dev, Thomas Gleixner, Guenter Roeck
On Wed, 2018-02-07 at 15:47 -0800, Craig Topper wrote:
> I'll save you a review iteration. This is updated to match llvm
> coding style. Capital camel case variable names and uses
> LLVM_FALLTHROUGH instead of the comment.

Thanks. How's this for a test?

; RUN: llc < %s -mtriple=i686-- -no-integrated-as | FileCheck -check-prefix=X86 %s
; RUN: llc < %s -mtriple=x86_64-- -no-integrated-as | FileCheck -check-prefix=X64 %s

; If the target does not have 64-bit integer registers, emit 32-bit register
; names.

; X86: calll __x86_indirect_thunk_e{{[abcd]}}x
; X64: callq __x86_indirect_thunk_r

define void @q_modifier(i32* %p) {
entry:
  tail call void asm sideeffect "call __x86_indirect_thunk_${0:V}", "r,~{dirflag},~{fpsr},~{flags}"(i32* %p)
  ret void
}

Guenter Roeck via llvm-dev

unread,
Feb 7, 2018, 8:00:01 PM2/7/18
to David Woodhouse, llvm-dev, Thomas Gleixner
On Wed, Feb 07, 2018 at 08:44:32PM +0000, David Woodhouse wrote:
> On Wed, 2018-02-07 at 10:11 -0800, Guenter Roeck wrote:
>
> > On Wed, Feb 07, 2018 at 10:49:25AM +0000, David Woodhouse wrote:
> > > Hm, please could we also have the %V asm constraint modifier? That
> > > allows us to emit calls to the thunks from inline asm using the
> > > register that the compiler chose for us:
> > >
> > >  asm volatile ("call __x86_indirect_thunk_%V[thunk_target]" : :
> > > [thunk_target] "r" (the_function)); 
> > >
> > > Other than that, I get the following errors with LLVM+Clang master, and
> > > my tree at
> > > http://git.infradead.org/users/dwmw2/linux-retpoline.git/shortlog/refs/heads/ibpb
> > >
> > I tried ToT clang with Linux upstream as well as chromeos-4.14,
> > with 'defconfig'. I don't see any errors when building x86_64.
> > Lots and lots of warnings, though.
>
> The defconfig doesn't build here either; it still includes the cpuidle
> bits which don't build. I'll update LLVM/clang and try again, in case
> it was fixed between your attempt and mine...
>

Here are my exact versions:


llvm: 3afd566557f3 ("AMDGPU: Add 32-bit constant address space")
clang: 848874aed95a ("[clang-format] Fix ObjC message arguments formatting.")

I used cmake-3.10.2, and the following commands:
mkdir build
cd build
../cmake-3.10.2/bin/cmake -DCMAKE_BUILD_TYPE=Release -G "Unix Makefiles" ../llvm
make -j80

To build the kernel:

PATH=${HOME}/src/clang/build/bin:${PATH}
make CC=clang defconfig
make CC=clang -j80

The build is a bit noisy right now due to 66f793099a63 ("x86/retpoline: Avoid
retpolines for built-in __init functions"), since clang doesn't understand
"__attribute__((indirect_branch("keep")))".

>

> > I can disable the i915 driver, but the "invalid output size for
> > constraint '=q'" happens all over the place. Ultimately this means
> > that I can not really test a 32-bit build, though it would not build
> > anyway because it requires the following symbols
> >
> > U __x86_indirect_thunk_esp
> > U __x86_indirect_thunk
>
> The latter I can live with, as discussed, for 32-bit only. We don't
> care about CET compatibility there, so I'm OK to implement the bare
> ret-equivalent __x86_indirect_thunk.
>
> The former... wtf? Can you show me the code that actually *calls* that.
> I am having difficulty imagining any situation in which that's sane.

Turns out this was PBKAC. That was with chromeos-4.14, which does not yet


include the upstream patch removing __x86_indirect_thunk_esp. I don't
see it with ToT. The "U" was from ./arch/x86/lib/lib-ksyms.o, meaning it
was internally generated. Sorry for the noise.

Guenter

Guenter Roeck via llvm-dev

unread,
Feb 7, 2018, 8:00:31 PM2/7/18
to Chandler Carruth, llvm-dev, Thomas Gleixner
On Wed, Feb 07, 2018 at 06:20:40AM +0000, Chandler Carruth wrote:
> I've landed the patch in r324449.
>
> Before we merge this into two different Clang release branches and almost
> immediately release one of them, I would really like someone to confirm
> that this patch works well with the Linux kernel. David, if you're up for
> that, it would be great. Alternatively, Guenter or someone else here can
> help.
>
It works pretty well for me, except for an endless sequence of

./include/linux/init.h:134:6: warning:
unknown attribute 'indirect_branch' ignored [-Wunknown-attributes]

This is due to

#define __noretpoline __attribute__((indirect_branch("keep")))

which was introduced with upstream commit 66f793099a636 ("x86/retpoline:
Avoid retpolines for built-in __init functions") a couple of days ago.

Guenter

Chandler Carruth via llvm-dev

unread,
Feb 8, 2018, 6:48:53 PM2/8/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
Bringing everything back to this thread -- we now have %V support landed in top-of-tree, so wanted to get confirmation that top-of-tree is healthy for the kernel, or see what else we need to do.

Also, huge thanks to David for jumping on several of the issues and the help from folks building Linux to check them.

David, you had offered to work on the attribute: I'm actually happy to do that if you want to focus on testing things. One question I have: how important is this? Specifically, does the attribute need to get backported? There are two approaches there, one of which matches a code review request anyways, but would be more disruptive to backport all the way to 5.0, so I'd like to get a sense of which approach to take (a somewhat hacky one that will definitely backport or the cleaner solution).

-Chandler

David Woodhouse via llvm-dev

unread,
Feb 8, 2018, 6:59:46 PM2/8/18
to Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck
On Thu, 2018-02-08 at 23:48 +0000, Chandler Carruth wrote:
> Bringing everything back to this thread -- we now have %V support
> landed in top-of-tree, so wanted to get confirmation that top-of-tree
> is healthy for the kernel, or see what else we need to do.

For 64-bit it's fine. For 32-bit we *think* the retpoline bits are OK
but it doesn't build for other reasons on 32-bit. But that isn't new
breakage — 5.0 has the same problem with the latest kernel. I'll see if
we can work around that in the kernel, instead of relying on certain
inline asms getting optimised away before the compiler ever notices
that they're bogus.

> David, you had offered to work on the attribute: I'm actually happy
> to do that if you want to focus on testing things.

That sounds good to me; thanks.

> One question I have: how important is this? Specifically, does the
> attribute need to get backported?

No, we can live without it. It's a slight performance optimisation,
that none of the init code in the kernel itself actually has to be
built with retpolines. But it isn't imperative.

Chandler Carruth via llvm-dev

unread,
Feb 8, 2018, 7:09:32 PM2/8/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
On Thu, Feb 8, 2018 at 3:59 PM David Woodhouse <dw...@infradead.org> wrote:
On Thu, 2018-02-08 at 23:48 +0000, Chandler Carruth wrote:
> Bringing everything back to this thread -- we now have %V support
> landed in top-of-tree, so wanted to get confirmation that top-of-tree
> is healthy for the kernel, or see what else we need to do.

For 64-bit it's fine. For 32-bit we *think* the retpoline bits are OK
but it doesn't build for other reasons on 32-bit. But that isn't new
breakage — 5.0 has the same problem with the latest kernel. I'll see if
we can work around that in the kernel, instead of relying on certain
inline asms getting optimised away before the compiler ever notices
that they're bogus.

Cool, let us know if we can help here, but this at least tells us that we're in pretty good shape to start backporting everything.
 

> David, you had offered to work on the attribute: I'm actually happy
> to do that if you want to focus on testing things.

That sounds good to me; thanks.

>  One question I have: how important is this? Specifically, does the
> attribute need to get backported?

No, we can live without it. It's a slight performance optimisation,
that none of the init code in the kernel itself actually has to be
built with retpolines. But it isn't imperative.

Makes sense, I'll do this as part of the requested cleanup to how we model whether or not to do retpolines.

Thanks,
-Chandler 

David Woodhouse via llvm-dev

unread,
Feb 8, 2018, 7:16:39 PM2/8/18
to Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck


On Fri, 2018-02-09 at 00:09 +0000, Chandler Carruth wrote:
> >
> > For 64-bit it's fine. For 32-bit we *think* the retpoline bits are OK
> > but it doesn't build for other reasons on 32-bit. But that isn't new
> > breakage — 5.0 has the same problem with the latest kernel. I'll see if
> > we can work around that in the kernel, instead of relying on certain
> > inline asms getting optimised away before the compiler ever notices
> > that they're bogus.
>
> Cool, let us know if we can help here, but this at least tells us
> that we're in pretty good shape to start backporting everything.

Test case (build with -m32):


long long foo(void)
{
long long ret;

switch(sizeof(ret)) {
case 1:
asm ("movb $5, %0" : "=q" (ret));
break;

case 2:
asm ("movw $5, %0" : "=r" (ret));
break;

case 4:
asm ("movl $5, %0" : "=r" (ret));
break;

case 8: // this bit was complex, but OK.
ret = 1;
}
return ret;
}

q.c:8:30: error: invalid output size for constraint '=q'
                asm ("movb $5, %0" : "=q" (ret));

Now looking up what the "q" constraint is, why it's being used for the
1-byte version, and why compilation fails for those and not "r" which
blatantly can't take a 'long long' *either* but Clang seems to cope
with that one.

David Woodhouse via llvm-dev

unread,
Feb 8, 2018, 8:19:05 PM2/8/18
to Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck

On Fri, 2018-02-09 at 00:16 +0000, David Woodhouse wrote:
>
> long long foo(void)
> {
>         long long ret;
>
>         switch(sizeof(ret)) {
>         case 1:
>                 asm ("movb $5, %0" : "=q" (ret));
>                 break;
>
>         case 2:
>                 asm ("movw $5, %0" : "=r" (ret));
>                 break;
>
>         case 4:
>                 asm ("movl $5, %0" : "=r" (ret));
>                 break;
>
>         case 8: // this bit was complex, but OK.
>                 ret = 1;
>         }
>         return ret;
> }
>
> q.c:8:30: error: invalid output size for constraint '=q'
>                 asm ("movb $5, %0" : "=q" (ret));
>
> Now looking up what the "q" constraint is, why it's being used for the
> 1-byte version, and why compilation fails for those and not "r" which
> blatantly can't take a 'long long' *either* but Clang seems to cope
> with that one.

It not only "copes" with "=r" but silently emits code where GCC would
(rightly) warn… if GCC ever got to the point of actually *emitting* it:

$ cat q.c


long long foo(void)
{
long long ret;

asm ("movl $5, %0" : "=r" (ret));

return ret;
}
$ clang -c -oq.o q.c -m32
$ gcc -c -oq.o q.c -m32
q.c: In function ‘foo’:
q.c:6:1: warning: unsupported size for integer register
 }
 ^

Would we get away with making "=q" also silently emit wrong code? :)
On the basis that right here and now it wasn't going to emit it
*anyway*; it gets optimised away.

For now I'm just going to attempt to work around it like this in the
kernel, so I can concentrate on the retpoline bits:
 http://david.woodhou.se/clang-percpu-hack.patch

David Woodhouse via llvm-dev

unread,
Feb 8, 2018, 9:21:46 PM2/8/18
to Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck
On Fri, 2018-02-09 at 01:18 +0000, David Woodhouse wrote:
>
> For now I'm just going to attempt to work around it like this in the
> kernel, so I can concentrate on the retpoline bits:
>  http://david.woodhou.se/clang-percpu-hack.patch

32-bit doesn't boot. Built without CONFIG_RETPOLINE and with Clang 5.0
(and the above patch) it does. I'm rebuilding a Release build of
llvm/clang so that experimental kernel builds hopefully take less than
an hour, and will prod further in the morning.

David Woodhouse via llvm-dev

unread,
Feb 9, 2018, 3:27:11 AM2/9/18
to Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck

What is the intended ABI of __x86_indirect_thunk which I have been
calling the "ret-equivalent" retpoline? I see this happening
(I ♥ 'qemu -d in_asm')...

----------------
IN: 
0xc136feea:  89 d8                    movl     %ebx, %eax
0xc136feec:  89 f2                    movl     %esi, %edx
0xc136feee:  8b 75 f0                 movl     -0x10(%ebp), %esi
0xc136fef1:  89 f1                    movl     %esi, %ecx
0xc136fef3:  ff 75 e0                 pushl    -0x20(%ebp)
0xc136fef6:  e8 c5 f3 58 00           calll    0xc18ff2c0 # __x86_indirect_thunk

----------------
IN: 
0xc18ff2c0:  c3                       retl     # Early boot, so it hasn't been turned into a proper retpoline yet

----------------
IN: 
0xc136fefb:  8d 34 7e                 leal     (%esi, %edi, 2), %esi


(gdb) list *0xc136fef6
0xc136fef6 is in sort (lib/sort.c:87).
82 if (c < n - size &&
83 cmp_func(base + c, base + c + size) < 0)
84 c += size;
85 if (cmp_func(base + r, base + c) >= 0)
86 break;
87 swap_func(base + r, base + c, size);
88 }
89 }
90
91 /* sort */

You're pushing the target (-0x20(%ebp)) onto the stack and then
*calling* __x86_indirect_thunk. So it looks like you're expecting
__x86_indirect_thunk to do something like

call *4(%esp)
ret

... except that final 'ret' still leaves the target address on the
stack, so there would also need to be a complicated dance, without
using any registers, to pop that too.

I expected the emitted code for a *call* using the thunk to look more
like

   jmp 2f 
1: pushl -0x20(%ebp)        # cmp_func
   jmp __x86_thunk_indirect # jmp, not call
2: call 1b                  # set up address for cmp_func to return to

Chandler Carruth via llvm-dev

unread,
Feb 9, 2018, 3:46:23 AM2/9/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
Yeah, we expect a complicated dance to re-order the stack to get the correct return address into the correct place.

You can see the sequence in the comments here:
 

I expected the emitted code for a *call* using the thunk to look more
like

   jmp 2f 
1: pushl -0x20(%ebp)        # cmp_func
   jmp __x86_thunk_indirect # jmp, not call
2: call 1b                  # set up address for cmp_func to return to

Yeah, the specific goal was to minimize the code size footprint at the call site even though it means a few more instructions in the thunk. Our pattern also has a minor reduction in the dynamic branches taken at the cost of the push/pop churn.

There was briefly a discussion of a different instruction sequence to minimize push/pop churn but it didn't end up happening.


Anyways, it appears that we have the first case where my suspicions were borne out and we have somewhat reasonably different ABIs for some of the thunks.

How should we name them to distinguish things? 

David Woodhouse via llvm-dev

unread,
Feb 9, 2018, 5:36:34 AM2/9/18
to Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck
On Fri, 2018-02-09 at 08:45 +0000, Chandler Carruth wrote:
> >
> > You're pushing the target (-0x20(%ebp)) onto the stack and then
> > *calling* __x86_indirect_thunk. So it looks like you're expecting
> > __x86_indirect_thunk to do something like
> >
> >   call *4(%esp)
> >   ret
> >
> > ... except that final 'ret' still leaves the target address on the
> > stack, so there would also need to be a complicated dance, without
> > using any registers, to pop that too.
>
> Yeah, we expect a complicated dance to re-order the stack to get the
> correct return address into the correct place.
>
> You can see the sequence in the comments here:
> https://github.com/llvm-project/llvm-project-20170507/blob/master/llvm/lib/Target/X86/X86RetpolineThunks.cpp#L179-L194

Great, thanks.

> Anyways, it appears that we have the first case where my suspicions
> were borne out and we have somewhat reasonably different ABIs for
> some of the thunks.
>
> How should we name them to distinguish things? 

For now it's only ever going to be used by LLVM so you can call it what
you like, although if do want to bring GCC in line to use it too then
perhaps it should be a more functional name and not explicitly mention
llvm? I don't know how weird GCC people would get if you ask them to
implement stuff that has 'llvm' in the name... :)

Did you get anywhere with the function attribute? Having isolated the
next boot failure to "it goes away if I compile io_apic.c without
retpoline", bisecting it per-function would help to further delay the
bit where I actually have to start *thinking*...

David Woodhouse via llvm-dev

unread,
Feb 9, 2018, 7:55:08 AM2/9/18
to Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck
On Fri, 2018-02-09 at 10:36 +0000, David Woodhouse wrote:
>
> Did you get anywhere with the function attribute? Having isolated the
> next boot failure to "it goes away if I compile io_apic.c without
> retpoline", bisecting it per-function would help to further delay the
> bit where I actually have to start *thinking*...

It's mp_register_ioapic(), and only when io_apic_unique_id() gets
inlined, at which point bad things start happening.

[    0.000000] mp_register_ioapic, 0 fec00000 0 c1b2fe88
[    0.000000] At line 412, gsi_base is 0
[    0.000000] At line 425, gsi_base is -1043715332
[    0.000000] At line 427, gsi_base is -1043715332

http://git.infradead.org/users/dwmw2/linux-retpoline.git/shortlog/refs/heads/clang
http://david.woodhou.se/clang32.config


http://david.woodhou.se/io_apic_b.i
http://david.woodhou.se/io_apic_b.noretpoline.s
http://david.woodhou.se/io_apic_b.retpoline.s

I don't *think* I screwed up copying and pasting the retpoline thunk.

David Woodhouse via llvm-dev

unread,
Feb 9, 2018, 10:37:41 AM2/9/18
to Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck
So, looking at the retpoline version...

gsi_base is in %edi, and gets spilled to the stack at about .Ltmp22
which is at line 412 right after the printk call:

.Ltmp22:
addl $12, %esp
movl %edi, 12(%esp)          # 4-byte Spill

At .Ltmp28 we then call __x86_indirect_thunk which *does* look like
it's doing the right thing (and using the LLVM-emitted thunk instead of
my own behaves the same; I don't think it's my copy-paste at fault).

At .Ltmp29 we call bad_ioapic_register() and then when returns zero (it
does) we je to .LBB0_10 aka .Ltmp34. At which point we happily stomp on
the value of 12(%esp) by spilling something *else* over it:

.LBB0_10:                               # %if.end28
.Ltmp34:
#DEBUG_VALUE: mp_register_ioapic:address <- $esi
#DEBUG_VALUE: mp_register_ioapic:cfg <- [DW_OP_plus_uconst 8] [$ebp+0]
.loc 2 0 1 is_stmt 0         # arch/x86/kernel/apic/io_apic_b.c:0:1
movl %ebx, 12(%esp)          # 4-byte Spill
movl %edi, 20(%esp)          # 4-byte Spill

Compare with the noretpoline.s version where those offsets from %esp
are different:

.LBB0_10:                               # %if.end28
.Ltmp33:
#DEBUG_VALUE: mp_register_ioapic:gsi_base <- [DW_OP_plus_uconst 12] [$esp+0]
#DEBUG_VALUE: mp_register_ioapic:address <- $esi
#DEBUG_VALUE: mp_register_ioapic:cfg <- [DW_OP_plus_uconst 8] [$ebp+0]
.loc 2 0 1 is_stmt 0         # arch/x86/kernel/apic/io_apic_b.c:0:1
movl %ebx, 8(%esp)           # 4-byte Spill
movl %edi, 16(%esp)          # 4-byte Spill


So... obviously the main preceding difference here is that the
retpoline version did this for an indirect call:

movl %esi, %edx
pushl pv_mmu_ops+132
.Ltmp28:
calll __x86_indirect_thunk

... while the noretpoline version did this:

movl %esi, %edx
calll *pv_mmu_ops+132
.Ltmp28:

Which leaves me suspecting that when we push the target onto the stack
for a retpoline call, that affects the %esp-based offsets of everything
that's been spilled, of course... and perhaps we're not correctly
adjusting those offsets *back* again by accounting for the fact that
calling the thunk actually moves the stack pointer back up again?

Breakpoint 1, mp_register_ioapic (id=0, address=4273995776, gsi_base=0, cfg=0xc1b2fe88 <init_thread_union+7816>) at arch/x86/kernel/apic/io_apic_b.c:389
389 bool hotplug = !!ioapic_initialized;
1: x/i $pc
=> 0xc10469c7 <mp_register_ioapic+23>: mov    0xc1d36170,%eax
2: gsi_base = 0
(gdb) ni
393 pr_warn("%s, %d %x %x %px\n", __func__, id, address, gsi_base, cfg);
1: x/i $pc
=> 0xc10469cc <mp_register_ioapic+28>: mov    %eax,0x18(%esp)
2: gsi_base = 0
(gdb) 
0xc10469d0 393 pr_warn("%s, %d %x %x %px\n", __func__, id, address, gsi_base, cfg);
1: x/i $pc
=> 0xc10469d0 <mp_register_ioapic+32>: pushl  0x8(%ebp)
2: gsi_base = 0
(gdb) 
0xc10469d3 393 pr_warn("%s, %d %x %x %px\n", __func__, id, address, gsi_base, cfg);
1: x/i $pc
=> 0xc10469d3 <mp_register_ioapic+35>: push   %ecx
2: gsi_base = 0
(gdb) 
0xc10469d4 393 pr_warn("%s, %d %x %x %px\n", __func__, id, address, gsi_base, cfg);
1: x/i $pc
=> 0xc10469d4 <mp_register_ioapic+36>: push   %edx
2: gsi_base = 0
(gdb) 
0xc10469d5 393 pr_warn("%s, %d %x %x %px\n", __func__, id, address, gsi_base, cfg);
1: x/i $pc
=> 0xc10469d5 <mp_register_ioapic+37>: push   %ebx
2: gsi_base = 0
(gdb) 
0xc10469d6 393 pr_warn("%s, %d %x %x %px\n", __func__, id, address, gsi_base, cfg);
1: x/i $pc
=> 0xc10469d6 <mp_register_ioapic+38>: push   $0xc1a3f457
2: gsi_base = 0
(gdb) 
0xc10469db 393 pr_warn("%s, %d %x %x %px\n", __func__, id, address, gsi_base, cfg);
1: x/i $pc
=> 0xc10469db <mp_register_ioapic+43>: push   $0xc1a3f443
2: gsi_base = 0
(gdb) 
0xc10469e0 393 pr_warn("%s, %d %x %x %px\n", __func__, id, address, gsi_base, cfg);
1: x/i $pc
=> 0xc10469e0 <mp_register_ioapic+48>: call   0xc10aa840 <printk>
2: gsi_base = 0
(gdb) 
0xc10469e5 393 pr_warn("%s, %d %x %x %px\n", __func__, id, address, gsi_base, cfg);
1: x/i $pc
=> 0xc10469e5 <mp_register_ioapic+53>: add    $0x18,%esp
2: gsi_base = 0
(gdb) 
395 if (!address) {
1: x/i $pc
=> 0xc10469e8 <mp_register_ioapic+56>: test   %esi,%esi
2: gsi_base = 0
(gdb) 
0xc10469ea 395 if (!address) {
1: x/i $pc
=> 0xc10469ea <mp_register_ioapic+58>: je     0xc1046a34 <mp_register_ioapic+132>
2: gsi_base = 0
(gdb) 
399 for_each_ioapic(ioapic)
1: x/i $pc
=> 0xc10469ec <mp_register_ioapic+60>: mov    0xc1d36144,%eax
2: gsi_base = 0
(gdb) 
0xc10469f1 399 for_each_ioapic(ioapic)
1: x/i $pc
=> 0xc10469f1 <mp_register_ioapic+65>: test   %eax,%eax
2: gsi_base = 0
(gdb) 
0xc10469f3 399 for_each_ioapic(ioapic)
1: x/i $pc
=> 0xc10469f3 <mp_register_ioapic+67>: jle    0xc1046a10 <mp_register_ioapic+96>
2: gsi_base = 0
(gdb) 
406 idx = find_free_ioapic_entry();
1: x/i $pc
=> 0xc1046a10 <mp_register_ioapic+96>: call   0xc1045670 <find_free_ioapic_entry>
2: gsi_base = 0
(gdb) 
407 if (idx >= MAX_IO_APICS) {
1: x/i $pc
=> 0xc1046a15 <mp_register_ioapic+101>: cmp    $0x40,%eax
2: gsi_base = 0
(gdb) 
0xc1046a18 407 if (idx >= MAX_IO_APICS) {
1: x/i $pc
=> 0xc1046a18 <mp_register_ioapic+104>: jl     0xc1046a4b <mp_register_ioapic+155>
2: gsi_base = 0
(gdb) 
0xc1046a4b 396 pr_warn("Bogus (zero) I/O APIC address found, skipping!\n");
1: x/i $pc
=> 0xc1046a4b <mp_register_ioapic+155>: mov    %ebx,0x4(%esp)
2: gsi_base = 0
(gdb) 
412 pr_warn("At line %d, gsi_base is %d\n", __LINE__,gsi_base);
1: x/i $pc
=> 0xc1046a4f <mp_register_ioapic+159>: push   %edi
2: gsi_base = 0
(gdb) 
0xc1046a50 412 pr_warn("At line %d, gsi_base is %d\n", __LINE__,gsi_base);
1: x/i $pc
=> 0xc1046a50 <mp_register_ioapic+160>: push   $0x19c
2: gsi_base = 0
(gdb) 
0xc1046a55 412 pr_warn("At line %d, gsi_base is %d\n", __LINE__,gsi_base);
1: x/i $pc
=> 0xc1046a55 <mp_register_ioapic+165>: push   $0xc1a3f4fd
2: gsi_base = 0
(gdb) 
0xc1046a5a 412 pr_warn("At line %d, gsi_base is %d\n", __LINE__,gsi_base);
1: x/i $pc
=> 0xc1046a5a <mp_register_ioapic+170>: mov    %eax,%ebx
2: gsi_base = 0
(gdb) 
0xc1046a5c 412 pr_warn("At line %d, gsi_base is %d\n", __LINE__,gsi_base);
1: x/i $pc
=> 0xc1046a5c <mp_register_ioapic+172>: call   0xc10aa840 <printk>
2: gsi_base = 0
(gdb) 
0xc1046a61 412 pr_warn("At line %d, gsi_base is %d\n", __LINE__,gsi_base);
1: x/i $pc
=> 0xc1046a61 <mp_register_ioapic+177>: add    $0xc,%esp
2: gsi_base = 0
(gdb) 
0xc1046a64 412 pr_warn("At line %d, gsi_base is %d\n", __LINE__,gsi_base);
1: x/i $pc
=> 0xc1046a64 <mp_register_ioapic+180>: mov    %edi,0xc(%esp)
2: gsi_base = 0
(gdb) p/x 0xc+$esp
$1 = 0xc1b2fe34
(gdb) disp *(int *)0xc1b2fe34
3: *(int *)0xc1b2fe34 = 120
(gdb) ni
414 ioapics[idx].mp_config.type = MP_IOAPIC;
1: x/i $pc
=> 0xc1046a68 <mp_register_ioapic+184>: imul   $0x2c,%ebx,%edi
2: gsi_base = 0
3: *(int *)0xc1b2fe34 = 0
(gdb) 
0xc1046a6b 414 ioapics[idx].mp_config.type = MP_IOAPIC;
1: x/i $pc
=> 0xc1046a6b <mp_register_ioapic+187>: movb   $0x2,-0x3e2cb1bc(%edi)
2: gsi_base = 0
3: *(int *)0xc1b2fe34 = 0
(gdb) 
415 ioapics[idx].mp_config.flags = MPC_APIC_USABLE;
1: x/i $pc
=> 0xc1046a72 <mp_register_ioapic+194>: movb   $0x1,-0x3e2cb1b9(%edi)
2: gsi_base = 0
3: *(int *)0xc1b2fe34 = 0
(gdb) 
416 ioapics[idx].mp_config.apicaddr = address;
1: x/i $pc
=> 0xc1046a79 <mp_register_ioapic+201>: mov    %esi,-0x3e2cb1b8(%edi)
2: gsi_base = 0
3: *(int *)0xc1b2fe34 = 0
(gdb) 
418 set_fixmap_nocache(FIX_IO_APIC_BASE_0 + idx, address);
1: x/i $pc
=> 0xc1046a7f <mp_register_ioapic+207>: lea    0x5(%ebx),%eax
2: gsi_base = 0
3: *(int *)0xc1b2fe34 = 0
(gdb) 
cachemode2protval (pcm=_PAGE_CACHE_MODE_UC) at ./arch/x86/include/asm/pgtable_types.h:439
439 return __cachemode2pte_tbl[pcm];
1: x/i $pc
=> 0xc1046a82 <mp_register_ioapic+210>: movzwl 0xc1b424c6,%ecx
2: gsi_base = 0
3: *(int *)0xc1b2fe34 = 0
(gdb) 
mp_register_ioapic (id=<optimized out>, address=4273995776, gsi_base=0, cfg=0xc1b2fe88 <init_thread_union+7816>) at arch/x86/kernel/apic/io_apic_b.c:418
418 set_fixmap_nocache(FIX_IO_APIC_BASE_0 + idx, address);
1: x/i $pc
=> 0xc1046a89 <mp_register_ioapic+217>: or     $0x163,%ecx
2: gsi_base = 0
3: *(int *)0xc1b2fe34 = 0
(gdb) 
0xc1046a8f 418 set_fixmap_nocache(FIX_IO_APIC_BASE_0 + idx, address);
1: x/i $pc
=> 0xc1046a8f <mp_register_ioapic+223>: mov    %eax,0x14(%esp)
2: gsi_base = 0
3: *(int *)0xc1b2fe34 = 0
(gdb) 
__set_fixmap (idx=5, phys=<optimized out>, flags=...) at ./arch/x86/include/asm/paravirt.h:660
660 pv_mmu_ops.set_fixmap(idx, phys, flags);
1: x/i $pc
=> 0xc1046a93 <mp_register_ioapic+227>: mov    %esi,%edx
2: gsi_base = 0
3: *(int *)0xc1b2fe34 = 0
(gdb) 
0xc1046a95 660 pv_mmu_ops.set_fixmap(idx, phys, flags);
1: x/i $pc
=> 0xc1046a95 <mp_register_ioapic+229>: pushl  0xc1ad6434
2: gsi_base = 0
3: *(int *)0xc1b2fe34 = 0
(gdb) 
0xc1046a9b 660 pv_mmu_ops.set_fixmap(idx, phys, flags);
1: x/i $pc
=> 0xc1046a9b <mp_register_ioapic+235>: call   0xc18ff3a0 <__x86_indirect_thunk>
2: gsi_base = <optimized out>
3: *(int *)0xc1b2fe34 = 0
(gdb) 
mp_register_ioapic (id=<optimized out>, address=4273995776, gsi_base=<optimized out>, cfg=0xc1b2fe88 <init_thread_union+7816>) at arch/x86/kernel/apic/io_apic_b.c:419
419 if (bad_ioapic_register(idx)) {
1: x/i $pc
=> 0xc1046aa0 <mp_register_ioapic+240>: mov    %ebx,%eax
2: gsi_base = <optimized out>
3: *(int *)0xc1b2fe34 = 0
(gdb) 
0xc1046aa2 419 if (bad_ioapic_register(idx)) {
1: x/i $pc
=> 0xc1046aa2 <mp_register_ioapic+242>: call   0xc10455f0 <bad_ioapic_register>
2: gsi_base = <optimized out>
3: *(int *)0xc1b2fe34 = 0
(gdb) 
0xc1046aa7 419 if (bad_ioapic_register(idx)) {
1: x/i $pc
=> 0xc1046aa7 <mp_register_ioapic+247>: test   %eax,%eax
2: gsi_base = <optimized out>
3: *(int *)0xc1b2fe34 = 0
(gdb) 
0xc1046aa9 419 if (bad_ioapic_register(idx)) {
1: x/i $pc
=> 0xc1046aa9 <mp_register_ioapic+249>: je     0xc1046ae1 <mp_register_ioapic+305>
2: gsi_base = <optimized out>
3: *(int *)0xc1b2fe34 = 0
(gdb) 
0xc1046ae1 482 }
1: x/i $pc
=> 0xc1046ae1 <mp_register_ioapic+305>: mov    %ebx,0xc(%esp)
2: gsi_base = <optimized out>
3: *(int *)0xc1b2fe34 = 0
(gdb) p/x $esp+0xc
$2 = 0xc1b2fe34

Reid Kleckner via llvm-dev

unread,
Feb 9, 2018, 2:24:35 PM2/9/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
I haven't read the all the emails in full detail, but it seems pretty clear that  __x86_indirect_thunk and __llvm_retpoline_push do not do the same things. It sounds like __llvm_retpoline_push is equivalent to __x86_indirect_thunk except first it swaps the two words on the top of the stack.

I arranged it this way because the x86 call instruction puts the intended return address on the top of the stack, and there's no easy way to put it anywhere else. We use this thunk when we want to make an indirect call and there are no available scratch registers, i.e. 32-bit -mregparm=3 and the call has three or more arguments, which happens in Linux. One way to avoid this would be to compile with -mregparm=2, but that would pessimize direct calls unnecessarily.

It sounds like Linux is already providing its own thunks, so it might be better to for us to go back to the __llvm_retpoline_push external thunk name and then get Linux to provide that thunk ifdef __clang__.

David Woodhouse via llvm-dev

unread,
Feb 9, 2018, 2:39:30 PM2/9/18
to Reid Kleckner, llvm-dev, Thomas Gleixner, Guenter Roeck
On Fri, 2018-02-09 at 11:24 -0800, Reid Kleckner wrote:
> I haven't read the all the emails in full detail, but it seems pretty
> clear that  __x86_indirect_thunk and __llvm_retpoline_push do not do
> the same things. It sounds like __llvm_retpoline_push is equivalent
> to __x86_indirect_thunk except first it swaps the two words on the
> top of the stack.
>
> I arranged it this way because the x86 call instruction puts the
> intended return address on the top of the stack, and there's no easy
> way to put it anywhere else. We use this thunk when we want to make
> an indirect call and there are no available scratch registers, i.e.
> 32-bit -mregparm=3 and the call has three or more arguments, which
> happens in Linux. One way to avoid this would be to compile with
> -mregparm=2, but that would pessimize direct calls unnecessarily.
>
> It sounds like Linux is already providing its own thunks, so it might
> be better to for us to go back to the __llvm_retpoline_push external
> thunk name and then get Linux to provide that thunk ifdef __clang__.

Yes, the thunks are different in this case. That one should indeed be
renamed, and I'll provide the correct one in the kernel.

However, even if I let LLVM emit the thunk for itself, code generation
seems hosed. It looks like after a call to __llvm_retpoline_push (or
perhaps more to the point after pushing the target address in
preparation for the call?) we're losing track of where %esp points, and
thus everything else that was spilled to the stack.

Please let me know if you need any more detail on that, than I provided
in my last email. You should have everything you need there to follow
along with my analysis and reproduce it for yourself. Except perhaps
the Grandma-suck-eggs command lines like

$ qemu-system-i386  -display none -serial stdio -kernel arch/x86/boot/bzImage  -append earlyprintk=ttyS0,keep -s -S

$ gdb vmlinux
(gdb) tar rem localhost:1234
(gdb) break mp_register_ioapic
(gdb) c

Reid Kleckner via llvm-dev

unread,
Feb 9, 2018, 5:23:49 PM2/9/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
I think I see what's going on, and I agree it looks like a bug. It was too much to hope that later passes weren't going to mess with the PUSH instruction. :(

While I was trying to reproduce your problem, I think I found another one that looks like this:
$ clang -S -O2 -m32 -mregparm=3 -mretpoline spill_across_rp.cpp -o - | grep _retpoline_push -B2
...
        movl    %eax, 8(%esp)           # 4-byte Spill
...
        pushl   %edi
        movl    8(%esp), %edi           # 4-byte Reload
        calll   __llvm_retpoline_push
That's obviously broken, it doesn't account for the SP adjustment in the push.

It's weird, because it's basically the opposite of the problem you're having, which looks like LLVM *is* accounting for the push and is trying to adjust its offsets accordingly.

Can you send along the full command line used to compile io_apic_b.i?

David Woodhouse via llvm-dev

unread,
Feb 9, 2018, 5:28:58 PM2/9/18
to Reid Kleckner, llvm-dev, Thomas Gleixner, Guenter Roeck
On Fri, 2018-02-09 at 14:23 -0800, Reid Kleckner wrote:
> I think I see what's going on, and I agree it looks like a bug. It
> was too much to hope that later passes weren't going to mess with the
> PUSH instruction. :(
>
> While I was trying to reproduce your problem, I think I found another
> one that looks like this:
> $ clang -S -O2 -m32 -mregparm=3 -mretpoline spill_across_rp.cpp -o -
> | grep _retpoline_push -B2
> ...
>         movl    %eax, 8(%esp)           # 4-byte Spill
> ...
>         pushl   %edi
>         movl    8(%esp), %edi           # 4-byte Reload
>         calll   __llvm_retpoline_push
> That's obviously broken, it doesn't account for the SP adjustment in
> the push.

Fun. Isn't that the *opposite* of the problem I was seeing? I thought
it *was* accounting for the SP adjustment in the push, even after
__llvm_retpoline_push had done its thing and effectively popped that
word back off the stack again.

I've now filed this as PR36329 and marked it blocking PR35804 (6.0
release).

> It's weird, because it's basically the opposite of the problem you're
> having, which looks like LLVM *is* accounting for the push and is
> trying to adjust its offsets accordingly.
>
> Can you send along the full command line used to compile io_apic_b.i?

  /home/dwmw2/git/llvm6/bin/clang -Wp,-MD,arch/x86/kernel/apic/.io_apic_b.o.d  -nostdinc -isystem /home/dwmw2/git/llvm6/lib/clang/6.0.0/include -I./arch/x86/include -I./arch/x86/include/generated  -I./include -I./arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/kconfig.h -D__KERNEL__ -Qunused-arguments -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -Werror-implicit-function-declaration -Wno-format-security -std=gnu89 -fno-PIE -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -m32 -msoft-float -mregparm=3 -freg-struct-return -fno-pic -mstack-alignment=4 -march=i686 -ffreestanding -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_SSSE3=1 -DCONFIG_AS_CRC32=1 -DCONFIG_AS_AVX=1 -DCONFIG_AS_AVX2=1 -DCONFIG_AS_SHA1_NI=1 -DCONFIG_AS_SHA256_NI=1 -pipe -Wno-sign-compare -fno-asynchronous-unwind-tables -mretpoline-external-thunk -DRETPOLINE -O2 -Wframe-larger-than=2048 -fno-stack-protector -Wno-unused-variable -Wno-format-invalid-specifier -Wno-gnu -Wno-address-of-packed-member -Wno-tautological-compare -mno-global-merge -no-integrated-as -fno-omit-frame-pointer -fno-optimize-sibling-calls -g -pg -mfentry -DCC_USING_FENTRY -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fno-stack-check -Werror=implicit-int -Werror=strict-prototypes -Werror=date-time -Werror=incompatible-pointer-types -Wno-initializer-overrides -Wno-unused-value -Wno-format -Wno-sign-compare -Wno-format-zero-length -Wno-uninitialized -mno-retpoline-external-thunk    -DKBUILD_BASENAME='"io_apic_b"'  -DKBUILD_MODNAME='"io_apic_b"' -c -o arch/x86/kernel/apic/.tmp_io_apic_b.o arch/x86/kernel/apic/io_apic_b.c

David Woodhouse via llvm-dev

unread,
Feb 10, 2018, 5:38:04 AM2/10/18
to Reid Kleckner, llvm-dev, Thomas Gleixner, Guenter Roeck

> I arranged it this way because the x86 call instruction puts the intended
> return address on the top of the stack, and there's no easy way to put it
> anywhere else. We use this thunk when we want to make an indirect call and
> there are no available scratch registers, i.e. 32-bit -mregparm=3 and the
> call has three or more arguments, which happens in Linux. One way to avoid
> this would be to compile with -mregparm=2, but that would pessimize direct
> calls unnecessarily.

I appreciate that x86 is horribly register-starved but there *are* more
than three. For many of the cases where LLVM will emit its _push
retpoline, GCC would have just used another register trampoline like %ebx
or %esi that *isn't* one of the parameters.


--
dwmw2

Hans Wennborg via llvm-dev

unread,
Feb 19, 2018, 10:18:34 AM2/19/18
to Reid Kleckner, Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck
Reid, is this what you fixed with r325049 that's also merged to 6.0?

If so, do we have everything that's needed, or is there anything else
I should wait for?

Chandler, do you have anything else? Also, release notes please :-)

David Woodhouse via llvm-dev

unread,
Feb 19, 2018, 10:31:55 AM2/19/18
to Hans Wennborg, Reid Kleckner, Chandler Carruth, llvm-dev, Thomas Gleixner, Guenter Roeck
On Mon, 2018-02-19 at 16:18 +0100, Hans Wennborg wrote:
> Reid, is this what you fixed with r325049 that's also merged to 6.0?

That's working for me, yes.

> If so, do we have everything that's needed, or is there anything else
> I should wait for?

I wouldn't mind an opinion on 
https://bugs.llvm.org/show_bug.cgi?id=33587

Clang for x86_32 will happily compile this:

long long ret;
asm ("movl $5, %0" : "=r" (ret)); // 32-bit reg for 64-bit val is fine.

... but not this:

long long ret;
asm ("movb $5, %0" : "=q" (ret)); // 8-bit reg for 64-bit val not fine.

In practice, those cases go away in optimisation but the problem for us
in building Linux is that the 8-bit case causes a build failure
*before* the optimiser realises it'd never have to emit it anyway.

I can hack that up so that we're pretending to have 8-bit variables:
https://lkml.org/lkml/2018/2/9/529

But before I go further with that rather icky approach I'd like to know
why it doesn't break for the "r" case, and if it's going to *keep*
working. And if we can make the "q" case behave the same...

Hans Wennborg via llvm-dev

unread,
Feb 19, 2018, 10:36:01 AM2/19/18
to David Woodhouse, llvm-dev, Thomas Gleixner, Guenter Roeck
On Mon, Feb 19, 2018 at 4:31 PM, David Woodhouse <dw...@infradead.org> wrote:
> On Mon, 2018-02-19 at 16:18 +0100, Hans Wennborg wrote:
>> Reid, is this what you fixed with r325049 that's also merged to 6.0?
>
> That's working for me, yes.
>
>> If so, do we have everything that's needed, or is there anything else
>> I should wait for?
>
> I wouldn't mind an opinion on
> https://bugs.llvm.org/show_bug.cgi?id=33587
>
> Clang for x86_32 will happily compile this:
>
> long long ret;
> asm ("movl $5, %0" : "=r" (ret)); // 32-bit reg for 64-bit val is fine.
>
> ... but not this:
>
> long long ret;
> asm ("movb $5, %0" : "=q" (ret)); // 8-bit reg for 64-bit val not fine.
>
> In practice, those cases go away in optimisation but the problem for us
> in building Linux is that the 8-bit case causes a build failure
> *before* the optimiser realises it'd never have to emit it anyway.

There are probably others better qualified to comment on the
functionality here, but from a 6.0 release point of view, this doesn't
sound like a regression. If a straight-forward fix emerges soon, maybe
we could merge it, but otherwise I'm not sure I can hold the release
for this.

> I can hack that up so that we're pretending to have 8-bit variables:
> https://lkml.org/lkml/2018/2/9/529
>
> But before I go further with that rather icky approach I'd like to know
> why it doesn't break for the "r" case, and if it's going to *keep*
> working. And if we can make the "q" case behave the same...

Ed Maste via llvm-dev

unread,
Mar 13, 2018, 4:51:44 PM3/13/18
to Chandler Carruth, llvm-dev, Dimitry Andric
On 7 February 2018 at 01:20, Chandler Carruth via llvm-dev

<llvm...@lists.llvm.org> wrote:
> I've landed the patch in r324449.

To confirm, this change *did not* make it into 6.0.0, correct? I'm
looking at adding the external thunk for FreeBSD now, and we have
6.0.0 in HEAD. Assuming that future Clang releases (major, and
updates) will all use __x86_indirect_thunk_r11 we presumably want to
patch r324449 into the 6.0.0 in our tree.

Ed Maste via llvm-dev

unread,
Mar 13, 2018, 5:02:34 PM3/13/18
to Chandler Carruth, llvm-dev, Dimitry Andric
On 13 March 2018 at 16:51, Ed Maste <ema...@freebsd.org> wrote:
> On 7 February 2018 at 01:20, Chandler Carruth via llvm-dev
> <llvm...@lists.llvm.org> wrote:
>> I've landed the patch in r324449.
>
> To confirm, this change *did not* make it into 6.0.0, correct? I'm
> looking at adding the external thunk for FreeBSD now, and we have
> 6.0.0 in HEAD. Assuming that future Clang releases (major, and
> updates) will all use __x86_indirect_thunk_r11 we presumably want to
> patch r324449 into the 6.0.0 in our tree.

Dimitry points out it did get merged (r325082), so I must have been
testing a version of 6.0.0 before the final release.

Testing again now, with:
FreeBSD clang version 6.0.0 (tags/RELEASE_600/final 326565) (based on
LLVM 6.0.0)
confirms it emits calls to __x86_indirect_thunk_r11; sorry for the false alarm.

Chandler Carruth via llvm-dev

unread,
Mar 13, 2018, 7:37:41 PM3/13/18
to Ed Maste, llvm-dev, Dimitry Andric
Whew! But thanks for testing!
Reply all
Reply to author
Forward
0 new messages