GIC access conflict

[Dan Zach]

Dear forum,

I run Jailhouse on Jetson-TK1 with its native L4T distribution based on kernel 3.10.40, which I patched to support PSCI. The trouble starts when I use full Ubuntu user interface on the HDMI screen. After few seconds, there is a freeze and usually Jailhouse manages to report an unhandled write at 0x50041fxx (GIC distributor) from its DABT trap routine. This is not a mapping issue of display or HDMI peripherals, because we can use the display for several seconds.

What I found that display control drivers extensively mask and unmask interrupts constantly, which lead to Jailhouse data abort trap to run very frequently.

In parallel there is software interrupt being raised for SMP management, which flows through the same data abort trap.

1.Is my understanding correct that GIC access from an inmate is always handled by DABT trap?

2.Is it possible that there is re-entrance in DABT trap processing , when there is a concurrent GIC assess attempts from different cores?

Thanks
Dan

[Dan Zach]

BTW, Jailhouse is at v0.5 tag

[Jan Kiszka]

On 2017-01-06 11:18, Dan Zach wrote:
> On Friday, 6 January 2017 12:16:23 UTC+2, Dan Zach wrote:
>> Dear forum,
>>
>> I run Jailhouse on Jetson-TK1 with its native L4T distribution based on kernel 3.10.40, which I patched to support PSCI.

FYI: You won't get inter-cell communication for that historic kernel. We
have a legacy 3.14 patched to enable this, and that was already a
horrible, unshareable hack.

>> The trouble starts when I use full Ubuntu user interface on the HDMI screen. After few seconds, there is a freeze and usually Jailhouse manages to report an unhandled write at 0x50041fxx (GIC distributor) from its DABT trap routine. This is not a mapping issue of display or HDMI peripherals, because we can use the display for several seconds.
>>
>> What I found that display control drivers extensively mask and unmask interrupts constantly, which lead to Jailhouse data abort trap to run very frequently.
>>
>> In parallel there is software interrupt being raised for SMP management, which flows through the same data abort trap.
>>
>> 1.Is my understanding correct that GIC access from an inmate is always handled by DABT trap?
>>
>> 2.Is it possible that there is re-entrance in DABT trap processing , when there is a concurrent GIC assess attempts from different cores?
>>
>>
>> Thanks
>> Dan
>
> BTW, Jailhouse is at v0.5 tag
>

Please update to next. v0.5 has to be considered broken for ARM.

The next branch can be seen as "v0.6" for ARM. I'm just shaking out some
nasty remaining issues on x86 and will then issue a new release.

Jan

--
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux

[Dan Zach]

Jan,
That worked!
Moving to 'next' made all the freezes to disappear.
Thank you so much, I tried to sort it out for weeks.
So now we enjoy full Nvidia GPU support on the root cell - hence using the L4T distribution.

Can you please elaborate on the difficulty of inter-cell communication on 3.1x kernels?
I cant move upstream, because of the GPU driver I would loose.
So I must find a way to share some memory region between the cells.

Dan

[Jan Kiszka]

Perfect!

> Moving to 'next' made all the freezes to disappear.
> Thank you so much, I tried to sort it out for weeks.

Golden rule with any project: try latest and greatest first. At least
ask the people behind it if it may have an impact.

> So now we enjoy full Nvidia GPU support on the root cell - hence using the L4T distribution.
>
> Can you please elaborate on the difficulty of inter-cell communication on 3.1x kernels?
> I cant move upstream, because of the GPU driver I would loose.
> So I must find a way to share some memory region between the cells.

Jailhouse provides shared memory devices via PCI. On ARM SOCs where
there is no physical PCI controller or where we can't hook into it (like
on the K1), we provide a virtual generic host controller.

First of all, the driver for this controller pre-dates 3.10. Then we
need to hot-plug the device, because it is only available after
Jailhouse started. We use a device tree overlay for that, but support
for this feature also pre-dates 3.10.

For that 3.14 kernel, I back-ported the driver and added a hack to only
succeed with probing on the second, then manually via sysfs triggered
attempt. Unloading, i.e. Jailhouse disabling is not supported at all
(but that has issues with upstream as well due to resource leaks).

[Dan Zach]

> I wish I could go upstream! But NVidia indicated they don't plan to upgrade to a newer kernel anytime soon. Do you know any Jailhouse user that uses full graphics on Jetson TK1?

[Dan Zach]

I wish I could go upstream! But NVidia indicated they don't plan to upgrade to a newer kernel anytime soon. Do you know any Jailhouse user that uses full graphics on Jetson TK1?

[Ralf Ramsauer]

I'd also like to make use of the graphics stack, and I hope that the
nouveau stack will be sufficient for my use case. But I didn't find any
time to work on that yet...

Did you actually get the nouveau stack working? Or is there anything
against the nouveau stack in your case?

Ralf