Regarding integrating external interrupt controller

172 views
Skip to first unread message

saravanan

unread,
Dec 26, 2017, 9:53:23 PM12/26/17
to RISC-V SW Dev (sw-dev@groups.riscv.org), RISC-V HW Dev
Hi ,

We are using rocket chip processor. It has two interrupt pins. As it is
not fulfilling our requirement. We have added external Interrupt
controller, Do we have take care any thing explicitly in BBL for linux
bring up ?

Thanks

Saravanan


--
Mobiveil INC., CONFIDENTIALITY NOTICE: This e-mail message, including any
attachments, is for the sole use of the intended recipient(s) and may
contain proprietary confidential or privileged information or otherwise be
protected by law. Any unauthorized review, use, disclosure or distribution
is prohibited. If you are not the intended recipient, please notify the
sender and destroy all copies and the original message.

Dr Jonathan Kimmitt

unread,
Dec 27, 2017, 3:57:40 AM12/27/17
to saravanan, RISC-V SW Dev (sw-dev@groups.riscv.org), RISC-V HW Dev
By default interrupts are delivered to BBL in machine mode. This then is delivered to Linux as a software interrupt. The tricky thing is that external interrupts in machine mode take priority over software interrupts, so to handle this BBL has to tell the external interrupt controller to mask the interrupt before returning from BBL, otherwise an infinite sequence of interrupts will follow.

Once Linux receives the software interrupt, it can transfer the data and then reenable the controller before the handler exits.

Depending on the version of Rocket that you have, it should also be possible to bypass BBL completely and deliver the external interrupt directly to supervisor mode by appropriate use of CSR registers. To my mind the documentation is a bit thin in this area though.

Sent from my iPhone
> --
> You received this message because you are subscribed to the Google Groups "RISC-V HW Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to hw-dev+un...@groups.riscv.org.
> To post to this group, send email to hw-...@groups.riscv.org.
> Visit this group at https://groups.google.com/a/groups.riscv.org/group/hw-dev/.
> To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/hw-dev/b0fc8b04-6e06-0825-f2ca-4382023f7382%40mobiveil.co.in.

saravanan

unread,
Dec 27, 2017, 4:29:16 AM12/27/17
to Dr Jonathan Kimmitt, RISC-V SW Dev (sw-dev@groups.riscv.org), RISC-V HW Dev
Hello Jonathan,

Thanks for your reply.

Yes! we are seeing the infinite loop inside Linux ISR handling, because
we were delegating the external interrupts from BBL to Linux.

We are using the risc-v processor version available in this link
"https://github.com/freechipsproject/rocket-chip"

If in case do we need to redirect the external interrupts into
supervisor mode (Linux Level) directly, could you please help us with
sequence or steps need to be followed.


Thanks!

Saravanan

Dr Jonathan Kimmitt

unread,
Dec 27, 2017, 6:12:46 AM12/27/17
to saravanan, RISC-V SW Dev (sw-dev@groups.riscv.org), RISC-V HW Dev
That version of Rocket should definitely support delegation however I don’t know where the documentation of this function is. If you can get the easier method going of masking the interrupt controller interrupt output in BBL before returning from machine mode, that solved the problem for me, even if it is a bit hacky.

Sent from my iPhone

saravanan

unread,
Jan 7, 2018, 8:55:09 PM1/7/18
to Dr Jonathan Kimmitt, RISC-V SW Dev (sw-dev@groups.riscv.org), RISC-V HW Dev

Hi Jonathan,

 If you can get the easier method going of masking the interrupt controller interrupt output in BBL before returning from machine mode, that solved the problem for me
- In the chip scope, we where able to interrupt controllers output is always zero after first interrupt. But still we see the ISR  is looping.
  https://github.com/freechipsproject/rocket-chip

Thanks
Saravanan

Dr Jonathan Kimmitt

unread,
Jan 8, 2018, 3:42:45 AM1/8/18
to RISC-V HW Dev

Sorry for the previous misleading information but in the latest Rocket the platform level interrupt controller is integrated by default.

Although documentation is scanty on how this works you can get a good idea by reading SiFive documentation

and applying mutatis mutandis (see chapter 10 of the link https://static.dev.sifive.com/FE310-G000.pdf).

The basic idea is that you read and write the claim

and complete register to satisfy the interrupt and prevent looping. You need to know the address of the controller

but this is printed on the log when building the Rocket, and put in the boot rom.

So your choices are to disable the internal PLIC if you want to use my original suggestion or remove the external

interrupt controller.

If you remove the external interrupt controller you are back to the original problem of not having sufficient interrupt

sources available. Have you discovered that the configuration parameter WithNExtTopInterrupts(nExtInts) configured in Configs.scala?

This defaults to 2 and I think this is why you are seeing the behaviour. So for example if you are using DefaultConfig,

you would change

class DefaultConfig extends Config(new WithBlockingL1 ++ new WithNBigCores(1) ++ new BaseConfig)

to

class DefaultConfig extends Config(new WithNExtTopInterrupts(8) ++ new WithBlockingL1 ++ new WithNBigCores(1) ++ new BaseConfig)

If you go with this approach the width of the interrupt input will widen and you will have room for all your sources, then you no longer
have need of the external interrupt controller.

saravanan

unread,
Jan 8, 2018, 4:53:42 AM1/8/18
to Dr Jonathan Kimmitt, RISC-V HW Dev

Hi Dr Jonathan Kimmitt,

We are using latest interrupt controller (PLIC) along with the our Interrupt controller in cascaded form. We are trying to use the default config.scala, we are not interested in changing it.

In this case, could you suggest as what would be the cause for ISR looping ?


Thanks

Saravanan

Dr Jonathan Kimmitt

unread,
Jan 8, 2018, 5:21:24 AM1/8/18
to saravanan, RISC-V HW Dev

As already mentioned, you need to read and write claim register in the PLIC otherwise it will

not know that you have serviced the interrupt. This is a simple software hack, no need to modify

any chisel or HDL code. This is how I do it:

// At this point we already checked that the mcause was negative as per the privilege spec.

void handle_interrupt(long cause)
{
  int mip;
  char code[20];
  cause &= 0x7FFFFFFF;
  if (cause==IRQ_M_DEV) // There are other sources, software interrupt, timer etc.
   external_interrupt(); 
}

volatile uint32_t *plic;

// So we know the interrupt is external

void external_interrupt(void)
{
  int i, claim, handled = 0;
#ifdef VERBOSE
  printf("Hello external interrupt! "__TIMESTAMP__"\n");
#endif 
  claim = plic[0x80001]; // pick out the highest priority interrupt that the PLIC knows about

  // poll the external sources that we know about, including any secondary interrupt controller

  if (uart_check_read_irq())
    {
      int rslt = uart_read_irq();
      printf("uart interrupt read %x (%c)\n", rslt, rslt);
      handled = 1;
    }

  // Das Tut Mir Leid, we couldn't find the interrupt source

  if (!handled)
    {
      printf("unhandled interrupt!\n");
    }
  plic[0x80001] = claim; // make that interrupt go away
}

// Consult the PLIC documentation for the boring details of this code

void init_plic(void)
{
  int i;
  for (i = 1; i <= 64; i++)
    {
      plic[i] = 1;
    }
  for (i = 1; i <= 64; i++)
    {
      plic[0x800+i/32] |= 1<<(i&31);
    }
  plic[0x80000] = 0;
}

int main()

{
  plic = (volatile uint32_t *)0x0c000000; // hard-wired since we don't want to change scary Chisel code
  init_plic(); 
  printf("Enabling interrupts\n");
  set_csr(mstatus, MSTATUS_MIE|MSTATUS_HIE);
  set_csr(mie, ~(1 << IRQ_M_TIMER)); // everything except the timer
  printf("Enabling UART interrupt\n");
  uart_enable_read_irq();

  for(;;)

    {   

    // background code goes here   

saravanan

unread,
Jan 8, 2018, 6:09:16 AM1/8/18
to Dr Jonathan Kimmitt, RISC-V HW Dev

Hi Dr Jonathan Kimmitt,

While entering into ISR we are reading claim id from the PLIC.

and while exiting from ISR we are witting back the claim id into PLIC register.

but still we see ISR is looping.

Note : Same thing works fine for us in machine mode. Only in supervisor mode we have this issue. Do we have to take care anything special in BBL.  Could you please suggest us any idea, how to proceed.

Thanks

Saravanan

Dr Jonathan Kimmitt

unread,
Jan 8, 2018, 6:27:56 AM1/8/18
to saravanan, RISC-V HW Dev

One obvious difference is that memory protection is usually switched on in supervisor mode, so the address

of the PLIC might be different or inaccessible. Another difference is that machine level interrupts take priority over

supervisor level interrupts, so if you rely on supervisor mode to clear the PLIC via software interrupt, that interrupt

will never take place unless machine mode has turned off the interrupt source (inside BBL). Check out the documentation for

medeleg and mideleg csr registers concerning delegating interrupts directly to supervisor mode.

You can use the PLIC to mask interrupt sources in BBL to prevent recursion until you have had a chance to service the request

in supervisor mode.

If all else fails, I recommend writing a minimal program that contains the essentials of what you are trying to do and run it in

the vsim subdirectory of Rocket using Verilator to dump the internal values of registers and find out the root cause of the looping.

Also have you checked out your memory protection regions are valid (pmpaddr?, pmpcfg?)

Dr Jonathan Kimmitt

unread,
Jan 9, 2018, 4:27:17 AM1/9/18
to Luke Kenneth Casson Leighton, saravanan, RISC-V HW Dev
That sounds truly heroic Luke. But since we are working with an open source

project we have access in simulation to the entire state of the machine
right

down to the flip-flop level. on the FPGA should it be necessary to use a
peripheral

that is not mapped yet then you can just trap into machine mode as needed.


On 09/01/18 09:18, Luke Kenneth Casson Leighton wrote:
> On Mon, Jan 8, 2018 at 11:27 AM, Dr Jonathan Kimmitt <jr...@cam.ac.uk> wrote:
>
>> One obvious difference is that memory protection is usually switched on in
>> supervisor mode, so the address
>> of the PLIC might be different or inaccessible.
> oh ah, virtual memory vs physical memory, whoops :)
>
> as a reverse-engineer who did not necessarily have direct access even
> to JTAG or other techiques i often had to use tools like HARET.EXE
> (Hand-Held Reverse-Engineering Tool) when doing WinCEouch exploration.
>
> often i and my peers would use whatever was available to us to do the
> debugging. this included *direct* programming of UARTs (similar to
> u-boot and linux kernel "earlyprintk") and as an absolute absolute
> last resort, flashing of an LED.
>
> you *may* be able to use something similar, inside the ISR. if you
> can read the ISR's address, you should be able to print it out to a
> buffer in hexadecimal format (slprintf), and dump that out to a UART
> routine.
>
> you could even.... if you were absolutely desperate.... check one bit
> of the ISR data at a time, and delay *deliberate* segfaulting for 10
> seconds if bit 0 is a 1, and no delay if it is 0, then recompile
> trying bit 1, then bit 2... i mean... it's utterly laborious but at
> the end of it you will have a dump of the full value of the ISR.
>
> if, on doing so, you find that the bit you were expecting to be set
> which indicates "Interrupt occurred" is NOT in fact set, then you KNOW
> for a fact that you have the wrong address.
>
> if on the other hand it *is* set, you still cannot guarantee that you
> have the right address, because it could be a random area of memory.
> you should however also be able to check in the immediate
> neighbourhood (by checking other addresses relative to the ISR
> address), see what registers *those* represent, and find something
> that should definitely be a value that it isn't.
>
> yes it's laborious, but it beats guessing, it beats "not knowing the
> answer", and it *definitely* beats banging your head against an
> intransigent brick wall of not being able to use a debugging method
> that you can't use for some reason :)
>
> good luck,
>
> l.

Tadas Aleksonis

unread,
Sep 18, 2018, 5:45:51 PM9/18/18
to RISC-V HW Dev, sara...@mobiveil.co.in
Hi Saravanan and co,

Sorry for chiming in such a long time after this thread has been active, but I'm experiencing a similar issue and there's scant amount of similar bugs to observe online. Is there any update to your system as to what a potential fix is? On my system, I have Linux on a RISC-V softcore, and I'm experiencing an infinite loop in plic_chained_handle_irq, despite interrupts being disabled and enabled before and after the primary claim/complete loop respectively. Was a solution ever found, or did you move towards another solution? Thanks for your time.
Reply all
Reply to author
Forward
0 new messages