UART bridge in FireSim

[Novo Bruna David]

Hi,

I'm interested in creating my own FireSim bridge and I'm currently examining the UART bridge as a reference. To be more specific, I'm running a basic code that calls the printf function to display text through the UART in a baremetal FireSim simulation of a Rocket core. 

I was expecting to find an implementation of the printf function using the UART registers. However, I seem to be unable to progress beyond the following syscall:

static uintptr_t syscall(uintptr_t which, uint64_t arg0, uint64_t arg1, uint64_t arg2)

{

volatile uint64_t magic_mem[8] __attribute__((aligned(64)));

magic_mem[0] = which;

magic_mem[1] = arg0;

magic_mem[2] = arg1;

magic_mem[3] = arg2;

__sync_synchronize();

tohost = (uintptr_t)magic_mem;

while (fromhost == 0)

;

fromhost = 0;

__sync_synchronize();

return magic_mem[0];

}

I also used TraceV to inspect the FireSim simulation hoping that the execution would jump to a different code segment after the syscall. However, it seems that the UART manipulation happens while the core is busy waiting here:

while (fromhost == 0)

;

Is it possible that a proxy kernel is used in baremetal? Where can I find baremetal code that uses the UART base address 0x5400000 as configured in PeripheralFragments.scala?

Best,

d.

[Jerry Zhao]

The bare metal binaries don’t use UART for prints. I don’t recall an example of a bare metal uart-test floating around, but it should be pretty straightforward to implement your own putchar that just writes into the TX register. 

-Jerry

--
You received this message because you are subscribed to the Google Groups "FireSim" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firesim+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firesim/01002eb8-5222-4ccb-bad8-316044c8d5d3n%40googlegroups.com.

[Novo Bruna David]

Hi Jerry,

Thanks for the confirmation. I've successfully implemented my own putchar and getchar functions using the UART (see below). 
However, it took some guessing and trial and error because I was missing the initial step of enabling both TX and RX control registers. Where can I find the information on UART address mapping? During chisel elaboration, I could see that the 0x54000000 -  0x54001000 address range is assigned to UART. However, I'm curious about how I could determine that I needed to configure the address 0x54000008 to a value of 0x01 in order to activate UART TX functionality. Where is this implemented? I fail to see anything related to those configuration registers in UARTBridge.scala.

[localhost] out: Generated Address Map

[localhost] out: 3000 - 4000 ARWX error-device@3000

[localhost] out: 4000 - 5000 ARW boot-address-reg@4000

[localhost] out: 10000 - 20000 R X rom@10000

[localhost] out: 110000 - 111000 ARW tile-reset-setter@110000

[localhost] out: 2000000 - 2010000 ARW clint@2000000

[localhost] out: 2010000 - 2011000 ARW cache-controller@2010000

[localhost] out: c000000 - 10000000 ARW interrupt-controller@c000000

[localhost] out: 10015000 - 10016000 ARW blkdev-controller@10015000

[localhost] out: 54000000 - 54001000 ARW serial@54000000

[localhost] out: 80000000 - 480000000 ARWXC memory@80000000

#include <stdio.h>

#include <string.h>

#include <stdbool.h>

#ifndef BAREMETAL

#include <sys/mman.h>

#endif

#define UART_REG_TXFIFO 0

#define UART_REG_RXFIFO 1

#define UART_REG_TXCTRL 2

#define UART_REG_RXCTRL 3

#define UART_TXCTRL_TXEN 0x1

#define UART_RXCTRL_RXEN 0x1

uint32_t* uart_base = (int *) 0x54000000;

void set_reg(uint32_t num, uint32_t val)

{

uart_base[num] = val;

}

static void uart_putc(char ch)

{

volatile uint32_t *uart = (volatile uint32_t*) uart_base;

while ((int32_t)(uart[UART_REG_TXFIFO]) < 0);

uart[UART_REG_TXFIFO] = ch;

}

int uart_getc()

{

uint32_t ch;

volatile uint32_t *uart = (volatile uint32_t*) uart_base;

do{

ch = uart[UART_REG_RXFIFO];

} while ((ch & 0xFF) <= 0);

return (int)ch;

}

int main (int argc, char * argv[]) {

#ifndef BAREMETAL

if (mlockall(MCL_CURRENT | MCL_FUTURE) != 0) {

perror("mlockall failed");

exit(1);

}

#endif

char ch;

/* Enable TX */

set_reg(UART_REG_TXCTRL, UART_TXCTRL_TXEN);

/* Enable Rx */

set_reg(UART_REG_RXCTRL, UART_RXCTRL_RXEN);

printf("We are starting...\n");

/* Test sending characters to the UART */

uart_putc('U');

uart_putc('A');

uart_putc('R');

uart_putc('T');

uart_putc(':');

uart_putc(' ');

uart_putc('H');

uart_putc('e');

uart_putc('y');

uart_putc('\n');

/* Test receiving characters from the UART */

printf("This is the character I received from the UART: |%c|\n", uart_getc());

printf("... and we are done.\n");

return 0;

}

[Novo Bruna David]

I've found the information I was looking for in ./sim/target-rtl/chipyard/generators/sifive-blocks/src/main/scala/devices/uart/UART.scala

I initially thought that the UARTBridge was replacing the UART, but it turns out it's actually interfacing with it. I still can't see how the two are connected. Could you please point me to the chisel code making that connection?

[Jerry Zhao]

The UART TXD/RXD ports are exposed through ChipTop through UARTPortIO. The UARTBridge.apply function should accept as input a UARTPortIO object, and instantiate the Bridge. 
You can look at the WithUARTBridge Bridge-attachment function in chipyard/generators/firechip/src/main/scala/BridgeBinders.scala.

-Jerry

To view this discussion on the web visit https://groups.google.com/d/msgid/firesim/026ede03-64a9-4fa1-a9ed-6826e684ba1cn%40googlegroups.com.

[Aleksandar Pajkanovic]

Hi David,

thanks for sharing your code - means a lot, as I am interested in bridges as well. I have a basic understanding that the bridge interfaces with the internal UART. I also (think I) understand how drivers work: they are part of the host side meaning that they will provide (say) a character that the Bridge sends through UART for the core to read it through its UART interface. 

So, my idea for now is to start with the simplest thing and just slowly build up. I thought I'd create something called AdderBridge - which is literally the same thing like UARTBridge, so just another bridge with the same UART interface connected to the very same UART bus, with the addition that it sums up two numbers that its driver sends and then pass on the sum to the core via UART. Managing to do so, I would confirm both the driver and bridge and communication between them and the core were successful.

For now, I've changed files:

• TargetConfigs.scala and BridgeBinders.scala in chipyard, and
• firesim_top.cc in firesim

Also, I've created files:

• AdderBridge.scala (literal copy of UARTBridge.scala, just replaced word UART inside with Adder, while keeping the UARTPortIO <== as I want this AdderBridge to connect with the rocket via UART, as well)
• adder.cc - literal copy of uart.cc, no changes for now
• adder.h - literal copy of uart.h, no changes for now

I see that UARTBridge uses file uart.c when being instantiated as a new Module, but I couldn't figure out what uart.c is this referring to (it is not uart.cc above, and there are multiple files called uart.c in firesim repo). Hence, I just did like:

val ep = Module(new AdderBridge(UARTParams(address = 0x20040000)))

as this field, address, in UARTParams is the only one without the default value. I chose that value arbitrarily, although my spidey-sense is telling me I shouldn't have... However, I am not sure what to base that choice on... so, could you help here, please?

Finally, in the firesim documentation, the Bridge Walkthrough page tells about Build-system modifications, mentions adding header/source files to driver.mk, but as far as I could tell the driver.mk file already includes everything there is in the firesim/sim/firesim-lib/src/main/cc/bridges <== and that is exactly the location of adder.cc and adder.h, so am I correct to assume there is nothing to do here?

I managed to get to the point where GoldenGate simulator gets compiled with AdderConfig into a binary, it even executes standard bmarks. However, there's no trace of creating adder during the elaboration phase (I see uart0 in terminal, but no adder at all) - so, please, what could I be missing here?

Also, any comments on the idea itself, i.e. am I even correct in my assumptions about the exercise itself - adding another UART-interfaced-bridge with a different name next to an existing UARTBridge, keeping the UARTPortIO, etc <== are this valid ideas? FInally, some explanations about ADDERBRIDGEMODULE_struct and ADDERBRIDGEMODULE_struct_guard and what do they actually do would be great, as well as regarding ADDERBRIDGEMODULE_0_PRESENT (these are all #ifdef, but where are they #define(d)?)

My point here is to understand the process of creating the most trivial bridge.

Thanks,

  Aleksandar

[Usman Zain]

The uart.c you mentioned is referring to the UARTParams. Here is what I understand: If you go to the definition of UARTPortIO in sifive-blocks, the 'c' variable in the parameter is of type UARTParams. In the apply function, the variable `uart` is of type UARTPortIO and hence, it access its 'c' variable. If you look at the definition of UARTBridge (AdderBridge, in your case) class, you will notice that it takes UARTParams as an argument. So in the apply function where we instantiate the UARTBridge(AdderBridge in your case), we pass it the UARTParams by accessing the c variable in UARTPortIO.

What do you mean by there is no trace of AdderBridge? From what I know, if the struct guard is present for AdderBridge in generated-src/f1/YOUR_DESIGN/firesim_consts.h file then your bridge has been compiled correctly and is present in the simulator.

[Aleksandar Pajkanovic]

Hi Usman,

thanks for the explanation regarding uart.c 

I think I got the UARTParam handled very well. You can take a look at my solution here. Please do and then comment - I'd be more than happy to hear any further comments.

Regarding the existence of AdderBridge, here's what I mean. During the elaboration phase, I see this:

[info] [0.008] Elaborating design...
L2 InclusiveCache Client Map:
        0 <= blkdev-tracker0
        1 <= serial
        2 <= Core 0 DCache
        3 <= Core 0 ICache

Interrupt map (2 harts 2 interrupts):
  [1, 1] => frontend
  [2, 2] => uart_0

/dts-v1/;

and again this:

Generated Address Map
                0 -      1000 ARWX  debug-controller@0

             3000 -      4000 ARWX  error-device@3000

            10000 -     20000  R X  rom@10000

          2000000 -   2010000 ARW   clint@2000000

          2010000 -   2011000 ARW   cache-controller@2010000

          c000000 -  10000000 ARW   interrupt-controller@c000000

         10015000 -  10016000 ARW   blkdev-controller@10015000

         54000000 -  54001000 ARW   serial@54000000

         80000000 - 480000000 ARWXC memory@80000000

Obviously I see there is the UART that existed in the SoC by defalt (the UARTBridge), but there is no the bridge I created (the AdderBridge). 

What I am worried about is the address. I assigned as follows:

val ep = Module(new AdderBridge(UARTParams(address = 0x20040000)))

Note I chose this address bluntly, I have no idea what I should choose really - is there a rule I should apply when choosing it?

As for FireSim-const.h (if that's the one you mean) - it does not show any traces of ADDERBRIDGEMODULE_0_substruct even though it was defined here as follows:

    #ifdef ADDERBRIDGEMODULE_0_PRESENT
    // Create an instance of the constructor argument (this has all of
    // addresses of the BridgeModule's memory mapped registers)
    ADDERBRIDGEMODULE_0_substruct_create;
    // Instantiate the driver; register it in the main simulation class
    add_bridge_driver(new adder_t(this, ADDERBRIDGEMODULE_0_substruct, 0));
    #endif

Does it require anything else to be registered?

Thanks,

  Aleksandar

[Aleksandar Pajkanovic]

Hi,

given that there was not much interaction regarding this question, I hope you don't mind me asking for help in the Stack overflow, as well. 

I'll make sure to provide the answer here as well, if reached overthere.

Thanks,

  Aleksandar