Connecting IOs of LazyModuleImps

[Todd Inglett]

I'm certainly not fully grokking how all the LazyModule stuff works, so maybe the answer to this question is (hopefully) way simpler than I think.

I have a few devices for which I need to interconnect IOs. But these devices are also using TileLink agents.   That means I need to structure them using LazyModule and LazyModuleImp.   It ends up like this if you'll excuse the ASCII art....

+-----------------------------+
| AggDevice                   |
+-----------------------------+
 Vec +------+ +------+ +------+
     |Dev   | |Dev   | |Dev   |
     +------+ +------+ +------+

+-----------------------------+
| AggDeviceImp                |
+-----------------------------+
 Vec +------+ +------+ +------+
     |DevImp| |DevImp| |DevImp|
     +------+ +------+ +------+

The only reason AggDevice is there was my assumption it would help me interconnect the IOs.    But, I think what I really have is more like this:

+-----------------------------+
| AggDevice                   |
+-----------------------------+
+-----------------------------+
| AggDeviceImp                |
+-----------------------------+

 +------+  +------+  +------+
 |Dev   |  |Dev   |  |Dev   |
 +------+  +------+  +------+

 +------+  +------+  +------+
 |DevImp|  |DevImp|  |DevImp|
 +------+  +------+  +------+

That is, the AggDevice (and AggDeviceImp) isn't really helping at all.  For example, AggDeviceImp can't reach up to the AggDevice which has a vector of Dev which has a reference to DevImp in order to tie the DevImps IOs together.

Is there another approach?   Each Dev needs a TileLink agent so I think I need the Dev+DevImp setup.    I'm wondering if I need to learn about BundleBridge as that seems to be what other rocket-chip code is doing to interconnect these types of things.

Todd

[Abraham Gonzalez]

Hi,

I think you are trying to do something like this? (Note: There is probably a bit cleaner way to connect up the Diplomatic nodes… but I think this is sufficient?

class Dev(...) extends LazyModule {
  val node = // Some Diplomatic Node

  // This is DevImp(lementation) (instead of inlining the mod. implementation you could separate it into its own class)
  lazy val module = new LazyModuleImp {
    val io = IO(...) // some non-diplomatic IOs (of Type DevIO)

    // connecting node up to HW stuff + connecting the io up to stuff
  }
}

class AggDevice(...) extends LazyModule {
  val xbarNode = // Some Diplomatic Node

  val mods = Seq.fill(nMods){ LazyModule(new Dev(...)) }

  // can either connect the diplomatic IOs to something like a xbar
  mods.foreach { xbarNode := _.node }
  // or you can connect them up in the level above by doing something like the following the LazyModule above
  // aggDeviceInstance.mods.foreach { someSinkNode := _.node }

  // This is AggDeviceImp
  lazy val module = new LazyModuleImp {
    val io = IO(Vec(nMods, new DevIO)) // some non-diplomatic IOs

    io.zip(mods).map (case (sub_io, mod_io) => sub_io := mod_io)
  }
}

You can definitely bundle up IOs from different levels… it just depends on what you want to do… I think my preference is to probably get rid of the AggDevice and just connect up the N devices in the level above. You shouldn't really need to learn about BundleBridges to do something like this.

Hope that helps.

--
You received this message because you are subscribed to the Google Groups "Chipyard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chipyard+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/chipyard/7300df1b-8140-45bd-b002-e24801502461n%40googlegroups.com.

--

Abraham J. Gonzalez

Electrical Engineering and Computer Science Ph.D. Student

The University of California at Berkeley

[Todd Inglett]

Abraham,

Yeah, that helps a lot -- in particular the bit about what I'm trying to do can be done :).  I just need to wrap my head around it.   I assume the diplomatic nodes and their connection is required to make this work so I have "diplomatic IOs".  Right now my Dev devices are diplomatically connected (if I'm using those words correctly) with the sbus, simply because I use TLManagerNode and I am connecting them to the sbus via a "CanHavePeripheryXYZ" trait for the BaseSubsystem.  But I also want them interconnected between each other -- or with an aggregate device if that's required to connect them (seems it is not).   Right now what has me puzzled is the "// Some Diplomatic Node" bit in your example.  I've tried messing with generic TLXbar and TLIdentity() just for grins, but I don't seem to be getting anywhere.  I know I have a huge hole in my knowledge here, and perhaps I have more background reading to do.

Todd

[Abraham Gonzalez]

Hi Todd,

Assuming you are talking about connecting them together non-diplomatically (i.e. kinda the DevIO thing in the other example), then you can just connect them up in the LazyModuleImp area. I adjusted the prior example to not pass out the non-diplomatic IOs of the Dev devices but instead connect them together.

class AggDevice(...) extends LazyModule {
  val xbarNode = // Some Diplomatic Node

  val mods = Seq.fill(nMods){ LazyModule(new Dev(...)) }

  // can either connect the diplomatic IOs to something like a xbar
  mods.foreach { xbarNode := _.node }
  // or you can connect them up in the level above by doing something like the following the LazyModule above
  // aggDeviceInstance.mods.foreach { someSinkNode := _.node }

  // This is AggDeviceImp
  lazy val module = new LazyModuleImp

 {
    // connect mod(i).io to mod(j).io s
    // you would have to set i/j here
    mod(i).io <> mod(j).io
  }
}

In this case, your DevIO would have to change (probably split in 2 parts: an input and output). But you could connect the Dev modules together.

If you are doing everything diplomatically (i.e. there is no DevIO), you could just have different diplomatic nodes in "Dev" for each purpose (i.e. one node to connect up to the SBUS eventually and one node to source/sink requests between other Dev modules). In the end, both are very possible… just depends on what you want to do… (I assume you are probably trying to do the DevIO case but I can expand on this one if need be).

Hope that helps.

To view this discussion on the web visit https://groups.google.com/d/msgid/chipyard/56854a48-6bb8-4cc3-b188-29101cc95122n%40googlegroups.com.

[Todd Inglett]

Abraham,

Yes, indeed that really did help!   Sometimes the biggest help of all is giving the confidence I'm really on the right path so I can start "seeing" through the error messages.   In some of my testing I see now I was really running into "dumb" issues like not using <> to connect Decoupled interfaces.  Well, yeah, I'll get an error but I kept focusing on the "current module" part of the error "Sink is unwriteable by current module", when the real issue is the "unwriteable" bit :(.  So I think I'm moving forward once more!

I also see I don't really need the AggDevice because the BaseSubSystem periphery trait I am using could create the Devs, and the periphery Imp trait could do the IO connections.  But I think for now I'll keep the aggregate device in case I find it useful for something (maybe debug IOs).

Again...thanks for the help.  I'm off the ground again!

Todd

[Todd Inglett]

I'm back into this, but now am integrating into a larger effort.  This includes RoCC, and therefore one end of this I/O is based in a tile-attached module.  When I try to connect my IOs in this case I'm getting an error message like this:

[error] chisel3.internal.ChiselException: Connection between sink (Bool(IO test_devout_ready in TestRoCC)) and source (Bool(false)) failed @: Sink or source unavailable to current module.

This kind of error is, I think, what led me astray in my previous questions in this thread.   Is there some sort of parent-child module relationship that chisel needs for IOs between modules?   I'm just trying to understand what "unavailable" might mean when connecting IOs between modules.  I maybe need to give more context for the question, but before posting a bunch of scala I figure there may be something I need to go read/learn.

Todd

[Jiuyang Liu]

I skim the mail list and think we have these solutions:

1. Use BundleBridge to turn non-diplomatic IO into diplomatic IO

2. Create a SinkNode(or SourceNode) to turn diplomatic IO into non-diplomatic IO

Regards

Jiuyang 

To view this discussion on the web visit https://groups.google.com/d/msgid/chipyard/2be22825-8b70-49f8-b265-52483e424a2dn%40googlegroups.com.

[Todd Inglett]

Jiuyang,

Thanks!  I was thinking I needed to use BundleBridge for this, but I'm having a difficult time finding a simple example of its use.   The module causing me trouble is my RoCC module because it is created by other tile modules out of my control.  I have other modules of my own that are built later in the build that need to connect to  a simple IO in each RoCC module.  I was hoping each RoCC instance could somehow declare an IO that the other module could connect to.

Todd