Recursive module instantiation

[Saltuk Akgül]

Hi everyone,

I'm working on a module that recursively instantiates itself with smaller port sizes. After a certain number of recursion levels, the compilation fails with a java.lang.OutOfMemoryError. The logic inside each module isn't complex. It's mostly instantiating child modules and wiring them up. My suspicion is that the compiler is performing aggressive global optimizations across the entire hierarchy, which overwhelms memory. Is there a way to prevent this behavior and make the compiler treat each child module more like a black box, preserving the hierarchy and focusing only on the connections?

Best Regards,
Saltuk

[Jack Koenig]

Howdy Saltuk,

How deep of recursion are we talking here? It depends on what version of Chisel you are using, but assuming you're on a newer version with the CIRCT FIRRTL compiler, almost no optimizations are performed at Chisel time so you would only see an OutOfMemoryError just from the size of your design filling up the memory allocated to the Chisel run. You can increase memory to the JVM but it depends on how you're using Chisel for how you should do that.

You can increase memory in SBT (https://www.scala-sbt.org/1.x/docs/Troubleshoot-Memory-Issues.html#Troubleshoot+memory+issues) or in Mill (https://mill-build.org/mill/cli/flags.html#_running_mill_with_custom_jvm_options)

 If you can create a reproduction of your failure with Scala CLI (https://www.chisel-lang.org/docs/installation#quickstart-with-scala-cli) that would make it easier to advise on what you should try.

~Jack

--
You received this message because you are subscribed to the Google Groups "chisel-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chisel-users...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/chisel-users/58554474-b010-4255-810e-4aadb15bc1f5n%40googlegroups.com.

[Saltuk Akgül]

Hi Jack,
Thanks for the suggestions. I tried increasing the memory but it didn't solve the problem. What I was hoping for is basically making each module independent from each other and independent of recursion level since it is just one variable (port size) that changes in actual verilog. I solved the problem by using BlackBox class and overriding desiredName of the Module in Chisel. Here is the pseudocode for the solution I used:

class InitialModule extends module {
// some registers and logic
}

class MyModuleBlackBox(val size: Int) extends BlackBox {
override val desiredName = s"MyModule_$size"

val io = IO(new Bundle {
    val clock = Input(Clock())
    val reset = Input(Bool())
    val in    = Input(UInt(size.W))
    val out   = Output(UInt(size.W))
    })
}

class MyModule(val size: Int) extends Module {
override val desiredName = s"MyModule_$size"

val io = IO(new Bundle {
    val clock = Input(Clock())
    val reset = Input(Bool())
    val in    = Input(UInt(size.W))
    val out   = Output(UInt(size.W))
    })

    if (size == 4) {
    val subModules = Seq.fill(4)(Module(new InitialModule()))
    for (i <- 0 until 4) {
      subModules(i).io.in := ...
      // just some connections between subModules and io of this module
    }
    } else {
    val subModules = Seq.fill(4)(Module(new MyModuleBlackBox(size/4)))
    for (i <- 0 until 4) {
      subModules(i).io.clock := clock
      subModules(i).io.reset := reset.asBool
      subModules(i).io.in := ...
      // just some connections between subModules and io of this module
    }
    }
}

object MyModuleRecursiveInit extends App {

val max_width = 1024
for (size <- Iterator.iterate(4)(x => x * 2).takeWhile(_ <= max_width)) {
    ChiselStage.emitSystemVerilogFile(
      new MyModule(size),
      firtoolOpts = Array(
        "-disable-all-randomization",
        "-strip-debug-info"
      )
}
}

I was wondering if there is a compiler option for that. I cannot post the exact code that caused the error, but I can create a similar chisel code and reproduce failure with Scala CLI if you think that is still important for the chisel community.
Best Regards,
Saltuk

7 Mayıs 2025 Çarşamba tarihinde saat 01:14:08 UTC+2 itibarıyla Jack Koenig şunları yazdı:

[ghut...@gmail.com]

I've used recursion in multiple occasions, if you're building something which is feasible to implement it normally works fine.  The first thing to check, as with all recursive algorithms, is that your recursion terminates correctly, and normally it shouldn't be too deep -- if you have 1024 inputs, you should only end up with 10 levels of hierarchy.

One issue could be that Chisel is making a large number of unique objects, so if you are processing 1024 elements as a binary tree, by the time you get to the bottom of your tree, you have 512 N=2 instances, all of which are unique objects.  You could potentially fix this by using the Definition/Instance mechanism, and have a factory class to provide instances.  The first time you call the factory with, say, N=4, it creates a Definition of your class with a N=4 parameter, and returns an Instance.  Each subsequent time it's called with the same parameters, it simply returns a new Instance.  

Guy