chisel 5.0.0-RC1 and chiseltest

[Jake Taylor]

I'm attempting to upgrade a chisel 3.6.0 project to 5.0.0-RC1, and according to the docs it looks as though chiseltest is not recommended for new projects. On a hunch I did also try to add a dependency on `org.chipsalliance:chiseltest` (not knowing any better, just updating the previous version), and ofc that didn't work.

What is the recommended way to upgrade such a project? Are there any helpful docs on the subject that I've missed (seeing as I haven't found any)?

P.S. I've also asked this question on StackOverflow as suggested in the "create issue" form on GitHub, but this communication channel seems to be more active.

[Kevin Laeufer]

Hi Jake,

I managed to get chiseltest working with chisel 5.0.0 today. All our
tests are passing, but there might be new chisel 5 features that we
haven't tested yet.

libraryDependencies += "edu.berkeley.cs" %% "chiseltest" % "5.0-SNAPSHOT"

This is the version that is compatible with chisel 5.0.0 (the stable
release). I hope to make a stable chiseltest release in the future, but
for now you have to use the snapshot.

We would appreciate bug reports with small examples if you encounter
chisel 5 features that are not supported properly in chiseltest or if
there are any regressions. https://github.com/ucb-bar/chiseltest/issues

Best,
Kevin

On 6/9/23 02:11, Jake Taylor wrote:
> I'm attempting to upgrade a chisel 3.6.0 project to 5.0.0-RC1, and
> according to the docs

> <https://github.com/chipsalliance/chisel#build-your-own-chisel-projects>

> it looks as though chiseltest is not recommended for new projects. On a
> hunch I did also try to add a dependency on
> `org.chipsalliance:chiseltest` (not knowing any better, just updating
> the previous version), and ofc that didn't work.
>
> What is the recommended way to upgrade such a project? Are there any
> helpful docs on the subject that I've missed (seeing as I haven't found
> any)?
>
> P.S. I've also asked this question on StackOverflow

> <https://stackoverflow.com/questions/76432675/chisel-5-0-0-rc1-and-chiseltest> as suggested in the "create issue" form on GitHub, but this communication channel seems to be more active.
>
> *CONFIDENTIALITY NOTICE* -- This email is intended only for the
> person(s) named in the message header. Unless otherwise indicated, it
> contains information that is confidential, privileged and/or exempt from
> disclosure without written approval by Ascenium Inc.
>
> --
> You received this message because you are subscribed to the Google
> Groups "chisel-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to chisel-users...@googlegroups.com
> <mailto:chisel-users...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/chisel-users/77eb58d7-29de-449d-8fbf-def7080fc662n%40googlegroups.com <https://groups.google.com/d/msgid/chisel-users/77eb58d7-29de-449d-8fbf-def7080fc662n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Martin Schoeberl]

Hi Kevin,

this is very good news! Does this also include multithreaded testing? Does it include your Chisel formal work? I would like to include a bit Chisel formal in teaching next year (spring 2024).

Cheers,
Martin

> To unsubscribe from this group and stop receiving emails from it, send an email to chisel-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/chisel-users/50340ee5-d2cd-ffda-e3aa-4fe6a84ff6df%40berkeley.edu.
>

[Kevin Laeufer]

Hi Martin,

There are no feature regressions from the previous chiseltest version.
Which means that the multi-threaded testbench API and the formal
verification is all working.

Upgrading to Chisel 6 might be a bit harder though since the number of
new features that need to be supported is much higher.

- Kevin

[Martin Schoeberl]

Hi Kevin,

this is great news! How come that you managed to get formal (and multithreaded testing) into Chisel 5. AFAIK Treadle is dead for the Circt/MLIR based Chisel backend.

Martin

> To view this discussion on the web visit https://groups.google.com/d/msgid/chisel-users/0434f7da-26ff-c7b9-8bd5-60c2ab94eeec%40berkeley.edu.
>

[Kevin Laeufer]

Hi Martin,

Treadle is now part of chiseltest [0]. Fortunately Chisel produces
standard compliant FIRRTL which can then be fed into the Berkeley fork
of the firrtl compiler [1]. This made it relatively easy to get
chiseltest working with Chisel 5, however, it does pose a little bit of
a challenge, because as the FIRRTL spec is expanded, we need to add
support for these new features to the firrtl2 compiler.

- Kevin



[0]: https://github.com/ucb-bar/chiseltest/tree/main/src/main/scala/treadle2
[1]: https://github.com/ucb-bar/firrtl2

[Schuyler Eldridge]

Thanks for extending the life of Chiseltest here, Kevin. This helps
provide continuity from Chisel 3.6 to Chisel 5 and provide some
additional runway for migration.

This will only work for Chisel v5, however. Chisel v5's only major
change was the removal of the SFC (Scala-based FIRRTL compiler) and
requiring all FIRRTL code to be compiled through CIRCT (the MLIR-based
FIRRTL compiler/MFC). The SFC was then put into "maintenance mode" and
is not intended to receive further updates. Forking the SFC into
ucb-bar/firrtl2 and moving Treadle into Chiseltest will then work, but
only for Chisel v5.

Major changes in newer, released versions of the FIRRTL specification
used by Chisel v6 will cause problems with this approach.

> To view this discussion on the web visit https://groups.google.com/d/msgid/chisel-users/da439ada-3169-95da-dc59-db3d92887db9%40berkeley.edu.

[Jake Taylor]

Hmm, seems my response to this was lost in the void somehow? I can see it in my email outbox, but not in the google groups page... strange. Anyways, copy/paste below:

Thanks for the info, Schuyler.

It seems to me, from your response as well as reading docs, that chiseltest is not and will not be the recommended tool for testing/verifying chisel designs in the future. However, it is not clear to me what the recommended alternative(s) is/are. Perhaps you could shed some light on this?

Thanks,

Jake

[Jake Taylor]

Ah, it seems another one of my emails was also lost in the void/outbox, pasting below:

Hi Kevin,

Thanks for the heads-up, and great timing on merging that! I've updated dependencies to use chisel `5.0.0` (not `-RC1`) and chiseltest `5.0-SNAPSHOT`, which both appear to resolve correctly.

However, when running tests, I hit the following exception:

[info]   scala.NotImplementedError: TODO: convert InlineAnnotation(ModuleName(<omitted>))

[info]   at chiseltest.simulator.ChiselBridge$.convert(ChiselBridge.scala:127)

[info]   at chiseltest.simulator.ChiselBridge$.$anonfun$annosToState$2(ChiselBridge.scala:87)

[info]   at scala.collection.immutable.List.map(List.scala:246)

[info]   at scala.collection.immutable.List.map(List.scala:79)

[info]   at chiseltest.simulator.ChiselBridge$.annosToState(ChiselBridge.scala:87)

[info]   at chiseltest.simulator.ChiselBridge$.elaborate(ChiselBridge.scala:67)

[info]   at chiseltest.simulator.Compiler$.elaborate(Compiler.scala:15)

[info]   at chiseltest.internal.BackendExecutive$.start(BackendExecutive.scala:19)

[info]   at chiseltest.defaults.package$.createDefaultTester(defaults.scala:21)

I can look into making a minimal (shareable) reproduction, but in the meantime, maybe the above is enough? And/or should I report in another channel?

EDIT: I see now that you posted a link to the github issue tracker, so I'll follow up there.

Thanks,

-Jake

[Schuyler Eldridge]

There is a temporarily less featureful replacement called `svsim`
which is shipped with Chisel v5. (See:
https://github.com/chipsalliance/chisel/pull/3121) The Chisel Template
and other resources need to be updated to use this.

`svsim` is much lower level than Chiseltest and is closer to
infrastructure on top of which libraries like Chiseltest can be built
than a full Scala environment for testing the outputs of Chisel
generators. I.e., it is focused on feeding data to and checking data
from some Verilog running on VCS or Verilator in a way that is
agnostic to the fact that Chisel exists. This is sufficient for simple
tests in Scalatest like shown here:
https://github.com/chipsalliance/chisel/pull/3121/files#diff-cb8ffbae50d3e50b4df2fbb6d2fe814ed46d49c57e082e65539bef60e3d99496R60
However, the more powerful testing methods (e.g., dequeing data from a
decoupled, one-line bounded model checking) are note supported.

Chiseltest's existing implementation, specifically for formal tests,
is a maintenance issue as it is very closely coupled to the
Scala-based FIRRTL compiler (SFC). Kevin, who is the sole, part-time
maintainer of Chiseltest, is keeping things going as best he can.
However, there is insufficient bandwidth to keep up with the flood of
activity on the Chisel/CIRCT side of things.

This was discussed in the Chisel Dev meeting yesterday and the
conclusion was to get more of the Chiseltest APIs moved to svsim.

There is preliminary work in both Chisel and CIRCT that will
eventually enable better Chisel verification support. Support for SVAs
and properties is coming
(https://github.com/chipsalliance/chisel/pull/3337) and native XMR
support is already available in v6. There is planned work to enable
"groups" which lower to SystemVerilog binds to enable a more standard
SystemVerilog verification flow
(https://github.com/chipsalliance/firrtl-spec/pull/108). On the CIRCT
side there is new tooling/infrastructure like circt-lec (which only
works for combinational logic at the moment), arc dialect (for
compiled simulation from CIRCT), and planned other efforts (e.g.,
directly interpreting arc dialect avoid compilation).

Sorry for the churn.

> CONFIDENTIALITY NOTICE -- This email is intended only for the person(s) named in the message header. Unless otherwise indicated, it contains information that is confidential, privileged and/or exempt from disclosure without written approval by Ascenium Inc.

>
> --
> You received this message because you are subscribed to the Google Groups "chisel-users" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to chisel-users...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/chisel-users/64aba130-795e-4418-8aa1-7217ab320ea1n%40googlegroups.com.

[Jake Taylor]

Hi Kevin,

Thanks for the heads-up, and great timing on merging that! I've updated dependencies to use chisel `5.0.0` (not `-RC1`) and chiseltest `5.0-SNAPSHOT`, which both appear to resolve correctly.

However, when running tests, I hit the following exception:

[info]   scala.NotImplementedError: TODO: convert InlineAnnotation(ModuleName(<omitted>))

[info]   at chiseltest.simulator.ChiselBridge$.convert(ChiselBridge.scala:127)

[info]   at chiseltest.simulator.ChiselBridge$.$anonfun$annosToState$2(ChiselBridge.scala:87)

[info]   at scala.collection.immutable.List.map(List.scala:246)

[info]   at scala.collection.immutable.List.map(List.scala:79)

[info]   at chiseltest.simulator.ChiselBridge$.annosToState(ChiselBridge.scala:87)

[info]   at chiseltest.simulator.ChiselBridge$.elaborate(ChiselBridge.scala:67)

[info]   at chiseltest.simulator.Compiler$.elaborate(Compiler.scala:15)

[info]   at chiseltest.internal.BackendExecutive$.start(BackendExecutive.scala:19)

[info]   at chiseltest.defaults.package$.createDefaultTester(defaults.scala:21)

I can look into making a minimal (shareable) reproduction, but in the meantime, maybe the above is enough? And/or should I report in another channel?

Thanks,

-Jake

To view this discussion on the web visit https://groups.google.com/d/msgid/chisel-users/da439ada-3169-95da-dc59-db3d92887db9%40berkeley.edu.

CONFIDENTIALITY NOTICE -- This email is intended only for the person(s) named in the message header. Unless otherwise indicated, it contains information that is confidential, privileged and/or exempt from disclosure without written approval by Ascenium Inc.

[Jake Taylor]

Thanks for the info, Schuyler.

It seems to me, from your response as well as reading docs, that chiseltest is not and will not be the recommended tool for testing/verifying chisel designs in the future. However, it is not clear to me what the recommended alternative(s) is/are. Perhaps you could shed some light on this?

Thanks,

Jake

To view this discussion on the web visit https://groups.google.com/d/msgid/chisel-users/CAG%3DZR1uuUeds67UZ8zRV9ihd8FRQhU%3DbZ53CWnjkB3jPA61v_Q%40mail.gmail.com.

CONFIDENTIALITY NOTICE -- This email is intended only for the person(s) named in the message header. Unless otherwise indicated, it contains information that is confidential, privileged and/or exempt from disclosure without written approval by Ascenium Inc.

[Jake Taylor]

Hi Schuyler,

Thanks again for the info. svsim (and co) are indeed interesting. Looking forward to when this matures a bit and we can start migrating to use it.

Thanks again (and sorry again for the email noise, no idea what's happened there),

-Jake