ucb-bar / chiseltest Goto Github PK
View Code? Open in Web Editor NEWThe batteries-included testing and formal verification library for Chisel-based RTL designs.
License: Other
The batteries-included testing and formal verification library for Chisel-based RTL designs.
License: Other
Should return a Bundle literal.
Implicit clock resolution works on a branch of firrtl and testers2 but code that depends on those features will fail to compile on master. Add a stub method so that tests will compile, though tests depending on that feature may exception out.
The TestAdapters
suggest to use implicit conversion to DecoupledDriver
https://github.com/ucb-bar/chisel-testers2/blob/ea5e81f2e54f2df4d5cd34d8e7ece2a69fab0f86/src/main/scala/chisel3/tester/TestAdapters.scala#L9
It woks fine for DecoupledIO
. However, it does not work for IrrevocableIO
. I have to force the type conversion to DecoupledIO
:
it should "run implicit convert" in {
test(new Module{
val io = IO( new Bundle{
val in = Flipped(Irrevocable(UInt(8.W)))
val out = Irrevocable(UInt(8.W))
})
}).withAnnotations(Seq(WriteVcdAnnotation)) { c =>
Decoupled(c.io.in).initSource.setSourceClock(c.clock)
}
}
Recent chisel3 changes (chipsalliance/chisel#1077) cause compilation errors for testers2 backends that use chisel3.stage.ChiselGeneratorAnnotation()
:
[info] Compiling 29 Scala sources to .../testers2/target/scala-2.11/classes ...
[error] .../testers2/src/main/scala/chisel3/tester/backends/treadle/TreadleExecutive.scala:30:49: value circuit is not a member of firrtl.AnnotationSeq
[error] val circuit = generatorAnnotation.elaborate.circuit
[error] ^
and
[error] .../testers2/src/main/scala/chisel3/tester/legacy/backends/verilator/VerilatorExecutive.scala:46:49: value circuit is not a member of firrtl.AnnotationSeq
[error] val circuit = generatorAnnotation.elaborate.circuit
[error] ^
Elboration can be quite time consuming, so this slows down Verilator as a backend significantly.
Here's a pull request with a test that reproduces the problem and fails if it happens: #98
Discussed here:
https://groups.google.com/forum/#!topic/chisel-users/31NHUTKm8gw
FIRRTL compile and Verilator only runs once though.
This test passes for Treadle, but fails for Verilator.
$ sbt "test:testOnly *SingleElaboration*"
[deleted]
[info] SingleElaborationTests:
[info] elaboration should happen once
[info] [0.004] Elaborating design...
[info] [0.124] Done elaborating.
[info] [0.000] Elaborating design...
[info] [0.009] Done elaborating.
HACK! new phases.WriteOutputAnnotations to save memory while waiting for Chisel 3.2.0 memory consumption fix
Total FIRRTL Compile Time: 631.7 ms
[deleted]
And by extension vcs, etc.
Are there any plans for dumping verilog testbench? It is always needed for post-simulation.
I think the VerilogTbDump
implementation in https://github.com/ucb-bar/dsptools/blob/master/src/main/scala/dsptools/tester/VerilogTbDump.scala is good enough. Any plans to include such dsptools in the testers packages?
printf
does not show any output in the recently committed (legacy/experimental) verilator back-end.
Java/Scala seems to cut the exception off before there's a line number of the step() printed, so I don't know which step() that timed out
[info] - should write to main memory *** FAILED *** (1 minute, 43 seconds)
[info] chisel3.tester.TimeoutException: timeout on Clock(IO clock in ThingAMaBob) at 1000 idle cycles
[info] at chisel3.tester.backends.treadle.TreadleBackend.$anonfun$run$5(TreadleBackend.scala:175)
[info] at chisel3.tester.backends.treadle.TreadleBackend.$anonfun$run$5$adapted(TreadleBackend.scala:172)
[info] at scala.collection.mutable.HashMap.$anonfun$foreach$1(HashMap.scala:149)
[info] at scala.collection.mutable.HashTable.foreachEntry(HashTable.scala:237)
[info] at scala.collection.mutable.HashTable.foreachEntry$(HashTable.scala:230)
[info] at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:44)
[info] at scala.collection.mutable.HashMap.foreach(HashMap.scala:149)
[info] at chisel3.tester.backends.treadle.TreadleBackend.run(TreadleBackend.scala:172)
[info] at chisel3.tester.internal.Context$.$anonfun$run$1(Testers2.scala:85)
[info] at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
[info] ...
There are a lot of useful printing features inside ScalaTest that are currently not usable inside a TestBuilder
environment:
Given
/When
/Then
of GivenWhenThen
info
of Informing
feature
/scenario
of FeatureSpec
variantsThis limits the ability of a user to write a descriptive test that provides ScalaTest-reported output about what the test is doing. Users can always rely on println
(which does work). However, println
isn't integrated with the ScalaTest environment in any way and doesn't expose at what point in a test something failed.
As one example, I'd like the following to be supported:
behavior of "Foo"
it should "bar" in {
def test(new Foo) { c =>
Given("a Foo initialized via reset for 4 cycles")
c.doReset(4)
When("a sequence is enqueued")
c.in.enqueSeq(Seq(1, 2, 3))
Then("the same sequence should be dequeued")
c.out.expectDequeueSeq(Seq(1, 2, 3))
}
}
Currently, this will just swallow the Given
/When
/Then
and not print them.
See: http://doc.scalatest.org/3.1.0/org/scalatest/GivenWhenThen.html
This is related to some comments I made in #68, but is really a disjoint issue.
This would have equivalent arguments to what is supported by expect. Could provide better debugging information on timeouts, and could be more efficient in simulation since threads can be put to sleep.
In the future this API could be expanded with BitPat matches and stuff.
ZIO? Akka?
I've heard that Scala prefers Monads / etc for the problems that coroutines solve. Need to do more research into options to speed up the concurrency.
This is a feature request.
It would be nice if testers2 was integrated with a waveform viewer in some way. My current workflow is something that I think could be automated and integrated with ChiselTest.
test(new Foo).withAnnotations(Seq(WriteVcdAnnotation)) { dut =>
dut.io.bar.poke(1.U)
dut.clock.step(1)
}
Postprocess the VCD file to generate a GTKWave save file that (1) adds non-temporary signals and (2) groups signals by module. I currently do this by running GTKWave with a TCL script, but this could just as easily be done using Scala and likely more intelligently than processing the VCD header.
Open GTKWave loading the save file to get something like the following:
(Signal names are from something I'm working on and don't match the code example above.)
I think this is entirely automatable behind some viewWaveform
method that could be added to ChiselTest. Modifying the example above:
test(new Foo).withAnnotations(Seq(WriteVcdAnnotation)) { dut =>
dut.io.bar.poke(1.U)
dut.clock.step(1)
dut.viewWaveform()
}
At minimum, this could blindly run through steps 1--3 above.
At best it could reload the waveform associated with that specific test or create one if it doesn't yet exist.
For the minimum implementation, this is just encapsulating the current workflow above into a method. This shouldn't be hard to do and could, at further minimum, use the scripts I already have.
For the best approach, ChiselTest would need to maintain some global state associated with each test and some Option[Pipe]
that connects to the TCL console of that test's waveform viewer. GTKWave supports a TCL console (that you can startup with -w
/--wish
) which can be used for things like reloading, etc. It would also be tractable to instead of using a script to setup the viewer, the TCL console could be used entirely.
Appendix E of the GTKWave manual documents the available TCL commands.
The major benefit of something like this is that it doesn't involve rolling a custom waveform viewer like has been talked about in the past. GTKWave is totally open source (GPLv2), so there's no barrier to entry for using it with ChiselTest.
The one difficulty I've had is finding documentation of the save file format (outside of the source code). Granted, this isn't too complicated and the existing effort to script this out has discovered most of the interesting options that are needed.
This arose out of support for -DwriteVcd=1
in ScalaTest being broken, and whether that breakage was intentional or not. At the chisel-dev meeting, this turned into a longer discussion about configurations, in particular addressing #65.
There seems to be agreement about using annotations as the internal mechanism for representing configurations, including (as currently supported) dumping VCDs and simulator backend.
The overall design requirements are:
That being said, there was a debate on what the interface should look like.
Under this proposal, there would be a configuration file, probably in the resources folder, formatted as annotations command line syntax, that would be read in to provide default annotations for all tests.
Pros:
.withAnnotations(...)
.Cons:
I personally think the cons outweigh the pros here, especially since the configuration is somewhat implicit.
This proposal provides both a temporary mechanism (akin to the debug printf - something that isn't pushed to version control) for changing global configurations, and a permanent mechanism (something that does get pushed to version control) for sharing annotations across multiple test classes while remaining in Scala.
Basically, bring back (and possibly expand) the -DwriteVcd=1
option that can be passed to ScalaTest in sbt.
Pros:
Cons:
.withAnnotations(...)
.
Currently, tests are largely written with
class BasicTest extends FlatSpec with ChiselScalatestTester { ...
This proposal would allow subclassing ChiselScalatestTester
with annotations, for example:
class MyTestConfig extends ChiselScalatestTester(annos=Seq(writeVcdAnnotataion))
...
class BasicTest extends FlatSpec with MyTestConfig { ...
Potential solution for #65.
Pros:
MyTestConfig
in the unittest class hierarchy provides a dead giveaway of some non-default configuration being applied, and IDEs can furthermore navigate to that link.withAnnotations(...)
, managed by existing infrastructure like sbt.Cons:
Still up for debate how to run multiple backends. Should this be a responsibility of the build system (and above the ChiselTest level)? Or should there be an annotation that runs a single test multiple times under different backends? Should all tests be written such that they can run multiple times in the same JVM invocation? How would this coexist with proposals to avoid multiple elaboration?
The treadle backend calls the report method which has a field for number of tests (i.e. expects) that succeeded. But because testers2 does not use treadle's expect this number is always zero. This is confusing to new users. Simple fix would be to remove the report call, but I think it would be warmer and fuzzier to give some actual stats.
This would be nice for other backends, so it should probably be handled at the testers2 level and not within the specific backend.
https://github.com/freechipsproject/chisel-template master will complain
symbol 'type chisel3.experimental.MultiIOModule' is missing from the classpath
The ready valid controllers I use have combinational paths through them. The decision on whether to perform a transaction depends on the valid and ready signals supplied to the controller in that cycle.
To test such a controller, our test benches check the combinationally generated output of the controller to also decided which transactions are to be performed. So there is a sequence like this:
What is the best way to write testbenches for these controllers?
Here is what I tried to do but fails because there are peeks in threads after there are pokes.
There are some more comments below the code.
package examples.testers2
import org.scalatest._
import chisel3.tester._
import chisel3._
import chisel3.util._
import scala.util.Random
class Join extends Module {
val io = IO( new Bundle {
val inp0 = Flipped( DecoupledIO( UInt(16.W)))
val inp1 = Flipped( DecoupledIO( UInt(16.W)))
val out = DecoupledIO( UInt(16.W))
})
io.inp0.nodeq
io.inp1.nodeq
io.out.noenq
when ( io.inp0.valid && io.inp1.valid && io.out.ready) {
io.out.bits := io.inp0.bits + io.inp1.bits
io.inp0.ready := true.B
io.inp1.ready := true.B
io.out.valid := true.B
}
}
class Tie extends Module {
val io = IO( new Bundle {
val inp = Flipped( DecoupledIO( UInt(16.W)))
val out = DecoupledIO( UInt(16.W))
})
io.inp.nodeq
io.out.noenq
when ( io.inp.valid && io.out.ready) {
io.out.bits := io.inp.bits
io.inp.ready := true.B
io.out.valid := true.B
}
}
import chisel3.internal.firrtl.{LitArg, ULit, SLit}
object AlternativeTestAdapters {
class RandomFlip( seed : Int, val num : Int, val den : Int) {
val rnd = new Random( seed)
def nextBoolean() : Boolean = {
val r = rnd.nextInt(den) < num
println( s"Random flip: ${r}")
r
}
}
val rflip = new RandomFlip( 47, 12, 16)
class BaseAdapter( clk: Clock, timeout : Int) {
var nsteps = 0
def step() : Unit = {
if ( timeout > 0 && nsteps >= timeout) throw new Exception( s"Exceeded clock tick limit (${timeout}).")
clk.step(1)
nsteps += 1
}
}
class ReadyValidSource[T <: Data](x: ReadyValidIO[T], clk: Clock, tossCoin : () => Boolean, timeout : Int = 0) extends BaseAdapter( clk, timeout) {
x.valid.poke(false.B)
def enqueue(data: T): Unit = timescope {
while ( true) {
if ( tossCoin()) {
x.bits.poke(data)
x.valid.poke(true.B)
while ( x.ready.peek().litToBoolean == false) {
step()
}
step()
return
}
step()
}
}
def enqueueSeq(data: Seq[T]): Unit = timescope {
for (elt <- data) {
enqueue(elt)
}
}
}
class ReadyValidSink[T <: Data](x: ReadyValidIO[T], clk: Clock, tossCoin : () => Boolean, timeout : Int = 0) extends BaseAdapter( clk, timeout) {
x.ready.poke(false.B)
def dequeueExpect(data : T): Unit = {
while ( true) {
if ( tossCoin()) {
x.ready.poke(true.B)
while ( x.valid.peek().litToBoolean == false) {
step()
}
x.bits.expect(data)
step()
return
}
step()
}
}
def dequeueExpectSeq(data: Seq[T]): Unit = timescope {
for (elt <- data) {
dequeueExpect(elt)
}
}
}
}
import AlternativeTestAdapters._
class TieTestProbablyWrong extends FlatSpec with ChiselScalatestTester {
val rnd = new Random()
behavior of "Testers2 with Tie"
it should "work with a tie" in {
test( new Tie) { c =>
val INP = IndexedSeq.fill( 100){ BigInt( 16, rnd)}
val OUT = INP
val source = new ReadyValidSource( c.io.inp, c.clock, rflip.nextBoolean)
val sink = new ReadyValidSink( c.io.out, c.clock, rflip.nextBoolean, 2000)
fork {
source.enqueueSeq( INP map (_.U))
}
sink.dequeueExpectSeq( OUT map (_.U))
}
}
}
class JoinTestProbablyWrong extends FlatSpec with ChiselScalatestTester {
val rnd = new Random()
behavior of "Testers2 with Join"
it should "work with a join" in {
test( new Join) { c =>
val INP0 = IndexedSeq.fill( 100){ BigInt( 16, rflip.rnd)}
val INP1 = IndexedSeq.fill( 100){ BigInt( 16, rflip.rnd)}
val OUT = (INP0 zip INP1) map { case (x,y) => (x+y) & ((1<<16)-1)}
val source0 = new ReadyValidSource( c.io.inp0, c.clock, rflip.nextBoolean)
val source1 = new ReadyValidSource( c.io.inp1, c.clock, rflip.nextBoolean)
val sink = new ReadyValidSink( c.io.out, c.clock, rflip.nextBoolean, 2000)
fork {
source0.enqueueSeq( INP0 map (_.U))
}
fork {
source1.enqueueSeq( INP1 map (_.U))
}
sink.dequeueExpectSeq( OUT map (_.U))
}
}
}
I do like the way you can use implicit program counters in the threads so you don't need to write explicit state machines for random injection and extraction as well as waiting for the transaction to be performed.
So how do we make this work. Can we add a phase_step or some other action? Could it look like this:
object AlternativeTestAdapters {
...
class ReadyValidSource[T <: Data](x: ReadyValidIO[T], clk: Clock, tossCoin : () => Boolean, timeout : Int = 0) extends BaseAdapter( clk, timeout) {
x.valid.poke(false.B)
def enqueue(data: T): Unit = timescope {
while ( true) {
if ( tossCoin()) {
x.bits.poke(data)
x.valid.poke(true.B)
phase_step()
while ( x.ready.peek().litToBoolean == false) {
step()
}
step()
return
}
step()
}
}
def enqueueSeq(data: Seq[T]): Unit = timescope {
for (elt <- data) {
enqueue(elt)
}
}
}
class ReadyValidSink[T <: Data](x: ReadyValidIO[T], clk: Clock, tossCoin : () => Boolean, timeout : Int = 0) extends BaseAdapter( clk, timeout) {
x.ready.poke(false.B)
def dequeueExpect(data : T): Unit = {
while ( true) {
if ( tossCoin()) {
x.ready.poke(true.B)
phase_step()
while ( x.valid.peek().litToBoolean == false) {
step()
}
x.bits.expect(data)
step()
return
}
step()
}
}
def dequeueExpectSeq(data: Seq[T]): Unit = timescope {
for (elt <- data) {
dequeueExpect(elt)
}
}
}
}
Every tick is made up of two phases. "phase_step()" will pass the first phase. "step()" will pass the second phase (or both phases if the first hasn't been passed yet.)
This is more a question, not an issue.
When returning a Chisel type on peek(), they are not directly usable in Scala testing code, right? Example:
val x = port.rdData.peek
println("compare Chisel type: " + (x == 1.U))
println("compare with Scala conversion: " + (x.litValue() == 1))
Only the second compare works. Is this the only way (except expect()) to use the peek values?
Cheers,
Martin
While I know that zero-width I/Os don't really exist, it might be nice as syntactic sugar to be able to peek (or even dummy poke?) zero-width wires and have it return 0. This is especially useful in generators where parameters might be set in a way such that some wires end up with zero-width.
This might also be a treadle issue... @chick ?
Not super high priority since it can be worked around with some if statements.
Currently it fails with a slightly cryptic message:
java.lang.AssertionError: assertion failed: Error: getValue("zero_width_io_name") : argument is not an element of this circuit
at scala.Predef$.assert(Predef.scala:219)
at treadle.executable.ExecutionEngine.getValue(ExecutionEngine.scala:165)
at treadle.TreadleTester.peek(TreadleTester.scala:245)
at chisel3.tester.TreadleBackend.peekBits(TreadleBackend.scala:90)
at chisel3.tester.TreadleBackend.expectBits(TreadleBackend.scala:99)
at chisel3.tester.package$testableData.expectWithStale(package.scala:72)
at chisel3.tester.package$testableData.expect(package.scala:91)
Sometimes it is very useful to be able to set options (notably VCD dumping) programmatically as opposed to from the command line. Right now there isn't an easy way to programmatically set it since TesterOptions
is private. It can be in an experimental or some other not-fully-supported API.
Is tester2
binding to ScalaTest
?
If I wanna use other framework like utest
, what should I suppose to do? Adding ChiselUtestTester
like ChiselScalatestTester
did?
Not even sure how this would be done, or even what the syntax would look like, but it might make sense to initialize things like boundary ReadyValid IOs implicitly.
Placeholder issue for constrained random testing.
This would check some signal holds for the duration of the enclosing timescope. Checks are made between timeslots (right before the clock edge) and handled by infrastructure. Motivating case is to check certain signals are held, specifying the held time with a timescope ("while this is happening, expect these signals to hold at x").
expectDuring (or similar) might also be a useful construct, where it expect some signal value during some period in time. Or perhaps more useful would be a wait-until-with-cap, which does the same but also advances time. Use case could be where a spec calls for some maximum latency without needing to be precise.
Random thoughts:
VCS testing also has to be figured out.
$subject
I git it but when run sbt "testOnly chisel3.tests.BasicTest -- -DwriteVcd=1" there are a lot of errors like
object stage is not a member of package chisel3
I think maybe i should install the latest chisel3 ,but in sbt is already "chisel3" -> "3.2-SNAPSHOT", so what's the problem
import chisel3._
import chisel3.tester._
import org.scalatest.FreeSpec
class VecIO extends Bundle {
val x = UInt(5.W)
}
class UsesVec extends MultiIOModule {
val in = IO(Input(Vec(4, new VecIO)))
val addr = IO(Input(UInt(8.W)))
val out = IO(Output(UInt(5.W)))
out := in(addr)
}
class UsesVecSpec extends FreeSpec with ChiselScalatestTester {
"run" in {
test(new UsesVec) { c =>
c.in(0).x.poke(5.U)
c.in(1).x.poke(5.U)
c.in(2).x.poke(4.U)
c.addr.poke(2.U)
c.clock.step()
c.out.expect(4.U)
}
}
}
Generates error
Connection between sink (UInt<5>(IO in unelaborated UsesVec)) and source (VecIO(IO in unelaborated UsesVec)) failed @: Sink (UInt<5>(IO in unelaborated UsesVec)) and Source (VecIO(IO in unelaborated UsesVec)) have different types.
chisel3.internal.ChiselException: Connection between sink (UInt<5>(IO in unelaborated UsesVec)) and source (VecIO(IO in unelaborated UsesVec)) failed @: Sink (UInt<5>(IO in unelaborated UsesVec)) and Source (VecIO(IO in unelaborated UsesVec)) have different types.
Is there a way to catch TimeoutException
in user code? It seems like the exception gets thrown and there is no way to catch it (user code is not even part of the stacktrace) and process it in user code (e.g. to reformat the message, provide extra context, etc).
- should connect ports and work *** FAILED ***
chisel3.tester.TimeoutException: timeout on Clock(IO clock in MyModule) at 1000 idle cycles
at chisel3.tester.TreadleBackend.$anonfun$run$5(TreadleBackend.scala:188)
at chisel3.tester.TreadleBackend.$anonfun$run$5$adapted(TreadleBackend.scala:185)
at scala.collection.mutable.HashMap.$anonfun$foreach$1(HashMap.scala:145)
at scala.collection.mutable.HashTable.foreachEntry(HashTable.scala:235)
at scala.collection.mutable.HashTable.foreachEntry$(HashTable.scala:228)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
at scala.collection.mutable.HashMap.foreach(HashMap.scala:145)
at chisel3.tester.TreadleBackend.run(TreadleBackend.scala:185)
at chisel3.tester.internal.Context$.$anonfun$run$1(Testers2.scala:25)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
Have checks happen on the poke, instead of in bulk at the end of the timestep.
And other cleanup.
Resolution from chisel-dev 2/8: get rid of the blocking Region concept, replace it with a forking version to avoid the weirdness.
Also, maybe fork creates a ThreadBuilder object so we could have fork.withName("lol").onRegion(Monitor) { ... }
This was briefly discussed today, with epoch
being the proposed replacement. While the dictionary definition matches what we want, there is a possible conflict with the Unix epoch time starting at Jan 1, 1970.
timescope
will go through a (probably lengthy) deprecation cycle, since it's pretty long-lived and a fundamental testers2 primitive.
Other possibilities discussed at this and previous meetings:
span
, timespan
period
(overloaded with clocks in terms of digital circuits), timePeriod
interval
(conflicts with interval feature for width analysis)era
(implies a very long amount of time)moment
(implies a very short amount of time)I think the SNAPSHOT of chisel-testers2 is not in sync with the other SNAPSHOTS
I cloned the master branch of https://github.com/freechipsproject/chisel-template/
I am running into the error below, and inspecting release dates of the SNAPSHOTS, it looks like chisel-testers2 was released 2 days before the latest chisel3 on https://oss.sonatype.org/content/repositories/snapshots/edu/berkeley/cs/.
It looks like MultiIOModule moved from experimental in this PR: chipsalliance/chisel#1162
Is the best course of action to just download all the repos and build/publish them locally?
[info] Updating ...
[warn] Choosing sonatype-snapshots for edu.berkeley.cs#chisel3_2.12;3.2-SNAPSHOT
[warn] Choosing sonatype-snapshots for edu.berkeley.cs#firrtl_2.12;1.2-SNAPSHOT
[warn] Choosing sonatype-snapshots for edu.berkeley.cs#chisel-iotesters_2.12;1.3-SNAPSHOT
[warn] Choosing sonatype-snapshots for edu.berkeley.cs#firrtl-interpreter_2.12;1.2-SNAPSHOT
[warn] Choosing sonatype-snapshots for edu.berkeley.cs#treadle_2.12;1.1-SNAPSHOT
[warn] Choosing sonatype-snapshots for edu.berkeley.cs#chisel-testers2_2.12;0.1-SNAPSHOT
[info] Done updating.
[info] Compiling 2 Scala sources to /k/tmp/chisel-test3/target/scala-2.12/classes ...
[warn] there were 5 feature warnings; re-run with -feature for details
[warn] one warning found
[info] Done compiling.
[info] Packaging /k/tmp/chisel-test3/target/scala-2.12/mytest_2.12-0.0.1.jar ...
[info] Done packaging.
[info] Compiling 3 Scala sources to /k/tmp/chisel-test3/target/scala-2.12/test-classes ...
[error] /k/tmp/chisel-test3/src/test/scala/gcd/GcdTesters2.scala:25:5: Symbol 'type chisel3.experimental.MultiIOModule' is missing from the classpath.
[error] This symbol is required by 'type chisel3.tester.ChiselScalatestTester.T'.
[error] Make sure that type MultiIOModule is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
[error] A full rebuild may help if 'ChiselScalatestTester.class' was compiled against an incompatible version of chisel3.experimental.
[error] test(new DecoupledGcd(16)) { dut =>
[error] ^
[error] /k/tmp/chisel-test3/src/test/scala/gcd/GcdTesters2.scala:25:10: type mismatch;
[error] found : gcd.DecoupledGcd
[error] required: T
[error] test(new DecoupledGcd(16)) { dut =>
[error] ^
[error] two errors found
[error] (Test / compileIncremental) Compilation failed
[error] Total time: 18 s, completed Sep 30, 2019, 4:40:07 AM
There should be the same statistics printed for all simulators.
test ThingAMaBob Success: 0 tests passed in 1008 cycles in 34.515736 seconds 29.20 Hz
Resolution from chisel-dev 2/8: should be able to peek internal signals, optionally needing an annotation to avoid DCE
Tracking issue.
One long-standing issue with the workflow with chisel-testers was that we always re-ran the chisel generator. Ideally, you'd be able to re-run all your unit tests on your top level design. We had some workarounds for this (e.g. saving the peeks/expects and generating a verilog testbench that checked the original behavior was precisely reproduced), but they weren't great.
I think the fundamental issue is that we want to be able to introspect on the module. Is there a good way to retain scala objects used to produce a circuit so it can be reused after a compilation run?
Although it's possible to just define the queue ops on the DUT directly and poke manually, it'd be great if the ReadyValid
wrappers could figure this out automatically!
It would be great if the tester could peek/poke FIRRTL circuits. Also, it would be nice to use it to test IOs which a FIRRTL transform added, which doesn't have a Chisel equivalent.
I'd recommend looking at FIRRTL's CompleteTarget
class hierarchy, and hopefully use them to represent signals. We should also probably have normal Chisel references be converted to these Targets.
Thoughts?
I'm able to run testOnly pkg.MyModuleSpec
in SBT about five times before getting an OutOfMemoryError: Metaspace
. Based on removing parts of the (very simple) test, it seems like there's a memory leak per test run irrespective of what happens in the test (this is to say that it doesn't depend on whether there are peeks/pokes/steps).
I can only imagine that I'm doing something wrong...
Chisel version: 3.2-SNAPSHOT
Scala version: 2.12.7
SBT version: 1.0
[error] java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Metaspace
[error] at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
[error] at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
[error] at sbt.ConcurrentRestrictions$$anon$4.take(ConcurrentRestrictions.scala:207)
[error] at sbt.Execute.next$1(Execute.scala:104)
[error] at sbt.Execute.processAll(Execute.scala:107)
[error] at sbt.Execute.runKeep(Execute.scala:84)
[error] at sbt.EvaluateTask$.liftedTree1$1(EvaluateTask.scala:387)
[error] at sbt.EvaluateTask$.run$1(EvaluateTask.scala:386)
[error] at sbt.EvaluateTask$.runTask(EvaluateTask.scala:405)
[error] at sbt.internal.Aggregation$.$anonfun$timedRun$4(Aggregation.scala:100)
[error] at sbt.EvaluateTask$.withStreams(EvaluateTask.scala:331)
[error] at sbt.internal.Aggregation$.timedRun(Aggregation.scala:98)
[error] at sbt.internal.Aggregation$.runTasks(Aggregation.scala:111)
[error] at sbt.internal.Aggregation$.$anonfun$applyDynamicTasks$4(Aggregation.scala:174)
[error] at sbt.Command$.$anonfun$applyEffect$2(Command.scala:130)
[error] at sbt.internal.Act$.$anonfun$actParser0$3(Act.scala:387)
[error] at sbt.MainLoop$.processCommand(MainLoop.scala:153)
[error] at sbt.MainLoop$.$anonfun$next$2(MainLoop.scala:136)
[error] at sbt.State$$anon$1.runCmd$1(State.scala:242)
[error] at sbt.State$$anon$1.process(State.scala:248)
[error] at sbt.MainLoop$.$anonfun$next$1(MainLoop.scala:136)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error] at sbt.MainLoop$.next(MainLoop.scala:136)
[error] at sbt.MainLoop$.run(MainLoop.scala:129)
[error] at sbt.MainLoop$.$anonfun$runWithNewLog$1(MainLoop.scala:107)
[error] at sbt.io.Using.apply(Using.scala:22)
[error] at sbt.MainLoop$.runWithNewLog(MainLoop.scala:101)
[error] at sbt.MainLoop$.runAndClearLast(MainLoop.scala:57)
[error] at sbt.MainLoop$.runLoggedLoop(MainLoop.scala:42)
[error] at sbt.MainLoop$.runLogged(MainLoop.scala:34)
[error] at sbt.StandardMain$.runManaged(Main.scala:113)
[error] at sbt.xMain.run(Main.scala:76)
[error] at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:109)
[error] at xsbt.boot.Launch$.withContextLoader(Launch.scala:128)
[error] at xsbt.boot.Launch$.run(Launch.scala:109)
[error] at xsbt.boot.Launch$$anonfun$apply$1.apply(Launch.scala:35)
[error] at xsbt.boot.Launch$.launch(Launch.scala:117)
[error] at xsbt.boot.Launch$.apply(Launch.scala:18)
[error] at xsbt.boot.Boot$.runImpl(Boot.scala:56)
[error] at xsbt.boot.Boot$.main(Boot.scala:18)
[error] at xsbt.boot.Boot.main(Boot.scala)
So that failures can give a better trace of where something went wrong. eg timescope.withName(s"bit $i") { ... }
Setting a maximum number of cycles would prevent infinite loops when testing buggy circuits.
A nice feature of Testers2 is that it bases the name of the work directory for a test from the scalaTest test name. This can lead to potential collisions if two different tests share the same tests name.
Proposed fix, incorporate class name of test into the test directory
Example
class X extends FreeSpec with ChiselScalatestTester {
"basic compile should work" in {}
}
class Y extends FreeSpec with ChiselScalatestTester {
"basic compile should work" in {}
}
The working directory for both these tests will be the same, i.e. "test_run_dir/basic_compile_should_work".
It might not matter but it can create weird errors if the the two tests are similar and have different backends.
Example of multi-clock circuits: https://github.com/edwardcwang/chisel-multiclock-demo
It would make testing these kinds of circuits more DRY and less tedious than manually creating a wrapper Module.
Ideas
Expect allowed on inputs and poke allowed on outputs seems like this should warn or error.
Apparently a requirement for practically building chips.
Ideas:
For simulation, it would be convenient to have access to a clock cycle counter. Adding a counter into the design and then routing it to an output is not the best workaround.
Adding another item from past dev meetings.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.