svunit (related) discussion topics

[neil johnson]

all, I have a session penciled in for dac in june involving svunit. it'll be very informal. we take a list of topics and cycle through them in a series of 5-10min discussions. I'm looking for some topics to get discussion rolling and am hoping you'll help me out. If you've got a few minutes, please respond with comments or questions you've heard regarding unit testing, test driven development, svunit and/or related. as many as you can think of. can be for or against unit testing, doesn't matter. tougher the better ;). I'll take 10-15 and and quote them in a series of slides. we'll show the slides and invite the audience to agree/disagree/comment.

thanks in advance!

-neil

[Bryan Morris]

Neil:

Excellent.  

The key comment I get back is that TDD is a good idea, but too much "overhead".

Other comments received when describing TDD:

• Too much overhead: "Really?!  You mean to test every single function... even the simple ones?" -or- "Sounds like it might be useful.  But we don't have time for that in the schedule."  (to which I usually ask how much time they have allocated to "debug" time).
  
• No value seen: "I can see where that's useful for some people if they're writing really complicated or long tasks/functions, but I design them such that they are so simple and short that there can be no bugs lurking there."  (no real reply here.. more of a cynical arch of the eyebrow :-)
• Perceived better alternatives: "I use code inspections. I think they're more valuable"   (to which I reply: Absolutely agree that code inspections are valuable!  Do you code inspect everything?  and what happens in the future when the code you inspected so carefully, changes?

I hope this helps.  If I think of other comments, I'll let you know.

-- Bryan

[Jonathan Bromley]

hi Neil, something I suspect a few people may find relevant - I certainly would:

When I was developing a library of assorted functions, SVUnit was just fantastic. Each function could be given a straightforward set of tests, and the whole thing ran in a few seconds whenever I made any edit. There was massive added value in terms of saved debug time because errors got caught early, and missing features or operations were highlighted from the start. In fact, the ONLY known bugs in that library that escaped were things that, with hindsight, I really should have found in the unit tests anyway - but of course tests are never as complete as you'd like to think.

On the other hand, it's much less obvious to me how you should apply unit testing to complicated interacting sets of classes that have lots of state that evolves as the classes are stimulated by various method calls. Yes, I know that probably means the classes have too much state - but c'mon, we live in the real world, stuff is often less simple than you'd like it to be. So, how would you best apply SVUnit to the development of, let's say, a typical UVC? Some classes, some pieces, can easily be tested stand-alone. Others require lots of activity from the external world to get them into a state where some specific interesting thing is happening. In that case, the "unit test" starts to look much more like a complete test environment, and there's a sense that some of the immediacy benefits of unit testing are not so accessible. What do you and other experts do to get real value out of unit testing in such a situation? Or, to put it another way, what am I missing here?

thanks
Jonathan

[Tudor Timi]

Even if setting up the testing infrastructure takes some effort, I find it pays out in the end when I want to make small changes in the implementation, without changing the behavior (i.e. refactor stuff). This, to me, makes it all worth while.

[neil johnson]

this is awesome guys. just the kind of stuff I'm looking for. anyone else with similar or wildly different comments, please continue :)

[Stan Sokorac]

By far the most common response I get is "I don't have time in the schedule for this", as if this is some extracurricular activity that is taking away from real work. Unfortunately, the argument that "you'll save much more time later" is a hard one because it's difficult to quantify the benefits. One has to experience it to believe it. Now I'm starting to sound like a cult leader, right? :)

So, I guess the question is, how can unit testing be made easier and more accessible to those that haven't done it before, and who are in a typical setting of having an already developed testbench / design that they are tweaking and adding features to?

[syed taahir]

Hi Neil,

Looks interesting but again how the complete complex scenarios are achieved such as
1) Virtual Sequences
2) The two agents active at a time.

There are lots of scenarios which looks absence in SVUnit, may be I have not explored much

Regards
Taahir

[Tudor Timi]

@syed SVUnit isn't for complicated scenarios. It's for very directed and targeted testcases.

[David Woo Hyeok Kang]

Decoupling 10k+ lines of testbench code--written without unit tests--into units is the toughest part.
Especially for code that you haven't written.

[Tudor Timi]

I assume you're talking about writing unit tests for UVM. This is something we tried to do at Freecellera UVM Bug Blitz, but it never really gained traction, unfortunately.

The decoupling is already done, basically. You can't change the structure of the code. As for what to unit test, I'd start with the public APIs. For this it shouldn't matter how the code is implemented.

[Tudor Timi]

I've started writing about this on my blog:

http://blog.verificationgentleman.com/2016/08/testing-uvm-drivers.html
http://blog.verificationgentleman.com/2016/08/testing-uvm-drivers-part-2.html

I only talked about drivers up to now. The next point will be SVA sequences/properties. I also wanted to do something on monitors, where I would touch on the testing standalone vs. using other classes.