[libre-riscv-dev] Tests in soc/fu

[Luke Kenneth Casson Leighton]

On Sat, Jun 20, 2020 at 4:56 PM Yehowshua <yimmanuel3 at gatech.edu> wrote:
>
> Michael and I were discussing about unit tests.
> I recently realized its possible to maintain simulator
> state while popping on more unit tests.

yes.  ah.  i'm glad you brought this up.

unit test classes in particular, have setup and teardown functions (i
think they're actually called that).
https://docs.python.org/3/library/unittest.html#setupclass-and-teardownclass

they are.  encountered them a couple of years ago.  with so much else
going on i just haven't had time to raise it / mention it.

> You can do the following as many times as you want:
>
> ```python3
>
> sim.add_sync_process(process1)
> sim.run()
>
> sim.add_sync_process(process2)
> sim.run()
>
> sim.add_sync_process(process3)
> sim.run()
> ```

yes.  and, thus, the the setup function would establish the "sim"
object, placing it as a member of the unit test class, and teardown
would do any cleanup such as deleting temporary files or etc. etc. (if
needed).  so it would be more:

class TestSim(UnitTest)
   def setupClass(self):
        self.dut = Module()
        self.dut.m.submodules.blah = SomethingToTest()
        self.sim = Simulator()

 def process1(self):
      yield something
      self.assertEqual something else

  def test_fred(self):
       self.sim.add_sync_process(self.process1)
       self.sim.run()

something like that.

> This comes in handy when running many very
> short tests on an rtl unit. You no longer have
> to rebuild the simulator for each test.

correct.

> Currently, we have some hacky code that
> tries to accomplish preserving states between units.

indeed.  test_data list which accumulates assembly code to run.  this
has the disadvantage of accumulating all the tests in one big job-lot,
where a single assert terminates all of them.  it also puts the
success/fail report under one function name.

> I have released part of my thesis with an example
> of how to structure a unit test in this manner:
> https://github.com/BracketMaster/MAERIV6/blob/master/tests/test_heuristic/test_multleave.py#L98

oo nice.  that's tidy and clear.  you shouuuld find this works, after
adding self. in front of dut, weights etc. which sigh will push things
over an 80 char limit and interefere with readability and clarity so i
don't recommend you do it for this code, the main thing being it
demonstrates the way to use the unittest API.

    @classmethod
    def setupClass(cls):
        # setup
        LENGTH = 4
        cls.weights = np.array([-2, -1, 0, 1],dtype=int)
        cls.features = np.array([-3, -4, 0, 8],dtype=int)
        cls.inject = np.array([0, 0, 0, 1],dtype=int)
        cls.dut =  MultLeaveChain(
            LENGTH=LENGTH,
            INPUT_WIDTH=8
            )
        m = Module()
        m.submodules.dut = cls.dut
        cls.sim = Simulator(m)
        cls.sim.add_clock(1/(1e9), domain="sync")

note btw that cls and self are just naming conventions!


> Pretty low priority, but at some point, it may be worth
> revising older unit tests.

yyehh actually it may be worthwhile doing now for the assembly code
unit tests, because it is quite a nuisance not to have the name of the
unit test which caused "F" or "E", and, also, the *entire* batch will
terminate because it's run from a list.

given that all the assembly code unit tests are now run from a common
base class, this is not so hard.

> Certainly, new unit tests from now on would
> benefit from this structure.

yes definitely.

> And Luke, my apologies for all this code, some on GIthub
> some in SOC. I’ll try to be more intentional on this.

no problem.

> Still transitioning to making LibreSOC my main commitment.

understood.

btw i changed everyone's about page bugzilla link to include "cc":
https://libre-soc.org/about_us/

so now if you go to let's say Cesar's page: https://libre-soc.org/Cesar_Strauss/

there are 2 more bugreports that he's cc'd on than are assigned *to* him:
https://bugs.libre-soc.org/buglist.cgi?email1=cestrauss%40gmail.com&emailassigned_to1=1&emailcc1=1&emailtype1=substring&list_id=927&resolution=---
https://bugs.libre-soc.org/buglist.cgi?email1=cestrauss%40gmail.com&emailassigned_to1=1&emailtype1=substring&list_id=927&resolution=---

typing or remembering those is quite inconvenient.  therefore, if you
are "losing track" of conversations, simply going to your homepage,
clicking on "bugtracker assignments", you can see immediately at a
glance the ongoing conversations relevant to you.

(i have to do things differently because that list is 239 assigned
bugs, and that's not useful.  i may end up using the "priority" field
or a tag of some kind at some point)

l.