[libre-riscv-dev] buffered pipeline

[Jacob Lifshay]

On Mon, Mar 25, 2019 at 4:17 PM Luke Kenneth Casson Leighton <lkcl at lkcl.net>
wrote:

> ok, so, jacob, this is difficult for me to say but it is also
> important.  i'm under financial pressure so there's not a lot of
> wiggle-room to do too much experimentation, although doing so as a
> learning process is, i appreciate, extremely important.
>
> i've noticed that you sometimes don't respond to my comments, which
> means that from that point onwards, you continue working without
> consensus and agreement.  this has happened more than once, which is
> making me extremely nervous, given the increased financial pressure
> i'm under.
>
Sorry, I often put off replying till later, and then never get to it.

>
> i do not mind if you have a justification for continuing in a
> particular direction, however it's *important that you express it and
> we agree it*.  please, speak up!  if you have a reason, it's really
> really important to express it and to convince me (all of us) of the
> merits of the idea.
>
Yeah, sounds good to me.

>
> for example, i am still waiting for you to come back to me about the
> assessment that a barrel processor is totally unsuitable for anything
> other than a low-clock-rate I/O processor, particularly that the lack
> of multi-issue capability needs to be compensated for with a
> *four*-fold increase in either SMP concurrency or SIMD capacity over
> the presently-agreed design, both of which have very strong
> disadvantages.
>
Will respond in-depth in a separate email.

>
> i'm also waiting for you to come up with a proposal that incorporates
> all of the types of data arrangements in the pipeline API - not just
> the one (Record), and for a response to the other comments that i
> made.
>
I was planning on changing the interfaces to allow a Signal or a Record in
the interface, though I'm not sure how to handle associating a Direction
with the Signal, maybe by allowing a (Signal, Direction) tuple in the
interface and having a Signal by itself be Direction.NONE
I honestly think it would be cleaner to not allow interface types other
than Signal (with optional Direction) or Record.

Related to that, I think that nmigen's Layout class (Record.layout) has a
bad internal design since the shape of each member Signal is not normalized
to a (width, sign) tuple, instead it is left as whatever the user passed
in, requiring all code that uses it to manually handle the multiple
different signal types (int and (int, bool)).

>
> instead, you went ahead with an extremely well-engineered and
> well-designed pipeline, which has let you explore many of the features
> of python that even i didn't know existed, despite using python for
> nearly 19 years.
>
> the problem is: it is completely over-engineered [and limited in
> capability].... *and i've created duplicate code that's more
> flexible*.
>
Other than allowing a Signal instead of a Record and having a Pipeline
class which allows you to easily compose multiple Stage classes (both of
which I intend to fix), I actually think the classes I designed have a
cleaner interface:

- They allow for pipeline stages to have named input and output ports (just
name the member something other than pred or succ) as opposed to multiple
ports requiring an array (though this point won't apply after you finish
the refactoring you had planned).
- They have the pipeline registers separated out from the pipeline logic,
allowing RegStage, BreakReadyChainStage, and all similar classes to not
have to handle also having additional user-specified logic internally.
- They have the input and output data shapes specified as parameters to
__init__ allowing users to specify shapes without having to create a new
class specifically to override the ishape and oshape members (can't
remember if those are the right names). If a class wants to have the shape
fixed, it just has to override __init__ and call super().__init__() with
the shapes it wants.
- They allow the process function to be passed in to CombStage as a
parameter, allowing invoking code to use CombStage to handle some simple
function without having to derive from it. If a derived class wants a fixed
process function, it can override the process function in CombStage.

>
> we *really haven't time* and i don't have the financial resources to
> explore too many things without keeping each other "in check".  yes
> i'm counting on you (and everyone else) to make sure that what i'm
> doing - no, what *all* of us are doing - is sane and on track.
>
Makes sense. I'll try to do that.

>
> each of us bring something different to the table.  me, i have nearly
> 28 years of software engineering experience (good god that's a hell of
> a long time), which gives me an ability to think and plan ahead from
> potential paths, assessing the time taken for each, and aggressively
> pruning options that would take too long.
>
> i also have a strong pathological attitude towards "not invented here"
> syndrome.  i've written code in eight days that took a *team* of
> *three people* over *six months* to write an equivalent, because i
> chose to pick and join code that came from python-gobject, netscape,
> webkit and other wildly disparate projects.
>
> the other team spent months duplicating code that did 95% of the job
> and would only have needed 5% adaptation.

That makes sense. I'll admit that a while ago, I really didn't like using
other's code, though that may be partially because combining C/C++ code is
a pain because everyone uses a different mostly-incompatible build system
(which is a nice thing about Rust, since most people use Cargo, which has
become a de-facto standard). Since then, I've come around to using other's
code since standardized package managers (such as pip, npm, and cargo) make
using code really easy.

these kinds of decisions come from experience, from necessity, and
> from having an extremely ambitious goal that's *not* about the "what"
> - it's about the *why*.  although i've wanted to do a processor for
> nearly 30 years, it wasn't until i had a reason *why* i wanted to do
> one that i actually went ahead and decided to commit to one.
>
> so please, please understand: talking about what we're going to do,
> and getting consensus, is much *much* more important than actually
> writing code.
>
> haven't quite covered everything, i know i've missed things out...
> getting too long... hope you understand.
>
Yeah, makes sense. Hopefully working with nlnet will work out which should
relieve some of the financial pressure.

Jacob