[libre-riscv-dev] [OpenPOWER-HDL-Cores] little-endian only power cores and spec compliance

[Luke Kenneth Casson Leighton]

---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68

On Wed, May 13, 2020 at 7:01 AM Benjamin Herrenschmidt <benh at ozlabs.org> wrote:
>
> On Wed, 2020-05-13 at 11:22 +1000, Hugh Blemings wrote:
> > Hi All,
> > I'm barely qualified to add much more than $0.20 to this thread, but
> > will give it a whirl...
> > I've not had a chance to dig into the v3.1 spec as yet but can offer
> > the observation that the intent (as demonstrated in the Microwatt
> > implementation for example) was that the simpler/simplest
> > implementations would be LE with BE optional.

yes.  this was the take-away i got from reading it.  the summaries
were as follows:

* AIX Platform - POWER9 and above

   must support 64 and 32 bit
   must support LE and BE

* UNIX Platform - (i call it UNIX because RTOSes and L4Ka are not
"Linux") - *also POWER9 and above*

   must support 64 bit (32-bit probably optional, not explicity stated)
   must support LE (BE explicity stated as optional)

* SFFS - basically this recognises and gives space to the NXP QuorlQ
series (which is V2.07B(?) compliant)

  must support 32-bit (64-bit optional, explicitly stated)
  must support BE (LE explicity stated as optional)

* SFS - basically this recognises and gives space to the NXP MC5xxx
32-bit embedded series

  must support 32-bit (64-bit optional, explicitly stated)
  must support BE (LE explicity stated as optional)


note that *only POWER9* meets the requirements of the AIX and UNIX
platforms.  microwatt, by being a LE-only and non-VMX/VSX processor,
does not fall into *any* Compliance Category.

this puts microwatt firmly into "custom-built distro" territory.  our
design likewise - even though it will have 3D and VPU Vector
capability - will *still* be in "custom distro" territory, which is a
massive burden on customers and end-users.

with the product intended for mass-production, that means it'll end up
with "Rasbian" community-driven distros being created. (Rasbian was -
and is - a damn nuisance.  the original Broadcom processor used in the
Pi v1 was neither armel nor armv7: consequently it had to have its own
complete and total recompiled Debian distro, annoying the hell out of
the Debian Community due to the mis-filed bugreports and
miscategorised - unpaid - end-user support requests)

now, for Android-based products and Chromebook-based products *this is
not a problem*.  Android and chromebook products always
custom-recompile the entire binaries from source anyway.

but for distros? debian? fedora?  as things stand, we're excluded -
and so is microwatt.  microwatt as it stands cannot run *either* the
ppc32 debian distro *or* ppc64le distro binaries.

that exclusion - being unable to run standard debian or fedora distros
- in turn makes community grass-roots development almost impossible to
kick-start a thriving community and knowledgeable user-base.


> >  SIMD also optional in these as well.

unfortunately that was left out: only the SFFS and SFS subsets
(intended primarily to be 32-bit BE) are permitted to have SIMD (VMX
and VSX) optional.

it means that we (and microwatt, and any other new processor designer)
have a whopping 600 additional instructions to add (in addition to the
appx-380), with a full Vector Engine if they (we) would like their
product to take advantage of the time and effort saved by availability
of distro pre-compiled binaries (Fedora, Debian).

this is a very high threshold, which, working through the
implications, would simply have quite a significant number of
otherwise interested parties in POWER from pursuing other options.
the extra register file (Vector Regfile is separate) alone will be
enough to make them think twice.

if you already *have* VSX and VMX implemented, it is perfectly
reasonable to have it on the AIX and UNIX Platforms.  however for a
new entrant?


> The main difficulty is that in practice, a powerpc64le toolchain will
> generate code with both FP and SIMD instructions. I don't think it's
> possible to generate a fully soft-float one and even if you managed
> with gcc, glibc somewhat assumes it too. At least from memory.

a solution exists (it involves modifications to gcc, binutils and glibc).

the RISC-V Foundation members have been working on support for
multiple-compilations of functions - the exact same function -
compiled multiple times and the resultant assembly code inserted under
the *same function name* into the object file, with sufficient
metadata to be able to discern it from "functions-with-the-same-name".

(this i specifically mention because it is not a new form of c++
name-mangling being deployed, here)

what then happens at dynamic link time is this:

* the capabilities of the processor are detected
* the existence of two (or more) versions of the same dynamic-linked
function is noted
* the *CORRECT AND REQUIRED* version of that function according to the
*hardware* capabilities is selected *dynamically*.

this includes defining *new ABIs* suitable for each type of "hardware
capability".  so for example for RISC-V Vectors, they are defining a
new glibc6 ABI for vectors...

... *and compiling functions twice - once to conform to the Vector ABI
and once to conform to the standard scalar ABI*.

thus when it comes to "who must win" this is now entirely moot:
everyone can be happy.

Questions:

1) does the executable size increase?  yes of course.

2) is it the same as multilib?  honestly i am not sure that it is.
or, if it is, it's the mechanism *behind* multilib.


> If OpenPower wants to go down a path more similar to RiscV here and
> support small(er) implementations, it's necessary to stop having the
> toolchain engineers push for only supporting the latest/greatest IBM
> server CPUs :-)

i can understand why they would want to advocate that :)
unfortunately, as you point out, it excludes "simpler" entrants, and,
worse, excludes entrants to markets that *POWER9 is never going to be
able to enter anyway* due to its massive size and huge power
requirements.

however with the above scheme, recently introduced somewhere in... the
past 18+ months(?) it is no longer necessary to fight over the space.

of course, JIT-compilation by using LLVM-IR *also* solves this problem
nicely.  however, LLVM is not the default, de-facto and sole exclusive
compiler used in the world.


> The Microwatt core is now starting to boot Linux as well (not merged
> yet)

we saw the ppcdev patches.  nice work :)

> but at the moment this requires some kernel hacks to work around
> the above, such as plumbing in FPU emulation, and I know Paul has
> issues generating a usable userspace...

the powerpc gem5 experiment already has one pre-compiled (an
initramfs) with a "hello world" buildroot of some kind (busybox).  and
associated modifications to skiboot?

https://github.com/power-gem5/gem5
[branch "gem5-experimental"]

and the other support repos are at
https://github.com/power-gem5/

l.