[libre-riscv-dev] offtopic: memory safety

[Jacob Lifshay]

On Wed, Jul 17, 2019, 22:26 Luke Kenneth Casson Leighton <lkcl at lkcl.net>
wrote:

> On Wed, Jul 17, 2019 at 10:41 PM Jacob Lifshay <programmerjake at gmail.com>
> wrote:
>
> >  of even deeper concern is that they're used to justify one language
> > > over another for completely inappropriate purposes.
> > >
> > yeah. though it is nice to have a programming language that is designed
> to
> > help you avoid UB, without needing extensive experience to get it right
> --
> > such as Rust (just don't use the unsafe kw), Python, Java, and others.
>
> agreed.  what will make me feel more comfortable about rust is when
> (if) it stabilises enough for the gcc team to create a frontend for it
> (like the fortran and java frontends, which some people aren't aware
> these exist).
>
I thought gfortran (gcc's fortran frontend) was the canonical fortran
compiler for linux? I'm somewhat disappointed that the java compiler was
dropped due to not being maintained.

>
> cutting out the llvm JIT intermediary and having the actual compiler
> directly generate the assembler would alleviate some potential
> security / safety concerns.  i'm much happier trusting something where
> there's less "pieces in the puzzle" basically.
>
actually, rust is normally an ahead-of-time compiler -- I haven't heard of
any JIT compiled modes yet, though it does have an interpreter (called miri
for MIR-interpreter -- see below for naming) for const evaluation or for
stuff like a more powerful version of valgrind.

rust has a runtime similar to C or C++ in that you don't pay for what you
don't use and it's minimal when needed (I think it'll work with just memcpy
and friends if you don't use FP, otherwise it needs libm too -- useful for
embedded). The compiler design philosophy includes that any feature should
be just as performant as hand-written code.

it can directly generate assembly, though it usually uses llvm's integrated
assembler to output machine code directly -- just like clang.

it actually uses an internal architecture similar to clang's: input code ->
AST -> HIR (high-level IR; generally for checking invariants closely
related to syntatical structure) -> MIR (medium-level IR; to enable
checking invariants not tied to syntatical structure; also used for
compile-time code evaluation like C++'s constexpr and for high-level
language-dependent optimizations) -> LLVM IR

see https://rust-lang.github.io/rustc-guide/high-level-overview.html

also see https://github.com/thepowersgang/mrustc
rust compiler written in C++ for code that is already known to have no
compile errors -- translates to C -- currently mostly used for
bootstrapping rustc

>
> > C (and C++) is powerful, but it's easy to mess up without extensive
> > experience -- I've worked on commercial production firmware where all the
> > non-reenterant file system code was called from the USB interrupt without
> > any kind of mutex, disabling interrupts elsewhere, or similar. that same
> > codebase used global variables modified by interrupts without using
> > volatile (pretty much the one spot other than memory-mapped hw where
> > volatile is the correct tool). They relied on optimizations not being
> ran.
> > I've even seen code that uses memset to initialize std::string after
> > running the constructor!
>
> i must have just totally lucked out with my choices of places to work
> at.  the first one was CEDAR Audio (real-time audio restoration),
> where they were doing TI (50MFLOP) DSP programming and had created
> their own mixed c++ and x86 assembler GUI that ran on top of DOS, and
> did its own INT33 handling, direct AT/XT bus assembly programming (to
> communicate with the DSP card), wrote direct to the VGA screen and so
> on.
>
neat!

>
> we created our own version of the Win16/32 "Message Passing" stack but
> added priorities to it.  there was a separate message-stack for
> interrupts and we had to implement context-switching (in x86
> assembler).  it worked extremely well, and even on 386sx16 machines
> the GUI was responsive and efficient.
>
> then i went to work for Pi Technology in Milton, and there they
> successfully adopted ISO9001 (BS5750).  why was it successful (and
> liked)? because they were already doing *five* man line-by-line
> code-reviews on the firmware that was to run in Engine Controllers
> (get that wrong and it could kill someone), and they already *had*
> coding standards that said "on no account use malloc or free in c++"
> and other extremely sensible things that everyone knew, understood
> *and respected*.
>
> so when i heard of how ISO9001 was quotes universally hated quotes i
> was seriously taken aback.
>
> and i hear horror stories like the one you tell above, and it just...
> i genuinely don't understand how people manage to get themselves into
> such holes.
>
i don't know either -- it was written before I started there and they
figured using code that already works is better than starting from scratch.

>
> > >  in the case of the samba team "leaders", they simply genuinely had no
> > > idea - and no respect for - the dual nature of the two OSes that they
> > > were working with (windows *and* unix).  it was too much for them, and
> > > they made - repeatedly - poor design choices as a result.
> > >
> > hopefully they've been refactored since then.
>
> mmm.... no.  they basically went down the rabbit hole of recreating
> openldap *and* heimdal, both of which are well over a quarter of a
> million lines of code, each, with really experienced *separate* teams
> behind each, and, with no other wider user-adoption, make them the
> sole exclusive maintainer of something that's not in the slightest bit
> config-compatible with either project.
>
sadness...

>
> [NT 5 aka Windows 2000 adopted LDAP and Kerberos, then broke both by
> f*****g about with the standards.  luke howard wrote some extremely
> simple patches to both OpenLDAP and Heimdal, and developed a complete
> Active Directory replacement called XAD in about 2 years flat.  it was
> *eventually* bought by Novell].
>
> > In the mean time, one can vainly hope that microsoft switches windows to
> > using unix and deprecates the weirder parts of win32 -- apple did when
> they
> > introduced OS X.
>
> yyeah, they're an "interesting" company.  they can't do that - they
> have to support their customers.

that could still be done -- notice I said deprecate, not remove.

>   there's even switches inside DOS
> CMD.EXE which say "if executable == SIMCITY.EXE enable deliberate
> malloc memory bug".
>
wow!

>
> no, really, not kidding, look it up!


> there's been an internal fight for decades, of two teams that want to
> make a *deliberate* incompatibility "clean sweep", to clear away the
> awful cruft that keeps older applications running, and those who say,
> "sorry, the customer comes first: legacy is king".
>
I think they may drop some legacy support when intel drops mmx support
(which I hear intel's planning on -- amd already dropped fp in mmx
(3dnow!)), which has been replaced by sse2+ and avx since pentium 4 and
amd64.

Jacob