[libre-riscv-dev] extremely busy crowdsupply update started

[Jock Tanner]

Hi!

On Thu, 2020-03-26 at 21:03 +0000, Luke Kenneth Casson Leighton wrote:
> * jock on how you got on with the coriolis2 thing (even if it's "it
> was total chaos")

> if you prefer just reply to this message with a paragraph or two and
> i

I also can not put my first impressions in just one or two paragraphs,
so I wrote two versions. Luke, please choose the most appropriate one.

Short version (TL/DR)
=====================

As a humble Python developer, I understand the unique status and
significance of the Coriolis project, nevertheless I cannot help but
notice that it has a huge room for improvement. I genuinely hope that
my participation in libre-riscv will also help improve Coriolis.

Detailed version (The Rant)
===========================

1. Installation

Coriolis 2 can be built and installed two ways:

  a) inside the home directory (“Fixed directory tree”), using a helper
script [1],
  b) as a package, either RPM or DEB [2].

I followed the (a) route, and it actually went well except for a couple
of quirks, that is considering the number of dependencies. As I
understand, the (b) way is not well maintained for now.

The method (a) can be used per se, at least on Debian GNU/Linux, or
within chroot or LXC environments.

Unfortunately, the Python extensions are coupled with the rest of the
Coriolis, so it is impossible to use the advantages of 'virtualenv' or
similar environments with Coriolis.

Thus, entering the development with Coriolis for pythonistas is
complicated by the combination of two factors:

- lack of native packages for their OSes,
- lack of PyPI package or pip/virtualenv support.

I understand that Coriolis is a niche product, so we can not really
expect much in terms of packaging for popular OSes. So it may be better
for Coriolis to take another route:

- decouple Python extensions from the main product,
- provide Python extensions as a pip package (via PyPI or git),
- supply a script and/or instructions for building the main product
with the pip package.

This is actually not a unique approach. For example, xapian-haystack
ships a script for building a whole Xapian search engine (C++) in
PYTHONPATH [3].

2. Documentation
.
All the basics are covered by Coriolis User's Guide [4] (general
workflow and GUI) and Python Tutorial [5], that are available online
and offline, in PDF and HTML formats.

Unfortunately, Coriolis Python API don't have its own reference guide.
There is a good reference on Hurricane C++ API [6] though. Since Python
API just copies the C++ API (which is a terrible desicion by itself,
see the explanation in 3.2), this reference alone has the great part of
both C++ and Python APIs covered. But other parts of the Python API
(most importantly, Etesian and Katana) stays undocumented beyond the
basics.

3. API

3.1. Supported Python versions

The Coriolis Python API is a Python 2 extension.

Python 2.7 was released in 2010, and it was meant to be the last 2.x
release back then. In the following years Python 2 had no new features
other than backports from Python 3. As of January 2020 Python 2 is
considered unsupported by PSF. [7]

Still clinging to outdated and abandoned language and not even having a
(publicly available, at least) transition plan seems a bit scary to me.

3.2. Best practices of Python APIs

To have a new API mimic the old API written in another programming
language seems like an excellent idea at first. You don't have to
design anything. Don't have to think of naming. You can reuse
documentation and tests. The new API is easy to use, if you already got
used to the old one. Everybody's happy, right?

Well, everybody except the end users whom the new API is created for.

Each programming language has its own best practices, and most of the
time they either contradict the ones accepted by another language users
or just don't have analogues in that language. Your new users will see
your API as alien, cumbersome, or just poorly designed. They may have
to cram their way around the least expected features. They may have to
write wrappers around your API to reduce the astonishment factor [8]
for the others. But most likely they will simply walk away.

Here I'll try to give a run-down of Coriolis API features that scares
Python developers (“bad”) along with their attractive and comfy
alternatives (“good”).

3.2.1. Singletons

You don't need a special methods to create and access a unique object
in Python, since you can just import it.

Bad: af = CRL.AllianceFramework.get()

Good: from CRL import af

3.2.2. Getters and setters

Getters and setters are considered an antipattern (not only in Python),
so they should be replaced by properties or collection-like interfaces.

Bad:
  env = af.getEnvironment()
  env.setCLOCK('^ck$|m_clock|^clk$')

Good:
  af.env['CLOCK'] = '^ck$|m_clock|^clk$'

3.2.3. Data types

Python use duck typing, so you should avoid any direct type indication
or querying in Python.

Bad: Cfg.getParamInt('katana.vTracksReservedLocal').setInt(6)

Good: Cfg['katana.vTracksReservedLocal'] = 6

3.2.4. Syntactic sugar

It's easier to account for the certain things that goes in pairs
(open/close, create/destroy) with context managers or decorators.

Bad:
  UpdateSession.open()
  do_my_thing()
  UpdateSession.close()

Good:
  with UpdateSession():
      do_my_thing()

Also good in some cases:
  @update_session
  def do_my_thing():
      ...

3.2.5. Self-documenting

Python have a built-in 'help()' function, that returns a docstring for
a given object. Python C extensions can emulate a docstring for use
with 'help()'. Static syntax checkers and IDEs also rely on this API.

It is very important that this emulated docstring contains not only the
description of the function or method, but also a signature (parameters
and return value) in reST format, as recommended by PEP287 [9], or in
older Epytext format at worst. This will greatly facilitate the use of
Python C extension with modern IDEs.

The rules have slightly changed since Python 3.4, but the idea remains.

3.2.6. Naming conventions

The Python API must adhere to naming conventions accepted in Python. I
can not stress enough the importance of this point, but let us address
the styling issues in general in 3.3.

3.3. Styling

I have to admit that Coriolis Python style is quite consistent, but it
looks like the only purpose of this consistency is to contradict the
PEP 8 [10] in any possible way. [11]

You might think that style is not that important, but after 15 minutes
spent in search of the 'ChipRoute' class definition, diagnosing and
restarting IDE, checking Moon phase and whatnot, find that 'ChipRoute' 
is not a class, but a module… How well the end user could perform with
that in mind?

Another thing is that it's nearly impossible to lure a junior developer
into a project that neglects styling. In modern development,
proprietary and opensource alike, an ability to write a consistent,
readable, uniform code is very much appreciated, and junior developers
expect to acquire good professional habits in their first projects. And
style is the first thing they learn to distinguish in someone else's
code.

[1] http://coriolis.lip6.fr/pages/users-guide.html#id16
[2] http://coriolis.lip6.fr/pages/users-guide.html#id19
[3] https://github.com/notanumber/xapian-haystack
[4] http://coriolis.lip6.fr/pages/users-guide.html
[5] http://coriolis.lip6.fr/pages/python-tutorial.html
[6] http://coriolis.lip6.fr/doc/hurricane/index.html
[7] https://www.python.org/dev/peps/pep-0373/
[8] https://en.wikipedia.org/wiki/Principle_of_least_astonishment
[9] https://www.python.org/dev/peps/pep-0287/
[10] https://www.python.org/dev/peps/pep-0008/
[11] 
https://www.opensourceshakespeare.org/views/plays/play_view.php?WorkID=hamlet&Act=2&Scene=2&Scope=scene&LineHighlight=1307#1307