[libre-riscv-dev] [Bug 280] POWER spec parser needs to support fall-through cases

[bugzilla-daemon at libre-riscv.org]

http://bugs.libre-riscv.org/show_bug.cgi?id=280

--- Comment #6 from Jacob Lifshay <programmerjake at gmail.com> ---
(In reply to Luke Kenneth Casson Leighton from comment #5)
> (In reply to Jacob Lifshay from comment #4)
> > The following algorithm should work to translate lines:
> > 
> > def lexer(lines):
> >     """ lines is a iterator that returns each line
> >         of text without the trailing newline """
> 
> unfortunately, stripping the newlines removes vital information,
> particularly when discerning small statements after a colon from "suites".
> 
> a suite is defined as the following tokens
> 
> NEWLINE INDENT stmts DEDENT
> 
> where the algorithm you wrote (minus NEWLINE stripping) identifies indent
> and dedent
> 
> oh... er, 1 sec...
> 
> >         yield from tokenize(line)
> >         yield NL
> 
> ah interesting.  so the NL *isn't* removed?

No, NL tokens are retained, however duplicate NL tokens coming from empty
lines/comment lines are stripped. The idea is that it should never produce two
NL tokens right after each other.

However, the way I wrote it you can't do the following:

if True:
    # just a comment here, no actual statements;

    # python has a `pass` statement for this case
next_statement

since it will produce "if" "True" COLON NL next_statement
rather than "if" "True" COLON NL INDENT NL DEDENT next_statement

However, the following will work:

if True:
              # empty line with comment

    pass
next_statement

it will produce "if" "True" COLON NL INDENT "pass" NL DEDENT next_statement

-- 
You are receiving this mail because:
You are on the CC list for the bug.