BNF Rules and Syntax Highlighters

[tajmone]

@thoni56, while sifting through the dev style preview docs, and updating them, I noticed that BNF rules are currently defined as code blocks in The ALAN Manual (and possibly in some ALAN Design Docs):

[source,bnf]
--------------------------------
class = 'EVERY' id
            [inheritance]
            {property}
        'END' 'EVERY' [id] ['.']
--------------------------------

I now remember that my original intention was to provide a BNF syntax for all highlighter — Highlight already has one, and for the FOPub toolchain I was planning to create one (see #38).

But now we have multiple syntax highlighters being used in this project, and most of them (i.e. all except Highlight) don't support BNF grammars, which means they'll simply not highlight the block, which would then be just a verbatim block.

But even when they do highlight the BNF rule (e.g. Highlight), we didn't define any custom them for it, so it will show as a two-coloured block anyway (background and foreground text only).

I'm wondering if we shouldn't just convert those blocks so that bnf becomes a role (class) rather than a language. This would require tweaking the Sass sources (no big deal) and the XSL stylesheet (not a huge deal either).

Alternatively, I could just go ahead with our original intention of providing syntax highlighting support for BNF, which would result in subtle colour differences (or bold vs normal) being shown in the block, according to tokens types:

• Create BNF syntax definitions for:
  • Rouge (hasn't got any BNF, EBNF, ABNF lexers)
  • XSLHL (asciidoctor-fopub PDF toolchain)
• Create custom styles for BNF blocks:
  • HTML Backend » Sass
  • PDF Backend » XSL stylesheet.

Since BNF grammars are really simple, it wouldn't involve much work at all (I've also created the one for Highlight, if I remember correctly).

But it's worth noting that these BNF rules in the ALAN Manual are actually EBNF grammars, not BNF! Not trying to pedant here, my concern is that if the official highlighter were to introduce a BNF syntax definition, they might end up clashing with our, that's all.

[thoni56]

But it's worth noting that these BNF rules in the ALAN Manual are actually EBNF grammars, not BNF! Not trying to pedant here, my concern is that if the official highlighter were to introduce a BNF syntax definition, they might end up clashing with our, that's all.

Fair point. I'll try to remember to update that in the manual. For sake of correctness.

Otherwise I don't have much to add here. My only concern is that EBNF block should have a separate rendering so that they are visually different. There's no need for highlighting, so whatever is the easiest to achieve that is fine by me. (A role sounds like the easiest route, but you're the expert.)

[tajmone]

Fair point. I'll try to remember to update that in the manual. For sake of correctness.

If it's OK with you, I'd rather to that myself, so I make it coincide with the stylesheets updates. As you mentioned:

My only concern is that EBNF block should have a separate rendering so that they are visually different.

yes, definitely, changing [source,bfn] to [source,ebfn] would break the color scheme if we don't also rename the stylesheets.

There's no need for highlighting, so whatever is the easiest to achieve that is fine by me. (A role sounds like the easiest route, but you're the expert.)

OK, then this is what I'll do: I'll first convert all BNF code blocks to EBNF verbatim blocks, so we keep the styles but detach from syntax highlighters (which spares us all the problems relating to which highlighter supports which features, like callouts, spans, etc.).

Then I can start to experiment elsewhere with EBNF syntaxes and themes, and see what the results are (i.e. if they're worth adding, considering potential future maintenance efforts).

Even once we have our EBNF syntaxes, I'd suggest to add their code styles to the stylesheets without dropping the styles for verbatim blocks, just in case someone prefers to use them without highlighter (e.g. for problems relating to callouts, etc.). Having dual identical styles on two different block types causes no redundancy in Sass sources, and little space difference in CSS files and XSL stylesheets.

Here's an example of EBNF syntax highlighting on Wikipedia:

https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form#Syntax_diagram

color choices aside, you can see that it makes it easier to quickly distinguish between nonterminals, terminals, rules, punctuation and comments. So, probably even with our monochrome soft-orange scheme we could achieve some nice contrast to make the rules seem less flat and easier to track.

[tajmone]

Leaving BNF in Text

I think we should only change BNF to EBNF in the code blocks syntax, to avoid highlighters mismatches, but that in the text we can leave BNF — after all, EBNF and ABNF are BNF, albeit variations in notation, but these are technicalities which we could spare the average reader, who might be unfamiliar enough with BNF (let alone EBNF).

Also, for the Index (PDF only) it's not worth having a "lookup-breaking change", for most users will be familiar with the original BNF term.

In any case, right now I'll focus only on the syntax name in code blocks (soon to become literal blocks) and the role (lang/class) in the themes, starting with XSL-FOP, which is an external submodule repository and has to be fixed first (then we only need to update the pointer).

[thoni56]

Yeah, I was only talking about the textual change, the formatting stuff I'll leave to you ;-)

And you are probaly right that the formal difference is of little consequence to the reader of the manual. The intent will be fully understood anyway.