Thanks for sharing your insights.
> Lex wrote:
> Replacing that with a recursive descent parser giving an in-order AST is
simple ... but the results are different to
existing processors. So for Asciidoc 1.0 this might need to stay as is
for compatibility, and documents that depend on it could be deprecated
ready for Asciidoc 2.0 that changes the processing order.
If you look closely at the kickoff message (
https://www.eclipse.org/lists/asciidoc-lang-dev/msg00001.html), you'll see that I addressed this challenge as Goal #5.
> In that message, Dan wrote:
> While we may need to keep the substitutor in the first version, we may
also introduce a separate parsing mode that will parse the inline syntax
during the parsing phase using a recursive descent parser.
> Lex wrote:
> "extension points". This needs careful definition and consideration if its to be included in any specification to avoid implementation specifics.
To leave out extensions in AsciiDoc would severely diminish the value of the language. And they are expected. So we absolutely must have them.
> languages like C or other fully compiled languages are not so
amenable to being changed dynamically. It would seem to be not a
good thing to specify the capability in a way that implementations in
those languages cannot reasonably be made to comply.
First, I'd state that even fully compiled languages like C have a dynamic linker, so we can assume that plugging in additional code at runtime is possible.
But that doesn't mean we have to mandate the how. That really is left up to the implementation. (For example, Java uses a service loader pattern). What I'm going to propose we specify is a) where in the language syntax extensions are permitted (e.g., a custom block), b) where in the processor lifecycle extensions should be permitted (e.g., post processor), and c) what information extensions have access to and d) how the return values are handled with respect to processing. We know that anything we specify has to be language independent, so anywhere that gets into APIs will be expressed in pseudo-code (with compliant implementations to validate it, of course).
Even with all that in mind, we are developing the spec in tandem with the implementations, so if the spec goes too far, we will course correct (part of the stated vision).
> Not many (actually not any AFAIK) programming languages allow the source
code to specify which language constructs are allowed in parts of the
program, making normal formal computer language methods difficult to
apply.
We know this is going to be challenging. No one has claimed otherwise. AsciiDoc has a long history of being very flexible. We are going to encounter situations where we have to make tough decisions about whether to restrict some of that flexibility or require that implementations to deal with context-dependent markup. I don't claim to have all the answers right now. Hopefully, creative solutions will emerge. But I do know that we need to take it a step at a time, iterate, and do our best to make the language as robust as we can. We're here doing this precisely to strike that balance.
Best Regards,
-Dan
--
Dan Allen, Vice President | OpenDevise Inc.
Pronouns: he, him, his
