> On Mar 15, 2021, at 2:03 AM, Sylvain Leroux <sylvain@xxxxxxxxxxx> wrote:
>
> On 13/03/2021 18:06, David Jencks wrote:
>>>> Maybe…. perhaps in the other direction I've considered trying to refactor Asciidoctor so that anything that looks syntax-wise like a extension is implemented as an extension. I wonder if requiring this in the spec is reasonable. Being able to replace behavior could be useful. There could be a set of required extensions. As usual, needs more thought :-)
>>>
>>> I have close to zero experience with extensions. Could you be more
>>> explicit regarding the standard features "that looks syntax-wise like an
>>> extension"?
>>>
>>
>> I'm thinking of things such as `image:…`, `xref:…` which look like inline macros and `image::…` that looks like a block macro.>
>> `include::…` needs to stay special as it does not necessarily introduce block content.
So long as everything that "looks like" an xxx: markup is able to be replaced by an extension it should be up to the implementation if the ones specified in the AsciiDoc specification are built-in or actually implemented as extensions.
I agree that include:: is special, but because its a preprocessor directive, it totally ignores other syntax (except perhaps that it can't be inline ATM, so its always generating lines). That goes for ifdef:: and friends too.
I would specify that these directives behave _as if_ they are executed in a separate pass before parsing to define what text is input to other processing. That is easy to test, and it doesn't mean that actual implementation is that way, just that a document or extension shouldn't be able to tell.
>
> Thanks for the reply, David.
>
> TL;DR:
>
> Macro-like directives make good extension points. And not all processors
> might want to implement those features. So I support the idea of making
> them extensions.
A set of such markups needs to be defined in the specification to allow existing documents to be processed, they are "part of AsciiDoc".
The one downside of some implementations adding some extensions is that they become what Dan calls "flavours" of AsciiDoc, and documents are no longer portable. Thats why my other thread voiced the issue of extension portability.
Adding input macro processing would significantly reduce the incidence of some types of extension (he says without having examined a lot of documents, but based on questions in AsciiDoc Python and Asciidoctor lists) and adding them should be examined in the specification process.
Input macros should act like mini include files with text defined in the document (or multi-line document attribute substitutions, your mental model may vary :). But because they are _in the document_ they are now portable.
(Yes Dan its a new thing, but purely additional, it won't change any current document processing :)
>
>
> For the long message now:
>
> This WE, I studied a little bit more AsciiDoc extensions. There appear
> ([1]) to have much more extension points than I believed it.
>
> FWIW, the `image:…`, `xref:…` and `image::…` have in common to link to
> (or include from) external sources. I can imagine a situation where a
> client application might want to limit or filter what's allowed here.
> Say, for example, in the case of a Forum application using AsciiDoc as
> its message input format: the processor might want to enforce a forum's
> policy restricting the allowed image format or origin.
I don’t think it’s any business of a syntax extension, built-in or not, to interpret tokens referring to other resources. I think this functionality needs to be extracted to a separately replaceable document set service. Refactoring Asciidoctor for a POC of this is proving to be... time consuming.
In general the sequence of code points between xxx:: and [] should be simply available to the extension to do what it wishes, other than it being valid UTF-8 and escaping for [ or alternative terminator code points, AsciiDoc should not define any semantic for that content. It should be up to the extension to document how it uses the `target` as it is known, if it substitutes attributes etc. Clearly when the AsciiDoc spec defines those markups that are "part of AsciiDoc" the semantics will be defined for _those_ markups.
>
>
> On 13/03/2021 18:06, David Jencks wrote:
>> `include::…` needs to stay special as it does not necessarily introduce block content.
>
>
> As far as I understand, `include::...` are different because they are
> resolved before the document is parsed (in the pure tradition of the C
> pre-processor). That being said, a processor might want to limit or
> filter those directives at a finer granularity than currently allowed by
> the "safe-mode" security-level control [2].
I think all the “safe-mode” stuff is the responsibility of the document set. You are asking, ‘is this document in the document set?’.
I suggest that "safe" mode might better be renamed "constrained" or similar and only apply to the preprocessor and macros that are "part of AsciiDoc" since once an extension is loaded all bets are off.
The include:: preprocessor instruction might be more complicated than the C preprocessor, which I’m not really familiar with. You can use attributes in the include:: instruction, which can be set externally, in the document header, in the document body before the include:: instruction, and by block and block macro processors ‘invoked’ before the include instruction. I don’t recall verifying that attributes added by block/block macro processors are taken into account by include::, but I think they should be for logical consistency.
I don't think extensions should be able to define document attributes, what do they do if a user already defined that attribute? Do they overwrite it, or error and force users to change documents? Neither is very attractive.
I think this means that the source has to be streamed through the parser perhaps line by line as each line may affect the content of the next line. (lines may not be the correct unit here, perhaps blocks? include:: directives just need to start a line, but attributes can only be influenced at block boundaries). I think this also puts some constraints on how much lookahead is possible, since the entire “preprocessed source” isn’t known until parsing is finished.
I’ll mention that I think the current IncludeProcessor and Preprocessor extensions don’t really make sense, but I don’t have a clear idea how to deal with these areas in a better way.
>
> Conditional directive [3] (`ifdef::..., endif::...`) should probably
> remain part of the core specs.
>
Even I haven’t thought of a way to sensibly change or extend their functionality :-)
Just because our combined expertise can't think of a way, somebody else will, guaranteed :-)
In fact a simple example might be that more complex conditional expressions are supported by an extension.
>
>
> On 13/03/2021 02:50, David Jencks wrote:
>> I wonder if requiring this in the spec is reasonable. Being able to> replace behavior could be useful. There could be a set of required
>> extensions.
>
> Could there be a set of "standard extensions" whose behavior is defined
> in the specs but whose implementation is not mandatory?
>
I’d need a concrete example to think this was a good idea. There aren’t that many macro-like behaviors built in and pretty much all seem essential to me.
Agree there needs to be that basic set that are "part of AsciiDoc".
Cheers
Lex
>
> Regards,
> - Sylvain Leroux
>
>
Thanks!
David Jencks
>
> [1]:
> https://www.rubydoc.info/github/asciidoctor/asciidoctor/Asciidoctor/Extensions
> [2]: https://docs.asciidoctor.org/asciidoctor/latest/safe-modes/
> [2]: https://docs.asciidoctor.org/asciidoc/latest/directives/conditionals/
> _______________________________________________
> asciidoc-lang-dev mailing list
> asciidoc-lang-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> https://dev.eclipse.org/mailman/listinfo/asciidoc-lang-dev
_______________________________________________
asciidoc-lang-dev mailing list
asciidoc-lang-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/asciidoc-lang-dev
