Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[asciidoc-lang-dev] Include expansion and block parsing [Was: Extension hooks in Asciidoc]

NOTE: I was alerted a few days ago that some users of this mailing list couldn't see my messages because they were getting caught by spam filters. Since then, some configuration changes have been applied to the list. If you happen to detect that my messages keep going to the spam folder, please let me know.


I take back this message from a few days ago to ask about the include:: directive. I haven't seen anyone question what David have said about it (which is congruent with my own experiments) and think that the consequences are quite far reaching. So I ask for clarification.

El 15/3/21 a las 18:26, David Jencks escribió:

The include:: preprocessor instruction might be more complicated than the C preprocessor, which I’m not really familiar with.  You can use attributes in the include:: instruction, which can be set externally, in the document header, in the document body before the include:: instruction, and by block and block macro processors ‘invoked’ before the include instruction.  I don’t recall verifying that attributes added by block/block macro processors are taken into account by include::, but I think they should be for logical consistency.
I think this means that the source has to be streamed through the parser perhaps line by line as each line may affect the content of the next line.  (lines may not be the correct unit here, perhaps blocks? include:: directives just need to start a line, but attributes can only be influenced at block boundaries). I think this also puts some constraints on how much lookahead is possible, since the entire “preprocessed source” isn’t known until parsing is finished.

I’ll mention that I think the current IncludeProcessor and Preprocessor extensions don’t really make sense, but I don’t have a clear idea how to deal with these areas in a better way.

To elaborate on what David says: there is a circular dependency between include expansion, attribute resolution and block parsing:

* A line with contents

  :key: value

is considered an attribute entry only in some places (e.g., not inside a passthrough or verbatim delimited block), so parsing affects attribute setting and resolution.

* Attribute value affects include expansion, as David pointed out:


* Include expansion affects both attribute resolution and parsing. An include file can left open any number of delimited blocks or other constructions, thus affecting subsequent parsing, or the context in which a potential attribute entry is parsed. And attributes can be defined in and set in included files.

By some tests I've done, even comment strip-out cannot easily being done as an actual preprocess: an include can left open a comment block, affecting lines external to the included file.

All that means that in current AsciiDoc it's not possible to have an actual preprocessing as a separate step. Include expansion needs to be performed at the same time of (block) parsing (as David says). Or am I missing something?

How are the so called "preprocessor directives" implemented in Asciidoctor and other tools?


Back to the top