[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [pdt-dev] remove project notion from all php lexers

Dawid your answer gave me some ideas.

For now, we (mainly PHPStructuredEditor in a dedicated thread) tell (in a hacky way) which highlighter should be used (depending on the provided project value). In all other cases we default to most recent PHP version with some default settings regarding ASP tags handling,...

Right way to do (I think) would be:

one (and only one) highlighter is used and it's the

highlighter who tells us in which php version the found tokens are supported.
We already use this concept in classes SimpleProposal and PHPSimpleTypes.

For example, after document scanning,
- the token XYZ was found and the highlighter tells us this token was supported from version 5.4 and deprecated in version 7.0 (but still valid)
- the token ABC was found and the highlighter tells us this token was supported from version 5.4 and invalid since version 7.0
- the token FOO was found and it has no version restrictions
- the token BAR was found and it's valid since version 5.1
...

We let then a semantic highlighter decide how to highlight those tokens depending on the version of the current project.
If there are complicated structures that were added (for example heredocs) after a specific php version, they should also be managed by separated highlighters, but maybe we could also encode this information somehow in the tokens (for example tell that some tokens act like begin-end sections and allow specific tokens in their section bodies).

That's just an unfinished idea for now ;)



From: zulus@xxxxxxxxx
Date: Fri, 12 Aug 2016 02:01:12 -0700
To: pdt-dev@xxxxxxxxxxx
Subject: Re: [pdt-dev] remove project notion from all php lexers

Hi, 

I like idea when lexers/parsers are haven't connection to specific project. Rather than some kind of API IPHPSettings ;)

Wy we need for highlighters for each php version for now:
1. New keywords should be highlighted based on php version.
2. Highlighting after short echo "<?=" should be always enabled since 5.4, on older version only when short tags are enabled
3. ASP tags are perfectly valid until 5.6. So yes, it's still required for 5 - 5.6
4. Context sensitive keywords, since PHP for ex. "forEach" can be used as method name. It's not clear what will happen in next PHP versions. I personally prefer correct highlighting in this cases..

Maybe in the future (couple years later), when we drop PHP 5.* support will be able to keep one highlighter between PHP Versions with configurable keyword visibility. Or maybe it's possible to write one universal, confgurable highlighter for ex. in ANTLRv4 ?  

-- 
Dawid Pakuła

From: Michal Niewrzal <michal.n@xxxxxxxx>
Reply: PDT Developers <pdt-dev@xxxxxxxxxxx>
Date: 11 August 2016 at 21:48:55
To: pdt-dev@xxxxxxxxxxx <pdt-dev@xxxxxxxxxxx>
Subject:  [pdt-dev] remove project notion from all php lexers

+1 from me for removing IProject from lexer related elements and one highlighter for all versions


I think removing IProject from code is very good idea. In general it would be nice to decouple lexer/parser code from Eclipse api as much as possible. I can imagine that lexer/parser/ast source code would be interesting as separate bundles without Eclipse/DLTK/WST dependencies.



At the moment I cannot imagine any problem with only one highlighter. Parser will do the validation and that should be enough from user point.


Michal


Od: pdt-dev-bounces@xxxxxxxxxxx <pdt-dev-bounces@xxxxxxxxxxx> w imieniu użytkownika thierry blind <thierryblind@xxxxxxx>
Wysłane: 11 sierpnia 2016 16:44
Do: pdt-dev@xxxxxxxxxxx
Temat: [pdt-dev] remove project notion from all php lexers
 
Hi team,
I have a technical question :)
While I worked on bug 498877, I saw that the lexers need project informations for various operations (like to retrieve task tags defined for a specific project or to retrieve the PHP version of a project to decide which highlighter version to use, ...).
Sadly there are cases where lexers are created on-the-fly by eclipse to do some specific operations (document reparsing, reconciling or workspace builds are done on temporary&buffered documents for example) that are "detached" from any project notion.
Technically speaking, in those cases we work with project properties set to null ;)
So I would like to remove in the future all traces of "project" properties inside of php tokenizer, highlighters, PhpScriptRegion, PhpTokenContainer,... as it is not the way to go (looking how eclipse works). A good example is the hacky way used to set the project for document lexers in the PHPStructuredEditor class :
PhpSourceParser.editFile.set(resource);
Clearly that's a hack to find a way to provide a project value to lexers created on-the-fly by eclipse ;)

For now in the PDT lexers, project informations seem only necessary to :
- see if a project supports ASP tags (is this still used?)
- retrieve task tags defined per project (patch for bug 498877 will remove that)
- choose the highlighter version depending on project settings

So my question is:

why do we actually need a highlighter per php version? Why just not use the highlighter for the most recent supported php version (7.1)?
Is there any reason to not highlight most recently added php keywords on older php versions (even if not "supported")?
It would simplify a lot of code just to use one highlighter + php tokenizer + phpdoc scanner everywhere ;)

Thierry.

_______________________________________________
pdt-dev mailing list
pdt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev

_______________________________________________ pdt-dev mailing list pdt-dev@xxxxxxxxxxx To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/pdt-dev