Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Rich Client Platform (RCP) » Text Editor - Customize word separators
Text Editor - Customize word separators [message #449047] Tue, 09 May 2006 14:23 Go to next message
Antonio Gurisatti is currently offline Antonio GurisattiFriend
Messages: 109
Registered: July 2009
Senior Member
Hi everybody

I'm building a highlighted Text Editor for SQL and it works well with a few exceptions.

1. I can't find which class is in charge of detecting where a word finishes (word separator). By default, the text editor is considering the underscore (_) character as a word separator and I need to change this so that a word like "select_column" is not displayed with "select" highlighted.

2. I'd like to highlight some punctuation characters. If I do so, the word separating behavior changes and strings like "count('x')" stops showing "count" highlighted.

So both issues depend on the behavior of the word separators.


Any help would be very much appreciated.

Thanks a heap to everybody.
Re: Text Editor - Customize word separators [message #449247 is a reply to message #449047] Thu, 11 May 2006 13:06 Go to previous messageGo to next message
Antonio Gurisatti is currently offline Antonio GurisattiFriend
Messages: 109
Registered: July 2009
Senior Member
Hi,

No one for this one?

Im missing Paul Webster, Alex Blewitt, Rohit Khariwal and some other gurus. Are they at holidays?

Please, I'd appreciate very much some light in the subject.

Thanks a lot
Re: Text Editor - Customize word separators [message #449257 is a reply to message #449047] Thu, 11 May 2006 13:43 Go to previous messageGo to next message
Tom Eicher is currently offline Tom EicherFriend
Messages: 11
Registered: July 2009
Junior Member
Antonio Gurisatti wrote:

> Im missing Paul Webster, Alex Blewitt, Rohit Khariwal and some other gurus. Are they at holidays?

I am neither of them, but will try to answer anyway ;-)
> 1. I can't find which class is in charge of detecting where a word
> finishes (word separator). By default, the text editor is considering
> the underscore (_) character as a word separator and I need to change
> this so that a word like "select_column" is not displayed with
> "select" highlighted.

Do you use a WordRule to hightlight text? See IWordDetector then
(WordRule takes one as constructor argument).

> 2. I'd like to highlight some punctuation characters. If I do so, the
> word separating behavior changes and strings like "count('x')" stops
> showing "count" highlighted.

For an example, see
org.eclipse.jdt.internal.ui.text.java.JavaCodeScanner.Operat orRule.

HTH, tom
Re: Text Editor - Customize word separators [message #449369 is a reply to message #449257] Thu, 11 May 2006 14:45 Go to previous messageGo to next message
Antonio Gurisatti is currently offline Antonio GurisattiFriend
Messages: 109
Registered: July 2009
Senior Member
Tom,

Sorry I missed you in my list of gurus. Now I remeber you helped me more than once.

Going to the subject, the answer to your question is:

I'm using Document, WordDetector, CommentScanner, CodeScanner, and all that stuff. I have my own customized word rule to detect words in a case insensitive way.

Looks like the class you suggested me to look at is the clue to my problem.

I'll study it and try to implement it. Hope not to bug you more with this.

Thanks a lot.
Re: Text Editor - Customize word separators [message #449378 is a reply to message #449257] Thu, 11 May 2006 16:05 Go to previous messageGo to next message
Antonio Gurisatti is currently offline Antonio GurisattiFriend
Messages: 109
Registered: July 2009
Senior Member
Tom,

I solved the first of my problems by doing:

I defined a new token "connector" with default TextAttributes and added it to may word rules (Adding just the character "_". This has the effect that this character stops being considered as a "typed in word separator" and so the highlighting is correct.

But I'm not satisfied. This is achieved as a side effect. What I'd really like to find is the class that decides when a typed in word ends. I've been looking quite a few and didn't find it yet.

The class you suggested me does not appear to have a clue for this but it does have for the second issue: Coloured punctuation.

As a matter of fact, the only clue I have by now is the sentence "c != ICharacterScanner.EOF && fDetector.isWordPart" in the "evaluate" method in the WorldRule (IRule class).

AG
Re: Text Editor - Customize word separators [message #449381 is a reply to message #449378] Thu, 11 May 2006 16:15 Go to previous messageGo to next message
Tom Eicher is currently offline Tom EicherFriend
Messages: 11
Registered: July 2009
Junior Member
Antonio Gurisatti wrote:
> Tom,
>
> I solved the first of my problems by doing:
>
> I defined a new token "connector" with default TextAttributes and added it to may word rules (Adding just the character "_". This has the effect that this character stops being considered as a "typed in word separator" and so the highlighting is correct.
>
> But I'm not satisfied. This is achieved as a side effect. What I'd really like to find is the class that decides when a typed in word ends. I've been looking quite a few and didn't find it yet.
>
> The class you suggested me does not appear to have a clue for this but it does have for the second issue: Coloured punctuation.
>
> As a matter of fact, the only clue I have by now is the sentence "c != ICharacterScanner.EOF && fDetector.isWordPart" in the "evaluate" method in the WorldRule (IRule class).
>
> AG

Well, what does your IWordDetector implementation look like? There is no
default implementation in the eclipse SDK. It should be something along
the lines of org.eclipse.jdt.internal.ui.text.JavaWordDetector.

-tom
Re: Text Editor - Customize word separators [message #449384 is a reply to message #449381] Thu, 11 May 2006 17:00 Go to previous messageGo to next message
Antonio Gurisatti is currently offline Antonio GurisattiFriend
Messages: 109
Registered: July 2009
Senior Member
Well, my WordDetector looks like this:

public boolean isWordStart(char c) {

for (int i = 0, n = SQLSyntax.KEYWORDS.length; i < n; i++)
if (c == ((String) SQLSyntax.KEYWORDS[i]).charAt(0))
return true;
for (int i = 0, n = SQLSyntax.FUNCTIONS.length; i < n; i++)
if (c == ((String) SQLSyntax.FUNCTIONS[i]).charAt(0))
return true;
for (int i = 0, n = SQLSyntax.DATATYPES.length; i < n; i++)
if (c == ((String) SQLSyntax.DATATYPES[i]).charAt(0))
return true;
for (int i = 0, n = SQLSyntax.CONNECTORS.length; i < n; i++)
if (c == ((String) SQLSyntax.CONNECTORS[i]).charAt(0))
return true;
// for (int i = 0, n = SQLSyntax.PUNCTUATION.length; i < n; i++)
// if (c == ((String) SQLSyntax.PUNCTUATION[i]).charAt(0))
// return true;
return false;
}

/**
* Gets whether the specified character is part of a word
*
* @return boolean
*/
public boolean isWordPart(char c) {
for (int i = 0, n = SQLSyntax.KEYWORDS.length; i < n; i++)
if (((String) SQLSyntax.KEYWORDS[i]).indexOf(c) != -1)
return true;
for (int i = 0, n = SQLSyntax.FUNCTIONS.length; i < n; i++)
if (((String) SQLSyntax.FUNCTIONS[i]).indexOf(c) != -1)
return true;
for (int i = 0, n = SQLSyntax.DATATYPES.length; i < n; i++)
if (((String) SQLSyntax.DATATYPES[i]).indexOf(c) != -1)
return true;
for (int i = 0, n = SQLSyntax.CONNECTORS.length; i < n; i++)
if (((String) SQLSyntax.CONNECTORS[i]).indexOf(c) != -1)
return true;
// for (int i = 0, n = SQLSyntax.PUNCTUATION.length; i < n; i++)
// if (((String) SQLSyntax.PUNCTUATION[i]).indexOf(c) != -1)
// return true;
return false;
}

As far as I understand, this is used to know if what you are typing is the start or part of one of your target words.

What I can smell in the air is that WorldRule somehow is (indirectly) responsable for the typed in word endings.

As I understand, WorldRule uses a buffer to save the characters you are typing and an algorithm to narrow the search for a special word (in combination with WordDetector).

Suppose I'm typyn the special word "select". In the moment I type the "t" the word gets highlighted. If I continue and type, for example an "x" the word is not highlited any more. Why? Because the "x" is part of the word.

But if I type a space, the word continues highlited. Why? Because the space character is considered as a "word separator" and so, somehow and somewhere, the buffers are cleaned, counters and other stuff are restored and starts a new cicle.

The proble here is that, by default, the character"_" is considered as a "word separator", so if I type "select_" the "select" part remains highligted. And I'm trying to change that. I need that caharacters like "_" are considered as part of a typed word and don't cause a word termination.

AG
Re: Text Editor - Customize word separators [message #449386 is a reply to message #449384] Thu, 11 May 2006 17:53 Go to previous messageGo to next message
Tom Eicher is currently offline Tom EicherFriend
Messages: 11
Registered: July 2009
Junior Member
Antonio,

I believe your WordDetector is too complicated - it's simply there to
tokenize the character stream into word chunks, preferably using
character classes like Character.is{Word,UnicodeIdentifier,...}Char().

In your case, you have to make sure that the underscore is also in the
set of character considered word chars.

HTH, tom
Re: Text Editor - Customize word separators [message #449391 is a reply to message #449386] Thu, 11 May 2006 18:19 Go to previous messageGo to next message
Antonio Gurisatti is currently offline Antonio GurisattiFriend
Messages: 109
Registered: July 2009
Senior Member
Tom,

I'm sure you are quite rigth. Th thing is that I'm rather new to all this and so I took that from an example using the monkeys principle (see and imitate). Know that I have a better understanding of the matter I'll try to do it in a better fashion.

By the way, thanks to your advice I managed to solve the other problem I had: Colouring punctuation chars. I built a class implementing IRule and works correctly.

My famous "word separator" case is also by now resolved using a dirty trick. I'll continue working on it until I find a really correct solution.

Thanks a lot for your unvaluable help.

By the way (he, he, he...) I've another post without an answer. I reposted it today. I'd like to put a SourceViewer that uses this same SQL Text Editor in a Dialog. As obviuous, it is shown with a default size (quite tiny) and I can't fin a way to resize it. If you have an idea of what to do, I'll be very, very grateful.

AG
Re: Text Editor - Customize word separators [message #1408255 is a reply to message #449391] Thu, 14 August 2014 15:54 Go to previous message
Dan Busenbarrick is currently offline Dan BusenbarrickFriend
Messages: 1
Registered: August 2014
Junior Member
Could someone help me make the edits necessary to alter the word rules outlined above? I'm a long-time Eclipse user, but don't know much Java and have no idea where to dig in and make changes.

I'd like to change the underscore to be considered a word character, not a separator, for all languages.

Thanks for any help you can offer.
Previous Topic:Error Export Eclipse Product
Next Topic:Extend Output of Eclipse Product Export Wizard
Goto Forum:
  


Current Time: Sat Sep 05 04:03:14 GMT 2015

Powered by FUDForum. Page generated in 0.13877 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software