Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Specify the encoding of attached source files
Specify the encoding of attached source files [message #1822021] Tue, 25 February 2020 22:50 Go to next message
Marc WaltersFriend
Messages: 11
Registered: November 2019
Junior Member
Hello everyone,

I am currently using Xtext (2.20.0.v20191202-1256) in Eclipse 2019-12 on macOS and experience issues related to the display of special characters in the attached source code. The default encoding of my workspace is set to UTF-8.

When I open an .xtend source file from the Xtext source packages, Eclipse interprets the encoding incorrectly. Jumping to the declaration of 'GeneratorFragment2', for instance, opens org.eclipse.xtext.xtext.generator.generator.GeneratorFragment2.xtend in an editor, but all the guillemets are shown as question marks (as REPLACEMENT CHARACTER, U+FFFD, to be more specific). Unfortunately, this also prevents Eclipse from parsing the enclosed code accordingly.

Setting the default encoding of the workspace to ISO-8859-1 and reopening the file solves the issue, but is not an option for the project that I am working on.

I have tried to set the encoding of the opened .xtend file to ISO-8859-1 via the Edit > Set Encoding dialog and to set the encoding of the org.eclipse.xtext.xtext.generator.source_2.20.0.v20191202-0915.jar source attachment to ISO-8859-1 (in the Java Build Path dialog). Both of these attempts do not seem to have any effect, though.

Is there anything I can do to let Eclipse know about the encoding of the attached .xtend source files?

Thanks!

[Updated on: Tue, 25 February 2020 22:52]

Report message to a moderator

Re: Specify the encoding of attached source files [message #1822024 is a reply to message #1822021] Wed, 26 February 2020 01:12 Go to previous messageGo to next message
Karsten Thoms is currently offline Karsten ThomsFriend
Messages: 739
Registered: July 2009
Location: Dortmund, Germany
Senior Member

I'm afraid that I do not see an option either. Xtend files in Xtext source are encoded with ISO-8859-1. Eclipse has no idea that those files should be opened with a different encoding than the workspace default. For files in projects you can specify their encoding on any resource.
It seems Eclipse ignores your setting on the source attachment. This may indicate a bug/missing feature in Eclipse Platform or JDT. This should be reproducible if you create some jar with files having a different encoding. If that is a repro, please file a bug to the JDT project.
Re: Specify the encoding of attached source files [message #1822026 is a reply to message #1822024] Wed, 26 February 2020 05:20 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13628
Registered: July 2009
Senior Member
unfortunately: https://github.com/eclipse/xtext-core/issues/282

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Specify the encoding of attached source files [message #1822034 is a reply to message #1822026] Wed, 26 February 2020 08:36 Go to previous messageGo to next message
Marc WaltersFriend
Messages: 11
Registered: November 2019
Junior Member
Thank you for your answers. Although I would personally prefer to have everything encoded as UTF-8, wouldn't such a transformation create the reverse situation for people with their default encodings set to anything else than UTF-8?

I have just tried the 'Encoding' option of a source attachment for a JAR with just regular Java files. For these files, the option seems to work quite well: Despite my default workspace encoding set to UTF-8, Eclipse opens the source attachment as ISO-8859-1 if this is the value that I set the option to. Is it possible that .xtend files are just exempt from this? And could it be a suitable middle ground to make this affect .xtend files as well?

[Updated on: Wed, 26 February 2020 08:38]

Report message to a moderator

Re: Specify the encoding of attached source files [message #1822035 is a reply to message #1822034] Wed, 26 February 2020 08:52 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13628
Registered: July 2009
Senior Member
i dont know who reads that encoding when. this woulds need to be debugged which i currently not have the time to

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Specify the encoding of attached source files [message #1822052 is a reply to message #1822035] Wed, 26 February 2020 10:41 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6874
Registered: July 2009
Senior Member
Hi

Relying on the workspace encoding is poor practice, since when your colleague works on your file, the setting may be different. Unless of course you are clever enough to ensure that OOMPH imposes cross-workspace uniformity.

I recommend setting all projects to explicitly UTF-8 and Unix line endings since this works everywhere. Ensure that these project settings are in GIT. You may need to do a remedial conversion of any existing rogue files. Thereafter redundant GIT changes will alert you to the bad tools. I recommend that you do not configure GIT to be smart; if GIT changes a file from one style to another, that is IMHO a corruption that is just waiting to cause trouble. UTF-8 + Unix never needs changing.

Which tools read/impose encoding is very tool dependent. I've reported many bugs, but a few years later only about half have been fixed. For Xtext under MWE2 control I use

https://git.eclipse.org/r/plugins/gitiles/ocl/org.eclipse.ocl/+/master/examples/org.eclipse.ocl.examples.build/src/org/eclipse/ocl/examples/build/utilities/ConvertToUnixLineEndings.java

(Using workspace setting is particular bad if any standalone tools are used, since standalone the workspace location is unknown.)

Regards

Ed Willink
Re: Specify the encoding of attached source files [message #1822071 is a reply to message #1822052] Wed, 26 February 2020 15:38 Go to previous messageGo to next message
Marc WaltersFriend
Messages: 11
Registered: November 2019
Junior Member
I'll keep that in mind, thank you! Are you also suggesting to use this as a ā€˛workaround" for the issue described above? That is, do you mean that since projects should always have an encoding set explicitly, I should change the workspace encoding to ISO-8859-1 whenever I am working on an Xtext project? Or does this create other issues?
Re: Specify the encoding of attached source files [message #1822074 is a reply to message #1822071] Wed, 26 February 2020 16:45 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13628
Registered: July 2009
Senior Member
if you want to c&p simpy do from the original source code

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Specify the encoding of attached source files [message #1822078 is a reply to message #1822074] Wed, 26 February 2020 19:55 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6874
Registered: July 2009
Senior Member
Hi

I'm certainly not advocating setting anything deliberately to anything other than UTF-8. Just that leaving the workspace at its default may ensure that you and your colleague have a similar experience since nothing important will use it.

Regards

Ed Willink
Re: Specify the encoding of attached source files [message #1822079 is a reply to message #1822078] Wed, 26 February 2020 19:59 Go to previous message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13628
Registered: July 2009
Senior Member
I don't think that the Xtext project has any capacities to do a encoding migration.
Change my mind


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Previous Topic:Xtext Hover Documentation
Next Topic:Xtext Hover Documentation [Duplicate - https://www.eclipse.org/forums/index.php/m/1822037/#msg_18220
Goto Forum:
  


Current Time: Fri Oct 23 05:43:47 GMT 2020

Powered by FUDForum. Page generated in 0.02711 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top