Eclipse Community Forums: TMF (Xtext) » Specify the encoding of attached source files

Help

Home

Home » Modeling » TMF (Xtext) » Specify the encoding of attached source files

Show: Today's Messages :: Show Polls :: Message Navigator

Specify the encoding of attached source files [message #1822021]

Tue, 25 February 2020 22:50

Marc Walters

Messages: 11
Registered: November 2019

Junior Member

Hello everyone,

I am currently using Xtext (2.20.0.v20191202-1256) in Eclipse 2019-12 on macOS and experience issues related to the display of special characters in the attached source code. The default encoding of my workspace is set to UTF-8.

When I open an .xtend source file from the Xtext source packages, Eclipse interprets the encoding incorrectly. Jumping to the declaration of 'GeneratorFragment2', for instance, opens org.eclipse.xtext.xtext.generator.generator.GeneratorFragment2.xtend in an editor, but all the guillemets are shown as question marks (as REPLACEMENT CHARACTER, U+FFFD, to be more specific). Unfortunately, this also prevents Eclipse from parsing the enclosed code accordingly.

Setting the default encoding of the workspace to ISO-8859-1 and reopening the file solves the issue, but is not an option for the project that I am working on.

I have tried to set the encoding of the opened .xtend file to ISO-8859-1 via the Edit > Set Encoding dialog and to set the encoding of the org.eclipse.xtext.xtext.generator.source_2.20.0.v20191202-0915.jar source attachment to ISO-8859-1 (in the Java Build Path dialog). Both of these attempts do not seem to have any effect, though.

Is there anything I can do to let Eclipse know about the encoding of the attached .xtend source files?

Thanks!

[Updated on: Tue, 25 February 2020 22:52]

Report message to a moderator

Re: Specify the encoding of attached source files [message #1822024 is a reply to message #1822021]

Wed, 26 February 2020 01:12

Karsten Thoms

Messages: 742
Registered: July 2009
Location: Dortmund, Germany

Senior Member

I'm afraid that I do not see an option either. Xtend files in Xtext source are encoded with ISO-8859-1. Eclipse has no idea that those files should be opened with a different encoding than the workspace default. For files in projects you can specify their encoding on any resource.
It seems Eclipse ignores your setting on the source attachment. This may indicate a bug/missing feature in Eclipse Platform or JDT. This should be reproducible if you create some jar with files having a different encoding. If that is a repro, please file a bug to the JDT project.

Report message to a moderator

Re: Specify the encoding of attached source files [message #1822026 is a reply to message #1822024]

Wed, 26 February 2020 05:20

Christian Dietrich

Messages: 13724
Registered: July 2009

Senior Member

unfortunately: https://github.com/eclipse/xtext-core/issues/282

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Specify the encoding of attached source files [message #1822034 is a reply to message #1822026]

Wed, 26 February 2020 08:36

Marc Walters

Messages: 11
Registered: November 2019

Junior Member

Thank you for your answers. Although I would personally prefer to have everything encoded as UTF-8, wouldn't such a transformation create the reverse situation for people with their default encodings set to anything else than UTF-8?

I have just tried the 'Encoding' option of a source attachment for a JAR with just regular Java files. For these files, the option seems to work quite well: Despite my default workspace encoding set to UTF-8, Eclipse opens the source attachment as ISO-8859-1 if this is the value that I set the option to. Is it possible that .xtend files are just exempt from this? And could it be a suitable middle ground to make this affect .xtend files as well?

[Updated on: Wed, 26 February 2020 08:38]

Report message to a moderator

Re: Specify the encoding of attached source files [message #1822035 is a reply to message #1822034]

Wed, 26 February 2020 08:52

Christian Dietrich

Messages: 13724
Registered: July 2009

Senior Member

i dont know who reads that encoding when. this woulds need to be debugged which i currently not have the time to

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Specify the encoding of attached source files [message #1822052 is a reply to message #1822035]

Wed, 26 February 2020 10:41

Ed Willink

Messages: 6999
Registered: July 2009

Senior Member

Hi

Relying on the workspace encoding is poor practice, since when your colleague works on your file, the setting may be different. Unless of course you are clever enough to ensure that OOMPH imposes cross-workspace uniformity.

I recommend setting all projects to explicitly UTF-8 and Unix line endings since this works everywhere. Ensure that these project settings are in GIT. You may need to do a remedial conversion of any existing rogue files. Thereafter redundant GIT changes will alert you to the bad tools. I recommend that you do not configure GIT to be smart; if GIT changes a file from one style to another, that is IMHO a corruption that is just waiting to cause trouble. UTF-8 + Unix never needs changing.

Which tools read/impose encoding is very tool dependent. I've reported many bugs, but a few years later only about half have been fixed. For Xtext under MWE2 control I use

https://git.eclipse.org/r/plugins/gitiles/ocl/org.eclipse.ocl/+/master/examples/org.eclipse.ocl.examples.build/src/org/eclipse/ocl/examples/build/utilities/ConvertToUnixLineEndings.java

(Using workspace setting is particular bad if any standalone tools are used, since standalone the workspace location is unknown.)

Regards

Ed Willink

Report message to a moderator

Re: Specify the encoding of attached source files [message #1822071 is a reply to message #1822052]

Wed, 26 February 2020 15:38

Marc Walters

Messages: 11
Registered: November 2019

Junior Member

I'll keep that in mind, thank you! Are you also suggesting to use this as a „workaround" for the issue described above? That is, do you mean that since projects should always have an encoding set explicitly, I should change the workspace encoding to ISO-8859-1 whenever I am working on an Xtext project? Or does this create other issues?

Report message to a moderator

Re: Specify the encoding of attached source files [message #1822074 is a reply to message #1822071]

Wed, 26 February 2020 16:45

Christian Dietrich

Messages: 13724
Registered: July 2009

Senior Member

if you want to c&p simpy do from the original source code

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Specify the encoding of attached source files [message #1822078 is a reply to message #1822074]

Wed, 26 February 2020 19:55

Ed Willink

Messages: 6999
Registered: July 2009

Senior Member

Hi

I'm certainly not advocating setting anything deliberately to anything other than UTF-8. Just that leaving the workspace at its default may ensure that you and your colleague have a similar experience since nothing important will use it.

Regards

Ed Willink

Report message to a moderator

Re: Specify the encoding of attached source files [message #1822079 is a reply to message #1822078]

Wed, 26 February 2020 19:59

Christian Dietrich

Messages: 13724
Registered: July 2009

Senior Member

I don't think that the Xtext project has any capacities to do a encoding migration.
Change my mind

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Previous Topic:	Xtext Hover Documentation
Next Topic:	Xtext Hover Documentation [Duplicate - https://www.eclipse.org/forums/index.php/m/1822037/#msg_18220

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Thu Jan 28 00:09:01 GMT 2021

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter