Eclipse Community Forums: TMF (Xtext) » Specify the encoding of attached source files

Home » Modeling » TMF (Xtext) » Specify the encoding of attached source files

Specify the encoding of attached source files [message #1822021]

Tue, 25 February 2020 17:50

Eclipse User

Hello everyone,

I am currently using Xtext (2.20.0.v20191202-1256) in Eclipse 2019-12 on macOS and experience issues related to the display of special characters in the attached source code. The default encoding of my workspace is set to UTF-8.

When I open an .xtend source file from the Xtext source packages, Eclipse interprets the encoding incorrectly. Jumping to the declaration of 'GeneratorFragment2', for instance, opens org.eclipse.xtext.xtext.generator.generator.GeneratorFragment2.xtend in an editor, but all the guillemets are shown as question marks (as REPLACEMENT CHARACTER, U+FFFD, to be more specific). Unfortunately, this also prevents Eclipse from parsing the enclosed code accordingly.

Setting the default encoding of the workspace to ISO-8859-1 and reopening the file solves the issue, but is not an option for the project that I am working on.

I have tried to set the encoding of the opened .xtend file to ISO-8859-1 via the Edit > Set Encoding dialog and to set the encoding of the org.eclipse.xtext.xtext.generator.source_2.20.0.v20191202-0915.jar source attachment to ISO-8859-1 (in the Java Build Path dialog). Both of these attempts do not seem to have any effect, though.

Is there anything I can do to let Eclipse know about the encoding of the attached .xtend source files?

Thanks!

[Updated on: Tue, 25 February 2020 17:52] by Moderator

Re: Specify the encoding of attached source files [message #1822024 is a reply to message #1822021]

Tue, 25 February 2020 20:12

Eclipse User

I'm afraid that I do not see an option either. Xtend files in Xtext source are encoded with ISO-8859-1. Eclipse has no idea that those files should be opened with a different encoding than the workspace default. For files in projects you can specify their encoding on any resource.
It seems Eclipse ignores your setting on the source attachment. This may indicate a bug/missing feature in Eclipse Platform or JDT. This should be reproducible if you create some jar with files having a different encoding. If that is a repro, please file a bug to the JDT project.

Re: Specify the encoding of attached source files [message #1822026 is a reply to message #1822024]

Wed, 26 February 2020 00:20

Eclipse User

unfortunately: https://github.com/eclipse/xtext-core/issues/282

Re: Specify the encoding of attached source files [message #1822034 is a reply to message #1822026]

Wed, 26 February 2020 03:36

Eclipse User

Thank you for your answers. Although I would personally prefer to have everything encoded as UTF-8, wouldn't such a transformation create the reverse situation for people with their default encodings set to anything else than UTF-8?

I have just tried the 'Encoding' option of a source attachment for a JAR with just regular Java files. For these files, the option seems to work quite well: Despite my default workspace encoding set to UTF-8, Eclipse opens the source attachment as ISO-8859-1 if this is the value that I set the option to. Is it possible that .xtend files are just exempt from this? And could it be a suitable middle ground to make this affect .xtend files as well?

[Updated on: Wed, 26 February 2020 03:38] by Moderator

Re: Specify the encoding of attached source files [message #1822035 is a reply to message #1822034]

Wed, 26 February 2020 03:52

Eclipse User

i dont know who reads that encoding when. this woulds need to be debugged which i currently not have the time to

Re: Specify the encoding of attached source files [message #1822052 is a reply to message #1822035]

Wed, 26 February 2020 05:41

Eclipse User

Hi

Relying on the workspace encoding is poor practice, since when your colleague works on your file, the setting may be different. Unless of course you are clever enough to ensure that OOMPH imposes cross-workspace uniformity.

I recommend setting all projects to explicitly UTF-8 and Unix line endings since this works everywhere. Ensure that these project settings are in GIT. You may need to do a remedial conversion of any existing rogue files. Thereafter redundant GIT changes will alert you to the bad tools. I recommend that you do not configure GIT to be smart; if GIT changes a file from one style to another, that is IMHO a corruption that is just waiting to cause trouble. UTF-8 + Unix never needs changing.

Which tools read/impose encoding is very tool dependent. I've reported many bugs, but a few years later only about half have been fixed. For Xtext under MWE2 control I use

https://git.eclipse.org/r/plugins/gitiles/ocl/org.eclipse.ocl/+/master/examples/org.eclipse.ocl.examples.build/src/org/eclipse/ocl/examples/build/utilities/ConvertToUnixLineEndings.java

(Using workspace setting is particular bad if any standalone tools are used, since standalone the workspace location is unknown.)

Regards

Ed Willink

Re: Specify the encoding of attached source files [message #1822071 is a reply to message #1822052]

Wed, 26 February 2020 10:38

Eclipse User

I'll keep that in mind, thank you! Are you also suggesting to use this as a „workaround" for the issue described above? That is, do you mean that since projects should always have an encoding set explicitly, I should change the workspace encoding to ISO-8859-1 whenever I am working on an Xtext project? Or does this create other issues?

Re: Specify the encoding of attached source files [message #1822074 is a reply to message #1822071]

Wed, 26 February 2020 11:45

Eclipse User

if you want to c&p simpy do from the original source code

Re: Specify the encoding of attached source files [message #1822078 is a reply to message #1822074]

Wed, 26 February 2020 14:55

Eclipse User

Hi

I'm certainly not advocating setting anything deliberately to anything other than UTF-8. Just that leaving the workspace at its default may ensure that you and your colleague have a similar experience since nothing important will use it.

Regards

Ed Willink

Re: Specify the encoding of attached source files [message #1822079 is a reply to message #1822078]

Wed, 26 February 2020 14:59

Eclipse User

I don't think that the Xtext project has any capacities to do a encoding migration.
Change my mind

Previous Topic:	Xtext Hover Documentation
Next Topic:	Xtext Hover Documentation [Duplicate - https://www.eclipse.org/forums/index.php/m/1822037/#msg_18220

Goto Forum:

-=] Back to Top [=-

Current Time: Wed Jul 23 17:41:12 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter