Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » weird encoding problem with properties-files(wrong encoding when generating properties-file the first time)
weird encoding problem with properties-files [message #899325] Tue, 31 July 2012 13:28 Go to next message
Annette Pohl is currently offline Annette Pohl
Messages: 7
Registered: July 2012
Junior Member
Hi!

I am working on a DSL that generates properties files in order to deal with translations. I came across a weird encoding problem:
The encoding for properties files must be ISO-8859-1. Eclipse sets the encoding automatically correct as soon as the properties file is created. But when the generator is called for the first time, the properties file is corrupted, although the encoding is correct.

You might want to try the following example:

grammar:

Model:
	keyAndValues+=KeyAndValue*;
	
KeyAndValue:
	name = ID value = STRING; 


generator:

class MyDslGenerator implements IGenerator {
	
	override void doGenerate(Resource resource, IFileSystemAccess fsa) {
		if (resource.contents.size > 0) {
			val model = resource.contents.get(0) as Model;
			fsa.generateFile("test.properties", generatePropertyFile(model));
		}
	}
	
	def generatePropertyFile(Model it) '''
		«FOR KeyAndValue keyAndValue : keyAndValues»
			«keyAndValue(keyAndValue)»
		«ENDFOR»
	'''
	
	def keyAndValue(KeyAndValue it) '''«name» = «value»'''
}


test.mydsl:

testKey1 = äöü


results in test.properties:

testKey1 = aöü


If you call fsa.generateFile twice, everything is fine.

I debugged into EclipseResourceFileSystemAccess2 and saw that there is a distinction if the file already exists or not. Maybe eclipse sets the encoding to ISO-8859-1 after the file was generated and the content written to it and corrupts it by doing so?
Re: weird encoding problem with properties-files [message #899415 is a reply to message #899325] Tue, 31 July 2012 18:10 Go to previous messageGo to next message
Sebastian Zarnekow is currently offline Sebastian Zarnekow
Messages: 2915
Registered: July 2009
Senior Member
Hi Annette,

we use IFile.getCharset to determine the encoding. You want to bind a
custom IFileSystemAccess that returns ISO-8859-1 if the file extension
is *.properties?

Regards,
Sebastian
--
Looking for professional support for Xtext, Xtend or Eclipse Modeling?
Go visit: http://xtext.itemis.com

Am 31.07.12 15:28, schrieb Annette Pohl:
> Hi!
>
> I am working on a DSL that generates properties files in order to deal
> with translations. I came across a weird encoding problem:
> The encoding for properties files must be ISO-8859-1. Eclipse sets the
> encoding automatically correct as soon as the properties file is
> created. But when the generator is called for the first time, the
> properties file is corrupted, although the encoding is correct.
>
> You might want to try the following example:
>
> grammar:
>
>
> Model:
> keyAndValues+=KeyAndValue*;
>
> KeyAndValue:
> name = ID value = STRING;
>
> generator:
>
>
> class MyDslGenerator implements IGenerator {
>
> override void doGenerate(Resource resource, IFileSystemAccess fsa) {
> if (resource.contents.size > 0) {
> val model = resource.contents.get(0) as Model;
> fsa.generateFile("test.properties",
> generatePropertyFile(model));
> }
> }
>
> def generatePropertyFile(Model it) '''
> «FOR KeyAndValue keyAndValue : keyAndValues»
> «keyAndValue(keyAndValue)»
> «ENDFOR»
> '''
>
> def keyAndValue(KeyAndValue it) '''«name» = «value»'''
> }
>
>
> test.mydsl:
>
>
> testKey1 = äöü
>
>
> results in test.properties:
>
>
> testKey1 = aöü
>
>
> If you call fsa.generateFile twice, everything is fine.
>
> I debugged into EclipseResourceFileSystemAccess2 and saw that there is a
> distinction if the file already exists or not. Maybe eclipse sets the
> encoding to ISO-8859-1 after the file was generated and the content
> written to it and corrupts it by doing so?
Re: weird encoding problem with properties-files [message #899526 is a reply to message #899415] Wed, 01 August 2012 09:48 Go to previous message
Annette Pohl is currently offline Annette Pohl
Messages: 7
Registered: July 2012
Junior Member
Hi Sebastian,

thanks for your answer. This is working now:

public class MyDslFileSystemAccess extends EclipseResourceFileSystemAccess2 {

	private static final Logger LOGGER = LoggerFactory.getLogger( MyDslFileSystemAccess.class );

	@Override
	protected IFile getFile( String fileName, String outputName ) {
		IFile file = super.getFile( fileName, outputName );
		if (fileName != null && fileName.endsWith( "properties" ) && !file.exists()) {
			try {
				file.create(getInputStream("", file.getCharset(true)), true, getMonitor());
			} catch (CoreException e) {
				LOGGER.error( "failed to create file {} : {}", fileName, e.getMessage() );
			}
		}
		return file;
	}
}


I tried first to set the charset for file, but I got an exception when I called the generator the first time and the properties file did not exist yet. The exception complained that the resource did not exist. When I called the generator the second time everything was working and the charset could be set - but this was working anyway. My problem was the first call of the code generator.

So I created the file before setting the charset and now it was working. Then I tried if it was necessary to set the charset and found out that it was not. Creating the file was enough.

Thanks for your help - it pointed me to the right direction.
Previous Topic:scoping problem
Next Topic:[Grammar] Declaration and definition of variables for a DSL
Goto Forum:
  


Current Time: Fri Oct 31 11:37:50 GMT 2014

Powered by FUDForum. Page generated in 0.01691 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software