Eclipse Community Forums: TMF (Xtext) » Unit Testing the Language

Help

Home

Home » Modeling » TMF (Xtext) » Unit Testing the Language(Unit Testing)

Show: Today's Messages :: Show Polls :: Message Navigator

Unit Testing the Language [message #1693190]

Wed, 22 April 2015 10:29

Anakreontas Mentis

Messages: 85
Registered: October 2014

Member

Hello.

I followed the instructions of the documentation at https://www.eclipse.org/Xtext/documentation/303_runtime_concepts.html#testing to create unit tests for the grammar.

According to the documentation I need to import an InjectorProvider specific to my language. I searched the files for occurrences of this name but the only class I could find is called StandaloneSetupGenerated which is probably not the class I need. When I created the XText project, a sub-project for tests was created. Is there some option I need to include to generate an Injector provider?

Moreover, I wonder if one can expand on the documentation regarding the relation between tests and files. In the documentation, an example for a test does not declare a class, just a method. What is the purpose of the ParserTest class? Should I create a new file for each test? Must this file declare a class (like ParserTest) or just a method would do? If it is the latter, can I declare multiple methods in a file where each method tests a particular use case of the parser?

Thank you in advance for your reply.

Report message to a moderator

Re: Unit Testing the Language [message #1693195 is a reply to message #1693190]

Wed, 22 April 2015 10:59

Christian Dietrich

Messages: 14496
Registered: July 2009

Senior Member

Hi,

the file is generated in the yourdsl.tests project. the place where the tests should reside.

maybe you removed the j

fragment = junit.Junit4Fragment auto-inject {}

from the workflow or removed some properties there.

(compare it with a newly created project)

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

[Updated on: Wed, 22 April 2015 11:10]

Report message to a moderator

Re: Unit Testing the Language [message #1693326 is a reply to message #1693195]

Thu, 23 April 2015 08:57

Anakreontas Mentis

Messages: 85
Registered: October 2014

Member

Thank you Christian. I did find the file and now testing works fine.

Let me know if I should create a new topic, but I wanted to ask about lexers in XText.

I need to provide a custom lexer and also have multiple lexers or use lexer modes for tokenizing different parts of the input file.
I have used in the past multiple lexers in Antlr2. Perhaps in the version of Antlr XText uses I can achieve my purpose with lexer modes.
In any case, I need to provide a custom lexer. At http://consoliii.blogspot.co.uk/2013/04/xtext-is-incredibly-powerful-framework.html I found instructions on how to provide a custom lexer. However, this method requires manual copying of the custom lexer and seems that there might be maintenance issues with updating the custom lexer when the grammar changes. Is there a better way of introducing modifications to the lexer?

Report message to a moderator

Re: Unit Testing the Language [message #1693327 is a reply to message #1693326]

Thu, 23 April 2015 09:05

Christian Dietrich

Messages: 14496
Registered: July 2009

Senior Member

Hi,

i dont know what you mean by better, since you are the guy that maintains the lexer manually then right?
i dont see the copying problem? you give the ExternalAntlrLexerFragment the place where the lexer grammar is located?!?
of course you have to keep track between your lexer grammar and the pseudo terminals in the grammar, but so what?

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Unit Testing the Language [message #1693328 is a reply to message #1693327]

Thu, 23 April 2015 09:06

Christian Dietrich

Messages: 14496
Registered: July 2009

Senior Member

P.S:

maybe you can elaborate on your usecase. maybe you do not really need an external lexer

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Unit Testing the Language [message #1693335 is a reply to message #1693328]

Thu, 23 April 2015 10:00

Anakreontas Mentis

Messages: 85
Registered: October 2014

Member

What could happen is that I might introduce an implicit lexer rule in the parser and forget to update the custom lexer. It would be ideal if I could include in the XText grammar a preamble for the lexer and provide code that executes when a token matches.

The language I'm building a parser for has three different sections in the same file, each with different lexing rules. For example, one section supports identifiers, other sections don't. Each section has different reserved words and so on. It is similar with java and javadoc comments. Both are in the same file and have different rules. A possible solution is to have lexical rules grouped in modes.

Report message to a moderator

Re: Unit Testing the Language [message #1693337 is a reply to message #1693335]

Thu, 23 April 2015 10:09

Christian Dietrich

Messages: 14496
Registered: July 2009

Senior Member

i dont know of any utils to support that.

=> do full unti testing + test first and you will be fine

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Unit Testing the Language [message #1695348 is a reply to message #1693337]

Wed, 13 May 2015 16:11

Anakreontas Mentis

Messages: 85
Registered: October 2014

Member

Now I can run unit tests on the parser but there is a problem. In the grammar I have rules where I do not assign the result in a variable and the result is not a member of the produced EMF model. To test these rules, I included them in a test input and the test was successful. However, if I put incorrect input, the parser does return an EMF object with the properties set up to the error line. Is it possible to check directly if the parser failed to parse all it's input?

Tests are written in XTend and use:
@Inject
ParseHelper<MyModel> parser

to create a parser.

Report message to a moderator

Re: Unit Testing the Language [message #1695352 is a reply to message #1695348]

Wed, 13 May 2015 16:50

Christian Dietrich

Messages: 14496
Registered: July 2009

Senior Member

Hi,

if you get an result you can ask it for its eresource and that for its errors.
parse helper is just a dump helper class. you can always switch back to plain emf

package org.xtext.example.mydsl4.tests

import javax.inject.Inject
import org.eclipse.xtext.junit4.InjectWith
import org.eclipse.xtext.junit4.XtextRunner
import org.eclipse.xtext.junit4.util.ParseHelper
import org.junit.runner.RunWith
import org.xtext.example.mydsl4.MyDslInjectorProvider
import org.xtext.example.mydsl4.myDsl.Model
import org.junit.Test
import org.eclipse.emf.ecore.resource.ResourceSet
import org.eclipse.emf.common.util.URI
import org.eclipse.xtext.resource.FileExtensionProvider
import org.eclipse.xtext.util.StringInputStream

@RunWith(XtextRunner)
@InjectWith(MyDslInjectorProvider)
class MyDslParserTest {
	
	@Inject
	ResourceSet rs
	
	@Inject
	extension FileExtensionProvider fep
	
	@Test
	def void negativeTest() {
		val resource = rs.createResource(URI.createURI("test." + primaryFileExtension))
		resource.load('''Bla Blub!'''.asStream, null)
		println(resource.errors)
	}
	
	def private asStream(CharSequence s) {
		new StringInputStream(s.toString)
	}
	
}

alteratively you could use more advanced testing technique like http://www.xpect-tests.org/

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Unit Testing the Language [message #1695409 is a reply to message #1695352]

Thu, 14 May 2015 09:47

Anakreontas Mentis

Messages: 85
Registered: October 2014

Member

Thank you, it works fine.

I had to append a random number to the url passed to createURI. Otherwise an exception was thrown when re-executing a test.

Report message to a moderator

Re: Unit Testing the Language [message #1745250 is a reply to message #1693195]

Thu, 06 October 2016 22:25

Hong-Thai Nguyen

Messages: 6
Registered: April 2016

Junior Member

Hi, when i put

fragment = junit.Junit4Fragment {}

in workflow, this line highlight with error saying that not found.

New generation test project don't contain this element neither.

Can you help me ?

Thanks,

Report message to a moderator

Re: Unit Testing the Language [message #1745256 is a reply to message #1745250]

Fri, 07 October 2016 03:47

Christian Dietrich

Messages: 14496
Registered: July 2009

Senior Member

Are you in xtext >= 2.9

Then this thread is outdated regarding workflow

Use the new project wizard to see how to configure the workflow for generating test support

If you use the default settings you will get this for free

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Previous Topic:	Limit scope provider to few files
Next Topic:	Linking issue while programmatic validation

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Sun Jun 04 07:34:55 GMT 2023

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter