Home » Modeling » TMF (Xtext) » Unit testing hidden terminal - ParseHelper returning null model
|
| Re: Unit testing hidden terminal - ParseHelper returning null model [message #986138 is a reply to message #986131] |
Mon, 19 November 2012 07:23   |
 Jan Koehnlein
Messages: 760
Registered: July 2009
Location: Hamburg |
Senior Member |
|
|
Not quite sure what you mean, but the parser gets fed wit the tokens
from the lexer which has run independently. So invalid tokens will never
create a semantic element (by the parser) but should yield a different
node model (parseResult.getRootNode()).
An empty semantic model does not mean an empty node model.
The leaf nodes contain information about the lexer rule they conform to.
If they don't match, they should have a syntax error message attached.
I'd guess this is the way to make the distinction you're looking for.
Am 19.11.12 07:35, schrieb Barrie Treloar:
> Step debugging through my unit tests, I can see that my grammar is being
> parsed correctly and the hidden nodes are being created correctly.
>
> At ParseHelper.parse(InputStream in, URI uriToUse, Map<?, ?> options,
> ResourceSet resourceSet), resource.getContents() is empty, but
> resource.parseResult contains the hidden tokens.
>
> I was attempting to test SL_COMMENT in isolation but this is returning a
> null from the parser, which means I can not tell the difference between
> junk input and SL_COMMENT.
>
> val usrprops = parser.parse('''This is JUNK!''')
> is equivalent to
>
> val usrprops = parser.parse('''// Single line comment''')
> My current work around is to wrap the SL_COMMENT with other valid rules
> and check that only those non-hidden rules are returned.
>
>
> parser.parse('''NOP
> // Single line comment
> NOP''')
>
> Is there a better way?
--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com
---
Get professional support from the Xtext committers at www.typefox.io
|
|
| | | | | | |
| Re: Unit testing hidden terminal - ParseHelper returning null model [message #987265 is a reply to message #987027] |
Sun, 25 November 2012 11:28 |
Jan Koehnlein
Messages: 760
Registered: July 2009
Location: Hamburg |
Senior Member |
|
|
If there's nothing that can be parsed, what should the parser helper
return? Did you inspect the errors and warnings of the parsed resource
resource? This is where we usually store the syntax errors.
Am 23.11.12 04:24, schrieb Barrie Treloar:
> Jan Kohnlein wrote on Thu, 22 November 2012 06:00
>> We're also testing what happens with broken files, so it's a good
>> thing the parse helper does not automatically validate.
>
>
> I'm probably using incorrect terminology, i.e validate.
>
> I'm assuming that broken files means they aren't available in the
> resource set, rather than the contents of the file are not valid
> examples of the grammar.
>
> In my case, I'm unintentionally creating invalid instances of the
> grammar, which ParserHelper is returning null rather than throwing an
> exception.
>
> Tell help out the newbies, it would be really beneficial if ParserHelper
> didn't do that. I wasted a bit of time tracking down what was going on.
>
> I'm sure ParserHelper can be enhanced to suit everyone.
>
--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com
---
Get professional support from the Xtext committers at www.typefox.io
|
|
|
| Re: Unit testing hidden terminal - ParseHelper returning null model [message #987313 is a reply to message #987265] |
Mon, 26 November 2012 00:58 |
Barrie Treloar
Messages: 55
Registered: July 2009 |
Member |
|
|
Jan Kohnlein wrote on Sun, 25 November 2012 06:28If there's nothing that can be parsed, what should the parser helper
return? Did you inspect the errors and warnings of the parsed resource
resource? This is where we usually store the syntax errors.
Sure, my point is that as a noob I just started using ParserHelper as per http://www.eclipse.org/Xtext/documentation.html and found my errors being swallowed with no feedback. This is not the best experience for a new person when using a product.
Isn't this something that could be fixed relatively easily?
Especially since ParserHelper doesn't expose the resource to call getErrors() on.
I must roll my own version, which is duplicating effort.
[Updated on: Mon, 26 November 2012 01:03] Report message to a moderator |
|
|
| Re: Unit testing hidden terminal - ParseHelper returning null model [message #987367 is a reply to message #987313] |
Mon, 26 November 2012 10:17 |
Jan Koehnlein
Messages: 760
Registered: July 2009
Location: Hamburg |
Senior Member |
|
|
No, errors are not swallowed by the parse helper. It just wraps checked
exceptions into unchecked ones.
The misunderstanding is that you expect an Exception. But there is none,
especially not from the underlying Resource API. This is because we use
the same API in our tool, e.g. in the editor. Note that while editing a
file a parse error is the rule, not the exception. Most of the time the
model is broken, as the user is in the midst of some typing. If we just
stopped parsing throwing an exception, you wouldn't get any feedback on
what's behind the (currently) unparseable section, and a huge amount of
follow up errors, e.g. form other resources trying to refer to something
in the unreachable part.
So instead, we provide an error resilient parser, taht tries to recover
from errors and that logs the syntax errors inside the resource such
that they can be handled gracefully by the clients (Resource.getErrors()).
Dealing with clean models is easy. We spent a lot more effort to keep
the tooling working for broken models, too. This is why we need a parse
helper that behaves as the tooling needs it.
For your own languages, you should also have that in mind, e.g. a NPE in
some custom component will likely lead to quite unacceptable tool
behavior. If you pretend no broken models will make it through, you are
on the wrong track.
Am 26.11.12 01:58, schrieb Barrie Treloar:
> [quote title=Jan Kohnlein wrote on Sun, 25 November 2012 06:28]If
> there's nothing that can be parsed, what should the parser helper
> return? Did you inspect the errors and warnings of the parsed resource
> resource? This is where we usually store the syntax errors.
>
> Sure, my point is that as a noob I just started using ParserHelper as
> per http://www.eclipse.org/Xtext/documentation.html and found my errors
> being swallowed with no feedback. This is not the best experience for a
> new person when using a product.
> Isn't this something that could be fixed relatively easily?
>
--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com
---
Get professional support from the Xtext committers at www.typefox.io
|
|
|
| Re: Unit testing hidden terminal - ParseHelper returning null model [message #987369 is a reply to message #987367] |
Mon, 26 November 2012 10:25 |
Barrie Treloar
Messages: 55
Registered: July 2009 |
Member |
|
|
Jan Kohnlein wrote on Mon, 26 November 2012 05:17No, errors are not swallowed by the parse helper. It just wraps checked
exceptions into unchecked ones.
The misunderstanding is that you expect an Exception. But there is none,
especially not from the underlying Resource API. This is because we use
the same API in our tool, e.g. in the editor. Note that while editing a
file a parse error is the rule, not the exception. Most of the time the
model is broken, as the user is in the midst of some typing. If we just
stopped parsing throwing an exception, you wouldn't get any feedback on
what's behind the (currently) unparseable section, and a huge amount of
follow up errors, e.g. form other resources trying to refer to something
in the unreachable part.
I dont expect an Exception, but I do expect to be able to get at the Errors/Warnings, which is not possible via ParserHelper.
The resource is scoped to the method and there is no state in this class.
If I am not meant to be doing it this way, alternative out-of-the-box suggestions are most welcome.
|
|
|
Goto Forum:
Current Time: Fri Apr 19 11:45:24 GMT 2024
Powered by FUDForum. Page generated in 0.02255 seconds |
] 
Search
Help
Register
Login
Home
