Home » Modeling » TMF (Xtext) » The Story of cq-markup-xtext (pt. 1)
| The Story of cq-markup-xtext (pt. 1) [message #556596] |
Thu, 02 September 2010 00:59  |
 Ralf Ebert
Messages: 72
Registered: July 2009 |
Member |
|
|
Created cq-markup-xtext
As I planned to implement a markup language parser in Xtext
for practicing purposes on my way to Xtext wizardy, I'm glad
that I can motivate myself now not only by doing that, but
also by participating in a challenge:
http://www.codequarterly.com/code-challenges/markup/
I plan to work on this for about an hour a day and if it
fully works until the deadline (October 15th, 2010), I will
enter it as an entry to the challenge. I think it will be
quite interesting to see different implementations of such a
parser. Also, if I end up with a result I like, I will
change+extend the syntax to Markdown (my wikitext format of
choice) and maintain it as a Java Markdown parser bundle for
everyone.
Doing this I thought I'd write a bit about my progress,
things that worked out of the box, things that didn't work
quite right on the first time, things that I found too hard
or too easy, behavior that is beyond my understanding,
things that I believe need to be improved, things that are
fine as they are. I hope it's not boring but interesting. As
always, I'm very interested in hearing your feedback and
suggestions.
btw, the code is located here:
http://github.com/ralfebert/cq-markup-xtext
Thanks to Holger Schill from itemis for the very helpful
Xtext coaching!
http://github.com/ralfebert/cq-markup-xtext/commit/ddbc54f
Provided target platform
plugins/com.codequarterly.markup.releng/markup.target
Eclipse and EMF and Xtext and JDOM and Jakarta Commons are
needed for this and provided by the target platform in
com.codequarterly.markup.releng. If the target platform
doesn't work, reload it from the .target file, reload it
from the preferences dialog, restart Eclipse, rename the
.target file, use a fresh workspace, file a bug against p2,
don't ask me about it.
http://github.com/ralfebert/cq-markup-xtext/commit/4d87482
Created com.codequarterly.markup.tests bundle with test files
As a start, I slapped the test documents based on the parser
spec:
http://www.codequarterly.com/code-challenges/markup/markup-s pec.html
in a bundle.
http://github.com/ralfebert/cq-markup-xtext/commit/7029166
Created Xtext project com.codequarterly.markup
I stumbled upon the generation of the plugin.xml_gen and the
apparent requirement to resolve xml_gen -> xml changes
myself, as I changed the file extension after creating the
new plug-in. Maybe the .xml_gen should be generated only if
different and contain a comment about its purpose.
http://github.com/ralfebert/cq-markup-xtext/commit/80ebcde
MarkupTests comparing MarkupParser output with xml files
Wrote a parameterized test "MarkupTests" (did you know:
http://junit.sourceforge.net/javadoc/org/junit/runners/Param eterized.html
) that throws the .txt files to the Markup parser of my
dreams and compares the output with the .xml.
Great, this yields 33 failing tests, so there's quite some
work ahead of me.
http://github.com/ralfebert/cq-markup-xtext/commit/0c91cda
Empty document working, Markup/MarkupParser classes created
Let's see. Empty Document is return "". One green.
http://github.com/ralfebert/cq-markup-xtext/commit/9eecee5
Ignore whitespace when comparing markup output with example xmls
Done by formatting both documents from a DOM tree and
comparing the result. Being the nerd I am, I even asked if
this is allowed according to the challenge rules. I like
<tags /> a lot more than <tags/>.
http://github.com/ralfebert/cq-markup-xtext/commit/ac75ee6
02_simple_paragraph working
Next one is one paragraph. The most contemporary way to
convert 'hello' to
'<p>hello</p>' is an Xtext grammar, use MWE2 that generates
a lexer and a parser that will eat all these delicious CHAR
tokens and convert them to Text EObjects and finally
building together a MarkupDocument EObject which then is
carefully mangled to an JDOM XML document containing a <p>
Element, and beautifully written out as text using JDOM's
Format.getPrettyFormat().
Of course I need to pray three times in the direction of EMF
to get an URI for my file on the classpath to get my
ResourceFactoryImpl to get an resource to get my EObject,
but I consider this now fixed because soon I can just google
for 'pray EMF' to find this commit containing the example
snippet. Then, finally, I will never mistake
ResourceFactoryImpl.createResource for
ResourceSetImpl.getResource again.
http://github.com/ralfebert/cq-markup-xtext/commit/664e9bd
03_multiline_paragraph and 04_two_paragraphs working
I'm proud that I can still recite '(A (sep A)*)?' a week
after learning about this. Should I hit a party of parsers
one day, I might last a minute or two.
http://github.com/ralfebert/cq-markup-xtext/commit/eaf9406
Support headlines: 06_header
This is the moment where all the work to setup EMF and Xtext
pays off. Extending the generated switch class to do a model
tree walk outputting XML elements is just priceless and
beautiful. Thanks to Sven Efftinge for mentioning to me that
these classes make great interpreters, I'd never had
concluded this myself.
I wonder if I can make Xtext generate an EMF model which has
a common text attribute in the Part class and not individual
ones in Headline and Paragraph.
http://github.com/ralfebert/cq-markup-xtext/commit/628874e
Headline depth, 07_headers and 08_crazy_header
Long headlines, short headlines, crazy headlines, if we
don't have it, you don't want it!
http://github.com/ralfebert/cq-markup-xtext/commit/2fb4c64
support tagged_markup like \i{} and \u{}
I almost weeped when 23_tagged_markup went green (I skipped
the other tests for now because they need to parse the
indentation information and I'm not in the mood for that
now). I would not want to write such a conversion without
Xtext as of now.
I'm still a bit sad that I don't get why it makes little
parsers happy when you tell them
Text:
parts += (PlainText|FormattedText) (parts += FormattedText
parts += PlainText?)*;
instead of:
Text:
parts += TextPart+;
(PlainText and FormattedText being subclasses of TextPart).
Thanks to Sebastian Zarnekow for giving me the
above-mentioned solution to get a similar example working.
Also, the error message is not really helpful for humble
humans, telling me:
Decision can match input such as "RULE_CHAR" using multiple
alternatives: 1, 3 As a result, alternative(s) 3 were
disabled for that input
Isn't there a way Xtext could figure out what's meant by
TextPart+ automagically? Which doubt do I remove by stating
(A|B)(A B?)* instead of C+ when A and B are subclasses of C?
http://github.com/ralfebert/cq-markup-xtext/commit/ea1e81e
|
|
| | | |
| Re: Idea: Helper methods to load Xtext resources [message #557167 is a reply to message #557130] |
Sun, 05 September 2010 16:13  |
Sven Efftinge
Messages: 1823
Registered: July 2009 |
Senior Member |
|
|
We have such convenience methods in our test utilities, which we plan to
polish and officially publish at sometime within this development stream.
Making StandaloneSetup a singleton is not something we would want to do.
That is because you should use guice to intialize your application and
therefore you never ever need to access the injector through static access.
I think the EMF Resource API isn't too inconvenient. It doesn't abstract
the three core concepts (Resource,ResourceSet and URI) away, which is
good. In inut tests you have to do a lot of calls to this API, therefore
it is nice to have convenience API.
Sven
Am 9/4/10 9:17 PM, schrieb Ralf Ebert:
>> Of course I need to pray three times in the direction of EMF
>> to get an URI for my file on the classpath to get my
>> ResourceFactoryImpl to get an resource to get my EObject,
>
> This was annoying me again, I was thinking about how this could be
> simplified. As a start, I made my StandaloneSetup class a singleton and
> added convenience methods to load resources by URL so that I can load a
> resource and get the root model object using:
>
> TodoList list =
> TodosStandaloneSetup.get().load(SomeClass.class.getResource( "test.todos"));
>
> Example implementation attached. This is especially helpful for tests, a
> version with load(File/URI) might be helpful as well.
>
> I deliberately didn't file a bug on this as of now because I wanted to
> discuss ideas at first:
>
> Do you think this might be worth generating code for this by default?
>
> Is the StandaloneSetup class the best place for such a thing?
> - Could this be helpful for non-standalone cases as well?
> - Maybe StandaloneSetup could get the root model type as generic type
>
> I changed the StandaloneSetup to do the initialization only once by
> making it a singleton, are there cases where someone potentially want to
> call createInjectorAndDoEMFRegistration() multiple times?
>
> In my cases I only ever wanted to load one single resource in a new
> ResourceSet, I wonder if a general solution should support more than that?
>
> Ralf
--
--
Need professional support for Xtext or other Eclipse Modeling technologies?
Go to: http://xtext.itemis.com
Twitter : @svenefftinge
Blog : http://blog.efftinge.de
|
|
| | |
Goto Forum:
Current Time: Mon Sep 23 08:34:59 GMT 2024
Powered by FUDForum. Page generated in 0.03629 seconds |
] 
Search
Help
Register
Login
Home
