Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Working With a User Generated Script
Working With a User Generated Script [message #1016577] Thu, 07 March 2013 00:34 Go to next message
Steve Kallestad is currently offline Steve Kallestad
Messages: 62
Registered: March 2013
Member
Bear with me, I'm just getting started.

A few days ago, I learned antlr - just enough to declare a simple scripting language.

The idea popped into my head that it would be great if I could have syntax highlighting, which led me to XText.

I've gone through the tutorials, but... they seem pretty advanced for my needs.

I've created a grammar and the editor works just dandy. As far as that goes, XText is pretty darn awesome - the learning curve was pretty slim.

I'm now at the point where it's time to parse the script.

My goal is the scriptable automation of an existing java application. Say, for instance, I have a java class with two methods
public class talktome{
  public void sayHello(String Name){
    system.out.println("Hello " + Name);
  }
  public void sayGoodbye(String Name){
    system.out.println("Goodbye " + Name);
  }
}


I want to have a script that looks like this:
automater Talk1{
  SaySomethingTo("Fred")
  SaySomethingTo("Wilma")
  SaySomethingTo("Betty")
}

automater Talk2{
  JustSayGoodbye("Daphne")
  JustSayGoodbye("Scooby")
}


The end result that I want is something along the lines of:

import com.example.TalkToMe.*;
import somethingFromXtext;

public class Main {

  public static void void main(String[] args){

    XtextMagicalThing script = loadMyDSLScript("fromearlierinthepost");

    for (automater : script.getAutomaters()){

       for(command : automater.getCommands()){

            if(command.name == "SaySomethingTo"){
                 SayHello(command.param);
                 SayGoodBye(command.param);
            } else if (command.name == "JustSayGoodbye"){
                 SayGoodBye(command.param);
            }

       }
    }
  }
}


It seems to me from my basic understanding of XText that the parser is being automatically generated, I just don't know how to get it.

I did find this link which talks about loading a model, but I think I might have a little bit of a terminology disconnect. That uses EMF, and my understanding was that using EMF in a standalone application was not encouraged.

In antlr, I'd build a tree walker that parsed tokens from a lexer. Then I'd walk the tree and from there validate and interpret the results.

I see in my src-gen tree that there is an "InternalDSLLexer.java" and an "InternalDSLParser.java" that I could potentially use, but I thought it would be a good idea to ask what the recommended path is.

I hope I made at least a little bit of sense here.
Re: Working With a User Generated Script [message #1016604 is a reply to message #1016577] Thu, 07 March 2013 06:20 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian Dietrich
Messages: 6248
Registered: July 2009
Senior Member
Hi,

of course standalone use/generation is possible and i think there are a lot of projects that do that (in fact all projects i know)
The link about loading standalone might be one starting point.
if you implement the IGenerator that is automatically created into
the project you get:

(1) a builder integration in eclipse (code gen runs on save/build in eclipse)

plus you can change the GeneratorFragment in your Language Workflow (the mwe file next to your language)
to let xtext generate you a (2) java main class and/or a (3) workflow to call the generator with a model file.

so just give it a try.

~Christian
Re: Working With a User Generated Script [message #1016606 is a reply to message #1016604] Thu, 07 March 2013 06:31 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian Dietrich
Messages: 6248
Registered: July 2009
Senior Member
Btw you can write an interpreter too Wink
Re: Working With a User Generated Script [message #1017107 is a reply to message #1016606] Sat, 09 March 2013 12:12 Go to previous messageGo to next message
Steve Kallestad is currently offline Steve Kallestad
Messages: 62
Registered: March 2013
Member
OK, so I came back to this a few days later and I'm still totally confused... and not productive.

I don't know anything about EMF, so I figured there's a library that I need to include in my project, but I don't know where to get it from.

The link I posted provided this code example:

new org.eclipse.emf.mwe.utils.StandaloneSetup().setPlatformUri("../");
Injector injector = new MyDslStandaloneSetup().createInjectorAndDoEMFRegistration();
XtextResourceSet resourceSet = injector.getInstance(XtextResourceSet.class);
resourceSet.addLoadOption(XtextResource.OPTION_RESOLVE_ALL, Boolean.TRUE);
Resource resource = resourceSet.getResource(
    URI.createURI("platform:/resource/org.xtext.example.mydsl/src/example.mydsl"), true);
Model model = (Model) resource.getContents().get(0);


Believe it or not, I have no idea (and can't figure out) where to get the dependency to make that code work.

I tried downloading the EMF runtime, but the jars I grabbed don't have any runnable code in them.

It seems to me that I've missed something very basic in the documentation.
Re: Working With a User Generated Script [message #1017110 is a reply to message #1017107] Sat, 09 March 2013 12:51 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian Dietrich
Messages: 6248
Registered: July 2009
Senior Member
HI,

the best thing is to simply add the main to the runtime project and to export your xtext runtime project using the Export -> Java - Runnable Jar File Wizard
Re: Working With a User Generated Script [message #1017124 is a reply to message #1017110] Sat, 09 March 2013 16:06 Go to previous messageGo to next message
Steve Kallestad is currently offline Steve Kallestad
Messages: 62
Registered: March 2013
Member
Thanks! That worked.

an 18mb jar and 115mb of ram to script hello world. Yikes.

I like what XText provides, but... I think it might just be better to build a parser in antlr directly for standalone processing. That's a heck of a lot of overhead.
Re: Working With a User Generated Script [message #1017138 is a reply to message #1017124] Sat, 09 March 2013 19:40 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian Dietrich
Messages: 6248
Registered: July 2009
Senior Member
You asked for an easy way not for a small one. depending on what of Xtext you might throw out several things

- mwe
- xtext.generator
- xbase

....
Re: Working With a User Generated Script [message #1017151 is a reply to message #1017138] Sun, 10 March 2013 00:27 Go to previous messageGo to next message
Steve Kallestad is currently offline Steve Kallestad
Messages: 62
Registered: March 2013
Member
Thank you.

I've been able to successfully reduce the size of the jar down to < 9mb fairly easily. Memory usage is still constant.

Taking a brief look, I see a bajillion language resources under ibm.icu.impl.data and I'm wondering if that might be the reason. (Maybe there are tons of strings being generated).

I'm going to spend some time profiling and seeing if I can't see if there isn't a relatively obvious reason it's eating up so much memory.

100mb isn't so much for a one-off application, but if I'm creating a series of things that can be concurrently running I don't want to require the end user to set aside a gig of RAM.
Re: Working With a User Generated Script [message #1017156 is a reply to message #1017151] Sun, 10 March 2013 03:54 Go to previous message
Steve Kallestad is currently offline Steve Kallestad
Messages: 62
Registered: March 2013
Member
Just a quick follow up.

My memory usage issue really spawns from the fact that I started long ago when memory was expensive and everything needed to be optimized.

I can see that it's possible to generate a more efficient parser, but the default output does the job for quick and easy.

Playing around with it for a bit, I did get the jar size reduced to < 3000 files and < 4mb.

Memory usage was a bit complicated.
Looking in Task Manager shows between 120 and 160mb of usage. Profiling showed that the bulk of that was unused space and the actual used space was around 75mb.
Command line options can reduce the usage down to ~40mb, but that's not just limiting the leveraged libraries, that's also limiting the application referencing them.

I definitely see the utility in xtext and EMF - it provides a heck of a lot more than just validation of a script, but... if someone wants to efficiently run a script in an external application... it looks like you have to do one of two things:


  1. Write the code to generate your interpreter, or
  2. Write your interpreter externally


I did look through the generated antlr .g file and the resulting InternalXXLexer and InternalXXParser, but it wasn't obvious if those could be leveraged independently. Nothing to do at this point but give it a shot.

I do want to thank you very much Christian. The direction you provided was very helpful to me.
Previous Topic:Duplicate Name Default Error
Next Topic:How to display ecore Eclass names for content assist in Xtext editor after generating artifacts ?
Goto Forum:
  


Current Time: Wed Sep 03 07:15:51 GMT 2014

Powered by FUDForum. Page generated in 0.09689 seconds