| Resolving references time [message #652157] |
Wed, 02 February 2011 19:27  |
 Rafael Angarita
Messages: 94
Registered: November 2010 |
Member |
|
|
Hello everybody,
Here is the description of my problem:
I have created 3 languages using Xtext called Table, Attribute and Domain.
A Table can reference many Attributes and an Attribute can reference only one Domain. All the cross-references are working great. I need to be able to work with projects with a large amout of files of type Table, Attribute, and Domain. Right now, I am testing with a project composed of 158 Tables, 4316 Attribute and 186 Domains. Each Table references approximately 6 Attributes and each Attribute references one Domain.
One of the features of my application is "compile" my DSL, so I have to read all the Tables, its Attributes and its Domains.
I'm doing this for the initial loading of the Xtext resources:
AttributeStandaloneSetup.doSetup();
DomainStandaloneSetup.doSetup();
TableStandaloneSetup.doSetup();
ResourceSetImpl rs = new ResourceSetImpl();
Then I load each of the Table,Attributes, and Domain resources this way:
Resource r = rs.getResource(URI.createURI("platform:/resource" + path)), true);
r.load(null);
I am not resolving anything here, since It takes too long and consumes too much memory.
Then, I iterate the list of my Tables and get its information. The problem is that it takes too much time to resolve references, for example I obtain the Attribute for a certain Table and then I want to get the Domain of that Attribute:
Domain domain = attribute.getDomain();
That line takes approximately 4 seconds to execute. I checked and I found that that time is used by the Atrribute Implementation code, in the method getDomain():
public Domain getDomain()
{
if (domain != null && domain.eIsProxy())
{
InternalEObject oldDomain = (InternalEObject)domain;
domain = (Domain)eResolveProxy(oldDomain);
if (domain != oldDomain)
{
if (eNotificationRequired()) {
eNotify(new ENotificationImpl(this, Notification.RESOLVE, AttributePackage.ATTRIBUTE__DOMAIN, oldDomain, domain));
}
}
}
return domain;
}
especifically here:
domain = (Domain)eResolveProxy(oldDomain);
If it takes 4 seconds to resolve this reference, and each Table reference 6 attributes, and I have 158 (or 667 in another project I haven't tested yet), you can imagine how long it takes.
I think that I could be doing something wrong. What do you think?
I would appreciate any help and advise.
Thank you very much!
|
|
|
| Re: Resolving references time [message #652180 is a reply to message #652157] |
Wed, 02 February 2011 21:44  |
|
Hi,
did you made some customizations e.g. to Scoping and Nameprovider. Or Linking etc
I use this simple DSLs
grammar org.xtext.example.mydsla.MyDslA with org.eclipse.xtext.common.Terminals
generate myDslA "http://www.xtext.org/example/mydsla/MyDslA"
Model:
domains+=Domain*;
Domain:
"domain" name=ID;
grammar org.xtext.example.mydslb.MyDslB with org.eclipse.xtext.common.Terminals
generate myDslB "[url]http://www.xtext.org/example/mydslb/MyDslB[/url]"
import " platform:/resource/org.xtext.example.mydsla/src-gen/org/xtex t/example/mydsla/MyDslA.ecore " as domain
Model:
attributes+=Attribute*;
Attribute:
"attribute" name=ID "in" domain=[domain::Domain];
grammar org.xtext.example.mydslc.MyDslC with org.eclipse.xtext.common.Terminals
import "platform:/resource/org.xtext.example.mydslb/src-gen/org/xtext/example/mydslb/MyDslB.ecore" as attr
generate myDslC "http://www.xtext.org/example/mydslc/MyDslC"
Model:
tables+=Table*;
Table:
"table" name=ID "{"
attrs+=[attr::Attribute]*
"}";
And this simple test
package test;
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.eclipse.emf.common.util.URI;
import org.eclipse.emf.ecore.resource.Resource;
import org.eclipse.emf.ecore.resource.ResourceSet;
import org.eclipse.emf.ecore.resource.impl.ResourceSetImpl;
import org.xtext.example.mydsla.MyDslAStandaloneSetup;
import org.xtext.example.mydslb.MyDslBStandaloneSetup;
import org.xtext.example.mydslb.myDslB.Attribute;
import org.xtext.example.mydslc.MyDslCStandaloneSetup;
import org.xtext.example.mydslc.myDslC.Model;
import org.xtext.example.mydslc.myDslC.Table;
public class Test {
public static final int NUM_OF_DOMAINS = 200;
public static final int NUM_OF_ATTRIBUTES = 5000;
public static final int NUM_OF_TABLES = 200;
public static final int MAX_DOMAINS_PER_FILE = 20;
public static final int MAX_ATTRIBUTES_PER_FILE = 50;
public static final int MAX_TABLES_PER_FILE = 20;
public static final int ATTRS_PER_TABLE = 10;
public static void main(String[] args) throws IOException {
List<String> ds = new ArrayList<String>();
List<String> dfs = new ArrayList<String>();
List<String> as = new ArrayList<String>();
List<String> afs = new ArrayList<String>();
List<String> ts = new ArrayList<String>();
List<String> tfs = new ArrayList<String>();
int i = 0;
int k = 0;
while (i < NUM_OF_DOMAINS) {
String filename = "model/d" + k + ".mydsla";
dfs.add(filename);
StringBuffer r = new StringBuffer();
for (int j=0; j < Math.random() * MAX_DOMAINS_PER_FILE; j++) {
String d = "d"+i;
ds.add(d);
r.append("domain " + d+"\n");
i++;
}
BufferedWriter bw = new BufferedWriter(new FileWriter(filename));
bw.write(r.toString());
bw.close();
k++;
}
i = 0;
k = 0;
while (i < NUM_OF_ATTRIBUTES) {
String filename = "model/a" + k + ".mydslb";
afs.add(filename);
StringBuffer r = new StringBuffer();
for (int j=0; j < Math.random() * MAX_ATTRIBUTES_PER_FILE; j++) {
String a = "a"+i;
String d = ds.get((int) (Math.random()*ds.size()));
as.add(a);
r.append("attribute " + a+" in " + d +"\n");
i++;
}
BufferedWriter bw = new BufferedWriter(new FileWriter(filename));
bw.write(r.toString());
bw.close();
k++;
}
k = 0;
i = 0;
while (i < NUM_OF_TABLES) {
String filename = "model/t" + k + ".mydslc";
tfs.add(filename);
StringBuffer r = new StringBuffer();
for (int j=0; j < Math.random() * MAX_TABLES_PER_FILE; j++) {
String t = "t"+i;
String attrs = "";
for (int m=0; m <Math.random() * ATTRS_PER_TABLE;m++) {
String a = as.get((int) (Math.random()*as.size()));
attrs+=a;
attrs+=" ";
}
ts.add(t);
r.append("table " + t +" { " + attrs +"}\n");
i++;
}
BufferedWriter bw = new BufferedWriter(new FileWriter(filename));
bw.write(r.toString());
bw.close();
k++;
}
System.out.println(ds.size());
System.out.println(as.size());
System.out.println(ts.size());
long start = System.currentTimeMillis();
MyDslAStandaloneSetup.doSetup();
MyDslBStandaloneSetup.doSetup();
MyDslCStandaloneSetup.doSetup();
ResourceSet rs = new ResourceSetImpl();
for (String s : dfs) {
Resource r = rs.getResource(URI.createURI(s), true);
r.load(null);
}
for (String s : afs) {
Resource r =rs.getResource(URI.createURI(s), true);
r.load(null);
}
List<Model> models = new ArrayList<Model>();
for (String s : tfs) {
Resource r = rs.getResource(URI.createURI(s), true);
r.load(null);
models.add((Model) r.getContents().get(0));
}
for (Model m : models) {
for (Table t : m.getTables()) {
for (Attribute a : t.getAttrs()) {
if( t.getName() == null) System.exit(1);
if( a.getName() ==null) System.exit(1);
if( a.getDomain().getName() == null) System.exit(1);
}
}
}
System.out.println((System.currentTimeMillis() - start)/ 1000);
}
}
In my case it is far less than 4 sec per attribute
~Christian
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
[Updated on: Wed, 02 February 2011 22:04] Report message to a moderator |
|
|
| Re: Resolving references time [message #652185 is a reply to message #652180] |
Wed, 02 February 2011 22:29 |
|
And if i leverage Xtexts Mwe reader i am even a factor 10 faster than before
(<10 vs <100 sec)
long start = System.currentTimeMillis();
Reader reader = new Reader();
reader.addRegister(new MyDslAStandaloneSetup());
reader.addRegister(new MyDslBStandaloneSetup());
reader.addRegister(new MyDslCStandaloneSetup());
reader.addPath("model");
SlotEntry se = new SlotEntry();
se.setSlot("tables");
se.setType("Table");
reader.addLoad(se);
IWorkflowContext ctx = new WorkflowContextImpl();
reader.invoke(ctx );
List<Table> tables = (List<Table>) ctx.get("tables");
for (Table t : tables) {
for (Attribute a : t.getAttrs()) {
String x = t.getName()+ a.getName() + a.getDomain().getName();
System.out.println(x);
}
}
System.out.println((System.currentTimeMillis() - start)/ 1000);
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
[Updated on: Wed, 02 February 2011 22:29] Report message to a moderator |
|
|
| Re: Resolving references time [message #652904 is a reply to message #652185] |
Mon, 07 February 2011 14:27 |
Rafael Angarita
Messages: 94
Registered: November 2010 |
Member |
|
|
Hello Christian! Thank you very much for your answer.
I did implement scoping and name provider. Here they are:
public class TableQualifiedNameProvider extends DefaultDeclarativeQualifiedNameProvider {
public String qualifiedName(Column col) {
if (col != null) {
CompositeNode cNode = NodeUtil.getNode(col);
EList<AbstractNode> anList = cNode.getChildren();
AbstractNode aNode = anList.get(1);
LeafNode lNode = (LeafNode) aNode;
return lNode.getText();
}
return null;
}
@Override
public String getQualifiedName(EObject obj) {
if(obj instanceof Column) {
Column col = ((Column) obj);
CompositeNode cNode = NodeUtil.getNode(col);
EList<AbstractNode> anList = cNode.getChildren();
if (anList.size() > 0){
AbstractNode aNode = anList.get(1);
LeafNode lNode = (LeafNode) aNode;
return lNode.getText();
}
}
return super.getQualifiedName(obj);
}
}
and:
public class TableScopeProvider extends AbstractDeclarativeScopeProvider {
@Inject
IQualifiedNameProvider nameProvider;
public IScope scope_IndexColumn_indexColumn(Index i, EReference r) {
EList<Column> list = ((Table) i.eContainer())
.getColumns();
return Scopes.scopeFor(list, nameProvider, IScope.NULLSCOPE);
}
public IScope scope_IndexColumn_indexColumn(IndexColumn i, EReference r) {
EList<Column> list = ((Table) i.eContainer().eContainer())
.getColumns();
return Scopes.scopeFor(list, nameProvider, IScope.NULLSCOPE);
}
}
I implemented them because I have to reference Attributes already defined inside the current table (for example to define indexes of the table).
I defined an Attribute inside a Table this way:
Column:
attribute=[att::Attribute]
therefore I had to get its qualified name for the references inside the same table.
Your code leveraging Xtexts Mwe reader works really fast!!!! However, I am a little lost concerning how to use it. I'm loading all the resources, but I get an exception if there is something worng with the code written using my dsl:
org.eclipse.emf.mwe.core.WorkflowInterruptedException: Validation problems:
1 error:
It doesn't give me an option to validate my model before that happen.
What can I do? Should I implement something like a custom reader?
I would appreciate if you can point me out the right direction.
Thank you very much for helping me.
[Updated on: Mon, 07 February 2011 14:37] Report message to a moderator |
|
|
|
|
Powered by
FUDForum. Page generated in 0.02041 seconds
] 
Search
Help
Register
Login
Home
