|
Re: Xtext Resource Storage [message #1841178 is a reply to message #1841177] |
Wed, 05 May 2021 17:07   |
Eclipse User |
|
|
|
Hi, can you give more details about your exact usecase
i am not sure if the storage aware feature solves anything for you
besides that binding
StorageAwareResourceDescriptionManager as IResourceDescriptionManager
and StorageAwareResource as XtextResource
bindings for IResourceStorageFacade to ResourceStorageFacade
and AbstractFileSystemAccess2 to EclipseResourceFileSystemAccess2 should do the trick
(maybe some more needed)
for reading you can steer with the SourceLevelURIsAdapter if resources should be loaded via storage or not (happens automatically during build)
public class MyDslUiModule extends AbstractMyDslUiModule {
public MyDslUiModule(AbstractUIPlugin plugin) {
super(plugin);
}
//.....
public Class<? extends IResourceDescription.Manager> bindIResourceDescription$Manager() {
return StorageAwareResourceDescriptionManager.class;
}
public Class<? extends XtextResource> bindXtextResource() {
return StorageAwareResource.class;
}
public Class<? extends IResourceStorageFacade> bindIResourceStorageFacade() {
return ResourceStorageFacade.class;
}
public Class<? extends AbstractFileSystemAccess2> bindAbstractFileSystemAccess2() {
return EclipseResourceFileSystemAccess2.class;
}
}
you may also have to do some corrective customization like org.eclipse.xtext.xbase.resource.BatchLinkableResourceStorageFacade.getSourceContainerURI(StorageAwareResource) / org.eclipse.xtext.resource.persistence.ResourceStorageFacade.computeOutputPath(StorageAwareResource) so that the storages land in a folder
e.g.
@Override
protected String computeOutputPath(StorageAwareResource resource) {
URI u = resource.getURI();
if (u.isPlatformResource()) {
URI base = u.trimFileExtension().trimSegments(u.segmentCount()-3);
URI binaryStorageURI = getBinaryStorageURI(resource.getURI());
String result = binaryStorageURI.deresolve(base)
.path();
return result;
}
return super.computeOutputPath(resource);
}
[Updated on: Thu, 06 May 2021 14:52] by Moderator Report message to a moderator
|
|
|
Re: Xtext Resource Storage [message #1841179 is a reply to message #1841178] |
Wed, 05 May 2021 19:09   |
Eclipse User |
|
|
|
Hi
I'm not sure you really need to use an Xtext facility. Binary storage is an EMF option, so you just need to set the correct save options when saving your parsed Resource. I've never used them, but you should find that the reverse just works automatically. Interactively you could add a Save-As-Binary action unless Xtext already does. For thousands of files you probably want to write a little tool/builder to load one or more XML files and store them in binary format.
Regards
Ed Willink
|
|
|
Re: Xtext Resource Storage [message #1841181 is a reply to message #1841178] |
Wed, 05 May 2021 19:33   |
Eclipse User |
|
|
|
Thanks Christian for the quick answer.
We have our own DSL, which is developed by Xtext, to do the data modelling.
The models are saved in the Git repository. Users are working on different Git branches. Every time when the user switch to another Git branch, the cross-references and many other parsed information are broken. User has to use "Project-Clean" in Eclipse to re-parse all the models again. This takes with current model size around 20 minutes.
The idea is, is that possible to persist parsed information in disk . When the model git branch is switched, the already pared models can be directly get from the disk. and not need to re-parse all the models again.
|
|
|
Re: Xtext Resource Storage [message #1841182 is a reply to message #1841181] |
Wed, 05 May 2021 19:42   |
Eclipse User |
|
|
|
Hi
I strongly recommend that you do not persist binary files in GIT unless you can guarantee never to evolve your DSL, since an evolution may easily break binary files that refer to feature "4" rather than feature "name", so if you insert a new feature at "3" all the subsequent ones are off by one. Crazy EMF crash.
Exactly this happened with *.xtextbin [1], although fortuitously the new feature was a last feature so in that case the incompatibility was only in one direction.
You can probably manage to make your textual DSL upward compatible so if you check in *.dsl to GIT and have a builder that refreshes *.dslbin from *.dsl, reparsing will only happen after changes. Your downstream tooling can try to open *.dslbin and fall back to *.dsl.
Regards
Ed Willink
[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=460978
|
|
|
Re: Xtext Resource Storage [message #1841183 is a reply to message #1841182] |
Wed, 05 May 2021 20:30   |
Eclipse User |
|
|
|
The storages won't speed up clean builds but incremental ones and open editors with many transitive dependencies only.
So you would have to give it a try
If you change 90% of the files by a switch branch profit will be likely zero
Same if you just have a hand full of deps for the files that change
|
|
|
|
Re: Xtext Resource Storage [message #1841207 is a reply to message #1841184] |
Thu, 06 May 2021 13:54   |
Eclipse User |
|
|
|
Thanks Christian and Ed for your helpful suggestions.
1. switch git branch break the models
I tried today again to find out the reason, why after switching git branch, the model need to be cleaned. I think I find out something.
Because we are using git command or the git client application (GitExtensions) to work with git. Every time when we switch the git branch. The changed files are not synchronized in Eclipse Model Explorer. Only when I refresh the folder in eclipse model explorer, the changed files are synchronized.
I tried with the git inside eclipse (EGit), with it after switching git branch, the files in Model Explorer are updated and the automatically build run successfully.
So it seems that it's a problem of Eclipse working together with third party git application
2. why the clean build takes 20 minutes
it was my type error. most time the clean build can be finished around 10 minutes. We have some time consuming validations with the models, and totally there are around 5000 models. These models have also many transitive dependencies with each other. So I think the clean build with 10 minutes is still acceptable.
But I will try to use the storaged resource to accelerate the incremental build.
One more question is, does Xtext persistent the parse result somewhere in the disk. I ask this, because even the computer restarted and when I reopen the models, the cross reference and other things still works. If the parsing result were saved in the memory, this should not happen. Please correct me if I am wrong.
|
|
|
Re: Xtext Resource Storage [message #1841210 is a reply to message #1841207] |
Thu, 06 May 2021 14:26   |
Eclipse User |
|
|
|
you may check auto refresh or refresh on access in your workspace preferences.
xtext will just store the Xtext in index in a file called builder.state in a subfolder of .metadata in the workspace.
this index data then will be used when you continue with the workspace.
with the storage face you will get a for a src-gen/.somefile.mydslbin somefile.mydsl for a somefile.mydsl (path is calculated by org.eclipse.xtext.resource.persistence.ResourceStorageFacade.computeOutputPath(StorageAwareResource))
|
|
|
Re: Xtext Resource Storage [message #1841216 is a reply to message #1841210] |
Thu, 06 May 2021 16:28   |
Eclipse User |
|
|
|
With the suggestion of Christian, I changed the refresh setting in eclipse.
It was only the option "Refresh on access" selected. Now I also selected another option "Refresh using native hooks or polling". Bingo, it works well with the third party git client now!!!! Thanks so much Christian.
The "builder.state" file I could not find under .metadata folder and its sub folders. I only found some history.index files, which is under .metadata/.plugins/org.eclipse.core.resources/.project/<xtext project name>/.indexes, I don't think these files are generated by xtext.
I'm using xtext version 2.19.
|
|
|
|
Re: Xtext Resource Storage [message #1841234 is a reply to message #1841178] |
Fri, 07 May 2021 09:55   |
Eclipse User |
|
|
|
hi Christian,
I tried to bind the required classes.
There are some warning message in the import part of codes.
Warning - Discouraged access: The type StorageAwareResourceDescriptionManager is not accessible due to restriction on required project com.xxxxxx.model.ui
Can I ignore them?
|
|
|
|
Re: Xtext Resource Storage [message #1841244 is a reply to message #1841235] |
Fri, 07 May 2021 14:55   |
Eclipse User |
|
|
|
hi christian,
finally the storable resources are working. I saw the "dslbin" files are created.
I want to make clear, what is the difference between these "dslbin" files and the "builder.state" file. As you said, the "builder.state"is the index file. do the "dslbin" files contains more information than the "builder.state" file?
another question is, do I need to add some codes to tell xtext when to load the dslbin file, when not? if yes, how can I do this?
|
|
|
Re: Xtext Resource Storage [message #1841249 is a reply to message #1841244] |
Fri, 07 May 2021 19:19  |
Eclipse User |
|
|
|
The builder state is just "the index"
It says there are these objects if that types with these names in these files
The mydsl bin are a binary version of the ast and nodemodel
The mydsl build are use automatically if you don't opt out using SourceLevelURIsAdapter
You can observe it debugging the storageawareresource class
|
|
|
Powered by
FUDForum. Page generated in 0.04319 seconds