Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Adding a standard library to a DSL
Adding a standard library to a DSL [message #1858957] Wed, 03 May 2023 10:17 Go to next message
Simon Cockx is currently offline Simon CockxFriend
Messages: 69
Registered: October 2021
Member
Written in my DSL, I have a couple of files (`basictypes.rosetta` and `annotations.rosetta`) that can be considered a "standard library" of my DSL, i.e., they should be auto-included in every project and auto-imported in every file.

The auto-importing is easy - I can override the `getImplicitImports` in my scope provider.

It's the auto-inclusion I'm having trouble with. Is there an example I can rely on? I've already tried the following two approaches:

Approach 1: Override `XtextResourceSet` and `SynchronizedXtextResourceSet` and add the standard library there.

E.g.,
public class RosettaResourceSet extends XtextResourceSet {
	@Inject
	public RosettaResourceSet(RosettaBuiltinsService builtinsService) {
		getResources().add(builtinsService.getBasicTypesResource());
		getResources().add(builtinsService.getAnnotationsResource());
	}
}

(similar for `SynchronizedXtextResourceSet`)

In the runtime module:
	override Class<? extends ResourceSet> bindResourceSet() {
		RosettaResourceSet
	}
	
	override Class<? extends XtextResourceSet> bindXtextResourceSet() {
		SynchronizedRosettaResourceSet
	}


This results in a Guice error, and I'm not sure why.
com.google.inject.ProvisionException: Unable to provision, see the following errors:

1) [Guice/CanNotProxyClass]: Tried proxying XtextResourceSet to support a circular dependency, but it is not an interface.
  while locating SynchronizedRosettaResourceSet
  while locating XtextResourceSet
  at TerminalsGrammarAccess.<init>(TerminalsGrammarAccess.java:36)
  at TerminalsGrammarAccess.class(TerminalsGrammarAccess.java:36)
  at RosettaGrammarAccess.<init>(RosettaGrammarAccess.java:9045)
      \_ for 2nd parameter
  at RosettaGrammarAccess.class(RosettaGrammarAccess.java:9045)
  while locating RosettaGrammarAccess
  at LazyLinker.grammarAccess(LazyLinker.java:300)
      \_ for field grammarAccess
  while locating LazyLinker
  at XtextResource.linker(XtextResource.java:474)
      \_ for field linker
  while locating DerivedStateAwareResource
  while locating XtextResource
  at SynchronizedRosettaResourceSet.<init>(SynchronizedRosettaResourceSet.java:11)
  while locating SynchronizedRosettaResourceSet
  while locating XtextResourceSet


Approach 2: Override `ResourceSetBasedResourceDescriptions` and `ResourceSetBasedAllContainersStateProvider` and add the standard library there.

E.g.,
public class RosettaResourceDescriptions extends ResourceSetBasedResourceDescriptions {
	private final Map<URI, IResourceDescription> standardLibrary;
		
	@Inject
	public RosettaResourceDescriptions(
			RosettaBuiltinsService builtinsService,
			IResourceDescription.Manager manager) {
		this.standardLibrary = Map.of(
			builtinsService.basicTypesURI, manager.getResourceDescription(builtinsService.getBasicTypesResource()),
			builtinsService.annotationsURI, manager.getResourceDescription(builtinsService.getAnnotationsResource())
		);
	}
	
	@Override
	public Iterable<IResourceDescription> getAllResourceDescriptions() {
		return Iterables.concat(
				super.getAllResourceDescriptions(),
				standardLibrary.values());
	}
	
	@Override
	public boolean isEmpty() {
		return false;
	}
	
	@Override
	public IResourceDescription getResourceDescription(URI uri) {
		IResourceDescription descr = super.getResourceDescription(uri);
		if (descr != null) {
			return descr;
		}
		return standardLibrary.get(uri);
	}
}


This works fine up to the point where `EcoreUtil.resolve` is called, which expects the referenced EObjects to be in the same resource set as the proxy. In this case however, the resources of the standard library don't belong to a resource set at all.

[Updated on: Wed, 03 May 2023 12:11]

Report message to a moderator

Re: Adding a standard library to a DSL [message #1858985 is a reply to message #1858957] Thu, 04 May 2023 06:25 Go to previous messageGo to next message
Lorenzo Bettini is currently offline Lorenzo BettiniFriend
Messages: 1815
Registered: July 2009
Location: Firenze, Italy
Senior Member
What I do in these cases is provide a project wizard for my DSL that also adds a dependency to the library.
In the past (several years ago), I tried something similar to what you describe. It never worked. I suggest you rely on the classpath/dependencies and let Xtext scoping and linking do the rest.


Re: Adding a standard library to a DSL [message #1858988 is a reply to message #1858985] Thu, 04 May 2023 08:46 Go to previous messageGo to next message
Simon Cockx is currently offline Simon CockxFriend
Messages: 69
Registered: October 2021
Member
Hmm, how does that work exactly? I don't have experience with managing dependencies between DSL projects.
Is this independent of Eclipse? (e.g., does this work with Maven? Our DSL is entirely used outside Eclipse, so I get scared when I hear Eclipse vocabulary such as "project wizard" :)
Will the files of dependent projects be part of the resource set? (might be important => the code generator should ignore them)
Does this also work when writing unit tests? (i.e., I would like the standard library to be automagically available when parsing using the `ParseHelper` class)
Re: Adding a standard library to a DSL [message #1858989 is a reply to message #1858988] Thu, 04 May 2023 08:52 Go to previous messageGo to next message
Lorenzo Bettini is currently offline Lorenzo BettiniFriend
Messages: 1815
Registered: July 2009
Location: Firenze, Italy
Senior Member
In Eclipse, you have to use Eclipse's specific dependencies mechanisms. Same for Maven.
If you want something completely outside Eclipse, then I guess you have to handle dependencies manually.
For ParseHelper, it's enough to make everything available in the same resource set.

In general, it should also work by properly configuring a resource set. the error you get from Guice maybe means you're doing something wrong with the binding leading to a cycle.

Customizing resource descriptions should not be necessary as long as everything is in the same resource set.


Re: Adding a standard library to a DSL [message #1858991 is a reply to message #1858989] Thu, 04 May 2023 12:25 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14736
Registered: July 2009
Senior Member
i also recommend to make your model java projects and put the std lib to its classpath.
if you want to continue with the resourcesets:
- why do you have two impls/bindings?
- you also need to bind it for the builder as this one has its own global resourceset.
maybe you also just want to look into resourcesetinitializers instead of the resourcesets directly

. you also might have to deal with what LiveScopeResourceSetInitializer does too as in bulder scope != resourceset. this is in standalone mode only


Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Day Job: https://www.everest-systems.com

[Updated on: Thu, 04 May 2023 12:28]

Report message to a moderator

Re: Adding a standard library to a DSL [message #1859004 is a reply to message #1858991] Fri, 05 May 2023 07:37 Go to previous messageGo to next message
Simon Cockx is currently offline Simon CockxFriend
Messages: 69
Registered: October 2021
Member
@Christian
Quote:
if you want to continue with the resourcesets:
- why do you have two impls/bindings?


The `DefaultRuntimeModule` provides two bindings, i.e.,
	public Class<? extends ResourceSet> bindResourceSet() {
		return XtextResourceSet.class;
	}
	
	public Class<? extends XtextResourceSet> bindXtextResourceSet() {
		return SynchronizedXtextResourceSet.class;
	}

To be on the safe side, I overrode both. I don't want my solution to fail depending on the way a resource set is injected.

Quote:
- you also need to bind it for the builder as this one has its own global resourceset.

I'm not sure I understand what you mean with "the builder".

Quote:
maybe you also just want to look into resourcesetinitializers instead of the resourcesets directly

This seems promising. Following this path, I noticed that `XtextGeneratorLanguage` exposes a field called `referencedResources`, which, if I'm not mistaken, are automatically added to resource sets. Would this be a good place to specify my standard lib resources?
I want to make sure that it is actually made available everywhere - that means in all resource sets including the ones created in unit tests and in Xtext's language server.

@Lorenzo
Quote:
For ParseHelper, it's enough to make everything available in the same resource set.

Our current approach was to add it manually everywhere. That means we had to create a wrapper for parsing in unit tests, we had to override the AbstractLanguageServerTest, and we need to copy the files whenever we create a new project. That's the dirty current situation I want to get rid of. I guess this isn't the only possible solution, but I thought it would be an elegant one. Apparently there aren't many Xtext DSL's out there which followed the same thought process. :')

Quote:
In general, it should also work by properly configuring a resource set. the error you get from Guice maybe means you're doing something wrong with the binding leading to a cycle.

I'm a bit confused. I tried debugging this, so I discarded all dependencies of my `SynchronizedRosettaResourceSet`, but even then the error remains. It only stops when I don't create `XtextResource`s in the constructor, which made me think that a `XtextResource` somehow depends on `XtextResourceSet`, but that doesn't make sense.


General update: I've tried following the route with ResourceDescriptions and Containers, and I actually got it working up to the point where indexes got involved - which is used in Xtext's language server.
Having gone down this road, I'm somehow more confused. It seems that the concept of containers in Xtext is an elegant way of describing dependencies, but the default implementation completely ignores it, and instead uses a "singleton" container called "all" (see `FlatResourceSetBasedAllContainersState.HANDLE`). Having implemented a generalization of this behaviour to define dependencies between resource sets, it's just a shame that this doesn't work when using an index.

It seems the Xtext language server also uses a concept of 'dependencies' (see `org.eclipse.xtext.resource.impl.ProjectDescription::getDependencies`), and briefly having gone through the code, these dependencies seem to be added to the index automatically. I'm not sure whether this is another potential route to follow, or that this will only bring along another rabbit hole. I suspect that I won't be able to generalize this enough to be used in a context outside the Xtext language server as well (e.g., in unit tests), so it's probably not worth following.

Anyway, thanks for the advice, and I look forward to hearing your opinion on the above.

[Updated on: Fri, 05 May 2023 07:40]

Report message to a moderator

Re: Adding a standard library to a DSL [message #1859007 is a reply to message #1859004] Fri, 05 May 2023 12:08 Go to previous messageGo to next message
Simon Cockx is currently offline Simon CockxFriend
Messages: 69
Registered: October 2021
Member
One final note: the reason why I prefer _not_ adding everything to the same resource set is twofold:

1. It's not necessary. You only need the resource description of dependencies, not the entire resources. This fact could be used to improve performance memory-wise.
2. Code generation is triggered on an entire resource set. Adding dependencies to that resource set results in performing code generation for those projects as well, which is superfluous.

... which is why I imagined using the concept of resource descriptions and containers would be a more ideal solution.

[Updated on: Fri, 05 May 2023 12:18]

Report message to a moderator

Re: Adding a standard library to a DSL [message #1859008 is a reply to message #1859007] Fri, 05 May 2023 12:46 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14736
Registered: July 2009
Senior Member
with builder i mean the eclipse build. or the lsp build. or the xtext maven or gradle plugin
(basically all what supports multiple dsls used together)
they use one resourceset for all languages. so binding stuff in one language will not work.

the referencedResources is a concept special for workflows.

its not clear. which usecases do you try to support? lsp, eclipse, standalone, all


Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Day Job: https://www.everest-systems.com
Re: Adding a standard library to a DSL [message #1859032 is a reply to message #1859008] Mon, 08 May 2023 07:44 Go to previous message
Simon Cockx is currently offline Simon CockxFriend
Messages: 69
Registered: October 2021
Member
The usecases I want to support is standalone (e.g., for code generation using the Xtext Maven plugin) and LSP (e.g., for running our DSL is VS code) - and internally for unit testing.

The particular case of the Xtext Maven plugin seems to work out-of-the-box, because it automatically scans the classpath for .rosetta files (and our standard lib is included in the classpath). Other "standalone" usecases such as custom scripts for analyzing Rosetta projects currently need to include it manually.

Ideally, I would like to find some common ground which allows me to define a standard library in a single place. What I would like to avoid is having to add support for all of these use cases separately.
Previous Topic:Missing fsa.deleteFile implementation in generator of language server
Next Topic:Check unique names only in the provided scope
Goto Forum:
  


Current Time: Tue Dec 10 09:33:18 GMT 2024

Powered by FUDForum. Page generated in 0.03443 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top