Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF "Technology" (Ecore Tools, EMFatic, etc)  » [EMFStore] Support for multiple workspaces
[EMFStore] Support for multiple workspaces [message #1724776] Thu, 25 February 2016 23:12 Go to next message
Scott Dybiec is currently offline Scott DybiecFriend
Messages: 146
Registered: July 2009
Senior Member
I've been testing the feasibility of using the EMFStore client in a web
app. My current design uses ESWorkspaceProviderImpl.getInstance(String
workspaceName) to create a dedicated workspace for each web session. The
workspace is disposed by a reaper thread when the user's session expires
or when the user logs out.

My prototype is now paritally functional. It required many hours of
tracing through the EMFStore code to figure out how it all worked and
find solutions to problems with minimal impact to the code base.

Perhaps there are easier ways to enable EMFStore client multi-threaded
access to multiple workspaces, so I'm interested in your thoughts. I saw
comments in the code that suggested that the appserver based model I'm
trying to implement was tried in the past. Was it?

Here are the problems I fixed/worked around:

1. To ensure internal usages of ESWorkspaceProviderImpl.getInstance()
work with the TransactionalEditingDomain (e.g.
AbstractEMFStoreCommand.aRun()), the workspace provider must be set in
the ESWorkspaceProvider's ThreadLocal variable. Unfortunately only the
first thread that calls ESWorkspaceProviderImpl.getInstance(String
workspaceName) gets the workspace provider added to its ThreadLocal.
Subsequent threads accessing the same workspace do not. Here's how I
modified this method:

public static ESWorkspaceProviderImpl getInstance(String
workspaceProviderId) {
ESWorkspaceProviderImpl ws;
if (WorkspaceLocator.hasId(workspaceProviderId)) {
ws = WorkspaceLocator.getWorkspaceById(workspaceProviderId);
} else {
ws = WorkspaceLocator.createWorkspaceProviderFor(workspaceProviderId);
}
WS_THREAD_LOCAL.set(ws);
return ws;
}

Is there a reason that the ThreadLocal is only set for the first thread
that creates the workspace and no subsequent threads? I couldn't see why
the threadlocal would not be set on every call.


2. Both ESWorkspaceProvider.INSTANCE.getWorkspace() and
ESWorkspaceProviderImpl.getInstance().getWorkspace() are used in the
EMFStore code base to access the workspace, but only the second is able
to provide the workspace stored in the ThreadLocal to the calling
thread. The INSTANCE variable never changes after it is first
initialized when the class is instantiated. As a workaround, I changed
all references from ESWorkspaceProvider.INSTANCE.getWorkspace() ->
ESWorkspaceProviderImpl.getInstance().getWorkspace() in the EMFStore
code base. Another possible solution is to a) create a new API method
(e.g. ESWorkspaceProvider.getWorkspace()) that, in turn, calls
ESWorkspaceProviderImpl.getInstance().getWorkspace(), and b) eliminate
the ESWorkspaceProvider.INSTANCE variable.

3. Disposing of a workspace does not clean up references in the static
map in WorkspaceLocator, and WorkspaceLocator does not provide a method
to manually remove these references. WorkspaceLocator needs a method
such as:

public static void removeWorkspaceProvider(String workspaceProviderId) {
WORKSPACE_PROVIDER_MAP.remove(workspaceProviderId);
}

And I added two lines to the ESWorkspaceProvider.dispose() method:

WS_THREAD_LOCAL.set(null);
WorkspaceLocator.removeWorkspaceProvider(getName());

4. Although it's possible to create named client workspaces using
ESWorkspaceProviderImpl.getInstance(String workspaceProviderId), each of
the corresponding workspace XMI resource URIs resolve/normalize to the
same file on disk. For my purposes, I'd like each client workspace
reside in its own directory. As it stands, the URI normalization
routines call out to a custom WorkspaceLocationProvider plugin making it
possible to append a 'profile' name to the workspace directory, but its
static so there's no way to pass workspace ID information to this plugin.

I found a way to fix this that minimizes impact to the code base by
taking advantage of the unused 'profile' parameter in the abstract
methods in ESAbstractClientURIConverter. XMIClientURIConverter currently
ignores this parameter when forming the workspace directory. None of the
changes should impact the current functionality or behavior.

In ESClientURIUtil added methods:

public static URI createWorkspaceURI(String profile) {
return URI.createURI(getClientPrefix(profile) + WORKSPACE_SEGMENT);
}


public static URI createProjectURI(IdentifiableElement projectSpace,
String profile) {
return URI.createURI(getProjectspacesPrefix(projectSpace, profile) +
PROJECT_SEGMENT);
}


public static URI createOperationsURI(IdentifiableElement projectSpace,
String profile) {
return URI.createURI(getProjectspacesPrefix(projectSpace, profile) +
OPERATIONS_SEGMENT);
}

and added/changed these methods:

private static String getClientPrefix() {
return getClientPrefix(ESServerURIUtil.getProfile());
}

private static String getProjectspacesPrefix(IdentifiableElement
projectSpace) {
return getClientPrefix() + getProjectspacesSuffix(projectSpace);
}

private static String getProjectspacesPrefix(IdentifiableElement
projectSpace, String profile) {
return getClientPrefix(profile) + getProjectspacesSuffix(projectSpace);
}

private static String getProjectspacesSuffix(IdentifiableElement
projectSpace) {
return PROJECTSPACES_SEGMENT + "/" + projectSpace.getIdentifier() +
"/"; //$NON-NLS-1$ //$NON-NLS-2$
}

In XMIClientURIConverter I added two methods:

protected static String getWorkspaceDirectory(String profile) {
String directory;
if (Behavior.isSupportMultipleWorkspaces()) {
if (profile != null && profile.length() > 0) {
directory = Configuration.getFileInfo().getWorkspaceDirectory() +
profile + File.separatorChar;
} else {
directory = Configuration.getFileInfo().getWorkspaceDirectory();
}
} else {
directory = Configuration.getFileInfo().getWorkspaceDirectory();
}
return directory;
}

public static String getFileCacheFolder(IdentifiableElement projectSpace) {
final String profile = ESClientURIUtil.getProfile();
final String workspaceDirectory = getWorkspaceDirectory(profile);
final String cacheDirectory = workspaceDirectory +
XMIClientURIConverter.PROJECT_SAPCE_DIRECTORY_PREFIX
+ projectSpace.getIdentifier()
+ File.separatorChar
+ "files" + File.separatorChar; //$NON-NLS-1$
return cacheDirectory;
}

and changed getProjectSpaceDirectory() and normalizeWorkspaceURI() to
call getWorkspaceDirectory().

In ESWorkspaceProviderImpl added method:

private URI createWorkspaceURI() {
URI workspaceURI;
if (id != null && id.length() > 0) {
workspaceURI = ESClientURIUtil.createWorkspaceURI(id);
} else {
workspaceURI = ESClientURIUtil.createWorkspaceURI();
}
return workspaceURI;
}

In FileTransferCacheManager replace one method:

public static String getCacheFolder(ProjectSpace projectSpace) {
return XMIClientURIConverter.getFileCacheFolder(projectSpace);
}

and in ESWorkspaceProviderImpl.load() changed:

final URI workspaceURI = ESClientURIUtil.createWorkspaceURI();

to

URI workspaceURI;
if (Behavior.isSupportMultipleWorkspaces()) {
workspaceURI = createWorkspaceURI();
} else {
workspaceURI = ESClientURIUtil.createWorkspaceURI();
}


In ProjectSpaceBase added methods:

private String getProfileName() {
String profileName;
final ESWorkspaceProviderImpl workspaceProvider =
ESWorkspaceProviderImpl.getInstance();
final String workspaceProviderName = workspaceProvider.getName();
if (workspaceProviderName != null && workspaceProviderName.length() >
0) {
profileName = workspaceProviderName;
} else {
profileName = ESServerURIUtil.getProfile();
}
return profileName;
}


private URI createProjectSpaceURI() {
URI projectSpaceURI;
if (Behavior.isSupportMultipleWorkspaces()) {
String profileName = getProfileName();
projectSpaceURI = ESClientURIUtil.createProjectSpaceURI(this,
profileName);
} else {
projectSpaceURI = ESClientURIUtil.createProjectSpaceURI(this);
}
return projectSpaceURI;
}

private URI createOperationsURI() {
URI operationsURI;
if (Behavior.isSupportMultipleWorkspaces()) {
String profileName = getProfileName();
operationsURI = ESClientURIUtil.createOperationsURI(this, profileName);
} else {
operationsURI = ESClientURIUtil.createOperationsURI(this);
}
return operationsURI;
}

private URI createProjectURI() {
URI projectURI;
if (Behavior.isSupportMultipleWorkspaces()) {
String profileName = getProfileName();
projectURI = ESClientURIUtil.createProjectURI(this, profileName);
} else {
projectURI = ESClientURIUtil.createProjectURI(this);
}
return projectURI;
}

and changed the URI initializers in ProjectSpaceBase.initResources() to
use these new local methods:

URI projectSpaceURI = createProjectSpaceURI();
URI operationsURI = createOperationsURI();
URI projectURI = createProjectURI();

Let me know if you think this is worth a contribution to the EMFStore
code base.


$cott
Re: [EMFStore] Support for multiple workspaces [message #1725162 is a reply to message #1724776] Tue, 01 March 2016 10:21 Go to previous messageGo to next message
Edgar Mueller is currently offline Edgar MuellerFriend
Messages: 89
Registered: March 2011
Member
Hi Scott,

sorry for the delay, comments inline,

> Perhaps there are easier ways to enable EMFStore client multi-threaded
> access to multiple workspaces, so I'm interested in your thoughts. I saw
> comments in the code that suggested that the appserver based model I'm
> trying to implement was tried in the past. Was it?

We included some external changes in the past that would allow the
workspace to be accessed by multiple threads, but we didn't continue
development on that feature ourselves, so I guess it's possible that
there are some things that don't work out for everyone.

> Is there a reason that the ThreadLocal is only set for the first thread
> that creates the workspace and no subsequent threads? I couldn't see why
> the threadlocal would not be set on every call.

I think the initial motivation was that only one thread at most can
access a specific workspace at any time, but this assumption might not
hold for every use case, of course.


> Let me know if you think this is worth a contribution to the EMFStore
> code base.

Yes, I do. Could you please open up a BR and contribute your changes
(https://wiki.eclipse.org/Development_Resources/Contributing_via_Git#via_Gerrit)?
I just skipped over them, but they seem to make sense. We can also
continue our discussion regarding your changes there, if necessary.

Thank you!

--
Edgar Mueller

Get Professional Eclipse Support:
http://eclipsesource.com/en/services/developer-support/
Re: [EMFStore] Support for multiple workspaces [message #1725293 is a reply to message #1725162] Wed, 02 March 2016 10:04 Go to previous messageGo to next message
Jonas Helming is currently offline Jonas HelmingFriend
Messages: 699
Registered: July 2009
Senior Member
Hi Scott,

I think, before you proceed, you should have a look at the contribution
we already accepted for multiple workspace support:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=440288
As far I know, Neil (the contributor) uses this appraoch already in his
project.

Best regards

Jonas

Am 01.03.2016 um 11:21 schrieb Edgar Mueller:
> Hi Scott,
>
> sorry for the delay, comments inline,
>
>> Perhaps there are easier ways to enable EMFStore client multi-threaded
>> access to multiple workspaces, so I'm interested in your thoughts. I saw
>> comments in the code that suggested that the appserver based model I'm
>> trying to implement was tried in the past. Was it?
>
> We included some external changes in the past that would allow the
> workspace to be accessed by multiple threads, but we didn't continue
> development on that feature ourselves, so I guess it's possible that
> there are some things that don't work out for everyone.
>
>> Is there a reason that the ThreadLocal is only set for the first thread
>> that creates the workspace and no subsequent threads? I couldn't see why
>> the threadlocal would not be set on every call.
>
> I think the initial motivation was that only one thread at most can
> access a specific workspace at any time, but this assumption might not
> hold for every use case, of course.
>
>
>> Let me know if you think this is worth a contribution to the EMFStore
>> code base.
>
> Yes, I do. Could you please open up a BR and contribute your changes
> (https://wiki.eclipse.org/Development_Resources/Contributing_via_Git#via_Gerrit)?
> I just skipped over them, but they seem to make sense. We can also
> continue our discussion regarding your changes there, if necessary.
>
> Thank you!
>


--
--

Jonas Helming

Get professional Eclipse developer support:

http://eclipsesource.com/en/services/developer-support/
Re: [EMFStore] Support for multiple workspaces [message #1725567 is a reply to message #1725293] Fri, 04 March 2016 13:15 Go to previous messageGo to next message
Scott Dybiec is currently offline Scott DybiecFriend
Messages: 146
Registered: July 2009
Senior Member
Thanks for the referral to that bugzilla. It's clear that Neil ran into
the same issue I did. I see now where the beginnings of the support for
multiple workspaces got started --- WorkspaceLocator, ThreadLocal, etc.

My additions are based on the ThreadLocal-based design that is currently
in the code, but my tests found some areas that need to be improved for
it to work.

Do you know if Neil ever got this working? I would imagine he ran into
many of the same problems I did.

$cott

On 3/2/2016 5:04 AM, Jonas Helming wrote:
> Hi Scott,
>
> I think, before you proceed, you should have a look at the contribution
> we already accepted for multiple workspace support:
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=440288
> As far I know, Neil (the contributor) uses this appraoch already in his
> project.
>
> Best regards
>
> Jonas
>
> Am 01.03.2016 um 11:21 schrieb Edgar Mueller:
>> Hi Scott,
>>
>> sorry for the delay, comments inline,
>>
>>> Perhaps there are easier ways to enable EMFStore client multi-threaded
>>> access to multiple workspaces, so I'm interested in your thoughts. I saw
>>> comments in the code that suggested that the appserver based model I'm
>>> trying to implement was tried in the past. Was it?
>>
>> We included some external changes in the past that would allow the
>> workspace to be accessed by multiple threads, but we didn't continue
>> development on that feature ourselves, so I guess it's possible that
>> there are some things that don't work out for everyone.
>>
>>> Is there a reason that the ThreadLocal is only set for the first thread
>>> that creates the workspace and no subsequent threads? I couldn't see why
>>> the threadlocal would not be set on every call.
>>
>> I think the initial motivation was that only one thread at most can
>> access a specific workspace at any time, but this assumption might not
>> hold for every use case, of course.
>>
>>
>>> Let me know if you think this is worth a contribution to the EMFStore
>>> code base.
>>
>> Yes, I do. Could you please open up a BR and contribute your changes
>> (https://wiki.eclipse.org/Development_Resources/Contributing_via_Git#via_Gerrit)?
>> I just skipped over them, but they seem to make sense. We can also
>> continue our discussion regarding your changes there, if necessary.
>>
>> Thank you!
>>
>
>
Re: [EMFStore] Support for multiple workspaces [message #1725628 is a reply to message #1725567] Sat, 05 March 2016 10:26 Go to previous messageGo to next message
Jonas Helming is currently offline Jonas HelmingFriend
Messages: 699
Registered: July 2009
Senior Member
Hi,

as mentioned, I think he uses this in his project. I will forward this
to him and ask him to comment here himself.

best regards

Jonas

Am 04.03.2016 um 14:15 schrieb Scott Dybiec:
> Thanks for the referral to that bugzilla. It's clear that Neil ran into
> the same issue I did. I see now where the beginnings of the support for
> multiple workspaces got started --- WorkspaceLocator, ThreadLocal, etc.
>
> My additions are based on the ThreadLocal-based design that is currently
> in the code, but my tests found some areas that need to be improved for
> it to work.
>
> Do you know if Neil ever got this working? I would imagine he ran into
> many of the same problems I did.
>
> $cott
>
> On 3/2/2016 5:04 AM, Jonas Helming wrote:
>> Hi Scott,
>>
>> I think, before you proceed, you should have a look at the contribution
>> we already accepted for multiple workspace support:
>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=440288
>> As far I know, Neil (the contributor) uses this appraoch already in his
>> project.
>>
>> Best regards
>>
>> Jonas
>>
>> Am 01.03.2016 um 11:21 schrieb Edgar Mueller:
>>> Hi Scott,
>>>
>>> sorry for the delay, comments inline,
>>>
>>>> Perhaps there are easier ways to enable EMFStore client multi-threaded
>>>> access to multiple workspaces, so I'm interested in your thoughts. I
>>>> saw
>>>> comments in the code that suggested that the appserver based model I'm
>>>> trying to implement was tried in the past. Was it?
>>>
>>> We included some external changes in the past that would allow the
>>> workspace to be accessed by multiple threads, but we didn't continue
>>> development on that feature ourselves, so I guess it's possible that
>>> there are some things that don't work out for everyone.
>>>
>>>> Is there a reason that the ThreadLocal is only set for the first thread
>>>> that creates the workspace and no subsequent threads? I couldn't see
>>>> why
>>>> the threadlocal would not be set on every call.
>>>
>>> I think the initial motivation was that only one thread at most can
>>> access a specific workspace at any time, but this assumption might not
>>> hold for every use case, of course.
>>>
>>>
>>>> Let me know if you think this is worth a contribution to the EMFStore
>>>> code base.
>>>
>>> Yes, I do. Could you please open up a BR and contribute your changes
>>> (https://wiki.eclipse.org/Development_Resources/Contributing_via_Git#via_Gerrit)?
>>>
>>> I just skipped over them, but they seem to make sense. We can also
>>> continue our discussion regarding your changes there, if necessary.
>>>
>>> Thank you!
>>>
>>
>>
>


--
--

Jonas Helming

Get professional Eclipse developer support:

http://eclipsesource.com/en/services/developer-support/
Re: [EMFStore] Support for multiple workspaces [message #1725638 is a reply to message #1725628] Sat, 05 March 2016 14:30 Go to previous messageGo to next message
Neil Mackenzie is currently offline Neil MackenzieFriend
Messages: 111
Registered: December 2013
Senior Member
Hi Scott. I will check if I have any further documentation or design discussions relating to why this approach was taken. It works for me, but I noticed recently that there is no explicit tidy up when workspaces are no longer needed. I have a fix for this that I am testing, looks like you have too! I am quite busy this week attending eclipsecon in Washington. After that I am making sure that the RAP version of the demo app works 100% correctly with emstore and multiple users in tomcat, there is a bugzilla issue in ecp for this work (i wll find the bug id) so if there are any remaining issues with multiple workspaces in emfstore then I will also be keen to fix and test those also. Happy to discuss with you any common problems we find. Regards, Neil
Re: [EMFStore] Support for multiple workspaces [message #1731452 is a reply to message #1725638] Thu, 05 May 2016 12:20 Go to previous message
Neil Mackenzie is currently offline Neil MackenzieFriend
Messages: 111
Registered: December 2013
Senior Member
I had a look for more documenation but couldnt find any, however the bug report contains a lot of detail about the design.
I have created a number of bug reports today , with attached working patches, in order to get ECP working with RAP and EMFStore. This requires EMFStore dealing with a workspace per user and also removing workspaces when web sessions end (so they dont use up memory resources). I think this is pretty similar to what you are doing Scott.
The primary new bug report of interest is https://bugs.eclipse.org/bugs/show_bug.cgi?id=493068
allowing emfstore workspaces to be removed
also of interest is
https://bugs.eclipse.org/bugs/show_bug.cgi?id=493071
which calls that code from ECP.

I have also added a number of bug reports with patches to get ECP working well with RAP and EMFStore, these might be useful for you or might not Scott. These are 493066 493067 493069 493073 493075 493074 475393
If you have any questions you can send me an email on neilswebmail@yahoo.co.uk
Regards,
Neil
Previous Topic:Add png as an emf resource?
Next Topic:[EMF Forms] Locales with country and variant code
Goto Forum:
  


Current Time: Tue Sep 18 23:13:29 GMT 2018

Powered by FUDForum. Page generated in 0.02313 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top