|
| Re: [CDO] Store multiple large files concurrently [message #1829518 is a reply to message #1829495] |
Mon, 06 July 2020 05:59   |
|
Quote:I'm creating a REST api for accessing a CDO repository. Multiple users can connect at the same time to store and retrieve models from the repo. I started out with caching a SessionConfiguration (which also caches an open session) for each user's HTTP session to speed up CDO access.
Hi Ewoud,
a CDOSession, through it's CDORevisionManager, caches CDORevisions which contain all the data of the objects. These revision caches can become big. Using multiple client sessions on the server and keeping them open can become a resource problem. As an alternative you could open just one session and open a separate transaction for each user of this session. Unfortunately the user name is currently associated with the session (if at all) and not with a transaction or commit. In theory this could be enhanced if you are interested in sponsoring a respective effort.
Quote:I'm now trying to store a larger set of files (>4MB XMI each, ~80MB in total) in CDO in parallel (in the same cached CDO user session) and having a problem with it.
Does that mean you open 1 transaction on that session and have multiple threads calling commit on that transaction? Note that all methods of CDOTransaction and CDOView, as well as all model getters and setters synchronize on the view/transaction monitor lock. They will not commit in parallel.
Quote:I lock the folder in which all files will be stored by using a folder.lockOption().lock() with a timeout of 10 minutes, but get a TimeoutRuntimeException while the next file is waiting for the lock during the commit of the curent file in the repo. I assume this has something to do with that this lock is in the current CDOSession. How can I improve this? Do I need to queue these files first?
I guess you mean folder.cdoWriteOption().lock()? This would not mean that you own a write lock. It means that you own the exclusive right to acquire a write lock. With this type of lock (i.e., lock option) you prevent others from acquiring a write lock, but you don't prevent others from acquiring read locks.
Quote:Additionally, it takes quite some time to store a single file in the repository (the commit itself takes quite some time). Are there ways to speed up this process?
It's not enirely clear to me what you're doing and how. Is it possible that you show the relevant code and the full stack traces that you receive?
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: [CDO] Store multiple large files concurrently [message #1829624 is a reply to message #1829518] |
Tue, 07 July 2020 19:11 |
 Ewoud Werkman
Messages: 28
Registered: January 2018 |
Junior Member |
|
|
Eike Stepper wrote on Mon, 06 July 2020 05:59
Quote:I'm creating a REST api for accessing a CDO repository. Multiple users can connect at the same time to store and retrieve models from the repo. I started out with caching a SessionConfiguration (which also caches an open session) for each user's HTTP session to speed up CDO access.
Hi Ewoud,
a CDOSession, through it's CDORevisionManager, caches CDORevisions which contain all the data of the objects. These revision caches can become big. Using multiple client sessions on the server and keeping them open can become a resource problem. As an alternative you could open just one session and open a separate transaction for each user of this session. Unfortunately the user name is currently associated with the session (if at all) and not with a transaction or commit. In theory this could be enhanced if you are interested in sponsoring a respective effort.
Thanks for your quick reply!
Yes, that is spot on. I tried first using one SessionConfiguration and a single session, but I could not set the user name, and that information is necessary for our use case (keep a history of commits by each user on the same model). The user name is set in the SessionConfiguration and not when opening a new session or transaction (it will generate a"Session is already open" error). Unfortunately, sponsoring is not an option for me.
Quote:Quote:I'm now trying to store a larger set of files (>4MB XMI each, ~80MB in total) in CDO in parallel (in the same cached CDO user session) and having a problem with it.
Does that mean you open 1 transaction on that session and have multiple threads calling commit on that transaction? Note that all methods of CDOTransaction and CDOView, as well as all model getters and setters synchronize on the view/transaction monitor lock. They will not commit in parallel.
Before committing the transaction I first lock the folder in which the resource will be placed. I do the following:
CDOTransaction transaction = session.openTransaction();
CDOResourceFolder resourceFolder = transaction.getResourceFolder(folder);
CDOLock cdoWriteOption = resourceFolder.cdoWriteOption();
try {
System.out.print("Acquiring write lock for folder "+ resourceFolder.getPath() + "...");
cdoWriteOption.lock(CONCURRENTWRITE_TIMEOUT_MINUTES, TimeUnit.MINUTES);
System.out.println("done");
CDOResource resource = transaction.getOrCreateResource(resourceName);
if (resource.getContents().size() == 1) {
// overwrite
resource.getContents().set(0, rootObject);
} else {
resource.getContents().add(rootObject);
}
System.out.println("Committing transaction");
CDOCommitInfo commitInfo = transaction.commit();
} catch (Exception e) {
System.out.println("Acquiring write lock: timed out");
throw e;
} finally {
cdoWriteOption.unlock();
}
The web browser (ajax calls) opens a HTTP connection to the CDO REST endpoint for each file that gets uploaded, so if the user uploads 5 files, it will do five parallel connections and the code above will be called 5 times concurrently.
The principle I thought would work is lock on the folder (the folder resource itself also changes due to adding or updating a resource in that folder) for a write option (so other clients can still read from the folder in their session). The first connection will get the lock and the others will wait for a maximum of CONCURRENTWRITE_TIMEOUT_MINUTES minutes before timing out. Unfortunately this doesn't work, as I get the following TimeoutException:
[INFO] [ERROR ] Error: org.eclipse.net4j.util.WrappedException: java.util.concurrent.TimeoutException
[INFO] [err] org.eclipse.net4j.util.WrappedException: java.util.concurrent.TimeoutException
[INFO] [err] at org.eclipse.net4j.util.WrappedException.wrap(WrappedException.java:54)
[INFO] [err] at org.eclipse.emf.cdo.internal.net4j.protocol.CDOClientProtocol.lockObjects2(CDOClientProtocol.java:337)
[INFO] [err] at org.eclipse.emf.internal.cdo.view.CDOViewImpl.lockObjects(CDOViewImpl.java:405)
[INFO] [err] at org.eclipse.emf.internal.cdo.view.CDOViewImpl.lockObjects(CDOViewImpl.java:353)
[INFO] [err] at org.eclipse.emf.internal.cdo.object.CDOLockImpl.lock(CDOLockImpl.java:84)
[INFO] [err] at nl.tno.esdl.hub.cdo.CDOManager.storeResource(CDOManager.java:200)
[INFO] [err] at nl.tno.esdl.hub.CDOServerResource.putResource(CDOServerResource.java:186)
[INFO] [err] at nl.tno.esdl.hub.CDOServerResource$Proxy$_$$_WeldClientProxy.putResource(Unknown Source)
[INFO] [err] at sun.reflect.GeneratedMethodAccessor955.invoke(Unknown Source)
[INFO] [err] at java.lang.reflect.Method.invoke(Method.java:498)
[INFO] [err] at com.ibm.ws.jaxrs20.cdi.component.JaxRsFactoryImplicitBeanCDICustomizer.serviceInvoke(JaxRsFactoryImplicitBeanCDICustomizer.java:339)
[INFO] [err] at [internal classes]
[INFO] [err] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[INFO] [err] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[INFO] [err] at java.lang.Thread.run(Thread.java:748)
[INFO] [err] Caused by: java.util.concurrent.TimeoutException
[INFO] [err] at org.eclipse.net4j.util.io.IOTimeoutException.createTimeoutException(IOTimeoutException.java:46)
[INFO] [err] at org.eclipse.net4j.signal.Signal.runSync(Signal.java:301)
[INFO] [err] at org.eclipse.net4j.signal.SignalProtocol.startSignal(SignalProtocol.java:527)
[INFO] [err] at org.eclipse.net4j.signal.RequestWithConfirmation.doSend(RequestWithConfirmation.java:90)
[INFO] [err] at org.eclipse.net4j.signal.RequestWithConfirmation.send(RequestWithConfirmation.java:76)
[INFO] [err] at org.eclipse.emf.cdo.internal.net4j.protocol.CDOClientProtocol.lockObjects2(CDOClientProtocol.java:318)
[INFO] [err] ... 57 more
[INFO] [err] Caused by: org.eclipse.net4j.util.io.IOTimeoutException
[INFO] [err] at org.eclipse.net4j.buffer.BufferInputStream.computeTimeout(BufferInputStream.java:354)
[INFO] [err] at org.eclipse.net4j.buffer.BufferInputStream.ensureBuffer(BufferInputStream.java:310)
[INFO] [err] at org.eclipse.net4j.buffer.BufferInputStream.read(BufferInputStream.java:135)
[INFO] [err] at java.io.DataInputStream.readBoolean(DataInputStream.java:242)
[INFO] [err] at org.eclipse.net4j.util.io.ExtendedDataInput$Delegating.readBoolean(ExtendedDataInput.java:79)
[INFO] [err] at org.eclipse.emf.cdo.internal.net4j.protocol.LockObjectsRequest.confirming(LockObjectsRequest.java:78)
[INFO] [err] at org.eclipse.emf.cdo.internal.net4j.protocol.LockObjectsRequest.confirming(LockObjectsRequest.java:1)
[INFO] [err] at org.eclipse.emf.cdo.internal.net4j.protocol.CDOClientRequest.confirming(CDOClientRequest.java:104)
[INFO] [err] at org.eclipse.net4j.signal.RequestWithConfirmation.doExtendedInput(RequestWithConfirmation.java:126)
[INFO] [err] at org.eclipse.net4j.signal.Signal.doInput(Signal.java:391)
[INFO] [err] at org.eclipse.net4j.signal.RequestWithConfirmation.doExecute(RequestWithConfirmation.java:106)
[INFO] [err] at org.eclipse.net4j.signal.SignalActor.execute(SignalActor.java:53)
[INFO] [err] at org.eclipse.net4j.signal.Signal.runSync(Signal.java:297)
[INFO] [err] ... 61 more
Quote:Quote:I lock the folder in which all files will be stored by using a folder.lockOption().lock() with a timeout of 10 minutes, but get a TimeoutRuntimeException while the next file is waiting for the lock during the commit of the curent file in the repo. I assume this has something to do with that this lock is in the current CDOSession. How can I improve this? Do I need to queue these files first?
I guess you mean folder.cdoWriteOption().lock()? This would not mean that you own a write lock. It means that you own the exclusive right to acquire a write lock. With this type of lock (i.e., lock option) you prevent others from acquiring a write lock, but you don't prevent others from acquiring read locks.
Ah, I misunderstood that, but folder.cdoWriteLock().lock() gives me the same issue (with the same user session and CDOSession). It seems that locking always gives the above Exception, even with one file. I seem to misunderstand the use of locks. Is locking on a ResourceFolder not a good idea?
Quote:Quote:Additionally, it takes quite some time to store a single file in the repository (the commit itself takes quite some time). Are there ways to speed up this process?
It's not enirely clear to me what you're doing and how. Is it possible that you show the relevant code and the full stack traces that you receive?
I hope I clarified it a bit. What I meant was that storing a single 4MB XMI takes significant time and I was wondering how I could speed this up. The database is a PostgreSQL 12 database and both the db and the cdo server run on a 8 core 32gb machine.
[Updated on: Tue, 07 July 2020 19:23] Report message to a moderator |
|
|
| Re: [CDO] Store multiple large files concurrently [message #1829634 is a reply to message #1829624] |
Wed, 08 July 2020 05:48 |
|
A few comments:
1. You need a cdoWriteLock() to exclude concurrent commits and other write locks.
2. A write lock does not exclude other reads, just other read locks.
3. The timeout on the lock() call should be smaller than Net4j's protocol timeout.
You can increase the protocol timeout like so:
((org.eclipse.emf.cdo.net4j.CDONet4jSession)session).options().getNet4jProtocol().setTimeout(lockTimeout + 10000);
This should fix your exception.
4. If all your model access goes through your REST endpoint then you could perhaps do the thread synchronization through
an own in-memory mechansim. Then you could leave the protocol timeout small.
5. Regarding the time needed for committing a large CLOB: Many things happen during a commit.
Network transport to the server, (implicit) locking, pre-commit checks, and actual DB persisting.
Can you use a profiler to measure the time spent in TransactionCommitContext.writeAccessor()?
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
|
|
Powered by
FUDForum. Page generated in 0.02375 seconds
] 
Search
Help
Register
Login
Home
