Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » EMF / GWT Datastore Contention Issues / Failed to Save
EMF / GWT Datastore Contention Issues / Failed to Save [message #776766] Mon, 09 January 2012 08:41 Go to next message
Pasquale Gervasi is currently offline Pasquale Gervasi
Messages: 12
Registered: July 2010
Junior Member
Hi Everyone,

When I try to send multiple http requests to my FrontController (ie bypassing EntryPont Editor) I was abe to pesist many requests in the datastore with:

registerPackages(resourceSet.getPackageRegistry());
resource = resourceSet.getResource(URI.createURI("datastore:/SessionLog"),true);

options.put(URIConverter.OPTION_UPDATE_ONLY_IF_TIME_STAMP_MATCHES,resource.getTimeStamp());
resource.getContents().add(commSession);
try{
resource.save(options);
}catch (IOException e){
System.out.println(e.getLocalizedMessage());
}



until I got these two types of errors:

TYPE I
Failed to save datastore:/SessionLog

TYPE II
java.util.ConcurrentModificationException: too much contention on these datastore entities. please try again.
at com.google.appengine.api.datastore.DatastoreApiHelper.translateError(DatastoreApiHelper.java:39)
at com.google.appengine.api.datastore.DatastoreApiHelper$1.convertException(DatastoreApiHelper.java:76)
at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:106)
at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:90)
at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:90)
at com.google.appengine.api.datastore.FutureHelper.getInternal(FutureHelper.java:72)
at com.google.appengine.api.datastore.FutureHelper.quietGet(FutureHelper.java:33)
at com.google.appengine.api.datastore.TransactionImpl.commit(TransactionImpl.java:105)
at org.eclipse.emf.server.ecore.resource.DatastoreUtil.store(DatastoreUtil.java:360)
at org.eclipse.emf.server.ecore.resource.DatastoreURIHandlerImpl$1.flush(DatastoreURIHandlerImpl.java:54)
at org.eclipse.emf.server.ecore.resource.DatastoreURIHandlerImpl$1.close(DatastoreURIHandlerImpl.java:46)
at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(ResourceImpl.java:1028)
at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(ResourceImpl.java:972)
at com.cloud.connect.session.integration.impl.CommManagerImpl.saveSession(CommManagerImpl.java:165)
at com.cloud.connect.session.integration.FrontController.doPost(FrontController.java:55)
Re: EMF / GWT Datastore Contention Issues / Failed to Save [message #776790 is a reply to message #776766] Mon, 09 January 2012 09:26 Go to previous messageGo to next message
Ed Merks is currently offline Ed Merks
Messages: 26048
Registered: July 2009
Senior Member
Pasquale,

It's hard to comment about the behavior of the Google AppEngine itself.
I'm not sure there's any discussion about commit producing this
particular exception. Perhaps there is a mechanism in place to prevent
what would amount to a DoS attack by overuse. I.e., if the rate at
which the requests for new commits comes in faster than the ability of
the engine to actually do the commits, some queue (one that's
sequentializing the transactions) is likely to overfill...


On 09/01/2012 9:41 AM, Pasquale Gervasi wrote:
> Hi Everyone,
>
> When I try to send multiple http requests to my FrontController (ie
> bypassing EntryPont Editor) I was abe to pesist many requests in the
> datastore with:
>
> registerPackages(resourceSet.getPackageRegistry());
> resource =
> resourceSet.getResource(URI.createURI("datastore:/SessionLog"),true);
>
> options.put(URIConverter.OPTION_UPDATE_ONLY_IF_TIME_STAMP_MATCHES,resource.getTimeStamp());
>
> resource.getContents().add(commSession);
> try{
> resource.save(options);
> }catch (IOException e){
> System.out.println(e.getLocalizedMessage());
> }
>
>
>
> until I got these two types of errors:
>
> TYPE I
> Failed to save datastore:/SessionLog
> TYPE II
> java.util.ConcurrentModificationException: too much contention on
> these datastore entities. please try again.
> at
> com.google.appengine.api.datastore.DatastoreApiHelper.translateError(DatastoreApiHelper.java:39)
> at
> com.google.appengine.api.datastore.DatastoreApiHelper$1.convertException(DatastoreApiHelper.java:76)
> at
> com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:106)
> at
> com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:90)
> at
> com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:90)
> at
> com.google.appengine.api.datastore.FutureHelper.getInternal(FutureHelper.java:72)
> at
> com.google.appengine.api.datastore.FutureHelper.quietGet(FutureHelper.java:33)
> at
> com.google.appengine.api.datastore.TransactionImpl.commit(TransactionImpl.java:105)
> at
> org.eclipse.emf.server.ecore.resource.DatastoreUtil.store(DatastoreUtil.java:360)
> at
> org.eclipse.emf.server.ecore.resource.DatastoreURIHandlerImpl$1.flush(DatastoreURIHandlerImpl.java:54)
> at
> org.eclipse.emf.server.ecore.resource.DatastoreURIHandlerImpl$1.close(DatastoreURIHandlerImpl.java:46)
> at
> org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(ResourceImpl.java:1028)
> at
> org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(ResourceImpl.java:972)
> at
> com.cloud.connect.session.integration.impl.CommManagerImpl.saveSession(CommManagerImpl.java:165)
> at
> com.cloud.connect.session.integration.FrontController.doPost(FrontController.java:55)
>
Re: EMF / GWT Datastore Contention Issues / Failed to Save [message #778815 is a reply to message #776790] Fri, 13 January 2012 22:11 Go to previous messageGo to next message
Pasquale Gervasi is currently offline Pasquale Gervasi
Messages: 12
Registered: July 2010
Junior Member
Hi Ed, sorry for the silence I was dealing with another issue on GAE.

Ok, back to your reply, would your generated editor have the same issue if there were enough users clicking the save button at the same time? Or is there something in the async process that would prevent this issue. Just wondering if you would recommend I trace the generated client side code all the way to the server side and see how I could use it.

Other options, which seems ugly, is having to implement task queues on the save operation. Any other suggestions?

Pasquale
Re: EMF / GWT Datastore Contention Issues / Failed to Save [message #778945 is a reply to message #778815] Sat, 14 January 2012 07:22 Go to previous messageGo to next message
Ed Merks is currently offline Ed Merks
Messages: 26048
Registered: July 2009
Senior Member
Pasquale,

Comments below.

On 13/01/2012 11:11 PM, Pasquale Gervasi wrote:
> Hi Ed, sorry for the silence I was dealing with another issue on GAE.
> Ok, back to your reply, would your generated editor have the same
> issue if there were enough users clicking the save button at the same
> time?
I suppose it's possible too. One user couldn't generate the volume
because they're waiting for the result of one save, but if many users
were saving the same thing at once, which of course one would expect to
lead to many failures because all but one would be trying to overwrite
some else's change, then it could be possible too. Of course such a
design would be frustrating to most users, so that's not really workable
anyway.
> Or is there something in the async process that would prevent this issue.
No, I doubt it
> Just wondering if you would recommend I trace the generated client
> side code all the way to the server side and see how I could use it.
I suspect it's an issue reproducible purely on the server side, but I
don't know...
> Other options, which seems ugly, is having to implement task queues on
> the save operation. Any other suggestions?
Given that colliding saves should fail anyway, you might do a time stamp
check before saving. Or am I not understanding something about your
application? I.e., should there be no failures at all because all your
saves are properly serialized?
>
> Pasquale
Re: EMF / GWT Datastore Contention Issues / Failed to Save [message #779227 is a reply to message #776766] Sun, 15 January 2012 03:53 Go to previous messageGo to next message
Pasquale Gervasi is currently offline Pasquale Gervasi
Messages: 12
Registered: July 2010
Junior Member
Thanks. I'll try the timestamp idea.
Pasquale
Re: EMF / GWT Datastore Contention Issues / Failed to Save [message #780095 is a reply to message #778945] Tue, 17 January 2012 07:08 Go to previous messageGo to next message
Pasquale Gervasi is currently offline Pasquale Gervasi
Messages: 12
Registered: July 2010
Junior Member
Hi Ed,

I'm pretty sure I'm missing something in my code. For every Http session (i.e. a different user(actually a plain java HTTP client) submitts a http POST request) i create an EMF object commSession and execute this method to save it in the datastore.



public void saveSession(RequestContext requestContext)

/* Create EMF object commSession based on http request */


Map<String, Object> options = null;
        options = new HashMap<String, Object>();
		ResourceSet resourceSet = new ResourceSetImpl();

		//Required
		
	    registerPackages(resourceSet.getPackageRegistry());
		resource = resourceSet.getResource(URI.createURI("datastore:/SessionLog"),true);	
        options.put(URIConverter.OPTION_UPDATE_ONLY_IF_TIME_STAMP_MATCHES, resource.getTimeStamp());
		String msgID = commSession.getRequest().getID();

		if (resource.isLoaded()){
			resource.getContents().add(commSession);
			
			try{
				resource.save(options);
				
			}catch (IOException e){
            	System.out.println(e.getLocalizedMessage());
            	System.out.println("Message ID: "+msgID+" not stored!");				
			}
						
		}else
		{
        	System.out.println("Session Resource: "+resource.getURI().toString()+" not loaded!");			
        	System.out.println("Message ID: "+msgID+" not stored!");			
		}


Errors I get:

java.util.ConcurrentModificationException: too much contention on these datastore entities. please try again.
at com.google.appengine.api.datastore.DatastoreApiHelper.translateError(DatastoreApiHelper.java:39)
at com.google.appengine.api.datastore.DatastoreApiHelper$1.convertException(DatastoreApiHelper.java:76)
at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:106)
at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:90)
at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:90)
at com.google.appengine.api.datastore.FutureHelper.getInternal(FutureHelper.java:72)
at com.google.appengine.api.datastore.FutureHelper.quietGet(FutureHelper.java:33)
at com.google.appengine.api.datastore.TransactionImpl.commit(TransactionImpl.java:105)
at org.eclipse.emf.server.ecore.resource.DatastoreUtil.store(DatastoreUtil.java:360)
at org.eclipse.emf.server.ecore.resource.DatastoreURIHandlerImpl$1.flush(DatastoreURIHandlerImpl.java:54)
at org.eclipse.emf.server.ecore.resource.DatastoreURIHandlerImpl$1.close(DatastoreURIHandlerImpl.java:46)
at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(ResourceImpl.java:1028)
at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(ResourceImpl.java:972)

Re: EMF / GWT Datastore Contention Issues / Failed to Save [message #780117 is a reply to message #780095] Tue, 17 January 2012 08:04 Go to previous messageGo to next message
Ed Merks is currently offline Ed Merks
Messages: 26048
Registered: July 2009
Senior Member
Pasquale,

How many times is this being called to have a problem? Are you
expecting all users to share a single log? I'd expect each user to have
their own log...


On 17/01/2012 8:08 AM, Pasquale Gervasi wrote:
> Hi Ed,
>
> I'm pretty sure I'm missing something in my code. For every Http
> session (i.e. a different user(actually a plain java HTTP client)
> submitts a http POST request) i create an EMF object commSession and
> execute this method to save it in the datastore.
>
>
>
>
> public void saveSession(RequestContext requestContext)
>
> /* Create EMF object commSession based on http request */
>
>
> Map<String, Object> options = null;
> options = new HashMap<String, Object>();
> ResourceSet resourceSet = new ResourceSetImpl();
>
> //Required
>
> registerPackages(resourceSet.getPackageRegistry());
> resource =
> resourceSet.getResource(URI.createURI("datastore:/SessionLog"),true);
>
> options.put(URIConverter.OPTION_UPDATE_ONLY_IF_TIME_STAMP_MATCHES,
> resource.getTimeStamp());
> String msgID = commSession.getRequest().getID();
>
> if (resource.isLoaded()){
> resource.getContents().add(commSession);
>
> try{
> resource.save(options);
>
> }catch (IOException e){
> System.out.println(e.getLocalizedMessage());
> System.out.println("Message ID: "+msgID+" not stored!");
> }
>
> }else
> {
> System.out.println("Session Resource:
> "+resource.getURI().toString()+" not loaded!");
> System.out.println("Message ID: "+msgID+" not stored!");
> }
>
>
> Errors I get:
>
> java.util.ConcurrentModificationException: too much contention on
> these datastore entities. please try again.
> at
> com.google.appengine.api.datastore.DatastoreApiHelper.translateError(DatastoreApiHelper.java:39)
> at
> com.google.appengine.api.datastore.DatastoreApiHelper$1.convertException(DatastoreApiHelper.java:76)
> at
> com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:106)
> at
> com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:90)
> at
> com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:90)
> at
> com.google.appengine.api.datastore.FutureHelper.getInternal(FutureHelper.java:72)
> at
> com.google.appengine.api.datastore.FutureHelper.quietGet(FutureHelper.java:33)
> at
> com.google.appengine.api.datastore.TransactionImpl.commit(TransactionImpl.java:105)
> at
> org.eclipse.emf.server.ecore.resource.DatastoreUtil.store(DatastoreUtil.java:360)
> at
> org.eclipse.emf.server.ecore.resource.DatastoreURIHandlerImpl$1.flush(DatastoreURIHandlerImpl.java:54)
> at
> org.eclipse.emf.server.ecore.resource.DatastoreURIHandlerImpl$1.close(DatastoreURIHandlerImpl.java:46)
> at
> org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(ResourceImpl.java:1028)
> at
> org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(ResourceImpl.java:972)
>
>
Re: EMF / GWT Datastore Contention Issues / Failed to Save [message #780291 is a reply to message #780117] Tue, 17 January 2012 14:44 Go to previous messageGo to next message
Pasquale Gervasi is currently offline Pasquale Gervasi
Messages: 12
Registered: July 2010
Junior Member
Hi Ed,

I should decribe the process at the java client (the user is actually a system). The system(or "user") is a communication app polling a local file directory. Each http session is a single file transfer to GAE using http post. A single user's polling directory can have hundreds files and the java client can spawn multiple http sessions each executing a single http post. Aggregating the volume across hundreds of "users" and you can get tens of thousands of file transfer requests. Keep in mind the "the user" is a system that just needs a single http response to end the session. I want to save the payload to the datastore and end the session as fast as possible.

On my side I will then process the payload after the session ends using task queues.

So, to answer your question the datastore:/SessionLog is shared across all users but I could create one per user but then I have hundreds(users) of datastores. No?

I do expect to queue the requests when the volume hits these large numbers but wanted to see how much I can get without queuing at this time by using the EMF/GWT components available.

Pasquale
Re: EMF / GWT Datastore Contention Issues / Failed to Save [message #780300 is a reply to message #780291] Tue, 17 January 2012 14:57 Go to previous messageGo to next message
Ed Merks is currently offline Ed Merks
Messages: 26048
Registered: July 2009
Senior Member
Pasquale,

Comments below.

On 17/01/2012 3:44 PM, Pasquale Gervasi wrote:
> Hi Ed,
>
> I should decribe the process at the java client (the user is actually
> a system). The system(or "user") is a communication app polling a
> local file directory. Each http session is a single file transfer to
> GAE using http post. A single user's polling directory can have
> hundreds files and the java client can spawn multiple http sessions
> each executing a single http post. Aggregating the volume across
> hundreds of "users" and you can get tens of thousands of file transfer
> requests. Keep in mind the "the user" is a system that just needs a
> single http response to end the session. I want to save the payload
> to the datastore and end the session as fast as possible.
>
> On my side I will then process the payload after the session ends
> using task queues.
>
> So, to answer your question the datastore:/SessionLog is shared across
> all users but I could create one per user but then I have
> hundreds(users) of datastores. No?
But how are the different user's changes aggregated with this approach,
as opposed to overwriting the changes made by someone else, which is how
your design looks to me?
>
> I do expect to queue the requests when the volume hits these large
> numbers but wanted to see how much I can get without queuing at this
> time by using the EMF/GWT components available.
I think each user's session should be saved under a different entity and
then you can query all those entities to produce an aggregate.

In the end, you simply can't design this so everyone is hammering away
at the same entity in the same data store...
>
> Pasquale
Re: EMF / GWT Datastore Contention Issues / Failed to Save [message #780437 is a reply to message #780300] Tue, 17 January 2012 21:38 Go to previous messageGo to next message
Pasquale Gervasi is currently offline Pasquale Gervasi
Messages: 12
Registered: July 2010
Junior Member
Hi Ed,

Each user is only adding a new session entity to the datastore - never changing existing entities. I never need to access the other entities but the only way I know to save the new entity correctly is:

resource = resourceSet.getResource(URI.createURI("datastore:/SessionLog"),true);

then adding a new entity to it:

resource.getContents().add(commSession);

and then save. Is there a way to save the new entity without loading the resource? I tried createResource without loading but when I save I lose previous saved entities and I tried
resource = resourceSet.getResource(URI.createURI("datastore:/SessionLog"),true);

resource.unload();

and got the same result as createResource.

When you say,
"I think each user's session should be saved under a different entity and
then you can query all those entities to produce an aggregate."

How would you construct the URI and prepare the resource before executing the resource.getContents().add command on the entity? Append obj.Hashcode() to the URI?

I'm not understanding sorry. I'm a junior member after all.

Pasquale

Re: EMF / GWT Datastore Contention Issues / Failed to Save [message #780973 is a reply to message #780437] Wed, 18 January 2012 08:08 Go to previous messageGo to next message
Ed Merks is currently offline Ed Merks
Messages: 26048
Registered: July 2009
Senior Member
Pasquale,

Comments below.

On 17/01/2012 10:38 PM, Pasquale Gervasi wrote:
> Hi Ed,
>
> Each user is only adding a new session entity to the datastore - never
> changing existing entities. I never need to access the other entities
> but the only way I know to save the new entity correctly is:
>
> resource =
> resourceSet.getResource(URI.createURI("datastore:/SessionLog"),true);
You show a fixed URI that everyone shares because it's the same for
everyone. It doesn't make sense to load one if you expect this to be
new information.
>
> then adding a new entity to it:
>
> resource.getContents().add(commSession);
>
> and then save. Is there a way to save the new entity without loading
> the resource?
Use a different URI for each client and each session. Have a look at
DatastoreUtil.OPTION_SESSION to see how that approach stores everything
under session-specific keys.
> I tried createResource without loading but when I save I lose previous
> saved entities and I tried
> resource =
> resourceSet.getResource(URI.createURI("datastore:/SessionLog"),true);
>
> resource.unload();
>
> and got the same result as createResource.
Keep in mind what OPTION_UPDATE_ONLY_IF_TIME_STAMP_MATCHES. Your save
will fail if the time stamp in the data store doesn't match the given
time stamp. But of course for a resource that's new , it's not in the
repo and using this option will fail every time. Don't use that
option unless your saving changes you made to a resource you loaded and
which you want to overwrite the previous version you loaded.
>
> When you say, "I think each user's session should be saved under a
> different entity and then you can query all those entities to produce
> an aggregate."
>
> How would you construct the URI and prepare the resource before
> executing the resource.getContents().add command on the entity?
For a new resource, the URI matters only when you save it.
> Append obj.Hashcode() to the URI?
How will you want to identify sessions later when you make use of them?
If you included the user identity and perhaps a creation timestamp as
part of the URI (the time the session you're recording started perhaps),
you'd end up with something unique. Using EcoreUtil.generateUUID is
another approach.
> I'm not understanding sorry. I'm a junior member after all.
>
> Pasquale
>
>
Re: EMF / GWT Datastore Contention Issues / Failed to Save [message #783098 is a reply to message #780973] Wed, 25 January 2012 03:49 Go to previous message
Pasquale Gervasi is currently offline Pasquale Gervasi
Messages: 12
Registered: July 2010
Junior Member
Thanks Ed. Concepts are starting to make sense. I found an interesting thread:

http://www.eclipse.org/forums/index.php/mv/msg/203479/650678/#msg_650678

Should have started there.

Pasquale
Previous Topic:[CDO] Notifications
Next Topic:[CDO] Offline mode
Goto Forum:
  


Current Time: Thu Sep 18 06:03:43 GMT 2014

Powered by FUDForum. Page generated in 0.07392 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software