Re: [udig-dev] ConcurrentModificationException on MemoryDataStore Layer

[Jody Garnett <jody.garnett@xxxxxxxxx>]

Not that major an upgrade, but yeah finally putting AbstractDataStore classes to bed. I am still waiting for some downstream projects to check in (uDig and GeoMesa) before deleting these classes.

Trodden may be able to answer specific questions about the ContentDataStore implementation of MemoryDataStore (but North America is on holiday today).

I did not expect to find that features( typeName ) method. It goes a bit against the design of ContentDataStore - but I expect it was done in order to have a more direct port from the AbstractDataStore original.

What would be better? I would of done a MemoryContentEntry (already arranged in a Map by type name) each one of which containing its FeatureCollection. This same approach (subclass ContentEntry and ContentState) is what is done by JDBCDataStore and the ProperyDataStore tutorial.

As it is I think you have found your concurrent modification error, and we have a couple days left before the beta closes to fix it. Even if we keep/modify that Map we would need to document how it should be locked with respect to ContentState 

--

Jody Garnett

On 7 September 2015 at 13:33, Frank Gasdorf <fgdrf@xxxxxxxxxxxxxxxxxxxxx> wrote:

Good to hear gt 14.x has been approved. Sounds like a major upgrade (from 11.2 right now) for uDig with some implications for Implementations. At the moment I'm upgrating a 1.2 Application to uDig 2.0 which is not straight forward too.

Tomorrow I'm goint to try 14.x MemoryDataStore in a Test-Setup to verify whether it would work. My first impression after I reviewed https://github.com/geotools/geotools/blob/aa0d0346e4bb4a5a2055559c22d2b12ded7cbd20/modules/library/data/src/main/java/org/geotools/data/memory/MemoryDataStore.java#L314 is:

* readers and writers are synchronized on internal map of featureTypes and Map of FeatureIds and Features (memory)

* please correct me, but if using accessing Features for FeatureType is not synchronized which would lead into Concurrent Modification Exceptions (memory.get(FeatureType).iterator())

However, I try to prepare a test scenario and let you know

- Frank

 

2015-09-07 21:36 GMT+02:00 Jody Garnett <jody.garnett@xxxxxxxxx>:

In general we try and have a Transaction to keep seperate threads independent, but in this case we are both rendering from (and writing) to the same transaction.

I would not put any time into debugging this where you are at, The GeoTools 14.x series has been reviewed and approved by Eclipse IP process. This branch has migrated MemoryDataStore to ContendDataStore - so we are up against a new thing to test against.

As part of the migration Torben was able to poke a few holes in the Diff data structure used to keep transactions independent. I would expect that would be the correct location to resolve any concurrent modification problems.

--

Jody

--

Jody Garnett

On 7 September 2015 at 12:23, Frank Gasdorf <fgdrf@xxxxxxxxxxxxxxxxxxxxx> wrote:

Hi folks,

Sorry for cross-posting, I'm unsure which forum is the right choise to address this 

I'm wondering about the current behavior adding features programatically to a layer. The layer has a MemoryGeoResourceImpl internally which uses MemoryDataStore (geotools 11.2 & uDig 2.0.0-SNAPSHOT).

I expected to get features stored to DataStore while uDig is (still) rendering features already added. But the during rendering ConcurrentModificatinExtepion's are thrown while writing new features or modifying exiting.

!ENTRY org.locationtech.udig.project 2 0 2015-09-07 16:06:32.517
!MESSAGE class java.util.ConcurrentModificationException occured during rendering: null
!STACK 0
org.locationtech.udig.project.render.RenderException: class java.util.ConcurrentModificationException occured during rendering: null
    at org.locationtech.udig.render.internal.feature.basic.BasicFeatureRenderer.render(BasicFeatureRenderer.java:489)
    at org.locationtech.udig.render.internal.feature.basic.BasicFeatureRenderer.render(BasicFeatureRenderer.java:320)
    at org.locationtech.udig.project.internal.render.impl.RenderJob.startRendering(RenderJob.java:117)
    at org.locationtech.udig.project.internal.render.impl.RenderJob.run(RenderJob.java:222)
    at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
Caused by: java.util.ConcurrentModificationException
    at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(Unknown Source)
    at java.util.LinkedHashMap$ValueIterator.next(Unknown Source)
    at org.geotools.data.memory.MemoryDataStore.getBounds(MemoryDataStore.java:609)
    at org.geotools.data.AbstractFeatureSource.getBounds(AbstractFeatureSource.java:318)
    at org.geotools.data.AbstractFeatureSource.getBounds(AbstractFeatureSource.java:286)
    at org.locationtech.udig.catalog.memory.internal.MemoryGeoResourceImpl$ScratchResourceInfo.getBounds(MemoryGeoResourceImpl.java:167)
    at org.locationtech.udig.project.internal.impl.GeoResourceInfoInterceptor$Wrapper.getBounds(GeoResourceInfoInterceptor.java:63)
    at org.locationtech.udig.project.internal.impl.LayerImpl.obtainBoundsFromResources(LayerImpl.java:2173)
    at org.locationtech.udig.project.internal.impl.LayerImpl.getBounds(LayerImpl.java:2144)
    at org.locationtech.udig.render.internal.feature.basic.BasicFeatureRenderer.validateBounds(BasicFeatureRenderer.java:567)
    at org.locationtech.udig.render.internal.feature.basic.BasicFeatureRenderer.render(BasicFeatureRenderer.java:363)
    ... 4 more

From a "user" perspective its quite hard to synchchronize access to the datastore while there are different extensions accessing the layer (getBounds, Iterator for Renderer, etc.) and its resources from uDig. I remember a discussion a few years ago about the same problem but cannot find the thread anymore.

Q: What's the best aproach to write features to MemoryDataStore while other having read-access in between? Would it be a good approach using a ConcurrentHashMap internally or using a different DataStore (e.g. h2)?

Thanks in advance

-- Frank

_______________________________________________
udig-dev mailing list
udig-dev@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://locationtech.org/mailman/listinfo/udig-dev