| Re: [geowave-dev] memory issues |
|
Hi,
For future reference, i manage to resolve this. The problem was a different SNAPSHOT geowave version in the accumulo classpath (general.vfs.context.classpath.geowave) and in the client. When i start testing geowave i build the full geowave-deploy-0.9.1-SNAPSHOT-accumulo-singlejar.jar to put in the accumulo classpath. The version was "0.9.1-20160201.220751-4". It seems after running some maven package operation in the client, the geowave 0.9.1-SNAPSHOT version got updated to "0.9.1-20160301.213533-5", so although both accumulo and the client had 0.9.1 version, they had in fact different classes that generate some classpath error and eventually an OutOfMemory. That's also why i remember i had this working in the past. Regards, De: geowave-dev-bounces@xxxxxxxxxxxxxxxx <geowave-dev-bounces@xxxxxxxxxxxxxxxx> en nombre de Jose M <soloninguno@xxxxxxxxxxx>
Enviado: lunes, 11 de abril de 2016 04:07 p.m. Para: geowave-dev@xxxxxxxxxxxxxxxx Asunto: Re: [geowave-dev] memory issues Sorry I didn't paste the full stack, here is it
2016-04-11 15:54:43 ERROR FeatureDataAdapter:485 - Unable to deserialized feature type org.geotools.feature.SchemaException: Could not type nagement{"configurations" as:[{"@class" at org.geotools.data.DataUtilities.createAttribute(DataUtilities.java:2717) at org.geotools.data.DataUtilities.createType(DataUtilities.java:1964) at mil.nga.giat.geowave.adapter.vector.util.FeatureDataUtils.decodeType(FeatureDataUtils.java:106) at mil.nga.giat.geowave.adapter.vector.FeatureDataAdapter.defaultTypeDataFromBinary(FeatureDataAdapter.java:455) at mil.nga.giat.geowave.core.store.adapter.AbstractDataAdapter.fromBinary(AbstractDataAdapter.java:282) at mil.nga.giat.geowave.core.index.PersistenceUtils.fromBinary(PersistenceUtils.java:93) at mil.nga.giat.geowave.datastore.accumulo.metadata.AbstractAccumuloPersistence.entryToValue(AbstractAccumuloPersistence.java:345) at mil.nga.giat.geowave.datastore.accumulo.metadata.AbstractAccumuloPersistence.objectExists(AbstractAccumuloPersistence.java:472) at mil.nga.giat.geowave.datastore.accumulo.metadata.AccumuloAdapterStore.adapterExists(AccumuloAdapterStore.java:46) at mil.nga.giat.geowave.datastore.accumulo.AccumuloDataStore.store(AccumuloDataStore.java:181) at mil.nga.giat.geowave.datastore.accumulo.AccumuloIndexWriter.writeInternal(AccumuloIndexWriter.java:240) at mil.nga.giat.geowave.datastore.accumulo.AccumuloIndexWriter.write(AccumuloIndexWriter.java:194) at mil.nga.giat.geowave.datastore.accumulo.AccumuloIndexWriter.write(AccumuloIndexWriter.java:168) at ar.palenque.backend.hadoop.LocalizationServiceHadoop.indexFeatures(LocalizationServiceHadoop.java:130) at ar.palenque.backend.hadoop.SiloServiceHadoop.addContent(SiloServiceHadoop.java:344) at ar.palenque.backend.stories.silo.AddContentToSiloWithGis.main(AddContentToSiloWithGis.java:88) Caused by: java.lang.ClassNotFoundException: [{"@class" at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:190) at org.geotools.data.DataUtilities.type(DataUtilities.java:2347) at org.geotools.data.DataUtilities.createAttribute(DataUtilities.java:2706) ... 15 more 2016-04-11 15:54:43 INFO ConnectionManager$HConnectionImplementation:1676 - Closing zookeeper sessionid=0x153f707d7072286 2016-04-11 15:54:43 INFO ZooKeeper:684 - Session: 0x153f707d7072286 closed 2016-04-11 15:54:43 INFO ClientCnxn:512 - EventThread shut down Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at mil.nga.giat.geowave.core.index.PersistenceUtils.fromBinary(PersistenceUtils.java:81) at mil.nga.giat.geowave.adapter.vector.FeatureDataAdapter.defaultTypeDataFromBinary(FeatureDataAdapter.java:490) at mil.nga.giat.geowave.core.store.adapter.AbstractDataAdapter.fromBinary(AbstractDataAdapter.java:282) at mil.nga.giat.geowave.core.index.PersistenceUtils.fromBinary(PersistenceUtils.java:93) at mil.nga.giat.geowave.datastore.accumulo.metadata.AbstractAccumuloPersistence.entryToValue(AbstractAccumuloPersistence.java:345) at mil.nga.giat.geowave.datastore.accumulo.metadata.AbstractAccumuloPersistence.objectExists(AbstractAccumuloPersistence.java:472) at mil.nga.giat.geowave.datastore.accumulo.metadata.AccumuloAdapterStore.adapterExists(AccumuloAdapterStore.java:46) at mil.nga.giat.geowave.datastore.accumulo.AccumuloDataStore.store(AccumuloDataStore.java:181) at mil.nga.giat.geowave.datastore.accumulo.AccumuloIndexWriter.writeInternal(AccumuloIndexWriter.java:240) at mil.nga.giat.geowave.datastore.accumulo.AccumuloIndexWriter.write(AccumuloIndexWriter.java:194) at mil.nga.giat.geowave.datastore.accumulo.AccumuloIndexWriter.write(AccumuloIndexWriter.java:168) at ar.palenque.backend.hadoop.LocalizationServiceHadoop.indexFeatures(LocalizationServiceHadoop.java:130) at ar.palenque.backend.hadoop.SiloServiceHadoop.addContent(SiloServiceHadoop.java:344) at ar.palenque.backend.stories.silo.AddContentToSiloWithGis.main(AddContentToSiloWithGis.java:88) It's strange the "Mismatched Feature Data Adapter version" and "ClassNotFoundException", as i already index a few geometries using the same index a few weeks ago.
De: geowave-dev-bounces@xxxxxxxxxxxxxxxx <geowave-dev-bounces@xxxxxxxxxxxxxxxx> en nombre de Jose M <soloninguno@xxxxxxxxxxx>
Enviado: lunes, 11 de abril de 2016 03:44 p.m. Para: geowave-dev@xxxxxxxxxxxxxxxx Asunto: [geowave-dev] memory issues Hi,
I'm having some memory issues when indexing one single polygon with geowave in accumulo (using SimpleFeatureTypeBuilder ad per geowave examples). This is the error I see
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
In accumulo Tablet Server Max Heapsize (tserver_max_heapsize) has 1 GB heap, and Master Max Heapsize (master_max_heapsize) 512 Mb as per geowave documentation.
Anyone knows if i need to tweak any other parameter in accumulo configuration?
Thanks! |