Re: [geowave-dev] memory issues

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]

Re: [geowave-dev] memory issues

From: Jose M <soloninguno@xxxxxxxxxxx>
Date: Tue, 19 Apr 2016 02:13:44 +0000
Accept-language: es-AR, en-US
Delivered-to: geowave-dev@xxxxxxxxxxxxxxxx
List-archive: <https://www.locationtech.org/mailman/private/geowave-dev>
List-help: <mailto:geowave-dev-request@locationtech.org?subject=help>
List-subscribe: <https://www.locationtech.org/mailman/listinfo/geowave-dev>, <mailto:geowave-dev-request@locationtech.org?subject=subscribe>
List-unsubscribe: <https://www.locationtech.org/mailman/options/geowave-dev>, <mailto:geowave-dev-request@locationtech.org?subject=unsubscribe>
Thread-index: AQHRlCHQrdBR+9RRe069KcBRqQ5GY5+FIffFgARlnXmABlVpgIAAYLDq
Thread-topic: [geowave-dev] memory issues

No problem Rich! Only post this as maybe could help someone with the same problem.

De: geowave-dev-bounces@xxxxxxxxxxxxxxxx <geowave-dev-bounces@xxxxxxxxxxxxxxxx> en nombre de Rich Fecher <rfecher@xxxxxxxxx>
Enviado: lunes, 18 de abril de 2016 11:57 a.m.
Para: geowave-dev@xxxxxxxxxxxxxxxx
Asunto: Re: [geowave-dev] memory issues

Jose,

Its good to see you were able to resolve this and thanks for the fix. Sorry we missed this and weren't able to help out sooner.

Rich

On Thu, Apr 14, 2016 at 10:22 AM, Jose M <soloninguno@xxxxxxxxxxx> wrote:

Hi,

For future reference, i manage to resolve this. The problem was a different SNAPSHOT geowave version in the accumulo classpath (general.vfs.context.classpath.geowave) and in the client.

When i start testing geowave i build the full geowave-deploy-0.9.1-SNAPSHOT-accumulo-singlejar.jar to put in the accumulo classpath. The version was "0.9.1-20160201.220751-4".

It seems after running some maven package operation in the client, the geowave 0.9.1-SNAPSHOT version got updated to "0.9.1-20160301.213533-5", so although both accumulo and the client had 0.9.1 version, they had in fact different classes that generate some classpath error and eventually an OutOfMemory. That's also why i remember i had this working in the past.

Regards,

De: geowave-dev-bounces@xxxxxxxxxxxxxxxx <geowave-dev-bounces@xxxxxxxxxxxxxxxx> en nombre de Jose M <soloninguno@xxxxxxxxxxx>
Enviado: lunes, 11 de abril de 2016 04:07 p.m.
Para: geowave-dev@xxxxxxxxxxxxxxxx
Asunto: Re: [geowave-dev] memory issues

Sorry I didn't paste the full stack, here is it

2016-04-11 15:54:43 WARN FeatureDataAdapter:428 - Mismatched Feature Data Adapter version
2016-04-11 15:54:43 ERROR FeatureDataAdapter:485 - Unable to deserialized feature type
org.geotools.feature.SchemaException: Could not type nagement{"configurations" as:[{"@class"
   at org.geotools.data.DataUtilities.createAttribute(DataUtilities.java:2717)
   at org.geotools.data.DataUtilities.createType(DataUtilities.java:1964)
   at mil.nga.giat.geowave.adapter.vector.util.FeatureDataUtils.decodeType(FeatureDataUtils.java:106)
   at mil.nga.giat.geowave.adapter.vector.FeatureDataAdapter.defaultTypeDataFromBinary(FeatureDataAdapter.java:455)
   at mil.nga.giat.geowave.core.store.adapter.AbstractDataAdapter.fromBinary(AbstractDataAdapter.java:282)
   at mil.nga.giat.geowave.core.index.PersistenceUtils.fromBinary(PersistenceUtils.java:93)
   at mil.nga.giat.geowave.datastore.accumulo.metadata.AbstractAccumuloPersistence.entryToValue(AbstractAccumuloPersistence.java:345)
   at mil.nga.giat.geowave.datastore.accumulo.metadata.AbstractAccumuloPersistence.objectExists(AbstractAccumuloPersistence.java:472)
   at mil.nga.giat.geowave.datastore.accumulo.metadata.AccumuloAdapterStore.adapterExists(AccumuloAdapterStore.java:46)
   at mil.nga.giat.geowave.datastore.accumulo.AccumuloDataStore.store(AccumuloDataStore.java:181)
   at mil.nga.giat.geowave.datastore.accumulo.AccumuloIndexWriter.writeInternal(AccumuloIndexWriter.java:240)
   at mil.nga.giat.geowave.datastore.accumulo.AccumuloIndexWriter.write(AccumuloIndexWriter.java:194)
   at mil.nga.giat.geowave.datastore.accumulo.AccumuloIndexWriter.write(AccumuloIndexWriter.java:168)
   at ar.palenque.backend.hadoop.LocalizationServiceHadoop.indexFeatures(LocalizationServiceHadoop.java:130)
   at ar.palenque.backend.hadoop.SiloServiceHadoop.addContent(SiloServiceHadoop.java:344)
   at ar.palenque.backend.stories.silo.AddContentToSiloWithGis.main(AddContentToSiloWithGis.java:88)
Caused by: java.lang.ClassNotFoundException: [{"@class"
   at java.lang.Class.forName0(Native Method)
   at java.lang.Class.forName(Class.java:190)
   at org.geotools.data.DataUtilities.type(DataUtilities.java:2347)
   at org.geotools.data.DataUtilities.createAttribute(DataUtilities.java:2706)
   ... 15 more
2016-04-11 15:54:43 INFO ConnectionManager$HConnectionImplementation:1676 - Closing zookeeper sessionid=0x153f707d7072286
2016-04-11 15:54:43 INFO ZooKeeper:684 - Session: 0x153f707d7072286 closed
2016-04-11 15:54:43 INFO ClientCnxn:512 - EventThread shut down
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
   at mil.nga.giat.geowave.core.index.PersistenceUtils.fromBinary(PersistenceUtils.java:81)
   at mil.nga.giat.geowave.adapter.vector.FeatureDataAdapter.defaultTypeDataFromBinary(FeatureDataAdapter.java:490)
   at mil.nga.giat.geowave.core.store.adapter.AbstractDataAdapter.fromBinary(AbstractDataAdapter.java:282)
   at mil.nga.giat.geowave.core.index.PersistenceUtils.fromBinary(PersistenceUtils.java:93)
   at mil.nga.giat.geowave.datastore.accumulo.metadata.AbstractAccumuloPersistence.entryToValue(AbstractAccumuloPersistence.java:345)
   at mil.nga.giat.geowave.datastore.accumulo.metadata.AbstractAccumuloPersistence.objectExists(AbstractAccumuloPersistence.java:472)
   at mil.nga.giat.geowave.datastore.accumulo.metadata.AccumuloAdapterStore.adapterExists(AccumuloAdapterStore.java:46)
   at mil.nga.giat.geowave.datastore.accumulo.AccumuloDataStore.store(AccumuloDataStore.java:181)
   at mil.nga.giat.geowave.datastore.accumulo.AccumuloIndexWriter.writeInternal(AccumuloIndexWriter.java:240)
   at mil.nga.giat.geowave.datastore.accumulo.AccumuloIndexWriter.write(AccumuloIndexWriter.java:194)
   at mil.nga.giat.geowave.datastore.accumulo.AccumuloIndexWriter.write(AccumuloIndexWriter.java:168)
   at ar.palenque.backend.hadoop.LocalizationServiceHadoop.indexFeatures(LocalizationServiceHadoop.java:130)
   at ar.palenque.backend.hadoop.SiloServiceHadoop.addContent(SiloServiceHadoop.java:344)
   at ar.palenque.backend.stories.silo.AddContentToSiloWithGis.main(AddContentToSiloWithGis.java:88)

It's strange the "Mismatched Feature Data Adapter version" and "ClassNotFoundException", as i already index a few geometries using the same index a few weeks ago.

De: geowave-dev-bounces@xxxxxxxxxxxxxxxx <geowave-dev-bounces@xxxxxxxxxxxxxxxx> en nombre de Jose M <soloninguno@xxxxxxxxxxx>
Enviado: lunes, 11 de abril de 2016 03:44 p.m.
Para: geowave-dev@xxxxxxxxxxxxxxxx
Asunto: [geowave-dev] memory issues

Hi,

I'm having some memory issues when indexing one single polygon with geowave in accumulo (using SimpleFeatureTypeBuilder ad per geowave examples). This is the error I see

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
   at mil.nga.giat.geowave.core.index.PersistenceUtils.fromBinary(PersistenceUtils.java:81)
   at mil.nga.giat.geowave.adapter.vector.FeatureDataAdapter.defaultTypeDataFromBinary(FeatureDataAdapter.java:490)
   at mil.nga.giat.geowave.core.store.adapter.AbstractDataAdapter.fromBinary(AbstractDataAdapter.java:282)
   at mil.nga.giat.geowave.core.index.PersistenceUtils.fromBinary(PersistenceUtils.java:93)
   at mil.nga.giat.geowave.datastore.accumulo.metadata.AbstractAccumuloPersistence.entryToValue(AbstractAccumuloPersistence.java:345)
   at mil.nga.giat.geowave.datastore.accumulo.metadata.AbstractAccumuloPersistence.objectExists(AbstractAccumuloPersistence.java:472)
   at mil.nga.giat.geowave.datastore.accumulo.metadata.AccumuloAdapterStore.adapterExists(AccumuloAdapterStore.java:46)
   at mil.nga.giat.geowave.datastore.accumulo.AccumuloDataStore.store(AccumuloDataStore.java:181)
   at mil.nga.giat.geowave.datastore.accumulo.AccumuloIndexWriter.writeInternal(AccumuloIndexWriter.java:240)
   at mil.nga.giat.geowave.datastore.accumulo.AccumuloIndexWriter.write(AccumuloIndexWriter.java:194)
   at mil.nga.giat.geowave.datastore.accumulo.AccumuloIndexWriter.write(AccumuloIndexWriter.java:168)

In accumulo Tablet Server Max Heapsize (tserver_max_heapsize) has 1 GB heap, and Master Max Heapsize (master_max_heapsize) 512 Mb as per geowave documentation.

Anyone knows if i need to tweak any other parameter in accumulo configuration?

Thanks!

_______________________________________________
geowave-dev mailing list
geowave-dev@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geowave-dev

References:
- [geowave-dev] memory issues
  - From: Jose M
- Re: [geowave-dev] memory issues
  - From: Jose M
- Re: [geowave-dev] memory issues
  - From: Jose M
- Re: [geowave-dev] memory issues
  - From: Rich Fecher

Prev by Date: Re: [geowave-dev] memory issues
Next by Date: [geowave-dev] Multiband geotiff problem
Previous by thread: Re: [geowave-dev] memory issues
Next by thread: [geowave-dev] Multiband geotiff problem
Index(es):
- Date
- Thread

Breadcrumbs