Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] Issue geomesa-hbase and geoserver plugin
Hello,

That jar list is out of date... I've opened a ticket to fix it here: https://geomesa.atlassian.net/browse/GEOMESA-2312.

I think that the error you're getting is coming from hbase 1.4 vs hbase 1.3 (which we compile against). I believe that if you use the geomesa shaded jar (which bundles hbase 1.3), and use the following additional jars, things should work:

commons-cli-1.2.jar
commons-configuration-1.6.jar
commons-io-2.5.jar
commons-logging-1.1.3.jar
hadoop-auth-2.7.4.jar
hadoop-client-2.7.4.jar
hadoop-common-2.7.4.jar
hadoop-hdfs-2.7.4.jar
htrace-core-3.1.0-incubating.jar
metrics-core-2.2.0.jar
netty-3.6.2.Final.jar
netty-all-4.0.41.Final.jar
servlet-api-2.4.jar
zookeeper-3.4.10.jar

I tested this out a little while ago, although not on aws. If you're using aws/s3, you'll need additional aws-specific hadoop jars (as you discovered).

If you get a list of required aws jars, please write back and I'll include them in the documentation.

Thanks,

Emilio

On 06/28/2018 01:48 PM, Gaetano Perrone wrote:
You must subscribe to this list before posting.  Please go to
https://locationtech.org/mailman/listinfo to subscribe to the list,
then try again.  Perhaps you are already subscribed using another
email address; if that is the case, please send email to this list
using the email address you've used to subscribe.



---------- Messaggio inoltrato ----------
From: Gaetano Perrone <gaetano.perrone@xxxxxxxxxxxxxxx>
To: geomesa-users@xxxxxxxxxxxxxxxx
Cc: 
Bcc: 
Date: Thu, 28 Jun 2018 15:09:44 +0200
Subject: Issue about geomesa-hbase and geoserver plugin (geomesa-hbase_2.11-2.0.2)
Hello,
I have an issue when trying to run geomesa-hbase plugin in geoserver:

I have inserted all the jars inside the geoserver/WEB-INF/lib directory as the guide says, and
I was able to ingest data through geomesa-hbase utility, but I had problems in geoserver gui:

the first problem that I resolved was that geomesa server was not able to read from s3 filesystem (used to read the hbase dynamic library path in order to use the distribuited runtime library for coprocessing).

I resolved by adding the following library:

hadoop-aws-2.8.4.jar

(no info about this in docs, so I don't know if it was the right way).

Now I am not able to resolve the following stack trace error:

java.lang.RuntimeException: Error getting FeatureType, this should never happen!
    at org.geoserver.wms.map.GetMapKvpRequestReader.checkStyle(GetMapKvpRequestReader.java:1076)
    at org.geoserver.wms.map.GetMapKvpRequestReader.read(GetMapKvpRequestReader.java:496)
    at org.geoserver.wms.map.GetMapKvpRequestReader.read(GetMapKvpRequestReader.java:87)
    at org.geoserver.ows.Dispatcher.parseRequestKVP(Dispatcher.java:1531)
    at org.geoserver.ows.Dispatcher.dispatch(Dispatcher.java:690)
    at org.geoserver.ows.Dispatcher.handleRequestInternal(Dispatcher.java:260)
...
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException
    at org.geoserver.catalog.ResourcePool.getDataStore(ResourcePool.java:690)
    at org.geoserver.catalog.ResourcePool.getCacheableFeatureType(ResourcePool.java:963)
    at org.geoserver.catalog.ResourcePool.tryGetFeatureType(ResourcePool.java:950)
    at org.geoserver.catalog.ResourcePool.getFeatureType(ResourcePool.java:944)
    at org.geoserver.catalog.ResourcePool.getFeatureType(ResourcePool.java:931)
    at org.geoserver.catalog.impl.FeatureTypeInfoImpl.getFeatureType(FeatureTypeInfoImpl.java:121)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.geoserver.catalog.impl.ModificationProxy.invoke(ModificationProxy.java:147)
    at com.sun.proxy.$Proxy124.getFeatureType(Unknown Source)
    at org.geoserver.security.decorators.DecoratingFeatureTypeInfo.getFeatureType(DecoratingFeatureTypeInfo.java:97)
    at org.geoserver.security.decorators.SecuredFeatureTypeInfo.getFeatureType(SecuredFeatureTypeInfo.java:54)
    at org.geoserver.wms.map.GetMapKvpRequestReader.checkStyle(GetMapKvpRequestReader.java:1072)
    ... 101 more
Caused by: java.util.concurrent.CompletionException: java.io.IOException: java.lang.reflect.InvocationTargetException
    at com.github.benmanes.caffeine.cache.UnboundedLocalCache$UnboundedLocalLoadingCache.lambda$new$0(UnboundedLocalCache.java:929)
    at com.github.benmanes.caffeine.cache.UnboundedLocalCache.lambda$computeIfAbsent$2(UnboundedLocalCache.java:235)
    at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
    at com.github.benmanes.caffeine.cache.UnboundedLocalCache.computeIfAbsent(UnboundedLocalCache.java:231)
    at com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:113)
    at com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:65)
    at org.locationtech.geomesa.hbase.data.HBaseConnectionPool$.getConnection(HBaseConnectionPool.scala:86)
    at org.locationtech.geomesa.hbase.data.HBaseDataStoreFactory.createDataStore(HBaseDataStoreFactory.scala:44)
    at org.locationtech.geomesa.hbase.data.HBaseDataStoreFactory.createDataStore(HBaseDataStoreFactory.scala:34)
    at org.vfny.geoserver.util.DataStoreUtils.getDataAccess(DataStoreUtils.java:90)
    at org.geoserver.catalog.ResourcePool.getDataStore(ResourcePool.java:650)
    ... 115 more
Caused by: java.io.IOException: java.lang.reflect.InvocationTargetException
    at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)
    at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
    at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
    at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:3312)
    at org.locationtech.geomesa.hbase.data.HBaseConnectionPool$$anon$2$$anon$4.run(HBaseConnectionPool.scala:57)
    at org.locationtech.geomesa.hbase.data.HBaseConnectionPool$$anon$2$$anon$4.run(HBaseConnectionPool.scala:53)
    at org.locationtech.geomesa.hbase.data.HBaseConnectionPool$$anon$2.load(HBaseConnectionPool.scala:64)
    at org.locationtech.geomesa.hbase.data.HBaseConnectionPool$$anon$2.load(HBaseConnectionPool.scala:50)
    at com.github.benmanes.caffeine.cache.UnboundedLocalCache$UnboundedLocalLoadingCache.lambda$new$0(UnboundedLocalCache.java:922)
    ... 125 more
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
    ... 133 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil
    at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)
    at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75)
    at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105)
    at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:947)
    at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:723)
    ... 138 more


This error was generated from geomesa-hbase-gs-plugin_2.11-2.0.2-shaded.jar, anyway the ProtobufUtil was present inside the jar and classpath was acquired by tomcat server .


I have already tried to:
  • Compile the plugin by changin the hbase-client version
  • Run the tomcat as root (no folder permissions issues)
  • Use 1.x geomesa-hbase-gs-plugin : it seems to work but cannot read data ingested with newer geomesa-hbase utility version, and I would like to use newer version)
Any suggestion about this problem?
Info about the environment:
Hadoop Hadoop 2.8.3-amzn-1
HBase HBase 1.4.2
GeoMesa Geomesa 2.0.2
GeoServer v2.13.1
GeoMesa Plugin
gs-plugin_2.11-2.0.2-shaded.jar

Thanks very much,
Best Regards,
Gaetano



_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users

Back to the top