| Re: [geomesa-users] Issue geomesa-hbase and geoserver plugin |
You can definitely run against emr hbase 1.3, using the normal geomesa shaded jar. I've done that before, but I don't have a list of required jars handy. What additional jars are you copying into the lib folder?_______________________________________________
I would guess that if you can get it working against 1.3, it would also work against hbase 1.4 with the same setup.
You might take a look at http://www.geomesa.org/documentation/tutorials/geomesa-hbase-s3-on-aws.html, which doesn't cover geoserver but might have some useful details.
Thanks,
Emilio
On 06/29/2018 06:19 AM, Gaetano Perrone wrote:
Hi Emilio, thanks for the answer;
I have followed all your steps (by using the built master git jar and the last release version 2.0.2) but unfortunately on emr environment I always have the same error:
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil
at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)
at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75)
at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:907)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:691)
I also have tried to change emr hbase version to 1.3 as you suggested (and the opposite, compile the plugin with 1.4 hbase version) ...
Do you have any other suggestion ?
Thanks in advance .Gaetano
2018-06-28 22:22 GMT+02:00 Emilio Lahr-Vivaz <elahrvivaz@xxxxxxxx>:
Hello,
That jar list is out of date... I've opened a ticket to fix it here: https://geomesa.atlassian.net/browse/GEOMESA-2312.
I think that the error you're getting is coming from hbase 1.4 vs hbase 1.3 (which we compile against). I believe that if you use the geomesa shaded jar (which bundles hbase 1.3), and use the following additional jars, things should work:
commons-cli-1.2.jar
commons-configuration-1.6.jar
commons-io-2.5.jar
commons-logging-1.1.3.jar
hadoop-auth-2.7.4.jar
hadoop-client-2.7.4.jar
hadoop-common-2.7.4.jar
hadoop-hdfs-2.7.4.jar
htrace-core-3.1.0-incubating.jar
metrics-core-2.2.0.jar
netty-3.6.2.Final.jar
netty-all-4.0.41.Final.jar
servlet-api-2.4.jar
zookeeper-3.4.10.jar
I tested this out a little while ago, although not on aws. If you're using aws/s3, you'll need additional aws-specific hadoop jars (as you discovered).
If you get a list of required aws jars, please write back and I'll include them in the documentation.
Thanks,
Emilio
On 06/28/2018 01:48 PM, Gaetano Perrone wrote:
You must subscribe to this list before posting. Please go to
https://locationtech.org/mailman/listinfo to subscribe to the list,
then try again. Perhaps you are already subscribed using another
email address; if that is the case, please send email to this list
using the email address you've used to subscribe.
---------- Messaggio inoltrato ----------
From: Gaetano Perrone <gaetano.perrone@xxxxxxxxxxxxxxx>
To: geomesa-users@xxxxxxxxxxxxxxxx
Cc:
Bcc:
Date: Thu, 28 Jun 2018 15:09:44 +0200
Subject: Issue about geomesa-hbase and geoserver plugin (geomesa-hbase_2.11-2.0.2)
Hello,
I have an issue when trying to run geomesa-hbase plugin in geoserver:
I have inserted all the jars inside the geoserver/WEB-INF/lib directory as the guide says, and
I was able to ingest data through geomesa-hbase utility, but I had problems in geoserver gui:
the first problem that I resolved was that geomesa server was not able to read from s3 filesystem (used to read the hbase dynamic library path in order to use the distribuited runtime library for coprocessing).
I resolved by adding the following library:
hadoop-aws-2.8.4.jar
(no info about this in docs, so I don't know if it was the right way).
Now I am not able to resolve the following stack trace error:
java.lang.RuntimeException: Error getting FeatureType, this should never happen!
at org.geoserver.wms.map.GetMapKvpRequestReader.checkStyle(GetMapKvpRequestReader.java:1076)
at org.geoserver.wms.map.GetMapKvpRequestReader.read(GetMapKvpRequestReader.java:496)
at org.geoserver.wms.map.GetMapKvpRequestReader.read(GetMapKvpRequestReader.java:87)
at org.geoserver.ows.Dispatcher.parseRequestKVP(Dispatcher.java:1531)
at org.geoserver.ows.Dispatcher.dispatch(Dispatcher.java:690)
at org.geoserver.ows.Dispatcher.handleRequestInternal(Dispatcher.java:260)
...
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException
at org.geoserver.catalog.ResourcePool.getDataStore(ResourcePool.java:690)
at org.geoserver.catalog.ResourcePool.getCacheableFeatureType(ResourcePool.java:963)
at org.geoserver.catalog.ResourcePool.tryGetFeatureType(ResourcePool.java:950)
at org.geoserver.catalog.ResourcePool.getFeatureType(ResourcePool.java:944)
at org.geoserver.catalog.ResourcePool.getFeatureType(ResourcePool.java:931)
at org.geoserver.catalog.impl.FeatureTypeInfoImpl.getFeatureType(FeatureTypeInfoImpl.java:121)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.geoserver.catalog.impl.ModificationProxy.invoke(ModificationProxy.java:147)
at com.sun.proxy.$Proxy124.getFeatureType(Unknown Source)
at org.geoserver.security.decorators.DecoratingFeatureTypeInfo.getFeatureType(DecoratingFeatureTypeInfo.java:97)
at org.geoserver.security.decorators.SecuredFeatureTypeInfo.getFeatureType(SecuredFeatureTypeInfo.java:54)
at org.geoserver.wms.map.GetMapKvpRequestReader.checkStyle(GetMapKvpRequestReader.java:1072)
... 101 more
Caused by: java.util.concurrent.CompletionException: java.io.IOException: java.lang.reflect.InvocationTargetException
at com.github.benmanes.caffeine.cache.UnboundedLocalCache$UnboundedLocalLoadingCache.lambda$new$0(UnboundedLocalCache.java:929)
at com.github.benmanes.caffeine.cache.UnboundedLocalCache.lambda$computeIfAbsent$2(UnboundedLocalCache.java:235)
at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
at com.github.benmanes.caffeine.cache.UnboundedLocalCache.computeIfAbsent(UnboundedLocalCache.java:231)
at com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:113)
at com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:65)
at org.locationtech.geomesa.hbase.data.HBaseConnectionPool$.getConnection(HBaseConnectionPool.scala:86)
at org.locationtech.geomesa.hbase.data.HBaseDataStoreFactory.createDataStore(HBaseDataStoreFactory.scala:44)
at org.locationtech.geomesa.hbase.data.HBaseDataStoreFactory.createDataStore(HBaseDataStoreFactory.scala:34)
at org.vfny.geoserver.util.DataStoreUtils.getDataAccess(DataStoreUtils.java:90)
at org.geoserver.catalog.ResourcePool.getDataStore(ResourcePool.java:650)
... 115 more
Caused by: java.io.IOException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:3312)
at org.locationtech.geomesa.hbase.data.HBaseConnectionPool$$anon$2$$anon$4.run(HBaseConnectionPool.scala:57)
at org.locationtech.geomesa.hbase.data.HBaseConnectionPool$$anon$2$$anon$4.run(HBaseConnectionPool.scala:53)
at org.locationtech.geomesa.hbase.data.HBaseConnectionPool$$anon$2.load(HBaseConnectionPool.scala:64)
at org.locationtech.geomesa.hbase.data.HBaseConnectionPool$$anon$2.load(HBaseConnectionPool.scala:50)
at com.github.benmanes.caffeine.cache.UnboundedLocalCache$UnboundedLocalLoadingCache.lambda$new$0(UnboundedLocalCache.java:922)
... 125 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
... 133 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil
at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)
at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75)
at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:947)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:723)
... 138 more
This error was generated from geomesa-hbase-gs-plugin_2.11-2.0.2-shaded.jar, anyway the ProtobufUtil was present inside the jar and classpath was acquired by tomcat server .
I have already tried to:
- Compile the plugin by changin the hbase-client version
- Run the tomcat as root (no folder permissions issues)
- Use 1.x geomesa-hbase-gs-plugin : it seems to work but cannot read data ingested with newer geomesa-hbase utility version, and I would like to use newer version)
Any suggestion about this problem?
Info about the environment:
Thanks very much,Best Regards,Gaetano
_______________________________________________ geomesa-users mailing list geomesa-users@xxxxxxxxxxxxxxxx To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.locationtech.org/mailman/listinfo/geomesa-users
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users
_______________________________________________ geomesa-users mailing list geomesa-users@xxxxxxxxxxxxxxxx To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.locationtech.org/mailman/listinfo/geomesa-users
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users