|
Currently we only support accumulo in our spark implementation - if you develop anything for hbase, please contribute it back as a pull request!
Thanks,
Emilio
On Tue, 2016-04-26 at 09:29 +0000, yingxin li wrote:
Hi Emilio,
I have another question, does the GeoMesaSpark support hbase data store? To improve the performance, I try to use spark to query the locations.
Thanks in advance!
Regards,
Jessica
From: geomesa-dev-bounces@xxxxxxxxxxxxxxxx <geomesa-dev-bounces@xxxxxxxxxxxxxxxx> on behalf of yingxin li <yingxinlee@xxxxxxxxxxx>
Sent: Thursday, April 21, 2016 6:25 PM
To: Discussions between GeoMesa committers
Subject: Re: [geomesa-dev] How to use HBase as datastore in GeoMesa?
Hi Emilio,
In last email I got error "Column family M does not exist in region", then I changed my column family name to "M" and run again, now I got another error:
Exception in thread "main" java.io.IOException: Schema 'polygon' does not exist.
at org.geotools.data.store.ContentDataStore.ensureEntry(ContentDataStore.java:621)
at org.geotools.data.store.ContentDataStore.getFeatureSource(ContentDataStore.java:393)
at org.geotools.data.store.ContentDataStore.getFeatureSource(ContentDataStore.java:376)
at org.geotools.data.store.ContentDataStore.getFeatureReader(ContentDataStore.java:425)
And this is my code:
String tablename="cdb_vessel:vessel_polygon_test";
Map map = new HashMap();
map.put( "bigtable.table.name", tablename);
DataStore ds = DataStoreFinder.getDataStore(map );
String typeName="polygon";
String cql = "bbox(polygon_new,-180,-90,180,90)";
Query query =new Query(typeName, ECQL.toFilter(cql));
FeatureReader fr = ds.getFeatureReader(query,Transaction.AUTO_COMMIT);
ArrayList<SimpleFeature> features = new ArrayList<SimpleFeature>();
for(int i=0;i<10;i++){
features.add((SimpleFeature) fr.next());
}
What the typeName should be? Shall I define a type in a config? Thanks in advance!
Regards,
Jessica
From: geomesa-dev-bounces@xxxxxxxxxxxxxxxx <geomesa-dev-bounces@xxxxxxxxxxxxxxxx> on behalf of yingxin li <yingxinlee@xxxxxxxxxxx>
Sent: Thursday, April 21, 2016 3:59 PM
To: Discussions between GeoMesa committers
Subject: Re: [geomesa-dev] How to use HBase as datastore in GeoMesa?
Hi Emilio,
Thank you very much for your quick response.
I can get the datastore now, but when I try to query data from the hbase table, I got below error:
Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException):
org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column family M does not exist in region cdb_vessel:latest_location,,1453259842944.0365e0ba025cfecd5d37e66ed47a8ef6. in table 'cdb_vessel:latest_location', {NAME => 'details', DATA_BLOCK_ENCODING
=> 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'SNAPPY', MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
at org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:6916)
at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2385)
at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2115)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31305)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:745)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1199)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:216)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:300)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:31751)
at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:332)
Could you please help on this?
By the way, is the typeName="testsft" the column family name in
https://github.com/locationtech/geomesa/blob/master/geomesa-hbase/geomesa-hbase-datastore/src/test/scala/org/locationtech/geomesa/hbase/data/HBaseIntegrationTest.scala?
Thanks,
Jessica
|
|
github.com
geomesa - GeoMesa is a suite of tools for working with big geo-spatial data in a distributed fashion.
|
From: geomesa-dev-bounces@xxxxxxxxxxxxxxxx <geomesa-dev-bounces@xxxxxxxxxxxxxxxx> on behalf of Emilio Lahr-Vivaz <elahrvivaz@xxxxxxxx>
Sent: Tuesday, April 19, 2016 8:24 PM
To: Discussions between GeoMesa committers
Subject: Re: [geomesa-dev] How to use HBase as datastore in GeoMesa?
Hi Jessica,
Our HBase support is new, and we haven't documented it very well, so sorry for that. In order to use it, you would need the 'geomesa-hbase-datastore' jar on your classpath, which is available here:
(if you're not using maven you will need the dependent jars as well, which are described in the pom).
You also need to have your hbase-site.xml on your classpath.
Once you have those two things, you can get an HBase data store using the GeoTools DataStoreFinder, passing in a map with the parameter "bigtable.table.name" and the name of the table you want to store your data in.
If you are using GeoServer, we have a module you can build to install there as well. It looks like currently that is not being published, but you can build it from source following the readme here:
Thanks,
Emilio
On Tue, 2016-04-19 at 08:44 +0000, yingxin li wrote:
Dear all,
We decided to use GeoMesa to process our spatial data. While we are using HBase to store data. And in the GeoMesa manual, I didnot find how to install GeoMesa with HBase. Could you please help on this? It will be very appreciated for any help.
Best Regards,
Jessica
_______________________________________________
geomesa-dev mailing list
geomesa-dev@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://locationtech.org/mailman/listinfo/geomesa-dev
_______________________________________________
geomesa-dev mailing list
geomesa-dev@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://locationtech.org/mailman/listinfo/geomesa-dev |
