Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] geomesa-native-api on BigTable

We will probably get to BigTable soon but it may be a few weeks away.

Also, note that the native API is under fairly heavy development so we have marked it as unstable. The upside of that is that if there are things you'd like to do you can let us know and we can try to roll them in or you can make PRs against it! Another thing to consider is the EMR HBase instance which allows you to easily spin up Hbase nodes.

Perhaps it would be useful after you've poked around to chat and see how it's going and what we can do to help solve your use case.

Andrew

On 04/20/2017 11:07 AM, Emilio Lahr-Vivaz wrote:
Hi David,

Sounds like you're doing well with spark, but if you're still interested, we've actually got a PR up now for hbase native API support: https://github.com/locationtech/geomesa/pull/1484

Bigtable would be the next step, but should be pretty easy, just would require some testing.

Thanks,

Emilio

On 04/13/2017 08:34 PM, David Lewis wrote:
Hi Emilio,

Thanks for the quick response! I have a better handle around the project structure. I am going to try and use the spark project to interact with my BigTableDataStore, and I'll see how that goes.

-David

On Thu, Apr 13, 2017 at 7:52 AM, Emilio Lahr-Vivaz <elahrvivaz@xxxxxxxx> wrote:
Hi David,

I don't believe we have any imminent plans to expand the native api to bigtable. That said, it might not be very complicated to do - all of our implementations center around the geotools DataStore, and more specifically the GeoMesaDataStore. So likely one could simply replace the AccumuloDataStore instances in AccumuloGeoMesaIndex with a generic GeoMesaDataStore, then the various implementations would just have to pass the correct data store type (accumulo vs bigtable). If you'd like to work on it, that would be great! We'd be glad to provide support here or also on our gitter channel (https://gitter.im/locationtech/geomesa) for more immediate responses.

Another alternative would be to not use the native API, and just use the DataStore directly. The native API is just a convenience wrapper around geotools that simplifies the interface, but you can use the data stores directly without involving geoserver. There are lots of unit tests that show how to do this, in particular check out:

https://github.com/locationtech/geomesa/blob/master/geomesa-hbase/geomesa-bigtable-datastore/src/test/scala/org/locationtech/geomesa/bigtable/BigTableIntegrationTest.scala

Thanks,

Emilio

(Our bigtable integration uses their HBase API, so the data


On 04/12/2017 07:21 PM, David Lewis wrote:
Hello,

I am trying to use the geomesa-native-api to write data to bigtable from one ingestion processor and to query that data from another large analytics application.
I don't want to go through the trouble of setting up GeoServer nor implementing an ingestion plugin through that system. All I want is to be able to write my data to the tables and to query it efficiently.
I have looked into S2 and have figured out how I could store my data so that my queries are efficient, but I believe the flexibility of GeoMesa would provide more "future proofing" so I'd like to use geomesa's indexing and querying.

I have read the documentation that seems to say that google big table is well supported, but when I look at the source code I cannot find the implementations of GeoMesaIndex that I need.
After looking at the Accumulo implementation I don't think it would be trivial to write it myself.
Are there any projects that do this? Am I just not looking in the right place?
If my best bet is to write it myself are there any resources for doing so?

This seems like a very cool project and I look forward to using it if I can get it running :)

Thanks in advance!
-David Lewis


_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users
_______________________________________________ geomesa-users mailing list geomesa-users@xxxxxxxxxxxxxxxx To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.locationtech.org/mailman/listinfo/geomesa-users
--
   David Lewis
   Backend Engineer
   Mobile 801.616.17581
Blyncsy, Inc. | www.Blyncsy.com 
This email contains confidential commercial information the disclosure of which would result in serious competitive and commercial injury. Please be advised that the unauthorized disclosure of the trade secret(s) contained herein would result in serious competitive and commercial injury and subject the disclosing party to liability. As such, it is a protected record under law. Specifically, under Utah Government Records Access Management Act, additionally it may be considered a trade secret as defined by Idaho Code § 74-107(1) and is exempt from the Idaho Public Records Law Idaho Code §§ 74-101 through 74-126 and Colorado Uniform Trade Secrets Act § 24-72-204(3)(a)(IV).
This email may contain confidential commercial information that: (a) derives independent economic value, actual or potential, from not being generally known to, and not being readily ascertainable by proper means by other persons who can obtain economic value from its disclosure or use; and (b) Is the subject of efforts that are reasonable under the circumstances to maintain its secrecy. 
This message is confidential. It may also be privileged or otherwise protected by work product immunity or other legal rules. If you have received it by mistake, please let us know by e-mail reply and delete it from your system; you may not copy this message or disclose its contents to anyone. Please send us by fax any message containing deadlines as incoming e-mails are not screened for response deadlines. The integrity and security of this message cannot be guaranteed on the Internet.

 P Please consider the environment before printing this email.

_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users

Back to the top