Re: [geomesa-users] Questions on Geomesa Kafka DataStore and Structured

[Emilio Lahr-Vivaz <elahrvivaz@xxxxxxxx>]

Hello,

1. We don't currently have any deep integration with Spark streaming. If you wanted to use it, you could attach to the Kafka topic and deserialize the resulting messages into SimpleFeatures using GeoMesa classes: https://github.com/locationtech/geomesa/blob/master/geomesa-kafka/geomesa-kafka-datastore/src/main/scala/org/locationtech/geomesa/kafka/utils/GeoMessageSerializer.scala#L180

2. The Kafka data store works by reading off a Kafka topic. If you aren't writing data, then you won't see any data in the feature reader. Also, you need to allow some amount of time for the reader to process the topic before the features will be queryable, so if you get a feature reader and immediately query it you might not see any data. There are configuration options to control how far-back on the topic the reader starts - by default it does not read-back at all: https://www.geomesa.org/documentation/user/kafka/usage.html#kafka-data-store-parameters

3. You can use the GeoMessageSerializer class I linked to above.

Thanks,

Emilio

On 9/20/19 3:29 AM, Yifan Wang wrote:

Hi, 

Recently I came across some problems on Geomesa Kafka DataStore. 

1. How can I read data from  Geomesa Kafka Datastore through Spark Streaming or Structured Streaming?
2. When I called 'getFeatureReader' function in  Geomesa Kafka Datastore, it could not return any data.
3. How can I deserialize Value of Consumer after I wrote SimpleFeature to Kafka using 'getFeatureWriterAppend' method in Geomesa Kafka Datastore.  

I am wondering if someone could give me a hand with that. Thank you!

Best Regards,

Evan

_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users