[geomesa-users] Advise on Geomesa data streaming

[Jose Bujalance <joseab56@xxxxxxxxx>]

Hi,

I have a Geomesa-Accumulo datastore containing 80M rows, and I would like to use that data to update a routing graph (GraphHopper).

In order to do that, I have to get all the Geomesa data row by row. I am working on Java with the Geomesa-Spark API (version 1.3.1). Obviously, I can't instantiate a Java list with 80M elements using the RDD.collect() method, because of obvious memory limitations.

So I am looking for a way to stream my Geomesa data row by row in an efficient way.

Right now, I am thinking about some tools like Spark Streaming, Geomesa-Stream, or Storm, but I have never used non of them, so I don't know if that's what I need and which tool would be more adapted to my problem.

Any idea?

Thanks a lot,

José