[geomesa-users] Instantiate a SpatialRDDProvider from Geomesa HBase (Geo

[Jose Bujalance <joseab56@xxxxxxxxx>]

Hi,

I am running Geomesa on top of HBase. I have created some SimpleFeatureTypes in a Geomesa catalog and I am trying now to access the features from Spark using the Geomesa-Spark libraries:

import java.io.IOException;

import java.io.Serializable;

import java.util.HashMap;

import java.util.Map;

import org.apache.hadoop.conf.Configuration;

import org.apache.spark.SparkConf;

import org.apache.spark.SparkContext;

import org.apache.spark.rdd.RDD;

import org.geotools.data.Query;

import org.geotools.filter.text.cql2.CQLException;

import org.geotools.filter.text.ecql.ECQL;

import org.locationtech.geomesa.spark.GeoMesaSpark;

import org.locationtech.geomesa.spark.SpatialRDDProvider;

import org.opengis.feature.simple.SimpleFeature;

public class Test {

public static void main(String[] args) throws IOException, CQLException {

//Spark configuration

SparkConf conf = new SparkConf().setAppName("MyAppName").setMaster("local[*]");

conf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer");

conf.set("spark.kryo.registrator", "org.locationtech.geomesa.spark.GeoMesaSparkKryoRegistrator");

SparkContext sc = new SparkContext(conf);

//Datastore configuration

Map<String, Serializable> parameters = new HashMap<>();

parameters.put("bigtable.table.name", "Geoloc_Praxedo_catalog");

SpatialRDDProvider provider = GeoMesaSpark.apply(parameters);

String predicate = "BBOX(coords, 48.815215, 2.249294, 48.904295, 2.419337)";

Query query = new Query("history_feature_nodate", ECQL.toFilter(predicate));

RDD<SimpleFeature> resultRDD = provider.rdd(new Configuration(), sc, (scala.collection.immutable.Map<String, String>) parameters, query);

System.out.println("Number of RDDs: " + resultRDD.count());

}

}

Either if I do this on Java or on Scala, I am getting the same error at the same line : SpatialRDDProvider provider = GeoMesaSpark.apply(parameters); The errror is:

Error in Java:

Exception in thread "main" java.util.ServiceConfigurationError: org.locationtech.geomesa.spark.SpatialRDDProvider: Provider org.locationtech.geomesa.spark.converter.ConverterSpatialRDDProviderorg.locationtech.geomesa.spark.geotools.GeoToolsSpatialRDDProvider not found

        at java.util.ServiceLoader.fail(ServiceLoader.java:239)

        at java.util.ServiceLoader.access$300(ServiceLoader.java:185)

        at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:372)

        at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)

        at java.util.ServiceLoader$1.next(ServiceLoader.java:480)

        at scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:43)

        at scala.collection.Iterator$class.find(Iterator.scala:944)

        at scala.collection.AbstractIterator.find(Iterator.scala:1336)

        at scala.collection.IterableLike$class.find(IterableLike.scala:79)

        at scala.collection.AbstractIterable.find(Iterable.scala:54)

        at org.locationtech.geomesa.spark.GeoMesaSpark$.apply(GeoMesaSpark.scala:32)

        at org.locationtech.geomesa.spark.GeoMesaSpark.apply(GeoMesaSpark.scala)

        at com.praxedo.geomesa.geomesa_spark.Test.main(Test.java:34)

Error in Scala:

scala> val spatialRDDProvider = GeoMesaSpark(params)

java.lang.RuntimeException: Could not find a SparkGISProvider

  at org.locationtech.geomesa.spark.GeoMesaSpark$$anonfun$apply$2.apply(GeoMesaSpark.scala:47)

  at org.locationtech.geomesa.spark.GeoMesaSpark$$anonfun$apply$2.apply(GeoMesaSpark.scala:47)

  at scala.Option.getOrElse(Option.scala:121)

  at org.locationtech.geomesa.spark.GeoMesaSpark$.apply(GeoMesaSpark.scala:47)

  ... 54 elided

I gess I am not accessing the HBase catalog properly.

Can someone give a hint on how to access a HBase catalog to treat it on Spark ?

Thank you,

José