Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] SparkSql for Geomesa Hbase
Hello,

We build our docs nightly against the current master branch and host them here: https://www.geomesa.org/documentation/current/user/index.html

As you saw, they aren't really meant for general consumption, as things like links to artifacts and such are not always correct. They are the current best effort, but there may be inconsistencies present. Usually before a release we go through all the tutorials, etc, to ensure that everything is working correctly, but that hasn't been done yet for 2.1.0.

You can definitely use 2.1.0-SNAPSHOT, and we're happy to help you work through any issues you encounter, but it's not official yet so things may change out from under you without warning. That said, we're fairly close to a release, so the changes shouldn't be too drastic. We've cut some milestone tags to give early adopters a consistent target for integration; the latest one is https://github.com/locationtech/geomesa/tree/geomesa_2.11-2.1.0-m.3. You'll have to build all the artifacts yourself, as they aren't hosted anywhere.

Thanks,

Emilio

On 10/17/18 4:53 AM, Martin Kellner wrote:
Hello,

@Emilio, you were right again. Going back to Spark 2.2.0 solved the problem.
I read the upgrade guide for 2.1.0. Can I use it already or is it not stable right now?

I also saw that there exists a tutorial that suggests getting the 2.1.0-SNAPSHOT for GeoMesa HBase from here:
But as far as I can see, this file is not available there.

Thanks,

Martin


Am Mo., 15. Okt. 2018 um 17:11 Uhr schrieb Emilio Lahr-Vivaz <elahrvivaz@xxxxxxxx>:
Hello,

The issue comes from a spark version mismatch. In geomesa 2.0.2, I believe we only support spark 2.2. In current master (2.1.0-SNAPSHOT), we've updated to spark 2.3, while maintaining partial compatibility with 2.2 (we will offer full compatibility before release).

Generally we don't recommend mixing versions of geomesa in a single workflow, but you could try just using the current master hbase-spark-runtime and it might work. Longer term, you could upgrade all components to geomesa 2.1.0, or downgrade the spark version on your cluster.

Thanks,

Emilio

On 10/15/2018 10:56 AM, Martin Kellner wrote:
Hey, 

I got another question on Geomesa Hbase.
Ingesting data works perfectly now, thanks for your help so far.

Now I want to write some queries for Spark (in spark-shell).
Geomesa list lots of SparkSql functions:

I tried this query:
"select * from table where st_contains(st_makeBBOX(0.0, 0.0, 90.0, 90.0), geom)"
And I get this error:
java.lang.NoSuchMethodError: org.apache.spark.sql.execution.datasources.LogicalRelation.copy(Lorg/apache/spark/sql/sources/BaseRelation;Lscala/collection/Seq;Lscala/Option;)Lorg/apache/spark/sql/execution/datasources/LogicalRelation;

I am running spark-shell with these jars:

I also tried to add the jar for spark-sql like this:

Unfortunately that did not solve the problem. Do you have any ideas what could cause that error?

Thank you,
Martin





_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users

_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users

_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users

Back to the top