Hello,
I’ve been
attempting to convert a geomesa datastore that was generated
using a 1.1.0-rc.7 build to the format of 1.1.0-rc.2, using
the SchemaCopyJob (https://github.com/locationtech/geomesa/tree/master/geomesa-jobs#updating-existing-data-to-the-latest-index-format).
The job is failing in
org.locationtech.geomesa.features.avro.FeatureSpecificReader.readAttributes(…)
– specifically on the ‘match’ statement on
serializationVersion. A bit of debugging shows that the
serializationVersion being passed in to readAttributes is 0,
which means that the code is not able to assign a suitable
deserializer for the data in the table being converted.
For reference,
here’s the contents of my table metadata (generated using
scant –t <tablename> in the Accumulo shell):
~METADATA_mast_sri
attributes: []
aisMessageType:Long:index=false,navStatus:Integer:index=false,rateOfTurn:Double:index=false,trueHeading:Integer:index=false,cargoDescription:String:index=false,shippingAgent:String:index=false,receiverClass:String:index=false,aidType:Long:index=false,vesselType:Long:index=false,destination:String:index=false,arrivalTime:Date:index=false,destinationVerified:Long:index=false,simonVesselType:String:index=false,unenhancedObservationId:Long:index=false,streamDataTypeId:Long:index=false,dataSourceId:Long:index=false,vesselName:String:index=false,mmsi:Integer:index=true,callSign:String:index=false,flag:Long:index=false,imo:Integer:index=false,sconum:String:index=false,breadth:Float:index=false,length:Double:index=false,netTonnage:Float:index=false,grossTonnage:Float:index=false,deadweight:Float:index=false,buildYear:Long:index=false,latitude:Double:index=false,longitude:Double:index=false,observationTime:Date:index=false,semiMajor:Float:index=false,semiMinor:Float:index=false,orientation:Flo
at:index=false,altitude:Double:index=false,trackId:Integer:index=false,imageId:Long:index=false,observationCatCode:Long:index=false,geofeasible:Integer:index=false,displacement:Float:index=false,depth:Float:index=false,height:Float:index=false,vesselPreviousName:String:index=false,yearNameChanged:Long:index=false,draft:Float:index=false,draught:Float:index=false,satTelex:String:index=false,satPhone:String:index=false,satFax:String:index=false,mobilePhone:String:index=false,previousFlag:Long:index=false,previousCallSign:String:index=false,vesselClass:String:index=false,hullNumber:Long:index=false,currentHeading:Float:index=false,currentSpeed:Double:index=false,*geom:Point:srid=4326:index=false
~METADATA_mast_sri
bounds: []
21.0593433380127:24.4429340362549:37.5012168884277:40.1512336730957
~METADATA_mast_sri
dtgfield: [] observationTime
~METADATA_mast_sri
featureEncoding: [] avro
~METADATA_mast_sri
schema: []
%~#s%2#r%mast_sri#cstr%0,3#gh%yyyyMMdd#d::%~#s%3,2#gh::%~#s%#id
~METADATA_mast_sri
tables.idx.attr.name: [] test_tail_mast_5fsri_attr_idx
~METADATA_mast_sri
tables.idx.st.name: [] test_tail_mast_5fsri_st_idx
~METADATA_mast_sri
tables.queries.name: [] test_tail_mast_5fsri_queries
~METADATA_mast_sri
tables.record.name: [] test_tail_mast_5fsri_records
~METADATA_mast_sri
visibilities: []
Can any of the
experienced geomesans tell if this particular table format
is likely to cause the issue of an unrecognized
serializationVersion in FeatureSpecifiedReader?
Thanks!
Ben
From:
Emilio Lahr-Vivaz [mailto:elahrvivaz@xxxxxxxx]
Sent: Wednesday, July 15, 2015 12:28 PM
To: Geomesa User discussions; Ben Southall
Subject: Re: [geomesa-users] Moving from
geomesa-accumulo1.5-1.0.0-rc.7 to geomesa-1.1.0-rc.2
Hi Ben,
We don't really have a document of the breaking changes, as
most users of geomesa interact with us at the geotools
interface level, so they aren't affected. The only change most
people would have to make is migrating data to take advantage
of new index improvements - that process is outlined here:
https://github.com/locationtech/geomesa/tree/master/geomesa-jobs#updating-existing-data-to-the-latest-index-format
If you're using our serializers directly, then you can take a
look at the unit tests for examples. The usage is pretty
simple:
https://github.com/locationtech/geomesa/blob/master/geomesa-features/geomesa-feature-kryo/src/test/scala/org/locationtech/geomesa/features/kryo/KryoFeatureSerializerTest.scala
Kryo is the main serialization format we're supporting going
forward, although we also have options to use avro or native
byte buffers.
As you noticed, we changed the names from encoders to
serializers to be clearer about what they are actually doing.
We also moved around several packages and modules (as you also
noticed).
We'll try to get the geomesa.org links sorted out, thanks
bringing that to our attention!
If you have any more questions, or things still aren't clear,
please let us know.
Thanks,
Emilio
On 07/15/2015 10:19 AM, Ben Southall
wrote:
Hello,
I’m attempting to transition some code
from a CCRI fork (specifically, tag geomesa-DCB-v3.1) based
upon 1.0.0-rc.7 to geomesa-1.1.0-rc.2. It seems that in the
interim, there have been a number of breaking changes. Some
appear obvious (package name changes), some less so.
In particular, my code was making use of
1.5-DBC’s
org.locationtech.geomesa.core.data.SimpleFeatureEncoder –
has this now been broken into the
SimpleFeatureSerializer/Deserializer classes? In fact, the
whole feature landscape seems to be more complex than it was
in earlier versions.
Is there a summary document of the
breaking changes available?
Thanks very much!
Ben
p.s. The download instructions on
geomesa.org are a bit wonky. The instructions to clone a
branch geomesa-accumulo1.5-1.1.0-rc.2 seem to be incorrect.
The clone works with the tag geomesa-1.1.0-rc.2.
Also, the link to the tar.gz for
v1.1.0-rc.2 is broken, and the link to 1.1.0-rc.3-SNAPSHOT
contains pom.xml files where the version number is
1.0.0-rc.8.
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geomesa-users
