Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] Unusual issue: No Double/Float values stored in Accumulo


David,

Below is what I have used in the past for GeoTools shading (w/ extra bits for JAI/JAI-Ext operations)

Tom

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-shade-plugin</artifactId>
  <version>2.4.1</version>
  <executions>
    <execution>
      <phase>package</phase>
      <goals>
        <goal>shade</goal>
      </goals>
      <configuration>
        [SNIP]
        <transformers>
          <!-- merges META-INF/services/ entries instead of overwriting —>
          <transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
          <!-- merges META-INF/javax.media.jai.registryFile.jai entries instead of overwriting —>
          <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
            <resource>META-INF/javax.media.jai.registryFile.jai</resource>
          </transformer>
          <!-- merges META-INF/registryFile.jai entries instead of overwriting —>
          <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
            <resource>META-INF/registryFile.jai</resource>
          </transformer>
          <!-- merges META-INF/registryFile.jaiext entries instead of overwriting —>
          <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
            <resource>META-INF/registryFile.jaiext</resource>
          </transformer>
        </transformers>
        [SNIP]
      </configuration>
    </execution>
  </executions>
</plugin>

On Jul 8, 2017, at 11:45 AM, David Boyd <dboyd@xxxxxxxxxxxxxxxxx> wrote:

Tom:

  Thanks my maven foo is weak.  Any good examples I should follow?


On 7/8/17 11:42 AM, Tom Kunicki wrote:

I didn’t see any META-INF/Services merge directives in your shade configuration.  GeoTools relies heavily on SPI, there will me some more magic if you need to pull in JAI registries also.

Tom

On Jul 8, 2017, at 11:40 AM, David Boyd <dboyd@xxxxxxxxxxxxxxxxx> wrote:

Jim:

   Thanks.   By 2am my eyes were crossing.  I updated to get rid of all the 13.1 geotools dependencies.
(They were getting pulled in from another project that fortunately I control and could update the version
required).

Same problem no float fields getting populated.

The geoapi stuff was being pulled in by apache.tika.  I excluded geoapi, and tried running the jar again.

And the same problem persists with the floats not getting populated.

And I am now getting this error when I start:


Jul 08, 2017 11:12:27 AM org.geotools.factory.FactoryRegistry scanForPlugins
WARNING: Can't load a service for category "MathTransformFactory". Cause is "NoClassDefFoundError: org/opengis/util/Factory".
Jul 08, 2017 11:12:27 AM org.geotools.factory.FactoryRegistry scanForPlugins
WARNING: Can't load a service for category "CRSAuthorityFactory". Cause is "ServiceConfigurationError: org.opengis.referencing.crs.CRSAuthorityFactory: Provider org.geotools.referencing.factory.epsg.ThreadedHsqlEpsgFactory could not be instantiated".
java.util.ServiceConfigurationError: org.opengis.referencing.crs.CRSAuthorityFactory: Provider org.geotools.referencing.factory.epsg.ThreadedHsqlEpsgFactory could not be instantiated
    at java.util.ServiceLoader.fail(ServiceLoader.java:232)
    at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
    at org.geotools.factory.FactoryRegistry.register(FactoryRegistry.java:826)
    at org.geotools.factory.FactoryRegistry.scanForPlugins(FactoryRegistry.java:772)
    at org.geotools.factory.FactoryRegistry.scanForPluginsIfNeeded(FactoryRegistry.java:805)
    at org.geotools.factory.FactoryRegistry.getServiceProviders(FactoryRegistry.java:196)
    at org.geotools.referencing.ReferencingFactoryFinder.getFactories(ReferencingFactoryFinder.java:180)
    at org.geotools.referencing.ReferencingFactoryFinder.getCRSAuthorityFactories(ReferencingFactoryFinder.java:455)
    at org.geotools.referencing.DefaultAuthorityFactory.getBackingFactory(DefaultAuthorityFactory.java:89)
    at org.geotools.referencing.DefaultAuthorityFactory.<init>(DefaultAuthorityFactory.java:69)
    at org.geotools.referencing.CRS.getAuthorityFactory(CRS.java:263)
    at org.geotools.referencing.CRS.decode(CRS.java:520)
    at org.geotools.referencing.CRS.decode(CRS.java:448)
    at org.locationtech.geomesa.utils.geotools.package$.liftedTree1$1(package.scala:25)
    at org.locationtech.geomesa.utils.geotools.package$.<init>(package.scala:25)
    at org.locationtech.geomesa.utils.geotools.package$.<clinit>(package.scala)
    at org.locationtech.geomesa.utils.geotools.AttributeSpec$GeomAttributeSpec.builderHook(SimpleFeatureSpec.scala:165)
    at org.locationtech.geomesa.utils.geotools.AttributeSpec$class.toDescriptor(SimpleFeatureSpec.scala:76)
    at org.locationtech.geomesa.utils.geotools.AttributeSpec$GeomAttributeSpec.toDescriptor(SimpleFeatureSpec.scala:156)
    at org.locationtech.geomesa.utils.geotools.SimpleFeatureTypes$$anonfun$createType$1.apply(SimpleFeatureTypes.scala:189)
    at org.locationtech.geomesa.utils.geotools.SimpleFeatureTypes$$anonfun$createType$1.apply(SimpleFeatureTypes.scala:189)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
    at scala.collection.immutable.List.foreach(List.scala:381)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
    at scala.collection.immutable.List.map(List.scala:285)
    at org.locationtech.geomesa.utils.geotools.SimpleFeatureTypes$.createType(SimpleFeatureTypes.scala:189)
    at org.locationtech.geomesa.utils.geotools.SimpleFeatureTypes$.createType(SimpleFeatureTypes.scala:99)
    at org.locationtech.geomesa.utils.geotools.SimpleFeatureTypes$.createType(SimpleFeatureTypes.scala:83)
    at org.locationtech.geomesa.index.geotools.GeoMesaDataStore$$anonfun$1.apply(GeoMesaDataStore.scala:174)
    at org.locationtech.geomesa.index.geotools.GeoMesaDataStore$$anonfun$1.apply(GeoMesaDataStore.scala:174)
    at scala.Option.map(Option.scala:146)
    at org.locationtech.geomesa.index.geotools.GeoMesaDataStore.getSchema(GeoMesaDataStore.scala:174)
    at org.locationtech.geomesa.accumulo.data.AccumuloDataStore.getSchema(AccumuloDataStore.scala:144)
    at com.incadencecorp.coalesce.framework.persistance.accumulo.AccumuloPersistor.createLinkageFeature(AccumuloPersistor.java:698)
    at com.incadencecorp.coalesce.framework.persistance.accumulo.AccumuloPersistor.<init>(AccumuloPersistor.java:208)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at java.lang.Class.newInstance(Class.java:442)
    at com.incadencecorp.oe.common.utils.OECoalesceFramework.<init>(OECoalesceFramework.java:42)
    at com.incadencecorp.oe.ingest.gdelt.GDELT_EntityExtractor.<init>(GDELT_EntityExtractor.java:70)
    at com.incadencecorp.oe.ingest.gdelt.GDELT_Consumer.<init>(GDELT_Consumer.java:40)
    at com.incadencecorp.oe.ingest.gdelt.GDELT_Ingester.initializeConsumer(GDELT_Ingester.java:96)
    at com.incadencecorp.oe.ingest.gdelt.GDELT_Ingester.main(GDELT_Ingester.java:88)
Caused by: org.geotools.factory.FactoryNotFoundException: No factory of kind "MathTransformFactory" found.
    at org.geotools.factory.FactoryRegistry.getServiceProvider(FactoryRegistry.java:375)
    at org.geotools.factory.FactoryCreator.getServiceProvider(FactoryCreator.java:145)
    at org.geotools.referencing.ReferencingFactoryFinder.getFactory(ReferencingFactoryFinder.java:197)
    at org.geotools.referencing.ReferencingFactoryFinder.getMathTransformFactory(ReferencingFactoryFinder.java:520)
    at org.geotools.referencing.factory.ReferencingFactoryContainer.getMathTransformFactory(ReferencingFactoryContainer.java:275)
    at org.geotools.referencing.factory.ReferencingFactoryContainer.initialize(ReferencingFactoryContainer.java:182)
    at org.geotools.referencing.factory.ReferencingFactoryContainer.getImplementationHints(ReferencingFactoryContainer.java:207)
    at org.geotools.factory.FactoryRegistry.usesAcceptableHints(FactoryRegistry.java:552)
    at org.geotools.factory.FactoryRegistry.isAcceptable(FactoryRegistry.java:506)
    at org.geotools.factory.FactoryRegistry.getServiceImplementation(FactoryRegistry.java:437)
    at org.geotools.factory.FactoryRegistry.getServiceProvider(FactoryRegistry.java:365)
    at org.geotools.factory.FactoryCreator.getServiceProvider(FactoryCreator.java:145)
    at org.geotools.referencing.factory.ReferencingFactoryContainer.instance(ReferencingFactoryContainer.java:172)
    at org.geotools.referencing.factory.epsg.ThreadedEpsgFactory.<init>(ThreadedEpsgFactory.java:195)
    at org.geotools.referencing.factory.epsg.ThreadedHsqlEpsgFactory.<init>(ThreadedHsqlEpsgFactory.java:141)
    at org.geotools.referencing.factory.epsg.ThreadedHsqlEpsgFactory.<init>(ThreadedHsqlEpsgFactory.java:127)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at java.lang.Class.newInstance(Class.java:442)
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
    ... 46 more
Attached is my latest dependency tree.


On 7/8/17 10:31 AM, Jim Hughes wrote:
Hi David,

From a cursory look, it appears that you have multiple versions of GeoTools hanging around as transitive dependencies in your project. 

[INFO] |  |  +- (org.geotools:gt-opengis:jar:13.1:compile - omitted for conflict with 15.1)

If tamping down on the dependencyManagement bits doesn't sort things out, there's one other option.  The package name org.opengis is used by GeoAPI and by GeoTools.  GeoTools has slightly extended the API.  If you have 'opengis' jars from GeoTools and the GeoAPI jars in the same project, that could lead to the error below.

Cheers,

Jim

On 7/8/2017 1:49 AM, David Boyd wrote:
All:

  I think I have a very strange dependency issue and could use
some help finding where to look.

If I run my application from the shaded jar built none of the floating point
values in my data get into the database.

If I run the same application from eclipse, everything works fine.

So I assumed a geotools dependency issue and copied the exact
dependencies from the https://github.com/locationtech/geomesa/blob/master/pom.xml

Now I get:

xception in thread "main" java.lang.NoSuchMethodError: org.opengis.feature.type.AttributeType.getName()Lorg/opengis/feature/type/Name;
    at org.geotools.feature.simple.SimpleSchema.<init>(SimpleSchema.java:225)
    at org.geotools.feature.simple.SimpleFeatureTypeBuilder.<init>(SimpleFeatureTypeBuilder.java:196)
    at org.geotools.feature.simple.SimpleFeatureTypeBuilder.<init>(SimpleFeatureTypeBuilder.java:185)
    at org.locationtech.geomesa.utils.geotools.SimpleFeatureTypes$.createType(SimpleFeatureTypes.scala:186)
    at org.locationtech.geomesa.utils.geotools.SimpleFeatureTypes$.createType(SimpleFeatureTypes.scala:99)
    at org.locationtech.geomesa.utils.geotools.SimpleFeatureTypes$.createType(SimpleFeatureTypes.scala:83)
    at org.locationtech.geomesa.index.geotools.GeoMesaDataStore$$anonfun$1.apply(GeoMesaDataStore.scala:174)
    at org.locationtech.geomesa.index.geotools.GeoMesaDataStore$$anonfun$1.apply(GeoMesaDataStore.scala:174)
    at scala.Option.map(Option.scala:146)
    at org.locationtech.geomesa.index.geotools.GeoMesaDataStore.getSchema(GeoMesaDataStore.scala:174)
    at org.locationtech.geomesa.accumulo.data.AccumuloDataStore.getSchema(AccumuloDataStore.scala:144)
    at com.incadencecorp.coalesce.framework.persistance.accumulo.AccumuloPersistor.createLinkageFeature(AccumuloPersistor.java:698)
    at com.incadencecorp.coalesce.framework.persistance.accumulo.AccumuloPersistor.<init>(AccumuloPersistor.java:208)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at java.lang.Class.newInstance(Class.java:442)
    at com.incadencecorp.oe.common.utils.OECoalesceFramework.<init>(OECoalesceFramework.java:42)
    at com.incadencecorp.oe.ingest.gdelt.GDELT_EntityExtractor.<init>(GDELT_EntityExtractor.java:70)
    at com.incadencecorp.oe.ingest.gdelt.GDELT_Consumer.<init>(GDELT_Consumer.java:40)
    at com.incadencecorp.oe.ingest.gdelt.GDELT_Ingester.initializeConsumer(GDELT_Ingester.java:96)
    at com.incadencecorp.oe.ingest.gdelt.GDELT_Ingester.main(GDELT_Ingester.java:88)
I am going cross eyed with all these dependencies.

So I will accept help with either my original issue or where
I am now.

Also attached is my maven build output and maven dependency tree output.






_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users




_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users

-- 
========= mailto:dboyd@xxxxxxxxxxxxxxxxx ============
David W. Boyd                     
VP,  Data Solutions       
10432 Balls Ford, Suite 240  
Manassas, VA 20109         
office:   +1-703-552-2862        
cell:     +1-703-402-7908
============== http://www.incadencecorp.com/ ============
ISO/IEC JTC1 WG9, editor ISO/IEC 20547 Big Data Reference Architecture
Chair ANSI/INCITS TC Big Data
Co-chair NIST Big Data Public Working Group Reference Architecture
First Robotic Mentor - FRC, FTC - www.iliterobotics.org
Board Member- USSTEM Foundation - www.usstem.org

The information contained in this message may be privileged 
and/or confidential and protected from disclosure.  
If the reader of this message is not the intended recipient 
or an employee or agent responsible for delivering this message 
to the intended recipient, you are hereby notified that any 
dissemination, distribution or copying of this communication 
is strictly prohibited.  If you have received this communication 
in error, please notify the sender immediately by replying to 
this message and deleting the material from any computer.

 
<depend.txt>_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users



_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users

-- 
========= mailto:dboyd@xxxxxxxxxxxxxxxxx ============
David W. Boyd                     
VP,  Data Solutions       
10432 Balls Ford, Suite 240  
Manassas, VA 20109         
office:   +1-703-552-2862        
cell:     +1-703-402-7908
============== http://www.incadencecorp.com/ ============
ISO/IEC JTC1 WG9, editor ISO/IEC 20547 Big Data Reference Architecture
Chair ANSI/INCITS TC Big Data
Co-chair NIST Big Data Public Working Group Reference Architecture
First Robotic Mentor - FRC, FTC - www.iliterobotics.org
Board Member- USSTEM Foundation - www.usstem.org

The information contained in this message may be privileged 
and/or confidential and protected from disclosure.  
If the reader of this message is not the intended recipient 
or an employee or agent responsible for delivering this message 
to the intended recipient, you are hereby notified that any 
dissemination, distribution or copying of this communication 
is strictly prohibited.  If you have received this communication 
in error, please notify the sender immediately by replying to 
this message and deleting the material from any computer.

 
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users


Back to the top