Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] What are the GeoMesa installation requirements and what are recommendations?

Accumulo has a port conflict with cloudera manager.  If you read the Accumulo installation instructions for Cloudera you will find instructions for the fixing the conflict on page 16.

 

http://www.cloudera.com/content/cloudera-content/cloudera-docs/Accumulo/latest/PDF/Apache-Accumulo-Installation-Guide.pdf

 

In short, you need to add the following properties to your accumulo-site.xml

 

<property>

<name>tserver.port.client</name>

<value>10011</value>

</property>

 

<property>

<name>master.port.client</name>

<value>10010</value>

</property>

 

 

 

GENERAL DYNAMICS
Advanced Information Systems
John T. Kalisz
Senior Principal Systems Engineer

Physical Security Systems
John.Kalisz@xxxxxxxxxx
Office: (413) 494-3376

 

From: geomesa-users-bounces@xxxxxxxxxxxxxxxx [mailto:geomesa-users-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Beau Lalonde
Sent: Friday, June 13, 2014 10:37 AM
To: GeoMesa Users Mailing List (geomesa-users@xxxxxxxxxxxxxxxx)
Subject: Re: [geomesa-users] What are the GeoMesa installation requirements and what are recommendations?

 

Hi,

 

I am definitely getting closer to getting the newer GeoMesa working, but I am having problems querying the data.

 

Here are the dependencies from my POM file that I am using to go along with the QuickStart.java example from http://geomesa.github.io/2014/05/28/geomesa-quickstart/.  I should note, that we changed the version of GeoMesa to 1.0.0-2 on our cloud, so we could differentiate it from an older version we were using.

 

<dependencies>

         <dependency>

            <groupId>geomesa</groupId>

            <artifactId>geomesa-core</artifactId>

            <version>1.0.0-2</version>

        </dependency>

        <dependency>

            <groupId>commons-cli</groupId>

            <artifactId>commons-cli</artifactId>

            <version>1.2</version>

        </dependency>

        <dependency>

            <groupId>org.apache.accumulo</groupId>

            <artifactId>accumulo-core</artifactId>

            <version>1.5.0</version>

        </dependency>

        <dependency>

            <groupId>org.apache.accumulo</groupId>

            <artifactId>accumulo-start</artifactId>

            <version>1.5.0</version>

        </dependency>

        <dependency>

            <groupId>org.apache.hadoop</groupId>

            <artifactId>hadoop-client</artifactId>

            <version>2.0.0-cdh4.5.0</version>

       </dependency>

        <dependency>

            <groupId>com.google.guava</groupId>

            <artifactId>guava</artifactId>

            <version>16.0.1</version>

        </dependency>

        <dependency>

            <groupId>org.geotools</groupId>

            <artifactId>gt-data</artifactId>

            <version>11.0</version>

        </dependency>

        <dependency>

            <groupId>org.geotools</groupId>

            <artifactId>gt-epsg-hsql</artifactId>

            <version>11.0</version>

        </dependency>

</dependencies>

 

 

It appears to me that the data is ingested into GeoMesa/accumulo correctly.  Here are the contents from scanning the table via the accumulo shell:

 

04~QuickStart~633~20140716 0d:Observation.2 []    \x00\x00\x00\x0DObservation.2\x00\x00\x00\x15\x00\x00\x00\x00\x01\xC0SN\xBE\xDF\xD5V\xF7\xC0B\xEF"I\xA2\xF0t\x00\x00\x01G>G\x1Ah

04~QuickStart~633~20140716 Observation.2:SimpleFeatureAttribute []    \x02\x1AObservation.2\x00\x0EClemens\x00\x04\x00\xD0\xE9\xB8\xE4\xE7Q\x00XPOINT (-77.23040004571872 -37.8682338758089)\x02

32~QuickStart~633~20140902 Observation.0:SimpleFeatureAttribute []    \x02\x1AObservation.0\x00\x0CAddams\x00\x00\x00\x80\xCC\xE7\xE8\x86R\x00ZPOINT (-76.0577293170671 -37.615979973322965)\x02

32~QuickStart~633~20140902 fj:Observation.0 []    \x00\x00\x00\x0DObservation.0\x00\x00\x00\x15\x00\x00\x00\x00\x01\xC0S\x03\xB1\xD6N4\xB6\xC0B\xCE\xD8n\x884\xE0\x00\x00\x01H6\x8C\xF3\x00

54~QuickStart~630~20140417 Observation.1:SimpleFeatureAttribute []    \x02\x1AObservation.1\x00\x0CBierce\x00\x02\x00\xE0\x9E\xD7\xF2\xADQ\x00ZPOINT (-77.48469253216224 -38.87427143505418)\x02

54~QuickStart~630~20140417 fu:Observation.1 []    \x00\x00\x00\x0DObservation.1\x00\x00\x00\x15\x00\x00\x00\x00\x01\xC0S_\x053\xD3\x90%\xC0Co\xE8 Z\xB1>\x00\x00\x01Eo*\xE7\xB0

~METADATA_QuickStart attributes: []    Who:String,What:java.lang.Long,When:Date,Where:Point:srid=4326,Why:String

~METADATA_QuickStart bounds: []    -77.48469253216224:-76.0577293170671:-38.87427143505418:-37.615979973322965

~METADATA_QuickStart dtgfield: []    When

~METADATA_QuickStart featureEncoding: []    avro

~METADATA_QuickStart schema: []    %~#s%99#r%QuickStart#cstr%0,3#gh%yyyyMMdd#d::%~#s%3,2#gh::%~#s%#id

 

 

When I perform a query, I get no results.  When I go to the tablet server log files, I see the following warnings and errors.  Doing some online research seems to indicate that this is an HSQL version compatibility problem – perhaps between GeoTools and Hadoop.  Has anyone encountered this problem?  Does anyone have any suggestions?

 

Thanks,

 

Beau

 

 

2014-06-13 12:35:54,839 [tabletserver.TabletServer] WARN : exception while doing multi-scan

java.lang.NoSuchMethodError: org.hsqldb.DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;

                at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)

                at org.hsqldb.jdbc.JDBCDataSource.getConnection(Unknown Source)

                at org.hsqldb.jdbc.JDBCDataSource.getConnection(Unknown Source)

                at org.geotools.referencing.factory.epsg.DirectEpsgFactory.getConnection(DirectEpsgFactory.java:3196)

                at org.geotools.referencing.factory.epsg.ThreadedEpsgFactory.createBackingStore(ThreadedEpsgFactory.java:436)

                at org.geotools.referencing.factory.DeferredAuthorityFactory.getBackingStore(DeferredAuthorityFactory.java:133)

                at org.geotools.referencing.factory.BufferedAuthorityFactory.isAvailable(BufferedAuthorityFactory.java:235)

                at org.geotools.referencing.factory.DeferredAuthorityFactory.isAvailable(DeferredAuthorityFactory.java:119)

                at org.geotools.factory.FactoryRegistry.isAvailable(FactoryRegistry.java:667)

                at org.geotools.factory.FactoryRegistry.isAcceptable(FactoryRegistry.java:501)

                at org.geotools.factory.FactoryRegistry$1.filter(FactoryRegistry.java:192)

                at javax.imageio.spi.FilterIterator.advance(ServiceRegistry.java:810)

                at javax.imageio.spi.FilterIterator.<init>(ServiceRegistry.java:804)

                at javax.imageio.spi.ServiceRegistry.getServiceProviders(ServiceRegistry.java:508)

                at org.geotools.factory.FactoryRegistry.getServiceProviders(FactoryRegistry.java:197)

                at org.geotools.referencing.ReferencingFactoryFinder.getFactories(ReferencingFactoryFinder.java:180)

                at org.geotools.referencing.ReferencingFactoryFinder.getCRSAuthorityFactories(ReferencingFactoryFinder.java:455)

                at org.geotools.referencing.DefaultAuthorityFactory.getBackingFactory(DefaultAuthorityFactory.java:88)

                at org.geotools.referencing.DefaultAuthorityFactory.<init>(DefaultAuthorityFactory.java:69)

                at org.geotools.referencing.CRS.getAuthorityFactory(CRS.java:257)

                at org.geotools.referencing.CRS.decode(CRS.java:514)

                at org.geotools.referencing.CRS.decode(CRS.java:442)

                at org.geotools.data.DataUtilities.createAttribute(DataUtilities.java:2646)

                at org.geotools.data.DataUtilities.createType(DataUtilities.java:1928)

                at org.geotools.data.DataUtilities.createType(DataUtilities.java:1901)

                at geomesa.core.iterators.SpatioTemporalIntersectingIterator.init(SpatioTemporalIntersectingIterator.scala:89)

                at org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:243)

                at org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:213)

                at org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:191)

                at org.apache.accumulo.server.tabletserver.Tablet$ScanDataSource.createIterator(Tablet.java:2084)

                at org.apache.accumulo.server.tabletserver.Tablet$ScanDataSource.iterator(Tablet.java:2022)

                at org.apache.accumulo.core.iterators.system.SourceSwitchingIterator.seek(SourceSwitchingIterator.java:163)

                at org.apache.accumulo.server.tabletserver.Tablet.lookup(Tablet.java:1600)

                at org.apache.accumulo.server.tabletserver.Tablet.lookup(Tablet.java:1705)

                at org.apache.accumulo.server.tabletserver.TabletServer$ThriftClientHandler$LookupTask.run(TabletServer.java:1036)

                at org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)

                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

                at org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)

                at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)

                at java.lang.Thread.run(Thread.java:744)

2014-06-13 12:35:54,839 [tabletserver.TabletServer] WARN : Failed to get multiscan result

java.util.concurrent.ExecutionException: java.lang.NoSuchMethodError: org.hsqldb.DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;

                at org.apache.accumulo.server.tabletserver.TabletServer$ScanTask.get(TabletServer.java:698)

                at org.apache.accumulo.server.tabletserver.TabletServer$ThriftClientHandler.continueMultiScan(TabletServer.java:1345)

                at org.apache.accumulo.server.tabletserver.TabletServer$ThriftClientHandler.startMultiScan(TabletServer.java:1310)

                at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source)

                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

                at java.lang.reflect.Method.invoke(Method.java:606)

                at org.apache.accumulo.trace.instrument.thrift.TraceWrap$1.invoke(TraceWrap.java:63)

                at com.sun.proxy.$Proxy10.startMultiScan(Unknown Source)

                at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$startMultiScan.getResult(TabletClientService.java:1985)

                at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$startMultiScan.getResult(TabletClientService.java:1969)

                at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)

                at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)

                at org.apache.accumulo.server.util.TServerUtils$TimedProcessor.process(TServerUtils.java:156)

                at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:478)

                at org.apache.accumulo.server.util.TServerUtils$THsHaServer$Invocation.run(TServerUtils.java:208)

                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

                at org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)

                at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)

                at java.lang.Thread.run(Thread.java:744)

Caused by: java.lang.NoSuchMethodError: org.hsqldb.DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;

                at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)

                at org.hsqldb.jdbc.JDBCDataSource.getConnection(Unknown Source)

                at org.hsqldb.jdbc.JDBCDataSource.getConnection(Unknown Source)

                at org.geotools.referencing.factory.epsg.DirectEpsgFactory.getConnection(DirectEpsgFactory.java:3196)

                at org.geotools.referencing.factory.epsg.ThreadedEpsgFactory.createBackingStore(ThreadedEpsgFactory.java:436)

                at org.geotools.referencing.factory.DeferredAuthorityFactory.getBackingStore(DeferredAuthorityFactory.java:133)

                at org.geotools.referencing.factory.BufferedAuthorityFactory.isAvailable(BufferedAuthorityFactory.java:235)

                at org.geotools.referencing.factory.DeferredAuthorityFactory.isAvailable(DeferredAuthorityFactory.java:119)

                at org.geotools.factory.FactoryRegistry.isAvailable(FactoryRegistry.java:667)

                at org.geotools.factory.FactoryRegistry.isAcceptable(FactoryRegistry.java:501)

                at org.geotools.factory.FactoryRegistry$1.filter(FactoryRegistry.java:192)

                at javax.imageio.spi.FilterIterator.advance(ServiceRegistry.java:810)

                at javax.imageio.spi.FilterIterator.<init>(ServiceRegistry.java:804)

                at javax.imageio.spi.ServiceRegistry.getServiceProviders(ServiceRegistry.java:508)

                at org.geotools.factory.FactoryRegistry.getServiceProviders(FactoryRegistry.java:197)

                at org.geotools.referencing.ReferencingFactoryFinder.getFactories(ReferencingFactoryFinder.java:180)

                at org.geotools.referencing.ReferencingFactoryFinder.getCRSAuthorityFactories(ReferencingFactoryFinder.java:455)

                at org.geotools.referencing.DefaultAuthorityFactory.getBackingFactory(DefaultAuthorityFactory.java:88)

                at org.geotools.referencing.DefaultAuthorityFactory.<init>(DefaultAuthorityFactory.java:69)

                at org.geotools.referencing.CRS.getAuthorityFactory(CRS.java:257)

                at org.geotools.referencing.CRS.decode(CRS.java:514)

                at org.geotools.referencing.CRS.decode(CRS.java:442)

                at org.geotools.data.DataUtilities.createAttribute(DataUtilities.java:2646)

                at org.geotools.data.DataUtilities.createType(DataUtilities.java:1928)

                at org.geotools.data.DataUtilities.createType(DataUtilities.java:1901)

                at geomesa.core.iterators.SpatioTemporalIntersectingIterator.init(SpatioTemporalIntersectingIterator.scala:89)

                at org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:243)

                at org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:213)

                at org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:191)

                at org.apache.accumulo.server.tabletserver.Tablet$ScanDataSource.createIterator(Tablet.java:2084)

                at org.apache.accumulo.server.tabletserver.Tablet$ScanDataSource.iterator(Tablet.java:2022)

                at org.apache.accumulo.core.iterators.system.SourceSwitchingIterator.seek(SourceSwitchingIterator.java:163)

                at org.apache.accumulo.server.tabletserver.Tablet.lookup(Tablet.java:1600)

                at org.apache.accumulo.server.tabletserver.Tablet.lookup(Tablet.java:1705)

                at org.apache.accumulo.server.tabletserver.TabletServer$ThriftClientHandler$LookupTask.run(TabletServer.java:1036)

                at org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)

                ... 5 more

2014-06-13 12:35:54,840 [thrift.ProcessFunction] ERROR: Internal error processing startMultiScan

java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.NoSuchMethodError: org.hsqldb.DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;

                at org.apache.accumulo.server.tabletserver.TabletServer$ThriftClientHandler.continueMultiScan(TabletServer.java:1358)

                at org.apache.accumulo.server.tabletserver.TabletServer$ThriftClientHandler.startMultiScan(TabletServer.java:1310)

                at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source)

                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

                at java.lang.reflect.Method.invoke(Method.java:606)

                at org.apache.accumulo.trace.instrument.thrift.TraceWrap$1.invoke(TraceWrap.java:63)

                at com.sun.proxy.$Proxy10.startMultiScan(Unknown Source)

                at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$startMultiScan.getResult(TabletClientService.java:1985)

                at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$startMultiScan.getResult(TabletClientService.java:1969)

                at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)

                at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)

                at org.apache.accumulo.server.util.TServerUtils$TimedProcessor.process(TServerUtils.java:156)

                at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:478)

                at org.apache.accumulo.server.util.TServerUtils$THsHaServer$Invocation.run(TServerUtils.java:208)

                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

                at org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)

                at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)

                at java.lang.Thread.run(Thread.java:744)

Caused by: java.util.concurrent.ExecutionException: java.lang.NoSuchMethodError: org.hsqldb.DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;

                at org.apache.accumulo.server.tabletserver.TabletServer$ScanTask.get(TabletServer.java:698)

                at org.apache.accumulo.server.tabletserver.TabletServer$ThriftClientHandler.continueMultiScan(TabletServer.java:1345)

                ... 18 more

Caused by: java.lang.NoSuchMethodError: org.hsqldb.DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;

                at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)

                at org.hsqldb.jdbc.JDBCDataSource.getConnection(Unknown Source)

                at org.hsqldb.jdbc.JDBCDataSource.getConnection(Unknown Source)

                at org.geotools.referencing.factory.epsg.DirectEpsgFactory.getConnection(DirectEpsgFactory.java:3196)

                at org.geotools.referencing.factory.epsg.ThreadedEpsgFactory.createBackingStore(ThreadedEpsgFactory.java:436)

                at org.geotools.referencing.factory.DeferredAuthorityFactory.getBackingStore(DeferredAuthorityFactory.java:133)

                at org.geotools.referencing.factory.BufferedAuthorityFactory.isAvailable(BufferedAuthorityFactory.java:235)

                at org.geotools.referencing.factory.DeferredAuthorityFactory.isAvailable(DeferredAuthorityFactory.java:119)

                at org.geotools.factory.FactoryRegistry.isAvailable(FactoryRegistry.java:667)

                at org.geotools.factory.FactoryRegistry.isAcceptable(FactoryRegistry.java:501)

                at org.geotools.factory.FactoryRegistry$1.filter(FactoryRegistry.java:192)

                at javax.imageio.spi.FilterIterator.advance(ServiceRegistry.java:810)

                at javax.imageio.spi.FilterIterator.<init>(ServiceRegistry.java:804)

                at javax.imageio.spi.ServiceRegistry.getServiceProviders(ServiceRegistry.java:508)

                at org.geotools.factory.FactoryRegistry.getServiceProviders(FactoryRegistry.java:197)

                at org.geotools.referencing.ReferencingFactoryFinder.getFactories(ReferencingFactoryFinder.java:180)

                at org.geotools.referencing.ReferencingFactoryFinder.getCRSAuthorityFactories(ReferencingFactoryFinder.java:455)

                at org.geotools.referencing.DefaultAuthorityFactory.getBackingFactory(DefaultAuthorityFactory.java:88)

                at org.geotools.referencing.DefaultAuthorityFactory.<init>(DefaultAuthorityFactory.java:69)

                at org.geotools.referencing.CRS.getAuthorityFactory(CRS.java:257)

                at org.geotools.referencing.CRS.decode(CRS.java:514)

                at org.geotools.referencing.CRS.decode(CRS.java:442)

                at org.geotools.data.DataUtilities.createAttribute(DataUtilities.java:2646)

                at org.geotools.data.DataUtilities.createType(DataUtilities.java:1928)

                at org.geotools.data.DataUtilities.createType(DataUtilities.java:1901)

                at geomesa.core.iterators.SpatioTemporalIntersectingIterator.init(SpatioTemporalIntersectingIterator.scala:89)

                at org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:243)

                at org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:213)

                at org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:191)

                at org.apache.accumulo.server.tabletserver.Tablet$ScanDataSource.createIterator(Tablet.java:2084)

                at org.apache.accumulo.server.tabletserver.Tablet$ScanDataSource.iterator(Tablet.java:2022)

                at org.apache.accumulo.core.iterators.system.SourceSwitchingIterator.seek(SourceSwitchingIterator.java:163)

                at org.apache.accumulo.server.tabletserver.Tablet.lookup(Tablet.java:1600)

                at org.apache.accumulo.server.tabletserver.Tablet.lookup(Tablet.java:1705)

                at org.apache.accumulo.server.tabletserver.TabletServer$ThriftClientHandler$LookupTask.run(TabletServer.java:1036)

                at org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)

                ... 5 more

 

 

From: Beau Lalonde
Sent: Thursday, June 12, 2014 1:36 PM
To: GeoMesa Users Mailing List (geomesa-users@xxxxxxxxxxxxxxxx)
Subject: RE: What are the GeoMesa installation requirements and what are recommendations?

 

John, Connor,

 

Thanks for your suggestions.  For the time being we have upgraded our cloud to use Accumulo 1.5.0 and Cloudera CDH 4.5.  CDH 4.5 uses Hadoop 2.0.0.  I am definitely getting further along, but I am not all of the way there.  I will post more questions if I cannot figure it out.

 

Thanks again,

Beau


Back to the top