|
Hi,
Sorry for not replying for Emilio's email and thank you for your
reply.
I'm using GeoMesa 1.2.3 (geomesa-dist-1.2.3-bin.tar.gz), Accumulo
1.6.2, Zookeeper 3.4.6 and GeoServer 2.9.0.
All servers are using the same version of GeoMesa.
Ingest command:
$
geomesa ingest -u van -c TDrive -i GeoMesa -s tdrive -C tdrive
geomesa/data/T_Drive.csv
Using
GEOMESA_HOME =
/home2/van/geomesa-1.2.3/dist/tools/geomesa-tools-1.2.3
Password
(mask enabled)>
Creating
schema tdrive
Running
ingestion in local mode
Ingesting
1 file with 1 thread
[============================================================]
100% complete 11661601 ingested 2 failed in 00:09:19
Local
ingestion complete in 00:09:19
Ingested
11661601 features and failed to ingest 2 features.
Catalog table scan:
van@GeoMesa> scan -t TDrive
~METADATA_tdrive attributes: []
taxiId:String,dtg:Date,*geom:Point:srid=4326;geomesa.index.dtg='dtg',geomesa.indexes.enabled='records,z3,attr_idx',geomesa.version='8',geomesa.table.sharing='false'
~METADATA_tdrive id: [] \x01
~METADATA_tdrive stats-date: [] 2016-07-08T05:30:51.844Z
~METADATA_tdrive tables.idx.attr.name: [] TDrive_tdrive_attr
~METADATA_tdrive tables.queries.name: [] TDrive_queries
~METADATA_tdrive tables.record.name: [] TDrive_tdrive_records
~METADATA_tdrive tables.z2.name: [] TDrive_tdrive_z2
~METADATA_tdrive tables.z3.name: [] TDrive_tdrive_z3
~METADATA_tdrive version: [] 8
I tried to ingest tdrive data again (with same command) to a new
table and work on the new table. The errors were gone but I did
not get results of the query. In the popup screen, there is only 1
word " unknown" appeared.
Here is info in GeoServer log:
15 Jul 10:12:21 INFO [geoserver.wps] -
Request: getServiceInfo
15 Jul 10:12:21 WARN [knn.KNearestNeighborSearchProcess] - The
provided data feature collection type may not support geomesa KNN
search: org.geoserver.feature.RetypingFeatureCollection
15 Jul 10:12:24 INFO [geoserver.wps] -
Request: execute
service = WPS
version = 1.0.0
baseUrl = http://localhost:8080/geoserver/
identifier:
value = geomesa:KNearestNeighborSearch
dataInputs:
input[0]:
identifier =
net.opengis.ows11.impl.CodeTypeImpl@4cdeeecb (value:
inputFeatures, codeSpace: null)
data =
"">
identifier =
net.opengis.ows11.impl.CodeTypeImpl@ce675b5 (value: dataFeatures,
codeSpace: null)
reference =
net.opengis.wps10.impl.InputReferenceTypeImpl@4add8c01 (body:
net.opengis.wfs.impl.GetFeatureTypeImpl@71ce8b27 (handle: null,
service: WFS, version: 1.0.0, baseUrl: null, providedVersion:
null, extendedProperties: {}) (maxFeatures: null, outputFormat:
GML2, resultType: <unset>, traverseXlinkDepth: null,
traverseXlinkExpiry: null, formatOptions: null, metadata: null,
startIndex: null, viewParams: null), encoding: null, href: http://geoserver/wfs,
method: POST, mimeType: text/xml, schema: null)input[0]:
identifier =
net.opengis.ows11.impl.CodeTypeImpl@3bca9388 (value: numDesired,
codeSpace: null)
data =
"">
identifier =
net.opengis.ows11.impl.CodeTypeImpl@22a52b9b (value:
estimatedDistance, codeSpace: null)
data =
"">
identifier =
net.opengis.ows11.impl.CodeTypeImpl@488e8ae6 (value:
maxSearchDistance, codeSpace: null)
data = "">
responseForm:
rawDataOutput:
identifier =
net.opengis.ows11.impl.CodeTypeImpl@3722ee46 (value: result,
codeSpace: null)
mimeType = text/xml; subtype=wfs-collection/1.0
Jul 15, 2016 10:12:24 AM
org.geoserver.platform.GeoServerExtensions checkContext
WARNING: Extension lookup 'PPIOFactory', but ApplicationContext is
unset.
Jul 15, 2016 10:12:24 AM
org.geoserver.platform.GeoServerExtensions checkContext
WARNING: Extension lookup 'ExtensionFilter', but
ApplicationContext is unset.
Jul 15, 2016 10:12:24 AM
org.geoserver.platform.GeoServerExtensions checkContext
WARNING: Extension lookup 'ExtensionProvider', but
ApplicationContext is unset.
Jul 15, 2016 10:12:24 AM
org.geoserver.platform.GeoServerExtensions checkContext
WARNING: Extension lookup 'ExtensionFilter', but
ApplicationContext is unset.
As you noted, ReTypingFeatureCollection is used so maybe the layer
name was not correct. Could you tell me how I can name the layer
properly.
Thanks,
Van
PS1: I'm interested in how spatiotemporal queries (kNN, window
query, range query...) are processed in GeoMesa but I only found
information in
https://github.com/locationtech/geomesa/tree/master/geomesa-process.
Is there any other tutorial about this (e.g., example of input,
output of each query...). Actually, I was not sure which value I
should set to each input field in the kNN query until I got Jim's
email.
PS2: the XML generated by the demo builder
<?xml version="1.0" encoding="UTF-8"?><wps:Execute version="1.0.0" service="WPS" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.opengis.net/wps/1.0.0" xmlns:wfs="http://www.opengis.net/wfs" xmlns:wps="http://www.opengis.net/wps/1.0.0" xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:gml="http://www.opengis.net/gml" xmlns:ogc="http://www.opengis.net/ogc" xmlns:wcs="http://www.opengis.net/wcs/1.1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xsi:schemaLocation="http://www.opengis.net/wps/1.0.0 http://schemas.opengis.net/wps/1.0.0/wpsAll.xsd">
<ows:Identifier>geomesa:KNearestNeighborSearch</ows:Identifier>
<wps:DataInputs>
<wps:Input>
<ows:Identifier>inputFeatures</ows:Identifier>
<wps:Data>
<wps:ComplexData mimeType="application/json"><![CDATA[{
"features" : [
{
"id" : "0",
"geometry" : {
"coordinates": [-77.034917, 38.894967],
"type" : "Point"
},
"type" : "Feature",
"properties" : {
"dtg" : "2014-05-17T15:33:16.000+0000",
}
}
],
"type" : "FeatureCollection"
}]]></wps:ComplexData>
</wps:Data>
</wps:Input>
<wps:Input>
<ows:Identifier>dataFeatures</ows:Identifier>
<wps:Reference mimeType="text/xml" xlink:href="" class="moz-txt-link-rfc2396E" href="http://geoserver/wfs">"http://geoserver/wfs" method="POST">
<wps:Body>
<wfs:GetFeature service="WFS" version="1.0.0" outputFormat="GML2" xmlns:it.geosolutions="http://www.geo-solutions.it">
<wfs:Query typeName="it.geosolutions:tdrive4"/>
</wfs:GetFeature>
</wps:Body>
</wps:Reference>
</wps:Input>
<wps:Input>
<ows:Identifier>numDesired</ows:Identifier>
<wps:Data>
<wps:LiteralData>5</wps:LiteralData>
</wps:Data>
</wps:Input>
<wps:Input>
<ows:Identifier>estimatedDistance</ows:Identifier>
<wps:Data>
<wps:LiteralData>1000</wps:LiteralData>
</wps:Data>
</wps:Input>
<wps:Input>
<ows:Identifier>maxSearchDistance</ows:Identifier>
<wps:Data>
<wps:LiteralData>10000</wps:LiteralData>
</wps:Data>
</wps:Input>
</wps:DataInputs>
<wps:ResponseForm>
<wps:RawDataOutput mimeType="text/xml; subtype=wfs-collection/1.0">
<ows:Identifier>result</ows:Identifier>
</wps:RawDataOutput>
</wps:ResponseForm>
</wps:Execute>
On 7/15/16 5:22 AM, Jim Hughes wrote:
Hi Van,
I checked out the KNN WPS today, and was able to get it to work.
I wanted to share some of the details.
We used the GeoServer Demos - WPS Request Builder on
KNearestNeighborSearch. For input features I selected 'Text' and
'application/json', and entered this block:
{
"features" : [
{
"id" : "0",
"geometry" : {
"coordinates": [-77.034917,
38.894967],
"type" : "Point"
},
"type" : "Feature",
"properties" : {
"dtg" :
"2014-05-17T15:33:16.000+0000",
}
}
],
"type" : "FeatureCollection"
}
From dataFeatures, I entered 'VECTOR_LAYER', I selected the
GeoServer layer for TDrive*. I set numDesired to 5,
estimatedDistance to 1000, and maxSearchDistance to 10000 (the
units are meters**, so this 1km with a bound of 10km). With all
that, I was able to execute the process and get back results.
To validate, I grabbed the feature IDs and built up a little query
for just those records (it looked like
IN('2594a5f2-f130-458d-94bc-16ebcfcc2755', ...)). In the
OpenLayers preview page, I was able to verify that those 5 points
appeared to make the 5 closest.
Anyhow, let us know how it goes for you. Emilio's previous email
about matching versions is likely the right path.
Cheers,
Jim
* NB: The layer has to be named what the SimpleFeatureType is
called. Otherwise, a ReTypingFeatureCollection will be used and
some things will go wrong. This is somewhat subtle issue.
** There are a few subtle points on distance calculations. Shout
if you have questions.
Sample XML generated by the demo builder:
<?xml version="1.0" encoding="UTF-8"?><wps:Execute
version="1.0.0" service="WPS" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://www.opengis.net/wps/1.0.0"
xmlns:wfs="http://www.opengis.net/wfs"
xmlns:wps="http://www.opengis.net/wps/1.0.0"
xmlns:ows="http://www.opengis.net/ows/1.1"
xmlns:gml="http://www.opengis.net/gml"
xmlns:ogc="http://www.opengis.net/ogc"
xmlns:wcs="http://www.opengis.net/wcs/1.1.1"
xmlns:xlink="http://www.w3.org/1999/xlink"
xsi:schemaLocation="http://www.opengis.net/wps/1.0.0
http://schemas.opengis.net/wps/1.0.0/wpsAll.xsd">
<ows:Identifier>geomesa:KNearestNeighborSearch</ows:Identifier>
<wps:DataInputs>
<wps:Input>
<ows:Identifier>inputFeatures</ows:Identifier>
<wps:Data>
<wps:ComplexData
mimeType="application/json"><![CDATA[{
"features" : [
{
"id" : "0",
"geometry" : {
"coordinates":
[116.30435943603516, 39.930667877197266],
"type" : "Point"
},
"type" : "Feature",
"properties" : {
"dtg" :
"2014-05-17T15:33:16.000+0000",
}
}
],
"type" : "FeatureCollection"
}]]></wps:ComplexData>
</wps:Data>
</wps:Input>
<wps:Input>
<ows:Identifier>dataFeatures</ows:Identifier>
<wps:Reference mimeType="text/xml" xlink:href=""
class="moz-txt-link-rfc2396E" href="http://geoserver/wfs">"http://geoserver/wfs"
method="POST">
<wps:Body>
<wfs:GetFeature service="WFS" version="1.0.0"
outputFormat="GML2" xmlns:it.geosolutions="http://www.geo-solutions.it">
<wfs:Query typeName="it.geosolutions:tdrive"/>
</wfs:GetFeature>
</wps:Body>
</wps:Reference>
</wps:Input>
<wps:Input>
<ows:Identifier>numDesired</ows:Identifier>
<wps:Data>
<wps:LiteralData>5</wps:LiteralData>
</wps:Data>
</wps:Input>
<wps:Input>
<ows:Identifier>estimatedDistance</ows:Identifier>
<wps:Data>
<wps:LiteralData>1000</wps:LiteralData>
</wps:Data>
</wps:Input>
<wps:Input>
<ows:Identifier>maxSearchDistance</ows:Identifier>
<wps:Data>
<wps:LiteralData>10000</wps:LiteralData>
</wps:Data>
</wps:Input>
</wps:DataInputs>
<wps:ResponseForm>
<wps:RawDataOutput mimeType="application/json">
<ows:Identifier>result</ows:Identifier>
</wps:RawDataOutput>
</wps:ResponseForm>
</wps:Execute>
On 07/11/2016 10:06 AM, Emilio Lahr-Vivaz wrote:
Hi Van,
As a first step, could you double check that you have the same
geomesa jar version deployed everywhere in your cluster? (e.g.
accumulo, geoserver, ingest, etc). Also, which geomesa version
are you using?
As a second step, could you scan the geomesa catalog table for
the schema that is failing, and send us the output? That should
allow us to re-create the bug in a unit test. Also, could you
send the XML that's getting sent to geoserver for the request?
Thanks,
Emilio
On 07/09/2016 05:59 AM, Le Hong Van wrote:
Hello,
I'm trying to run KNearestNeighborSearch in geomesa-process (https://github.com/locationtech/geomesa/tree/master/geomesa-process)
using "WPS request builder" on GeoServer but I got below
error:
Request: getServiceInfo
09 Jul 18:43:28 WARN [index.RecordIdxStrategy] - Running full
table scan for schema tdrive with filter BBOX(geom,
116.30435943603516,39.930667877197266,116.30470275878906,39.93083953857422)
09 Jul 18:43:30 WARN [impl.TabletServerBatchReaderIterator] -
Error on server hadoop120:9997
org.apache.accumulo.core.client.impl.AccumuloServerException:
Error on server hadoop120:9997
at
org.apache.accumulo.core.client.impl.TabletServerBatchReaderIterator.doLookup(TabletServerBatchReaderIterator.java:723)
at
org.apache.accumulo.core.client.impl.TabletServerBatchReaderIterator$QueryTask.run(TabletServerBatchReaderIterator.java:376)
at
org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at
org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
at
org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.TApplicationException: Internal
error processing continueMultiScan
at
org.apache.thrift.TApplicationException.read(TApplicationException.java:111)
at
org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
at
org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.recv_continueMultiScan(TabletClientService.java:340)
at
org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.continueMultiScan(TabletClientService.java:326)
at
org.apache.accumulo.core.client.impl.TabletServerBatchReaderIterator.doLookup(TabletServerBatchReaderIterator.java:691)
... 7 more
Log in the server that had error:
2016-07-09 18:43:30,460 [tserver.TabletServer] WARN : Failed
to get multiscan result
java.util.concurrent.ExecutionException:
com.esotericsoftware.kryo.KryoException: Buffer too small:
capacity: 1, required: 4
at
org.apache.accumulo.tserver.TabletServer$ScanTask.get(TabletServer.java:795)
at
org.apache.accumulo.tserver.TabletServer$ThriftClientHandler.continueMultiScan(TabletServer.java:1459)
at
org.apache.accumulo.tserver.TabletServer$ThriftClientHandler.continueMultiScan(TabletServer.java:1445)
at
sun.reflect.GeneratedMethodAccessor42.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.accumulo.trace.instrument.thrift.RpcServerInvocationHandler.invoke(RpcServerInvocationHandler.java:46)
at
org.apache.accumulo.server.util.RpcWrapper$1.invoke(RpcWrapper.java:47)
at com.sun.proxy.$Proxy16.continueMultiScan(Unknown
Source)
at
org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$continueMultiScan.getResult(TabletClientService.java:2278)
at
org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$continueMultiScan.getResult(TabletClientService.java:2262)
at
org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at
org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.accumulo.server.util.TServerUtils$TimedProcessor.process(TServerUtils.java:168)
at
org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:516)
at
org.apache.accumulo.server.util.CustomNonBlockingServer$1.run(CustomNonBlockingServer.java:77)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
at
org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.esotericsoftware.kryo.KryoException: Buffer too
small: capacity: 1, required: 4
at
com.esotericsoftware.kryo.io.Input.require(Input.java:152)
at
com.esotericsoftware.kryo.io.Input.readInt(Input.java:325)
at
org.locationtech.geomesa.features.kryo.KryoBufferSimpleFeature.setBuffer(KryoBufferSimpleFeature.scala:53)
at
org.locationtech.geomesa.accumulo.iterators.KryoLazyFilterTransformIterator.findTop(KryoLazyFilterTransformIterator.scala:98)
at
org.locationtech.geomesa.accumulo.iterators.KryoLazyFilterTransformIterator.next(KryoLazyFilterTransformIterator.scala:82)
at
org.apache.accumulo.core.iterators.system.SourceSwitchingIterator.readNext(SourceSwitchingIterator.java:125)
at
org.apache.accumulo.core.iterators.system.SourceSwitchingIterator.next(SourceSwitchingIterator.java:110)
at
org.apache.accumulo.tserver.Tablet.lookup(Tablet.java:1541)
at
org.apache.accumulo.tserver.Tablet.lookup(Tablet.java:1630)
at
org.apache.accumulo.tserver.TabletServer$ThriftClientHandler$LookupTask.run(TabletServer.java:1155)
at
org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
... 5 more
Is there any idea to solve this problem?
By the way, I tried with Point2Point query and it worked
without any error. So I don't think it is a problem of
accumulo configuration.
Thank you in advance.
Van
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users
|
