Van --
The data point you should use as input to the KNN procedure should
be near:
"coordinates": [116.30435943603516, 39.930667877197266],
not:
"coordinates": [-77.034917, 38.894967]
as that's very far from Beijing* and the algorithm (understandably)
can't find any features within 10 km. (It was right in the XML
output at the bottom of Jim's last reply message.) Could you please
try it again with the query point above?
-- Matt
* Actually downtown Washington DC.
On 07/15/2016 03:51 AM, Le Hong Van wrote:
Hi,
Sorry for not replying for Emilio's email and thank you for
your reply.
I'm using GeoMesa 1.2.3 (geomesa-dist-1.2.3-bin.tar.gz),
Accumulo 1.6.2, Zookeeper 3.4.6 and GeoServer 2.9.0.
All servers are using the same version of GeoMesa.
Ingest command:
$ geomesa ingest -u van -c TDrive -i
GeoMesa -s tdrive -C tdrive geomesa/data/T_Drive.csv
Using GEOMESA_HOME =
/home2/van/geomesa-1.2.3/dist/tools/geomesa-tools-1.2.3
Password (mask enabled)>
Creating schema tdrive
Running ingestion in local mode
Ingesting 1 file with 1 thread
[============================================================]
100% complete 11661601 ingested 2 failed in 00:09:19
Local ingestion complete in 00:09:19
Ingested 11661601 features and failed to
ingest 2 features.
Catalog table scan:
van@GeoMesa> scan -t TDrive
~METADATA_tdrive attributes: []
taxiId:String,dtg:Date,*geom:Point:srid=4326;geomesa.index.dtg='dtg',geomesa.indexes.enabled='records,z3,attr_idx',geomesa.version='8',geomesa.table.sharing='false'
~METADATA_tdrive id: [] \x01
~METADATA_tdrive stats-date: [] 2016-07-08T05:30:51.844Z
~METADATA_tdrive tables.idx.attr.name: [] TDrive_tdrive_attr
~METADATA_tdrive tables.queries.name: [] TDrive_queries
~METADATA_tdrive tables.record.name: [] TDrive_tdrive_records
~METADATA_tdrive tables.z2.name: [] TDrive_tdrive_z2
~METADATA_tdrive tables.z3.name: [] TDrive_tdrive_z3
~METADATA_tdrive version: [] 8
I tried to ingest tdrive data again (with same command) to a
new table and work on the new table. The errors were gone but
I did not get results of the query. In the popup screen, there
is only 1 word " unknown" appeared.
Here is info in GeoServer log:
15 Jul 10:12:21 INFO [geoserver.wps] -
Request: getServiceInfo
15 Jul 10:12:21 WARN [knn.KNearestNeighborSearchProcess] - The
provided data feature collection type may not support geomesa
KNN search: org.geoserver.feature.RetypingFeatureCollection
15 Jul 10:12:24 INFO [geoserver.wps] -
Request: execute
service = WPS
version = 1.0.0
baseUrl = http://localhost:8080/geoserver/
identifier:
value = geomesa:KNearestNeighborSearch
dataInputs:
input[0]:
identifier =
net.opengis.ows11.impl.CodeTypeImpl@4cdeeecb (value:
inputFeatures, codeSpace: null)
data =
"">
identifier =
net.opengis.ows11.impl.CodeTypeImpl@ce675b5 (value:
dataFeatures, codeSpace: null)
reference =
net.opengis.wps10.impl.InputReferenceTypeImpl@4add8c01 (body:
net.opengis.wfs.impl.GetFeatureTypeImpl@71ce8b27 (handle:
null, service: WFS, version: 1.0.0, baseUrl: null,
providedVersion: null, extendedProperties: {}) (maxFeatures:
null, outputFormat: GML2, resultType: <unset>,
traverseXlinkDepth: null, traverseXlinkExpiry: null,
formatOptions: null, metadata: null, startIndex: null,
viewParams: null), encoding: null, href: http://geoserver/wfs,
method: POST, mimeType: text/xml, schema: null)input[0]:
identifier =
net.opengis.ows11.impl.CodeTypeImpl@3bca9388 (value:
numDesired, codeSpace: null)
data =
"">
identifier =
net.opengis.ows11.impl.CodeTypeImpl@22a52b9b (value:
estimatedDistance, codeSpace: null)
data =
"">
identifier =
net.opengis.ows11.impl.CodeTypeImpl@488e8ae6 (value:
maxSearchDistance, codeSpace: null)
data =
"">
responseForm:
rawDataOutput:
identifier =
net.opengis.ows11.impl.CodeTypeImpl@3722ee46 (value: result,
codeSpace: null)
mimeType = text/xml; subtype=wfs-collection/1.0
Jul 15, 2016 10:12:24 AM
org.geoserver.platform.GeoServerExtensions checkContext
WARNING: Extension lookup 'PPIOFactory', but
ApplicationContext is unset.
Jul 15, 2016 10:12:24 AM
org.geoserver.platform.GeoServerExtensions checkContext
WARNING: Extension lookup 'ExtensionFilter', but
ApplicationContext is unset.
Jul 15, 2016 10:12:24 AM
org.geoserver.platform.GeoServerExtensions checkContext
WARNING: Extension lookup 'ExtensionProvider', but
ApplicationContext is unset.
Jul 15, 2016 10:12:24 AM
org.geoserver.platform.GeoServerExtensions checkContext
WARNING: Extension lookup 'ExtensionFilter', but
ApplicationContext is unset.
As you noted, ReTypingFeatureCollection is used so maybe the
layer name was not correct. Could you tell me how I can name the
layer properly.
Thanks,
Van
PS1: I'm interested in how spatiotemporal queries (kNN, window
query, range query...) are processed in GeoMesa but I only found
information in https://github.com/locationtech/geomesa/tree/master/geomesa-process.
Is there any other tutorial about this (e.g., example of input,
output of each query...). Actually, I was not sure which value I
should set to each input field in the kNN query until I got
Jim's email.
PS2: the XML generated by the demo builder
<?xml version="1.0" encoding="UTF-8"?><wps:Execute version="1.0.0" service="WPS" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.opengis.net/wps/1.0.0" xmlns:wfs="http://www.
op
engis.net/wfs" xmlns:wps="http://www.opengis.net/wps/1.0.0" xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:gml="http://www.opengis.net/gml" xmlns:ogc="http://www.opengis.net/ogc" xmlns:wcs="http://www.opengis.net/wcs/1.1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xsi:schemaLocation="http://www.opengis.net/wps/1.0.0 http://schemas.opengis.net/wps/1.0.0/wpsAll.xsd">
<ows:Identifier>geomesa:KNearestNeighborSearch</ows:Identifier>
<wps:DataInputs>
<wps:Input>
<ows:Identifier>inputFeatures</ows:Identifier>
<wps:Data>
<wps:ComplexData mimeType="application/json"><![CDATA[{
"features" : [
{
"id" : "0",
"geometry" : {
"coordinates": [-77.034917, 38.894967],
"type" : "Point"
},
"type" : "Feature",
"properties" : {
"dtg" : "2014-05-17T15:33:16.000+0000",
}
}
],
"type" : "FeatureCollection"
}]]></wps:ComplexData>
</wps:Data>
</wps:Input>
<wps:Input>
<ows:Identifier>dataFeatures</ows:Identifier>
<wps:Reference mimeType="text/xml" xlink:href="" moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="http://geoserver/wfs">"http://geoserver/wfs" method="POST">
<wps:Body>
<wfs:GetFeature service="WFS" version="1.0.0" outputFormat="GML2" xmlns:it.geosolutions="http://www.geo-solutions.it">
<wfs:Query typeName="it.geosolutions:tdrive4"/>
</wfs:GetFeature>
</wps:Body>
</wps:Reference>
</wps:Input>
<wps:Input>
<ows:Identifier>numDesired</ows:Identifier>
<wps:Data>
<wps:LiteralData>5</wps:LiteralData>
</wps:Data>
</wps:Input>
<wps:Input>
<ows:Identifier>estimatedDistance</ows:Identifier>
<wps:Data>
<wps:LiteralData>1000</wps:LiteralData>
</wps:Data>
</wps:Input>
<wps:Input>
<ows:Identifier>maxSearchDistance</ows:Identifier>
<wps:Data>
<wps:LiteralData>10000</wps:LiteralData>
</wps:Data>
</wps:Input>
</wps:DataInputs>
<wps:ResponseForm>
<wps:RawDataOutput mimeType="text/xml; subtype=wfs-collection/1.0">
<ows:Identifier>result</ows:Identifier>
</wps:RawDataOutput>
</wps:ResponseForm>
</wps:Execute>
On 7/15/16 5:22 AM, Jim Hughes
wrote:
Hi
Van,
I checked out the KNN WPS today, and was able to get it to
work. I wanted to share some of the details.
We used the GeoServer Demos - WPS Request Builder on
KNearestNeighborSearch. For input features I selected 'Text'
and 'application/json', and entered this block:
{
"features" : [
{
"id" : "0",
"geometry" : {
"coordinates": [-77.034917,
38.894967],
"type" : "Point"
},
"type" : "Feature",
"properties" : {
"dtg" :
"2014-05-17T15:33:16.000+0000",
}
}
],
"type" : "FeatureCollection"
}
From dataFeatures, I entered 'VECTOR_LAYER', I selected the
GeoServer layer for TDrive*. I set numDesired to 5,
estimatedDistance to 1000, and maxSearchDistance to 10000 (the
units are meters**, so this 1km with a bound of 10km). With
all that, I was able to execute the process and get back
results.
To validate, I grabbed the feature IDs and built up a little
query for just those records (it looked like
IN('2594a5f2-f130-458d-94bc-16ebcfcc2755', ...)). In the
OpenLayers preview page, I was able to verify that those 5
points appeared to make the 5 closest.
Anyhow, let us know how it goes for you. Emilio's previous
email about matching versions is likely the right path.
Cheers,
Jim
* NB: The layer has to be named what the SimpleFeatureType is
called. Otherwise, a ReTypingFeatureCollection will be used
and some things will go wrong. This is somewhat subtle issue.
** There are a few subtle points on distance calculations.
Shout if you have questions.
Sample XML generated by the demo builder:
<?xml version="1.0" encoding="UTF-8"?><wps:Execute
version="1.0.0" service="WPS" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://www.opengis.net/wps/1.0.0"
xmlns:wfs="http://www.opengis.net/wfs"
xmlns:wps="http://www.opengis.net/wps/1.0.0"
xmlns:ows="http://www.opengis.net/ows/1.1"
xmlns:gml="http://www.opengis.net/gml"
xmlns:ogc="http://www.opengis.net/ogc"
xmlns:wcs="http://www.opengis.net/wcs/1.1.1"
xmlns:xlink="http://www.w3.org/1999/xlink"
xsi:schemaLocation="http://www.opengis.net/wps/1.0.0
http://schemas.opengis.net/wps/1.0.0/wpsAll.xsd">
<ows:Identifier>geomesa:KNearestNeighborSearch</ows:Identifier>
<wps:DataInputs>
<wps:Input>
<ows:Identifier>inputFeatures</ows:Identifier>
<wps:Data>
<wps:ComplexData
mimeType="application/json"><![CDATA[{
"features" : [
{
"id" : "0",
"geometry" : {
"coordinates":
[116.30435943603516, 39.930667877197266],
"type" : "Point"
},
"type" : "Feature",
"properties" : {
"dtg" :
"2014-05-17T15:33:16.000+0000",
}
}
],
"type" : "FeatureCollection"
}]]></wps:ComplexData>
</wps:Data>
</wps:Input>
<wps:Input>
<ows:Identifier>dataFeatures</ows:Identifier>
<wps:Reference mimeType="text/xml" xlink:href=""
moz-do-not-send="true" class="moz-txt-link-rfc2396E"
href="http://geoserver/wfs">"http://geoserver/wfs"
method="POST">
<wps:Body>
<wfs:GetFeature service="WFS" version="1.0.0"
outputFormat="GML2" xmlns:it.geosolutions="http://www.geo-solutions.it">
<wfs:Query
typeName="it.geosolutions:tdrive"/>
</wfs:GetFeature>
</wps:Body>
</wps:Reference>
</wps:Input>
<wps:Input>
<ows:Identifier>numDesired</ows:Identifier>
<wps:Data>
<wps:LiteralData>5</wps:LiteralData>
</wps:Data>
</wps:Input>
<wps:Input>
<ows:Identifier>estimatedDistance</ows:Identifier>
<wps:Data>
<wps:LiteralData>1000</wps:LiteralData>
</wps:Data>
</wps:Input>
<wps:Input>
<ows:Identifier>maxSearchDistance</ows:Identifier>
<wps:Data>
<wps:LiteralData>10000</wps:LiteralData>
</wps:Data>
</wps:Input>
</wps:DataInputs>
<wps:ResponseForm>
<wps:RawDataOutput mimeType="application/json">
<ows:Identifier>result</ows:Identifier>
</wps:RawDataOutput>
</wps:ResponseForm>
</wps:Execute>
On 07/11/2016 10:06 AM, Emilio Lahr-Vivaz wrote:
Hi Van,
As a first step, could you double check that you have the
same geomesa jar version deployed everywhere in your
cluster? (e.g. accumulo, geoserver, ingest, etc). Also,
which geomesa version are you using?
As a second step, could you scan the geomesa catalog table
for the schema that is failing, and send us the output? That
should allow us to re-create the bug in a unit test. Also,
could you send the XML that's getting sent to geoserver for
the request?
Thanks,
Emilio
On 07/09/2016 05:59 AM, Le Hong Van wrote:
Hello,
I'm trying to run KNearestNeighborSearch in
geomesa-process (https://github.com/locationtech/geomesa/tree/master/geomesa-process)
using "WPS request builder" on GeoServer but I got below
error:
Request: getServiceInfo
09 Jul 18:43:28 WARN [index.RecordIdxStrategy] - Running
full table scan for schema tdrive with filter BBOX(geom,
116.30435943603516,39.930667877197266,116.30470275878906,39.93083953857422)
09 Jul 18:43:30 WARN
[impl.TabletServerBatchReaderIterator] - Error on server
hadoop120:9997
org.apache.accumulo.core.client.impl.AccumuloServerException:
Error on server hadoop120:9997
at
org.apache.accumulo.core.client.impl.TabletServerBatchReaderIterator.doLookup(TabletServerBatchReaderIterator.java:723)
at
org.apache.accumulo.core.client.impl.TabletServerBatchReaderIterator$QueryTask.run(TabletServerBatchReaderIterator.java:376)
at
org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at
org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
at
org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.TApplicationException:
Internal error processing continueMultiScan
at
org.apache.thrift.TApplicationException.read(TApplicationException.java:111)
at
org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
at
org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.recv_continueMultiScan(TabletClientService.java:340)
at
org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.continueMultiScan(TabletClientService.java:326)
at
org.apache.accumulo.core.client.impl.TabletServerBatchReaderIterator.doLookup(TabletServerBatchReaderIterator.java:691)
... 7 more
Log in the server that had error:
2016-07-09 18:43:30,460 [tserver.TabletServer] WARN :
Failed to get multiscan result
java.util.concurrent.ExecutionException:
com.esotericsoftware.kryo.KryoException: Buffer too small:
capacity: 1, required: 4
at
org.apache.accumulo.tserver.TabletServer$ScanTask.get(TabletServer.java:795)
at
org.apache.accumulo.tserver.TabletServer$ThriftClientHandler.continueMultiScan(TabletServer.java:1459)
at
org.apache.accumulo.tserver.TabletServer$ThriftClientHandler.continueMultiScan(TabletServer.java:1445)
at
sun.reflect.GeneratedMethodAccessor42.invoke(Unknown
Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at
java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.accumulo.trace.instrument.thrift.RpcServerInvocationHandler.invoke(RpcServerInvocationHandler.java:46)
at
org.apache.accumulo.server.util.RpcWrapper$1.invoke(RpcWrapper.java:47)
at
com.sun.proxy.$Proxy16.continueMultiScan(Unknown Source)
at
org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$continueMultiScan.getResult(TabletClientService.java:2278)
at
org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$continueMultiScan.getResult(TabletClientService.java:2262)
at
org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at
org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.accumulo.server.util.TServerUtils$TimedProcessor.process(TServerUtils.java:168)
at
org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:516)
at
org.apache.accumulo.server.util.CustomNonBlockingServer$1.run(CustomNonBlockingServer.java:77)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
at
org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.esotericsoftware.kryo.KryoException: Buffer
too small: capacity: 1, required: 4
at
com.esotericsoftware.kryo.io.Input.require(Input.java:152)
at
com.esotericsoftware.kryo.io.Input.readInt(Input.java:325)
at
org.locationtech.geomesa.features.kryo.KryoBufferSimpleFeature.setBuffer(KryoBufferSimpleFeature.scala:53)
at
org.locationtech.geomesa.accumulo.iterators.KryoLazyFilterTransformIterator.findTop(KryoLazyFilterTransformIterator.scala:98)
at
org.locationtech.geomesa.accumulo.iterators.KryoLazyFilterTransformIterator.next(KryoLazyFilterTransformIterator.scala:82)
at
org.apache.accumulo.core.iterators.system.SourceSwitchingIterator.readNext(SourceSwitchingIterator.java:125)
at
org.apache.accumulo.core.iterators.system.SourceSwitchingIterator.next(SourceSwitchingIterator.java:110)
at
org.apache.accumulo.tserver.Tablet.lookup(Tablet.java:1541)
at
org.apache.accumulo.tserver.Tablet.lookup(Tablet.java:1630)
at
org.apache.accumulo.tserver.TabletServer$ThriftClientHandler$LookupTask.run(TabletServer.java:1155)
at
org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
... 5 more
Is there any idea to solve this problem?
By the way, I tried with Point2Point query and it worked
without any error. So I don't think it is a problem of
accumulo configuration.
Thank you in advance.
Van
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password,
or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users
|
