[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [geomesa-users] geomesa-accumulo - Ingesting files with command line without editing application.conf
|
You shouldn't need to build anything custom. Since you're using 1.9.3,
you wouldn't want to run the install-dependencies script with the
default values, you would want to update the versions at the top to
match your install, but you should be able to just use the default
ACCUMULO_HOME config. I've been running geomesa 3 against various
accumulo 1.9.3 instances and not seen any problems, so I would guess
it's something configuration-related. Is it possible that there's an old
geomesa jar in the accumulo lib dir? I'd recommend using the accumulo
namespace classpath isolation[1]. If there is a geomesa jar in the
accumulo lib dir, that could conflict though. And it's possible that
there are some classpath issues if you don't use the namespace install,
as I don't think we generally test that anymore.
Thanks,
Emilio
[1]:
https://www.geomesa.org/documentation/stable/user/accumulo/install.html#namespace-install
On 7/30/20 4:32 PM, David Boyd wrote:
Emilio:
So I think I found at least some of the issue.
I am using the available geomesa-accumulo-dist for 3.0.0 from the
download directory.
I am running on my cluster:
hadoop 2.9.2
accumulo 1.9.3
If I run with the jars in my HADOOP_HOME and ACCUMULO_HOME I get the
error.
If I run the install-dependencies with the default accumulo and hadoop
versions (2.0.0 for accumulo
and 2.8.5 for hadoop)
the same error occurs but on my tablet server.
Does this mean I need to build a custom version of the accumulo-dist
from source specifically
with my clusters hadoop and accumulo (and spark) versions?
On 7/30/2020 3:42 PM, Emilio Lahr-Vivaz wrote:
Hello,
I don't think that the converter is the issue - are you possibly on
Accumulo 1.6? There appears to be an error checking the table
namespace, which I believe was only added in 1.7 (which is the
minimum supported version for GeoMesa). Otherwise, check your cluster
logs, there should be more details on the error there.
Thanks,
Emilio
On 7/30/20 3:06 PM, David Boyd wrote:
All:
I have a small issue. My geomesa-accumulo is managed by puppet.
So I can't edit
the baseline application.conf file in the ${GEOMESA_HOME}/conf dir.
I tried putting the converter definition and sft definition in files
and referencing them from the command line.
But either I have the syntax/format wrong or something else.
Here is my command I am running:
[root@dspcnode11 jwacdata]# geomesa-accumulo ingest -z dspcnode01
-u xxxxx -p xxxxxxxxx -i dspc -c power -C tranlines.conf -f
tranlines.conf transmission_lines/Transmission_Lines.shp
INFO Creating schema 'transmission-lines'
ERROR org.apache.thrift.TApplicationException: Internal error
processing checkNamespaceClass
org.apache.accumulo.core.client.AccumuloException:
org.apache.thrift.TApplicationException: Internal error processing
checkNamespaceClass
at
org.apache.accumulo.core.client.impl.NamespaceOperationsImpl.testClassLoad(NamespaceOperationsImpl.java:268)
at
org.locationtech.geomesa.accumulo.data.AccumuloDataStore.preSchemaCreate(AccumuloDataStore.scala:166)
at
org.locationtech.geomesa.index.geotools.MetadataBackedDataStore.createSchema(MetadataBackedDataStore.scala:136)
at
org.locationtech.geomesa.index.geotools.MetadataBackedDataStore.createSchema(MetadataBackedDataStore.scala:40)
at
org.locationtech.geomesa.tools.ingest.AbstractConverterIngest.run(AbstractConverterIngest.scala:31)
at
org.locationtech.geomesa.tools.ingest.IngestCommand$$anonfun$execute$2.apply(IngestCommand.scala:106)
at
org.locationtech.geomesa.tools.ingest.IngestCommand$$anonfun$execute$2.apply(IngestCommand.scala:105)
at scala.Option.foreach(Option.scala:257)
at
org.locationtech.geomesa.tools.ingest.IngestCommand$class.execute(IngestCommand.scala:105)
at
org.locationtech.geomesa.accumulo.tools.ingest.AccumuloIngestCommand.execute(AccumuloIngestCommand.scala:24)
at
org.locationtech.geomesa.tools.Runner$class.main(Runner.scala:28)
at
org.locationtech.geomesa.accumulo.tools.AccumuloRunner$.main(AccumuloRunner.scala:19)
at
org.locationtech.geomesa.accumulo.tools.AccumuloRunner.main(AccumuloRunner.scala)
Caused by: org.apache.thrift.TApplicationException: Internal error
processing checkNamespaceClass
at
org.apache.thrift.TApplicationException.read(TApplicationException.java:111)
at
org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
at
org.apache.accumulo.core.client.impl.thrift.ClientService$Client.recv_checkNamespaceClass(ClientService.java:1022)
at
org.apache.accumulo.core.client.impl.thrift.ClientService$Client.checkNamespaceClass(ClientService.java:1005)
at
org.apache.accumulo.core.client.impl.NamespaceOperationsImpl$4.execute(NamespaceOperationsImpl.java:252)
at
org.apache.accumulo.core.client.impl.NamespaceOperationsImpl$4.execute(NamespaceOperationsImpl.java:249)
at
org.apache.accumulo.core.client.impl.ServerClient.executeRaw(ServerClient.java:83)
at
org.apache.accumulo.core.client.impl.NamespaceOperationsImpl.testClassLoad(NamespaceOperationsImpl.java:249)
... 12 more
Here is the sft file content:
"transmission-lines" = {
attributes = [
{ name = "fid", type = "String", index = false }
{ name = "inferred", type = "String", index = true }
{ name = "naics_code", type = "String", index = true }
{ name = "naics_desc", type = "String", index = false }
{ name = "owner", type = "String", index = true }
{ name = "shape_length", type = "Double", index = false }
{ name = "source", type = "String", index = true }
{ name = "sourcedate", type = "Date", index = true }
{ name = "status", type = "String", index = true }
{ name = "sub_1", type = "String", index = true }
{ name = "sub_2", type = "String", index = true }
{ name = "type", type = "String", index = true }
{ name = "val_date", type = "Date", index = true }
{ name = "val_method", type = "String", index = true }
{ name = "volt_class", type = "String", index = true }
{ name = "voltage", type = "Double", index = true }
{ name = "geom", type = "LineString", index =
true, srid = 4326, default = true }
]
}
}
Here is the converter:
"transmission-shp" = {
type = "shp"
id-field = "$0"
fields = [
{ name = "inferred", transform =
"INFERRED" }
{ name = "naics_code", transform =
"NAICS_CODE" }
{ name = "naics_desc", transform =
"NAICS_DESC" }
{ name = "owner", transform = "OWNER" }
{ name = "shape_length", transform =
"SHAPE_Leng" }
{ name = "source", transform =
"SOURCE" }
{ name = "sourcedate", transform =
"SOURCEDATE" }
{ name = "status", transform =
"STATUS" }
{ name = "sub_1", transform = "SUB_1" }
{ name = "sub_2", transform = "SUB_2" }
{ name = "type", transform = "TYPE" }
{ name = "val_date", transform =
"VAL_DATE" }
{ name = "val_method", transform =
"VAL_METHOD" }
{ name = "volt_class", transform =
"VAL_CLASS" }
{ name = "voltage", transform =
"VOLTAGE" }
{ name = "geom", transform =
"multilinestring(the_geom)" }
]
}
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxx
To unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/geomesa-users
