Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geowave-dev] problems with deploy-geowave-to-hdfs.sh

I'm a newbie at this, so please tell me if I'm way off here:

I have the script working now as far as it goes. I commented out the last section where it is changing ownershop to accumulo. I don't have an accumulo user account at this point. I hope this won't cause a problem.

Mostly what I did is everywhere the script does an su <user> -c command

I added logic to first check to see if the user was cloudera-scm. If so, it:

sudo -u <user> command

else the original su line

You can check the script out for this.

There was also a bug in the path it was attemting to create -- at least I think it was a bug because it didn't match the usr/local path:

#ACCUMULO_LIB_DIR="$(determine_accumulo_hdfs_root)/classpath/geowave/$VENDOR_VERSION"
ACCUMULO_LIB_DIR="$(determine_accumulo_hdfs_root)/classpath/geowave-$VENDOR_VERSION"

Please let me know what you think. You can keep the script if you think it will be useful.


I had hoped that getting this jar file deployed would solve this problem that I encounter when doing the ingest-example, see:

http://ngageoint.github.io/geowave/documentation.html#ingest-example

But I still end up getting this accumulo log error:

Error checking object types
	java.lang.ClassNotFoundException: mil.nga.giat.geowave.datastore.accumulo.MergingCombiner
		at org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:175)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
		at org.apache.accumulo.server.client.ClientServiceHandler.checkTableClass(ClientServiceHandler.java:350)
		at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
		at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
		at java.lang.reflect.Method.invoke(Method.java:606)
		at org.apache.accumulo.trace.instrument.thrift.TraceWrap$1.invoke(TraceWrap.java:63)
		at com.sun.proxy.$Proxy22.checkTableClass(Unknown Source)
		at org.apache.accumulo.core.client.impl.thrift.ClientService$Processor$checkTableClass.getResult(ClientService.java:2957)
		at org.apache.accumulo.core.client.impl.thrift.ClientService$Processor$checkTableClass.getResult(ClientService.java:2941)
		at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
		at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
		at org.apache.accumulo.server.util.TServerUtils$TimedProcessor.process(TServerUtils.java:171)
		at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:478)
		at org.apache.accumulo.server.util.TServerUtils$THsHaServer$Invocation.run(TServerUtils.java:231)
		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
		at org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
		at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
		at java.lang.Thread.run(Thread.java:745)


Thanks,
Mike

From: geowave-dev-bounces@xxxxxxxxxxxxxxxx [geowave-dev-bounces@xxxxxxxxxxxxxxxx] on behalf of Freeman, Michael [FreemanM@xxxxxxxxxxxxx]
Sent: Friday, January 29, 2016 9:42 AM
To: geowave-dev@xxxxxxxxxxxxxxxx
Subject: EXT: Re: [geowave-dev] problems with deploy-geowave-to-hdfs.sh


Hi Andy,

Thanks for you quick response. I did "yum remove" the 0.8.7 release and installed the 0.8.9 release. This is what the log files says after having done this:

Accumulo application directory not found in HDFS, tried '/accumulo /apps/accumulo'. Aborting.
Accumulo application directory not found in HDFS, tried '/accumulo /apps/accumulo'. Aborting.
This account is currently not available.
This account is currently not available.
Unable to create /classpath/geowave/0.8.9-cdh5 directory in hdfs. Aborting.

There is a /accumulo HDFS directory that is owned by cloudera-scm. So, other than modifying the deploy script, not sure what else to do at this point.

Also, I am not authorized to view any of the links you sent.

Thanks,
Mike


From: geowave-dev-bounces@xxxxxxxxxxxxxxxx [geowave-dev-bounces@xxxxxxxxxxxxxxxx] on behalf of Andy Spohn [andy@xxxxxxxxxxxxx]
Sent: Friday, January 29, 2016 9:14 AM
To: geowave-dev@xxxxxxxxxxxxxxxx
Subject: EXT: Re: [geowave-dev] problems with deploy-geowave-to-hdfs.sh

Mike,

  If I remember that username correctly it's used when you select the "Single User Install Mode" option during a Cloudera install. We resolved that issue with the following ticket: https://github.com/ngageoint/geowave/pull/424 and if you look in any of the releases after 0.8.7 the fix has already been applied.

If you've got everything else working and you want to stick with 0.8.7 then it's a simple fix to the script. If you're just starting out I'd recommend you uninstall those 0.8.7 RPMs and try the latest 0.8.9 release.

Andy


On Fri, Jan 29, 2016 at 8:55 AM, Freeman, Michael <FreemanM@xxxxxxxxxxxxx> wrote:


Hello,

I must have missed something here: http://ngageoint.github.io/geowave/documentation.html#examples
It says:

# Install GeoWave Accumulo iterator on a host (probably a namenode)
yum --enablerepo=geowave install geowave-0.8.7-cdh5-accumulo

I am running on a namenode. My HDFS location for accumulo is /accumulo. However the /accumulo location is owned by cloudera-scm, a nologin account. So when the deploy-geowave-to-hdfs.sh script tries to su to the supplied account (cloudera-scm) it prompts for a password. There is no password. There is no associated shell. Generally anytime the account is used, it is used via sudo -u, e.g.:

sudo -u cloudera-scm hadoop fs -ls /accumulo

I also tried:

sudo -u cloudera-scm bash

And then executed the script (no args). It prompts for a password.

I can modify the script to get it to work, but I can't be the first person to encounter this. And I want to do this in a sane fashion. Can you tell me how you intend for the user to accomplish this with the cloudera-scm nologin account?

Thanks,
Mike

_______________________________________________
geowave-dev mailing list
geowave-dev@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geowave-dev


Attachment: deploy-geowave-to-hdfs.sh
Description: deploy-geowave-to-hdfs.sh

Back to the top