Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Equinox » Equinox and JNI(How does Equinox finds the native libraries)
Equinox and JNI [message #632316] Tue, 12 October 2010 14:37 Go to next message
Benoît Thiébault is currently offline Benoît ThiébaultFriend
Messages: 11
Registered: October 2010
Junior Member
Hi everyone,

I am trying to understand how Equinox works with native applications.
I have created a HelloWorld application in java that calls a print() method written in C. All this displays a wonderful HelloWorld message on screen.

The Java code looks like this:
package org.test.jni;

public class HelloWorld {
   public native void nativePrint();

  static {
	System.loadLibrary("HelloWorld");
    }

   public static void main(String[] args) {
	(new HelloWorld()).start();
   }

   public void start() {
        System.out.println("java.library.path: " + System.getProperty("java.library.path"));
	nativePrint();
   }

   public void stop() {
	System.out.println("Goodbye");
   }
}


The C file like that:
#include <jni.h>
#include <stdio.h>
#include "org_test_jni_HelloWorld.h"

JNIEXPORT void JNICALL 
Java_org_test_jni_HelloWorld_nativePrint(JNIEnv *env, jobject obj)
{
    printf("Hello World!\n");
    return;
}


I packaged my OSGi application with Maven Bundle Plugin and it works perfectly.

<plugin>
	<groupId>org.apache.felix</groupId>
	<artifactId>maven-bundle-plugin</artifactId>
	<version>1.4.3</version>
	<extensions>true</extensions>
	<configuration>
		<instructions>
			<Bundle-SymbolicName>${pom.artifactId}</Bundle-SymbolicName>
			<Private-Package>org.test.jni</Private-Package>
			<Bundle-Activator>org.test.jni.Activator</Bundle-Activator>
			<Import-Package>org.osgi.framework</Import-Package>
			<Export-Package></Export-Package>
			<Include-Resource>
				{maven-resources},
				lib=src/main/c/lib
                    </Include-Resource>
			<Bundle-NativeCode>
				lib/libHelloWorld.dylib;
				osname=MacOSX
                       </Bundle-NativeCode>
		</instructions>
	</configuration>
</plugin>


I noticed that my jar bundle is stored in the configuration/org.eclipse.osgi/bundles/1/1/ configuration directory (its the only bundle loaded as you can see).

I also noticed that the native library (libHelloWorld.jnilib) is stored in the configuration/org.eclipse.osgi/bundles/1/1/.cp/lib directory.

Everything works fine. However, when I print the java.library.path property, the path to the native library does not appear.

How does Equinox locates native libs? I ask this because I have an UnsatisfiedLinkError with a more complex library and I would like to check if the path is correctly set to this lib.

Kind regards

Ben

[Updated on: Tue, 12 October 2010 14:38]

Report message to a moderator

Re: Equinox and JNI [message #632384 is a reply to message #632316] Tue, 12 October 2010 17:59 Go to previous messageGo to next message
BJ Hargrave is currently offline BJ HargraveFriend
Messages: 60
Registered: July 2009
Member
See http://download.oracle.com/javase/1.5.0/docs/api/java/lang/C lassLoader.html#findLibrary(java.lang.String)

The bundle's class loader is called by the VM when a class does System.loadLibrary and can return the path to the jni lib.

[Updated on: Tue, 12 October 2010 18:00]

Report message to a moderator

Re: Equinox and JNI [message #632411 is a reply to message #632384] Tue, 12 October 2010 19:42 Go to previous messageGo to next message
Benoît Thiébault is currently offline Benoît ThiébaultFriend
Messages: 11
Registered: October 2010
Junior Member
Does Equinox overrides this method?
We I look at the source code of this method, I get:
/**
     * Returns the absolute path name of a native library.  The VM invokes this
     * method to locate the native libraries that belong to classes loaded with
     * this class loader. If this method returns <tt>null</tt>, the VM
     * searches the library along the path specified as the
     * "<tt>java.library.path</tt>" property.  </p>
     *
     * @param  libname
     *         The library name
     *
     * @return  The absolute path of the native library
     *
     * @see  System#loadLibrary(String)
     * @see  System#mapLibraryName(String)
     *
     * @since  1.2
     */
    protected String findLibrary(String libname) {
        return null;
    }


As the java.library.path property does not point to my bundle lib, I guess in Equinox, this method is overriden in order not to return null.

Report message to a moderator

Re: Equinox and JNI [message #632450 is a reply to message #632411] Wed, 13 October 2010 01:46 Go to previous messageGo to next message
BJ Hargrave is currently offline BJ HargraveFriend
Messages: 60
Registered: July 2009
Member
Of course it does Smile Equinox implements class loaders which support the OSGi specification including native code support.

Report message to a moderator

Re: Equinox and JNI [message #632755 is a reply to message #632450] Thu, 14 October 2010 07:37 Go to previous messageGo to next message
Benoît Thiébault is currently offline Benoît ThiébaultFriend
Messages: 11
Registered: October 2010
Junior Member
Ok, thank you for your answer.

Now here is the problem I have with my native lib.
I included in the jar bundle all jnilib files of my library. I check with "jar tvf mybundle.jar" and they are where they are supposed to be (in the lib/osx64b directory).

I have declared these libs in the <Bundle-NativeCode> configuration of Maven Bundle Plugin and find these declarations in the MANIFEST.MF file.

However, when I install and start the bundle in Equinox, I have an UnsatisfiedLinkError exception. When I check the configuration/org.eclipse.osgi/bundles/1/1/.cp directory, my libs are not there, which explains the UnsatisfiedLinkError.

How come Equinox does not extract my native libs from the jar? Does Equinox first checks that no .jnilib is missing and in that case does not extract any of them?

Report message to a moderator

Re: Equinox and JNI [message #632829 is a reply to message #632755] Thu, 14 October 2010 12:28 Go to previous messageGo to next message
BJ Hargrave is currently offline BJ HargraveFriend
Messages: 60
Registered: July 2009
Member
Are you using the right extension for Mac OS X? The vm on OS X seems to require the file extension be .jnilib. The example above:

<Bundle-NativeCode>
lib/libHelloWorld.dylib;
osname=MacOSX
</Bundle-NativeCode>

did not use the proper .jnilib extension.

Report message to a moderator

Re: Equinox and JNI [message #632956 is a reply to message #632829] Thu, 14 October 2010 19:19 Go to previous messageGo to next message
Benoît Thiébault is currently offline Benoît ThiébaultFriend
Messages: 11
Registered: October 2010
Junior Member
In the HelloWorld example, I indeed used dylib files first and then switched to jnilib files. I forgot to update my post.

I switched to Linux to avoid MacOSX specific problems (I am not comfortable with the double jnilib and dylib library extension names: both are supported by the JVM, but only .jnilib files work with OSGi).

My HelloWorld example works fine (with .so libs now), but when I use another native library, with plenty of lib files (vtk to name it, that has around 50 libs), I come across another problem.

All my .so files are correctly packaged in the jar, but when I load the bundle in Equinox, only one .so file (vtkCommonJava.so) is extracted in the configuration/org.eclipse.osgi/bundles/1/1/.cp directory.

Why OSGi only extracts this lib and not the others (its not even the first in the list)?

Report message to a moderator

Re: Equinox and JNI [message #633160 is a reply to message #632956] Fri, 15 October 2010 14:20 Go to previous messageGo to next message
BJ Hargrave is currently offline BJ HargraveFriend
Messages: 60
Registered: July 2009
Member
OSGi only knows about the native libs listed in the Bundle-NativeCode header. It does not know the platform details to read those native libs and deduce all dependencies.

You can list all the native libs in the Bundle-NativeCode header and OSGi will extract them. But then you will need to get them loaded. Obviously you will need to call System.loadLibrary on the native lib holding the JNI code. But that lib will have dependencies on other native libs and the folder where OSGi extracted the native libs listed in the Bundle-NativeCode header is not on the native library path used by the OS.

This is a difficult problem where there is no easy solution. One can either place those dependent native libs on the OS' native library path a priori or you can list them all in the Bundle-NativeCode header and try to load them "manually" with calls to System.loadLibrary for each native lib. These calls need to be in the right order so that the native dependents of each native lib have already been loaded by a prior call to System.loadLibrary.

The latter technique is not very useful when these dependent native libs need to be shared in the JVM process by other JNI libs. The JVM support for native libraries are not very helpful.

Report message to a moderator

Re: Equinox and JNI [message #633321 is a reply to message #633160] Sat, 16 October 2010 11:47 Go to previous messageGo to next message
Benoît Thiébault is currently offline Benoît ThiébaultFriend
Messages: 11
Registered: October 2010
Junior Member
Quote:

you can list them all in the Bundle-NativeCode header and try to load them "manually" with calls to System.loadLibrary for each native lib. These calls need to be in the right order so that the native dependents of each native lib have already been loaded by a prior call to System.loadLibrary.



Wow, thats not very handy...
Using the OS native library path is not a very good solution either as I want to simplify the application deployment on multiple platform. What OSGi proposes with native libraries declaration in the Bundle-NativeCode header for each platform is a far preferable approach according to me... If it works of course. You have one big bundle containing all libs for the different platforms, but at least, the user can load it without having to think about it.

I tried what you suggested and it indeed seems to work better for most of the libs I load.

I however come across a new exception with one of the libraries depending on jawt.so: UnsatisfiedLinkError looking for libmawt.so.

I am not lucky, its a know bug of the JVM: http://bugs.sun.com/view_bug.do?bug_id=6539705.
So I used their workaround and loaded awt before all the other libraries.

Here is my Activator class, for those interested:
package org.test.vtk;

import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;

public class Activator implements BundleActivator {

	static {
		System.loadLibrary("awt");
		System.loadLibrary("vtksys");
		System.loadLibrary("vtkCommon");
		System.loadLibrary("vtkCommonJava");
		System.loadLibrary("vtkFiltering");
		System.loadLibrary("vtkexpat");
		System.loadLibrary("vtkjpeg");
		System.loadLibrary("vtkzlib");
		System.loadLibrary("vtktiff");
		System.loadLibrary("vtkpng");
		System.loadLibrary("vtksqlite");
		System.loadLibrary("vtkmetaio");
		System.loadLibrary("vtkNetCDF");
		System.loadLibrary("vtkDICOMParser");
		System.loadLibrary("vtkFilteringJava");
		System.loadLibrary("vtkIO");
		System.loadLibrary("vtkIOJava");
		System.loadLibrary("vtkImaging");
		System.loadLibrary("jawt");
		System.loadLibrary("vtkImagingJava");
		System.loadLibrary("vtkverdict");
		System.loadLibrary("vtkGraphics");
		System.loadLibrary("vtkfreetype");
		System.loadLibrary("vtkftgl");
		System.loadLibrary("vtkGraphicsJava");
		System.loadLibrary("vtkRendering");
		System.loadLibrary("vtkRenderingJava");
		try {
			System.loadLibrary("vtkexoIIc");
			System.loadLibrary("vtkParallel");
			System.loadLibrary("vtkHybrid");
			System.loadLibrary("vtkHybridJava");
		} catch (Throwable e) {
			System.out.println("cannot load vtkHybrid, skipping...");
		}
		
		try {
			System.loadLibrary("vtkVolumeRendering");
			System.loadLibrary("vtkVolumeRenderingJava");
		} catch (Throwable e) {
			System.out.println("cannot load vtkVolumeRendering, skipping...");
		}
	}

	TestVTK testVTK;

	public void start(BundleContext context) throws Exception {
		testVTK = new TestVTK();
		testVTK.start();
	}

	public void stop(BundleContext context) throws Exception {
		testVTK.stop();
	}

}


The TestVTK class is a very simple JFrame displaying a cone in 3D. Its constructor is empty, the start method setups and displays the JFrame.

When the bundle is started, it crashes when calling the constructor, with the following error:
org.osgi.framework.BundleException: Exception in org.test.vtk.Activator.start() of bundle test-vtk.
	at org.eclipse.osgi.framework.internal.core.BundleContextImpl.startActivator(BundleContextImpl.java:806)
	at org.eclipse.osgi.framework.internal.core.BundleContextImpl.start(BundleContextImpl.java:755)
	at org.eclipse.osgi.framework.internal.core.BundleHost.startWorker(BundleHost.java:370)
	at org.eclipse.osgi.framework.internal.core.AbstractBundle.start(AbstractBundle.java:284)
	at org.eclipse.osgi.framework.internal.core.AbstractBundle.start(AbstractBundle.java:276)
	at org.eclipse.osgi.framework.internal.core.FrameworkCommandProvider._start(FrameworkCommandProvider.java:252)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.eclipse.osgi.framework.internal.core.FrameworkCommandInterpreter.execute(FrameworkCommandInterpreter.java:155)
	at org.eclipse.osgi.framework.internal.core.FrameworkConsole.docommand(FrameworkConsole.java:156)
	at org.eclipse.osgi.framework.internal.core.FrameworkConsole.runConsole(FrameworkConsole.java:141)
	at org.eclipse.osgi.framework.internal.core.FrameworkConsole.run(FrameworkConsole.java:105)
	at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.UnsatisfiedLinkError: Native Library /usr/lib/jvm/java-6-sun-1.6.0.21/jre/lib/i386/libawt.so already loaded in another classloader
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1768)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1720)
	at java.lang.Runtime.loadLibrary0(Runtime.java:823)
	at java.lang.System.loadLibrary(System.java:1028)
	at sun.security.action.LoadLibraryAction.run(LoadLibraryAction.java:50)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.awt.NativeLibLoader.loadLibraries(NativeLibLoader.java:38)
	at sun.awt.DebugHelper.<clinit>(DebugHelper.java:29)
	at java.awt.Component.<clinit>(Component.java:560)
	at org.test.vtk.Activator.start(Activator.java:58)
	at org.eclipse.osgi.framework.internal.core.BundleContextImpl$1.run(BundleContextImpl.java:783)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.eclipse.osgi.framework.internal.core.BundleContextImpl.startActivator(BundleContextImpl.java:774)
	... 14 more
Nested Exception:
java.lang.UnsatisfiedLinkError: Native Library /usr/lib/jvm/java-6-sun-1.6.0.21/jre/lib/i386/libawt.so already loaded in another classloader
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1768)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1720)
	at java.lang.Runtime.loadLibrary0(Runtime.java:823)
	at java.lang.System.loadLibrary(System.java:1028)
	at sun.security.action.LoadLibraryAction.run(LoadLibraryAction.java:50)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.awt.NativeLibLoader.loadLibraries(NativeLibLoader.java:38)
	at sun.awt.DebugHelper.<clinit>(DebugHelper.java:29)
	at java.awt.Component.<clinit>(Component.java:560)
	at org.test.vtk.Activator.start(Activator.java:58)
	at org.eclipse.osgi.framework.internal.core.BundleContextImpl$1.run(BundleContextImpl.java:783)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.eclipse.osgi.framework.internal.core.BundleContextImpl.startActivator(BundleContextImpl.java:774)
	at org.eclipse.osgi.framework.internal.core.BundleContextImpl.start(BundleContextImpl.java:755)
	at org.eclipse.osgi.framework.internal.core.BundleHost.startWorker(BundleHost.java:370)
	at org.eclipse.osgi.framework.internal.core.AbstractBundle.start(AbstractBundle.java:284)
	at org.eclipse.osgi.framework.internal.core.AbstractBundle.start(AbstractBundle.java:276)
	at org.eclipse.osgi.framework.internal.core.FrameworkCommandProvider._start(FrameworkCommandProvider.java:252)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.eclipse.osgi.framework.internal.core.FrameworkCommandInterpreter.execute(FrameworkCommandInterpreter.java:155)
	at org.eclipse.osgi.framework.internal.core.FrameworkConsole.docommand(FrameworkConsole.java:156)
	at org.eclipse.osgi.framework.internal.core.FrameworkConsole.runConsole(FrameworkConsole.java:141)
	at org.eclipse.osgi.framework.internal.core.FrameworkConsole.run(FrameworkConsole.java:105)
	at java.lang.Thread.run(Thread.java:619)
Nested Exception:
java.lang.UnsatisfiedLinkError: Native Library /usr/lib/jvm/java-6-sun-1.6.0.21/jre/lib/i386/libawt.so already loaded in another classloader
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1768)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1720)
	at java.lang.Runtime.loadLibrary0(Runtime.java:823)
	at java.lang.System.loadLibrary(System.java:1028)
	at sun.security.action.LoadLibraryAction.run(LoadLibraryAction.java:50)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.awt.NativeLibLoader.loadLibraries(NativeLibLoader.java:38)
	at sun.awt.DebugHelper.<clinit>(DebugHelper.java:29)
	at java.awt.Component.<clinit>(Component.java:560)
	at org.test.vtk.Activator.start(Activator.java:58)
	at org.eclipse.osgi.framework.internal.core.BundleContextImpl$1.run(BundleContextImpl.java:783)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.eclipse.osgi.framework.internal.core.BundleContextImpl.startActivator(BundleContextImpl.java:774)
	at org.eclipse.osgi.framework.internal.core.BundleContextImpl.start(BundleContextImpl.java:755)
	at org.eclipse.osgi.framework.internal.core.BundleHost.startWorker(BundleHost.java:370)
	at org.eclipse.osgi.framework.internal.core.AbstractBundle.start(AbstractBundle.java:284)
	at org.eclipse.osgi.framework.internal.core.AbstractBundle.start(AbstractBundle.java:276)
	at org.eclipse.osgi.framework.internal.core.FrameworkCommandProvider._start(FrameworkCommandProvider.java:252)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.eclipse.osgi.framework.internal.core.FrameworkCommandInterpreter.execute(FrameworkCommandInterpreter.java:155)
	at org.eclipse.osgi.framework.internal.core.FrameworkConsole.docommand(FrameworkConsole.java:156)
	at org.eclipse.osgi.framework.internal.core.FrameworkConsole.runConsole(FrameworkConsole.java:141)
	at org.eclipse.osgi.framework.internal.core.FrameworkConsole.run(FrameworkConsole.java:105)
	at java.lang.Thread.run(Thread.java:619)


I know that the is one Classloader per bundle and it seems its not good to load the same lib with two classloaders. But in my case, I only have two bundles and I don't think the first one is supposed to load awt...
Framework is launched.

id	State       Bundle
0	ACTIVE      org.eclipse.osgi_3.6.1.R36x_v20100806
1	RESOLVED    test-vtk_2.0.0.SNAPSHOT


I am now stuck here... any idea where this comes from? and how to solve this issue?

Report message to a moderator

Re: Equinox and JNI [message #633741 is a reply to message #632829] Tue, 19 October 2010 08:11 Go to previous messageGo to next message
Benoît Thiébault is currently offline Benoît ThiébaultFriend
Messages: 11
Registered: October 2010
Junior Member
Hi again,

You said:

BJ Hargrave wrote on Thu, 14 October 2010 08:28
The vm on OS X seems to require the file extension be .jnilib.


Where did you get this information?
I run for quite some time now Java applications using the VTK native library and always successfully used .dylib extensions.

I just have problems when I use OSGi, where it seems that OSGi doesn't want to extract .dylib files from the bundle jar.

I find this behaviour surprising, as I have seen on another forum someone that seems to have used both jnilib and dylib (
http://www.mail-archive.com/qt-jambi-interest@trolltech.com/msg00905.html
).

Is this a limitation of Equinox? of OSGi ?

Ben

[Updated on: Tue, 19 October 2010 08:13]

Report message to a moderator

Re: Equinox and JNI [message #633799 is a reply to message #633741] Tue, 19 October 2010 13:32 Go to previous messageGo to next message
BJ Hargrave is currently offline BJ HargraveFriend
Messages: 60
Registered: July 2009
Member
No Real Name wrote on Tue, 19 October 2010 04:11
Hi again,

You said:

BJ Hargrave wrote on Thu, 14 October 2010 08:28
The vm on OS X seems to require the file extension be .jnilib.


Where did you get this information?



I just understand this from some limited empirical evidence. I updated the native code tests in the OSGi Compliance Tests for native code support to include the Mac (as I have a Mac). The JNI sample created by Xcode used the .jnilib extension, so I mimicked that. I don't really have an authoritative source for the required extension for jni libs on Mac OS X.

Report message to a moderator

Re: Equinox and JNI [message #634038 is a reply to message #633799] Wed, 20 October 2010 11:04 Go to previous message
Benoît Thiébault is currently offline Benoît ThiébaultFriend
Messages: 11
Registered: October 2010
Junior Member
Hi,

I read in "OSGi and Equinox" book (http://equinoxosgi.org/) that it is possible to load Bundles as folders.

Has anyone tried this with native libraries?

Report message to a moderator

Previous Topic:Classloading issue for overwritten classes
Next Topic:'Problem determining user request' error on p2 Admin UI
Goto Forum:
  

[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Fri Sep 20 08:48:44 GMT 2024

Powered by FUDForum. Page generated in 0.04743 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top

Sign up to our Newsletter

A fresh new issue delivered monthly