Dynamic Languages and Java/Bridges in Java
From JVMLanguages
| Table of contents |
|
|
Introduction
As we mentioned in Chapter 3, there are many different kinds of bridges. In this section we will be focusing on both internal bridges and external bridges.
Java-to-C Bridges
No matter how powerful a particular programming language is, it is only a matter of time before you will need to integrate with some component written in C. It may be a part of the operating system that your language does not allow you direct access to (e.g., file locking). Or it may be a device driver or some other library that is distributed as a shared object.
There are two Java-to-C bridges in particular that we will discuss: the Java Native Interface (JNI) and GCJ's Compiled Native Interface (CNI). They provide very similar APIs but there are benefits and drawbacks to each.
Java Native Interface
JNI is a bridge that can be used to communicate between Java code and C or C++ code. JNI provides a set of header files against which you can compile a shared library (a .so on UNIX, .dll on Windows). This shared library must define functions that have the appropriate names and signatures. Adding the native keyword to a Java methods will replace it with a stub that locates the corresponding C function and wraps all input parameters and the return value with structures defined in JNI's header files.
Implementing Native Methods in Java
To implement a Java method in C, you need to do two things. First, you must declare your method using the keyword native and no implementation.
package com.oreilly.javalangint.jni;
public class JNITest {
public native int doSomethingNatively (String str, long l);
}
Next, you will need to define a method in C with the appropriate signature. In this case, the name must be Java_com_oreilly_javalangint_jni_JNITest_doSomethingNatively. The easiest way to do this is to use the javah tool included with your Java Developer Kit to generate a stub. javah takes a compiled Java class as input and generates a C header file.
$ javah -classpath . com.oreilly.javalangint.jni.JNITest
You should now have a file called com_oreilly_javalangint_jni_JNITest.h that contains the following declarations:
/*
* Class: com_oreilly_javalangint_jni_JNITest
* Method: doSomethingNatively
* Signature: (Ljava/lang/String;J)I
*/
JNIEXPORT jint JNICALL Java_com_oreilly_javalangint_jni_JNITest_doSomethingNatively
(JNIEnv *, jobject, jstring, jlong);
The JNI headers provide Java-specific data types for each of Java's primitive types, and for java.lang.String. Each JNI call also receives two additional arguments: JNIEnv and a this pointer. An instance of JNIEnv can be used to call back into the Java interpreter, and to receive various state information. This is described in more detail below. The jobject that is passed into each JNI call is a pointer to the Java object on which the method was invoked. It can be used to look up fields or call other methods (native or non-native).
Maintaining Peer Objects
It is often useful, especially when using an object-oriented C++ API, to have a one-to-one correspondence between Java classes and C++ classes. In this case, each Java object has a corresponding C++ object, and their lifecycles are usually linked. Each call to a native Java method must have some way to locate the C++ peer object. The way this is usually done is to store a reference to the C++ object on the Java object. However, Java has no concept of pointers, so this value is usually stored on a numeric field in Java. int is sometimes used, although since they are only 32-bits, this will cause problems for 64-bit JVM's. long fields are preferred.
It may be tempting to put a finalize method on your Java classes that free their corresponding C++ peer. This effectively lets you not worry about the memory management for your C++ classes. However, bear in mind that finalization in Java can be extremely expensive. If you are creating a large number of objects, there is a good chance that the finalization thread will fall behind. Objects that declare finalize methods must be seen at least twice during garbage collection before their memory can be reclaimed. This will very likely affect performance. Interacting with Java Objects from C
The JNIEnv pointer that is passed into each JNI call can be used as a starting point for many Java operations that you may wish to perform from native code. In C++, you can use the format:
env->FindClass("java/lang/String")
But in C, you must use the following syntax:
(*env)->FindClass(env, "java/lang/String")
The majority of the functions available on JNIEnv parallel those available through Java's java.lang.Class class and java.lang.reflect package. For example, to locate a class by name, use:
jclass FindClass(const char *name);
From here you can locate and interact with fields and methods:
jmethodID GetStaticMethodID(jclass clazz, const char *name, const char *sig);
jmethodID GetMethodID(jclass clazz, const char *name, const char *sig);
jfieldID GetStaticFieldID(jclass clazz, const char *name, const char *sig);
jfieldID GetFieldID(jclass clazz, const char *name, const char *sig);
void CallVoidMethod(jobject obj, jmethodID methodID, ...);
jobject CallObjectMethod(jobject obj, jmethodID methodID, ...);
j... Call...Method(jobject obj, jmethodID methodID, ...);
void CallNonvirtualVoidMethod(jobject obj, jclass clazz, jmethodID methodID, ...);
jobject CallNonvirtualObjectMethod(jobject obj, jclass clazz, jmethodID methodID, ...);
j... CallNonvirtual...Method(jobject obj, jclass clazz, jmethodID methodID, ...);
jobject GetObjectField(jobject obj, jfieldID fieldID);
j... Get...Field(jobject obj, jfieldID fieldID);
void SetObjectField(jobject obj, jfieldID fieldID, jobject val);
void Set...Field(jobject obj, jfieldID fieldID, j... val);
void CallStaticVoidMethod(jclass cls, jmethodID methodID, ...);
jobject CallStaticObjectMethod(jclass clazz, jmethodID methodID, ...);
j... CallStatic...Method(jclass clazz, jmethodID methodID, ...);
jobject GetStaticObjectField(jclass clazz, jfieldID fieldID);
j... GetStatic...Field(jclass clazz, jfieldID fieldID);
void SetStaticObjectField(jclass clazz, jfieldID fieldID, jobject value);
void SetStatic...Field(jclass clazz, jfieldID fieldID, j... value);
There are also functions available to perform Java-specific features such as throwing exceptions and interacting with strings and arrays.
jint Throw(jthrowable obj);
jint ThrowNew(jclass clazz, const char *msg);
void FatalError(const char *msg);
jstring NewString(const jchar *unicode, jsize len);
jsize GetStringLength(jstring str);
const jchar *GetStringChars(jstring str, jboolean *isCopy);
void GetStringRegion(jstring str, jsize start, jsize len, jchar *buf);
void ReleaseStringChars(jstring str, const jchar *chars);
jstring NewStringUTF(const char *utf);
jsize GetStringUTFLength(jstring str);
const char* GetStringUTFChars(jstring str, jboolean *isCopy);
void GetStringUTFRegion(jstring str, jsize start, jsize len, char *buf);
void ReleaseStringUTFChars(jstring str, const char* chars);
jsize GetArrayLength(jarray array);
jobjectArray NewObjectArray(jsize len, jclass clazz, jobject init);
jbooleanArray NewBooleanArray(jsize len);
j...Array New...Array(jsize len);
jobject GetObjectArrayElement(jobjectArray array, jsize index);
void SetObjectArrayElement(jobjectArray array, jsize index, jobject val);
j... * Get...ArrayElements(j...Array array, jboolean *isCopy);
void Release...ArrayElements(j...Array array, j... *elems, jint mode);
void Get...ArrayRegion(j...Array array, jsize start, jsize len, j... *buf);
void Set...ArrayRegion(j...Array array, jsize start, jsize len, const j... *buf);
There are also functions available to instantiate objects and compare them to other objects.
jobject AllocObject(jclass clazz);
jobject NewObject(jclass clazz, jmethodID methodID, ...);
jclass GetObjectClass(jobject obj);
jclass GetSuperclass(jclass sub);
jboolean IsAssignableFrom(jclass sub, jclass sup);
jboolean IsSameObject(jobject obj1, jobject obj2);
jboolean IsInstanceOf(jobject obj, jclass clazz);
Generation of Wrapper Code
If you have an existing C or C++ library that you want to integrate with a Java program, JNI will get you halfway there. JNI essentially provides a way to create proxy methods in Java that will forward on to methods in C. However, you have no control over the name of the functions that Java will look for and the signatures are very Java-centric. For example, JNI will convert a java.lang.String argument into a jstring type, rather than the char * or a std::string that your C/C++ library is expecting. If your C/C++ library was written without Java in mind, this will not be the case. Thus, it is up to you to write a set of wrappers in C that bridge between JNI's naming conventions and those of the library that you're planning to use.
Luckily, there is another tool that can save you a great deal of time. The Simplified Wrapper and Interface Generator (SWIG) is a code generator that can create bridges between any arbitrary C or C++ library and one of many target languages. SWIG is very popular for creating Perl, Python, Ruby, and TCL extensions; however, Java wrappers are also supported. This means that you can take the header files for an ordinary C/C++ library and generate both the C wrapper functions as well as the Java wrapper classes.
SWIG Input Files
SWIG can be given header files directly if no customization is necessary, but generally you will want to execute SWIG on an input file. This input file is allowed to import other input files, including many utility files that come with SWIG itself, and also to specify header files that SWIG will analyze. Proxy Objects
If you're using a C++ library, SWIG will manage the creation of peer objects for you. However, in this case it makes more sense to think of the native objects as the "real" objects and the Java objects as proxies for them. Memory Management
For each Java proxy object, SWIG maintains a flag that indicates whether that proxy object owns the native object that it is associated with. Typically, this flag is set to true whenever you call a wrapped constructor. This is the only case where it is safe for SWIG to assume that it can delete this memory. SWIG has no way to differentiate between accessor methods that return pointers to objects owned by a parent object and factory methods that create new objects and return them. If you have methods like this, you will need to inform SWIG of them as follows:
%newobject CPPClass::factoryMethod;
The native memory owned by a proxy object can be reclaimed by calling its delete() method. This method will not allow you to delete the memory twice -- after the first use the flag is cleared and the memory address of the native peer is thrown away.
By default, SWIG will create finalize methods on your Java proxy classes that will call delete() for you when the Java object is garbage collected. However, if you are creating a large number of wrapper objects, there can be a significant performance penalty to this. You can override the javafinalize typemap to control this.
%typemap("javafinalize") SWIGTYPE %{%}
Arrays
There are two simple ways of accessing C arrays from Java. The first is to map the C array to a Java array. This means that whenever a C array is passed to or returned from a function, all of the elements will be individually converted and copied across JNI.
The second method of accessing C arrays from Java is to wrap the entire array in a generated class that provides a similar API. This is analagous to passing the array by reference -- any attempt to retrieve or set individual elements represents a method call and must cross JNI. Generally, you will want to use this method if you are not going to access every element in the array in each method call.
What we would ideally like is to make the decision of which of these two approaches to use a configuration decision. However, an array and a class containing size() and get(int) methods are used very differently. As an example of how to extend SWIG to support custom mappings, I'll demonstrate how you can implement the List interface in both of these situations.
Do this later as an example?
STL Data Structures
std::vector is usually mapped to a typesafe generated class.
The typesafety was a nice reason to do this prior to Java 1.5, but now with the advent of generics there is no reason not to use the standard Java collection classes. This can be done also.
Smart Pointers
One way to deal with smart pointers is to create Java proxy objects for them. As of version XXX, SWIG will automatically create methods on the proxy object that forward on to the underlying object, but the pointer class and the underlying class are still completely distinct from a typing point of view. You will not be able to pass them around interchangably.
Another approach to dealing with smart pointers is to eliminate them from the Java API entirely, and make SWIG create and destroy them as part of each method call. This can be done with std::auto_ptr and with std::intrusive_ptr but not with std::shared_ptr.
Explain why. It's fascinating. :-)
Enumerations
The most intuitive way to map enumerations into Java is to simply use Java 1.5 enumerations. SWIG supports this via ...
If you're stuck on an earlier version of Java or you have some aversion to enumerations, you can also tell SWIG to create typesafe enumeration classes.
There are other options here, but they suck -- bother mentioning them?
Serialization
Unlike ordinary Java classes that store their data in fields, Java proxy objects simply provide accessor and mutator methods which retrieve data out of a native C++ object. But what if you want to serialize these proxy objects out to a file or send them across a socket to another process? We can modify SWIG to do this for us, ...
You can also integrate with existing serialization libraries, like boost serialization...
GCJ
GCJ is an enhancement to the popular GNU C Compiler to support compiling Java bytecode into machine code. This is useful for a number of reasons:
- Java applications compiled with GCJ do not need to be deployed with a Java Runtime Environment. This means that there is no potential problems with users running the wrong JRE.
- Your application does not need to wait for the HotSpot dynamic compiler to translate all of its critical code segments into machine code -- they are loaded off of disk in that format.
- Perhaps the most important benefit is that the Java Native Interface is no longer relevant. Because your Java code becomes ordinary machine code just like C and C++ code do, GCJ can manage communication between your Java and C/C++ code via something it calls the Compiled Native Interface (CNI). This does not have the same level of overhead that is comes with use of the JNI; thus, language integration with GCJ is far more efficient.
Calling C Functions from Java
Calling Java Methods from C
Cocoa
Cocoa provides a Java to Objective-C bridge. This allows developers to write code in Java that not only looks like a native Aqua application (as the new Aqua Look-and-Feel does), but actually lets us use Mac OS X functionality. For example...
Calling Objective-C Methods from Java
Calling Java Methods from Objective-C
BSFPerl
BSFPerl is a project that provides a Java-to-Perl bridge that can be used with BSF. BSFPerl works with any standard Perl installation (5.0 or greater) and requires no Perl code installed. When invoked, BSFPerl will spawn off a Perl process using Runtime.exec(), stream the necessary Perl bootstrap code via the STDIN stream, and then carry out all further communications on the STDIN and STDOUT streams. STDERR is reserved for output coming from the script itself, and from the script's point of view, STDOUT will also be redirected there.
Invoking BSFPerl
BSFPerl currently has no public interface other than a BSFEngine implementation. Future versions may include a separate API so that JSR-223 can also be supported.
You must have a Perl interpreter installed on your local machine and BSFPerl must be able to find it. This means that the path to perl must either by in your PATH environmental variable (or your operating system's equivalent) or you must set the perl.exe Java system property to the fully-qualified path to the Perl executable.
Interacting with Java objects from Perl
All objects passed into Perl from Java will be available as scalars. Objects implementing java.util.Collection will be translated into a reference to an array. Objects implementing java.util.Map will become a reference to a hashtable. All other objects will become a reference blessed into the BSF::JavaObject package, which acts as a proxy into Java. Any method calls will be forwarded on to Java, which will attempt to call a method with the specified name using reflection.
BSFPerl adds a global function to Perl called import that is capable of importing Java packages into Perl. Package names are specified with Perl's standard package separator (::) in place of Java's package separator (.) because . is not a valid character in a Perl identifier. This means that to import the Java class com.oreilly.javalangint.perl.SomeClass you would use:
import com::oreilly::javalangint::perl::SomeClass;
However, after this point you can refer to SomeClass with any package declaration. There is currently no way to import an entire package.
Once a Java class has been imported, you can call static methods on it. For example:
import java::lang::Math; my $my_sin = Math->sin($my_angle);
You can also instantiate the imported class by calling the new method, which BSFPerl interprets to mean that you want to look for a constructor rather than a static method.
import java::util::Random; my $r = new Random(); my $n1 = $r->nextDouble; my $n2 = $r->nextDouble; my $n3 = $r->nextInt;
Since BSFPerl is an ordinary Perl interpreter, everything else should work as expected. In particular, you can use any Perl modules that are installed in your Perl installation.
Interacting with Perl objects from Java
When objects are passed from Perl into Java, BSFPerl does its best to convert the data types in an intuitive way. BSFPerl will only ever try to pass scalar values, however the scalar value in Perl is used to represent numbers (both integer and floating-point), strings, references to arrays, references to maps (hashtables), and references to objects. BSFPerl uses the following rules to distinguish these cases.
- Is the scalar value a reference to an array? If so, recursively convert the elements of the array and construct a java.util.ArrayList to store the results.
- Is the scalar value a reference to a hashtable? If so, recursively convert the values of the array and construct a java.util.HashMap where the keys are the keys in Perl (these are always a string) and the values are the converted values.
- Is the scalar value a reference to a reference? If so, recursively convert the referent and construct an org.sourceforge.bsfperl.PerlReference object containing the converted object.
- Is the scalar value a reference to an object (i.e., a 'blessed' reference)? If so, construct an org.sourceforge.bsfperl.PerlObject object that can be used (e.g., by BSFEngine.call() to make further method calls.
- Was the reference used most recently as a number [1] ? If so, use a regular expression to distinguish between floating-point and integer numbers. Construct a java.lang.Double or java.lang.Long as appropriate.
- Otherwise construct a java.lang.String representing the string.
BSF Support
As of BSFPerl 0.2 and BSF 2.3.1, BSFPerl will be registered automatically (via an included Languages.properties file). If you are using BSFPerl 0.1 or BSF 2.3.0, you will need to register the engine explicitly.
BSFManager.registerScriptingEngine("bsfperl", "net.sourceforge.bsfperl.PerlEngineImpl", new String[] {"pl"});
Limitations
There is currently no way to access Java fields, either static or instance-based, from Perl. Support for this is planned for a future version.
