Introduction

JavaConnect 2.0beta is available!

JavaConnect is a Smalltalk library that allows to seamlessly interact with Java objects from Smalltalk. The current implementation is available for Visualworks and a version for Pharo is being prepared. Using JavaConnect, a Smalltalk application can access any Java object and send messages to it, just as if it were a Smalltalk object. Similarly, any Smalltalk object can be passed as an argument to a Java method. JavaConnect's implementation relies on a connection between the Smalltalk environment and a standard Java VM environment using Visualworks' DLLCC (or Squeak/Pharo's Alien FFI) and Java's JNI. The Java application thus executes on a regular Java VM and the Smalltalk application executes on the regular Smalltalk VM.

JavaConnect was inspired by the JNIPort for Dolphin and JavaInSt work. The JNIPort now also has a Visualworks version, but we also continue to maintain and develop JavaConnect. Please feel free to explore, test and use this work! We do appreciate to hear from you if you use it, like it or having problems with it! Just drop us a note... (see 'About me').

JavaConnect is released under the MIT Open Source License.

On this webpage:

Download and Get Started!

JavaConnect is written for Visualworks Smalltalk, works with the latest Java versions and has been tested on Mac (PPC and Intel), Linux and Windows. Remark: the JNI library of Java 6 on Mac OS X seems to have a problem... we are looking into the issue. In the meantime, on Mac OS X

Language Integration Basics

Java packages

JavaConnect maps Java packages onto Visualworks' namespaces beneath the JavaWorld root namespace. The java package java.lang.util, for example, is represented by the JavaWorld.java.lang.util namespace.

In addition, the refactoring browser's package tab shows the java packages that are currently available on the classpath of the loaded and running Java VM. These Java packages are not to be confused with the Smalltalk packages and are therefore rooted in JavaWorld label instead of the Local Image label.

Java classes

Regular Java classes are represented using regular Smalltalk classes with the same name and are accessible via the namespace that represents their corresponding Java package. The Java Date class, for example, is accessible as JavaWorld.java.lang.util.Date . Member classes are also represented using regular Smalltalk classes but their name is prefixed with their parent class(es) name(s) and includes the separator '_' between all the names. The Iterator class, member class of the List class in the java.lang.util package, for example, is represented by the JavaWorld.java.lang.util.List_Iterator class in Smalltalk.

The refactoring browser displays the list of classes contained in a Java package in the class tab of the refactoring browser.

Java objects

Java objects can be created by invoking the constructors defined on the class via different variants of the new message. This is explained in the next section. The resulting Java objects can be manipulated like any Smalltalk object. Java objects are, of course, also obtained as return results from the invocation of Java methods. JavaConnect guarantees that identical Java objects are uniquely represented in the Smalltalk image. This means that object comparison of Java objects in Smalltalk works and that the same actual Java object (living in the Java VM) is consistently represented by a unique Java object in the Smalltalk image.

Java objects can be inspected using the regular Smalltalk inspectors.

Java methods

All public and private methods of a Java class are represented as regular Smalltalk methods, except for a pre-defined and automatic conversion from Java's method-name syntax onto Smalltalk's message syntax. This means that the Java methods are available as regular Smalltalk methods. Furthermore, similar to any Smalltalk class, the static methods of the Java class can only be invoked on the class itself, while the normal methods can only be invoked on instance objects of the class.

The method-name conversion scheme is as follows:

Public methods, private methods and constructors are classified in different protocols by the refactoring browser. Instance methods and class (static) methods are displayed in their corresponding tab. In addition, JavaConnect will show (decompiled) Java source code in the Refactoring browser's source code pane whenever it was able to decompile the Java method you are browsing.

Java constructors

The public constructors of a Java class are mapped onto corresponding Smalltalk class methods. The name conversion scheme for constructors is identical to the one for methods, but the name of a constructor is always mapped onto 'new'. This means that zero-argument constructors are mapped onto the new class-method and multiple argument constructors are mapped onto keyworded messages of which the first keyword is prefixed with 'new'. For example, the Java constructor Date(int year, int month, int date) is represented as the Smalltalk class method with selector new_int:int:int: (implemented on the Date class).

The constructors of each Java class are displayed by the refactoring browser as class methods in the 'java native constructors' protocol.

Java fields

Public fields of a Java object are made available by automatically generated accessors and mutators. The accessor and mutator methods are prefixed with 'get_' and 'put_' respectively, following the name of the field. Static fields are accessible via class methods, while instance fields are accessible via instance methods.

Typing

Smalltalk programmers are only confronted with Java's static typing through the names of the messages that are sent to Java objects and classes. As we explained, these names are composed of the simple type names of each method parameter. From within Smalltalk, we are not concerned about the static return type of the method we invoke because JavaConnect always returns an object of the dynamic return type of the method. Of course, we must also ensure to pass the right argument objects to the methods, as the Java VM will throw a typing exception otherwise. But, this is no different from invoking a Smalltalk method.

Conversion of Java values into Smalltalk values, and vice-versa.

As Java makes a difference between native and reference types, JavaConnect also differentiates between these two kinds of values.

All native values (int, float, double, char, ...) are automatically converted into their Smalltalk counterparts and vice-versa. You can thus pass any such Smalltalk value where a native Java value is expected and you will get a Smalltalk value back when a method returns a native type.

All reference values (i.e. class types), are represented in Smalltalk as instances of their corresponding classes. For example, a Java String object will thus not be converted automatically into a Smalltalk String object but will be represented as an instance of the JavaWorld.java.lang.String class.

In order to convert a Java String object into a corresponding Smalltalk String object, you can send the #asSmalltalkValue message to the Java String object. This pre-defined Smalltalk method on the Java String class will return a new Smalltalk String object with the same textual content as the Java String object. Inversely, it is possible to pass a Smalltalk String object where a Java String object is expected. This is because JavaConnect always tries to convert any Smalltalk object being passed to a Java method into a corresponding Java object. This is done by invoking the #asJavaValue message on the Smalltalk object, which should result in an appropriate Java object. JavaConnect includes a predefined implementation for Smalltalk values of String, Boolean, Integer, Character, Array, and many more. Just take a look at the implementors of #asJavaValue.

Developers can also implement custom #asSmalltalkValue and #asJavaValue methods on any Java or Smalltalk class using the prototyping mechanism, which is explained later on.

Interfaces?

JavaConnect does not care about Java interfaces. The only exception being for values of anonymous member classes, which are then represented in Smalltalk as objects of the static return type of the method through which they are obtained. As this static return type can be an interface type, an interface can appear as a Smalltalk class in the image.

Java Exceptions

Exceptions thrown by the Java methods are converted into Smalltalk exceptions and can thus be addressed in exactly the same way as Smalltalk exceptions. Catching a Java java.io.FileNotFound exception is as easy as:

[ ... do something ... ] on: JavaWorld.java.io.FileNotFound do:[:e | ... handle the error ... ]

Adding Smalltalk code and state to Java classes

Smalltalk methods and (instance) variables can be added to any Java class. This makes those methods and variables available to the Smalltalk-side of the communication only, of course. You can easily add these methods to the class via the refactoring browser and even publish them to the store.

Passing Smalltalk objects to Java methods and call backs from Java into Smalltalk

JavaConnect allows to pass any Smalltalk object as an argument to a Java method. Obviously, the Smalltalk class of that object should implement the messages that can be sent by Java. At this time, JavaConnect only supports the passing of Smalltalk objects as arguments to Java if the static type of the parameter to which it is passed is of a Java interface type. JavaConnect will generate a dynamic proxy class in Java to represent the Smalltalk object. Whenever this dynamic proxy receives a message, it will invoke the corresponding Smalltalk method, hence executing a callback. The corresponding method is found by following the same translation scheme from Java method signatures to Smalltalk selectors as is explained above. In a future version, JavaConnect will also allow to pass Smalltalk objects to Java methods where a parameter of a class type is expected, using a similar mechanism involving dynamic class generation.

Smalltalk exceptions in callbacks

A callback from Java into Smalltalk might very well end up triggering a Smalltalk exception. JavaConnect will propagate this exception back into Java and (probably) it will get returned to Smalltalk by the invocation of the Java method that eventually triggered the callback. As such, Java methods might handle the Smalltalk exception but, most importantly, your Smalltalk code can handle exceptions around code that makes a call to Java.

System.out and System.err

JavaConnect redirects the JVM's System.out and System.err printstreams so you can access them from Smalltalk. They are available as: JavaConnect.JavaVM current err and JavaConnect.JavaVM current out

Java VM configurations

When you start JavaConnect, the JVM is started with a set of options that correspond to the command-line options you can pass to the 'java' executable. The most important option of a configuration is the classpath parameter. It determines which Java classes you can load and use. A fresh download of JavaConnect contains some pretty useless configurations with empty classpaths. This means that only the standard Java classes are available. Fortunately, you can create your own configurations very easily.

The class JavaVMOptions has a class-side protocol called 'predefined configurations'. Each method in this protocol defines a configuration. You can add a configuration by defining a new method in this protocol. Upon execution, the method MUST return a JavaVMOptions instance that holds all desirable options for your configuration. Take a look at the existing methods how to name your configuration and how to add options to it. The most important option, the classpath, has a convenience method to set it using a string.

Applications that use JavaConnect provide more configuration examples. Check those out if you need more In the future, we also plan to implement a settings tool.

Screenshots (click to enlarge)

Browsing Java classes and their methods in the standard Smalltalk Refactoring Browser.

The JavaConnect settings tab in the Visualworks settings. In the current version it allows to change the library locations.

JavaConnect loaded in the Visualworks Smalltalk environment !

Some Code Examples

Let's illustrate some usages of JavaConnect with some code examples.

       zipfile := JavaWorld.java.util.zip.ZipFile new_String:'foo.zip'.
       zipfile size inspect.
       entries := zipfile entries.
       [entries hasMoreElements] whileTrue:[Transcript show: entries nextElement]

Applications of JavaConnect

Penumbra

Check out the Penumbra website.

Reasoning over Java code.

JavaConnect was actually developed to enable SOUL to reason over Java programs without reimplementing many of the readily available Java libraries and utilities that exist for this purpose. In particular, we are using the Java libraries Soot and the Eclipse JDT to reason about Java programs from within Smalltalk. JavaConnect makes these Java utilities and libraries available as Smalltalk libraries, and thus makes them directly usable by tools implemented in Smalltalk. Cava is the SOUL extension containing a set of logic rules and all required functionality to use the abovementioned Java libraries.

JavaConnect is a good example to demonstrate the advantages of computational reflection in dynamic programming languages. Check out the presentation that I used during the course on reflection in programming languages.

Comparison with JNIPort

Namespaces in Visualworks

The most visible difference when looking at code excerpts is that JavaConnect integrates the Java class lookup mechanism in Visualworks' namespaces. Addressing a Java class such as org.dmp.javaconnect.Foobar is done by writing JavaWorld.org.dmp.javaconnect.Foobar. In JNIPort, one must write JVM current findClass:'org.dmp.javaconnect.Foobar'. The ongoing port of JavaConnect to Pharo will not feature such a mechanism of course (although Pharo does envision namespaces, if I understood correctly).

Callbacks

Although the overall use of JavaConnect and JNIPort focuses on sending messages from Smalltalk to Java objects, messages can very likely be sent from Java to Smalltalk as well. Whenever a Smalltalk object is passed as an argument to a Java method, the Java method will most likely send messages to the incoming Smalltalk object. JNIPort and JavaConnect feature vastly different mechanisms to achieve this. When a Smalltalk object resides in the argument list of a Java method invocation, JavaConnect will automatically generate a Java proxy class for the Smalltalk object. Any message sent by Java to the Smalltalk object will trigger a callback of Java into Smalltalk, executing the corresponding Smalltalk method. The corresponding Smalltalk method is found using the same translation scheme as the Java method signature <-> Smalltalk selectors scheme explained previously. In version 2.0, this mechanism will only work if the static type of the parameter (to which the argument is assigned) is of a Java interface type. This is because Java limits the generation of dynamic proxies to interface types. However, in an upcoming release, we are working on addressing this limitation such that it works for class types as well.

In contrast, JNIPort does not allow to just pass any Smalltalk object to any Java method. In order to allos callbacks, the Java class should be implemented to allow it. A developer basically needs to setup a significant amount of infrastructure in the Java class itself to achieve this. Callbacks from Java to Smalltalk are not mere Java method calls. In addition, the Smalltalk methods to be called by Java follow a different scheme than the Java signature -> Smalltalk selector translation and also require some infrastructure. However, JNIPort does treat issues with multithreading involved with callbacks from Java occuring in a different thread than the original thread that called to Java, which might lead to a deadlock. We have not yet required to address these issues in our uses of JavaConnect but we should address them in a future version of JavaConnect as they are of course there. Interestingly, we do have the Java garbage collection thread that is calling back to Smalltalk to release the Smalltalk objects passed to Java and no issue has (yet) occurred. It might also be the case that the DLLCC of Visualworks is different than Dolphin's mechanism (which was the original environment for JNIPort).

JVM running in a separate thread

Mac OS X has the (nasty) particularity that any JNI program that wants to use the GUI (AWT/Swing) needs to start the JVM on a different native thread. If your Java program opens a window, you need to tick the 'Threaded JVM' option in JavaConnect's settings to allow this. A downside of this, is that JavaConnect becomes much slower, mostly due to the slow threaded calls of DLLCC. JavaConnect has specifically addressed this issue by integrating with Smalltalk processes for this. As far as I can see, it will require a lot of code changes to make JNIPort support this.

Unary selector naming

A minor difference, but JavaConnect does not append '_null' at the end of the unary selector that represents a parameterless Java method. The Java method length() is simply #length in Smalltalk.

Standard settings

JNIPort has many settings that can be customized by the user. Such settings concern which Java methods are visible from Smalltalk (private, public, ...), which fields, wether Ghost classes are generated, and so on. JavaConnect chose not allow such customization by the user. It generates all methods by default and places them in appropriate categories. 'Ghost classes' (or Smalltalk proxy classes for the Java classes) are generated by default. As such, any JavaConnect installation is guaranteed to have the same settings. I believe this to be more convenient since we are not ommiting parts.

One of the most subtle differences is that, by default, JNIPort does not guarantee that a unique Smalltalk object represents unique Java objects. Unless specifically asked for (on a class-by-class basis), different Smalltalk objects can represent identical Java objects (this option is called 'canonical instances' in JNIPort). The result is that the following test will fail in JNIPort when you do not select the option (the Java method 'returnSameObject_Object' merely returns its argument. Selecting canonical instances does have a performance impact though.

object := (JVM default findClass: 'org.dmp.javaconnect.tests.ClassAndObjectTests') new.
self assert: (object == ((JVM default findClass: #'org.dmp.javaconnect.tests.ClassAndObjectTests') returnSameObject_Object: object))

 

Decompilation and Translation

JavaConnect packages a Java decompiler, which allows you to browse (decompiled) Java source code for the methods from the Refactoring browser. An experimental feature that stems from this is a translator that automatically translates such decompiled Java code to Smalltalk, allowing you to 'shift' a Java method to a Smalltalk method. Of course, this does not work for all Java code and the implementation is bound to improve a lot in future versions of JavaConnect.

Performance

JavaConnect has very few performance enhancements. From version 2.0 onwards, we included a major improvement concerned with (Java) class lookup. This improvement had a considerable impact on overall performance (3x better than before) since every value that is returned from Java needs to be 'wrapped' in its appropriate Smalltalk proxy class, requiring its lookup. However, there are plenty of opportunities left for performance upgrades, such as using the FastCMethodPointers package and optimizing the object registry. We intend to spend some effort in the improvement of performance for the upcoming 2.x versions. In any case, for reference, here are the current performance measurements and comparisons we did.

Using the TimeProfiler, we conducted the following tests and recorded the time needed to execute some code excerpts. We always wrote down the execution time for the process itself, thus excluding the time taken by other processes. For some tests, we executed two similar code excerpts: one where the class name is written 'inline' and one where the class is fetched outside of the profiled code excerpt. The code that is profiled is in bold and the unprofiled code in italics. The difference is due to the fact that class lookup mechanism induces overhead too, although it would be strange to write the code that way in an actual Smalltalk program, of course. Furthermore, each test was run 3 times and we wrote down the average.

Another important item to mind is that JavaConnect always ensures that identical Java objects are represented as identical Smalltalk objects (proxies). In JNIPort, this option is set to false by standard (canonical instances). However, this induces a performance overhead, which is why we also give the timings for JNIPort with canonical instances set to true when appropriate. In our experience, you generally do want a single Java object to be represented by different Smalltalk objects.

Need to know more?

If you're interested to learn about the implementation, or if you want to make adaptations, or anything else: Please contact me

Here is a presentation explaining the basics of JavaConnect

Updated 18 August 2009