It has recently become popular to differentiate between the Java platform and the Java language, but many Java developers still are unsure of how to incorporate scripting into Java application development. In this article, Gregor Roth introduces you to scripting on the Java platform. Learn when and how to use scripts in your Java applications, whether you are gluing together various Java application modules with Groovy or Jython, or writing your first JRuby-based application for the Java platform.

[@more@]

As a Java developer, you may have noticed that the Java language is no longer the sole proprietor of the Java platform. Groovy is also a programming language for the Java platform, and innumerable other languages now have interpreters and compilers to run on top of the JVM. Among these, Jython and JRuby are two of the most popular. In addition, Java SE 6 includes built-in support for script engines, and JSR 223 defines a standard interface to interact with dynamic languages running on the Java platform.

In order to take advantage of the many possibilities opened up by scripting on the Java platform, you should clearly understand the challenges and benefits involved. You need to understand what happens when calling Java classes from scripts written in JRuby or Jython, and vice verse. You also need to be aware of the difficulties of integrating a scripting language into the Java platform, and what impact that will have on your development process. Finally, you should know the different characteristics of at least a handful of popular scripting languages, so that you can evaluate where they might fit into your programs.

This article presents an overview of what differentiates scripting languages such as Groovy, JRuby, and Jython from the Java language, discusses some of the challenges involved in scripting on the Java platform, and introduces you to the various ways you can integrate scripts originally written in Ruby or Python with your Java code, as well as some scenarios where doing so is useful.

Static versus dynamic typing

Most scripting languages are dynamically typed, whereas Java is a static-typed language. Static-typed languages require the type of a variable to be defined at declaration time, and the variable can only be set with the data of the declared type. In contrast, dynamically typed languages do not require the programmer to write explicit type declarations, because the type information is attached to values, not to variables. As you can see in Listing 1 the Groovy-defined variable v1 can be reset with values of different types at any time.

Listing 1. Dynamic typing in Groovy

def v1 = '66'      // sets the variable with a value of type String
println v1.dump()
// prints out:
// 

v1 = 9            // resets the variable with a value of type Integer
println v1.dump()
// prints out:
//

A typical implementation of a dynamically typed scripting language will keep the value of variables tagged with a type. This type will be checked immediately before using any value in an operation. The disadvantage of this approach is the need for extra processing cycles to determine the value type and to perform type checks at runtime. Advocates of static typing point out that using dynamic languages always leads to performance penalties.

Weak versus strong typing

Some popular scripting languages are weakly typed. Weakly typed languages allow operations on incompatible types. To do this, weakly typed languages support implicit type conversion or ad-hoc polymorphism. For example, Groovy allows you to do an add operation on an integer and string value, as shown in Listing 2.

Listing 2. Groovy-based example of weak typing

def v1 = '66'         // set v1 with a String typed value
def v2 = 5            // set v2 with a Integer typed value
def v3 = v1 + v2
println v3
// prints out: 
// 665

Like Groovy, Python is also a dynamically typed language. But in contrast to Groovy, Python is strongly typed, so it won't support the above operation. Strong typing is much less lenient than weak typing, and prevents mixing operations between mismatched types. You can see this in Listing 3.

Listing 3. Python-based example of strongly typing

v1 = '66'              # set v1 with a String typed value
v2 = 5                 # set v2 with a Integer typed value
v3 = v1 + v2
# prints out:
# TypeError: cannot concatenate 'str' and 'int' objects

Using a static, strongly typed language like Java puts the strongest possible constraint on the type of object at declaration time. For example, if you want to implement a callback pattern in Java, you will start by writing a callback interface that defines the callback methods and declares the return type, as well as all the argument and exception types. The concrete callback implementation will be referenced and called using the interface shown in Listing 4.

Listing 4. A callback pattern implementation in Java

// the call back handler definition
        interface IDataHandler {
                public boolean onData(INonBlockingConnection nbc) throws IOException;
        }


        // the server receives data and performs the handler's call back method
        class MultithreadedServer {
                 private volatile boolean isRunning = true;
                 private IDataHandler dataHandler = null;

                // accepts only an object which implements the IDataHandler interface
                MultithreadedServer(IDataHandler dataHandler) {
                        this.dataHandler = dataHandler;
                }


                public void run() {
                       while (isRunning) {
                           // waiting for data
                           // ...
            
                           // ... and dispatch it
                           dataHandler.onData(nbc);
                       }
                }
        }


        // the call back implementation
        class SmtpProtocolHandler implements IDataHandler {
                public boolean onData(INonBlockingConnection nbc) throws IOException {
                     SmtpSession session = (SmtpSession) nbc.getAttachment();
                     //...
                }
        }

        MultithreadedServer server = new MultithreadedServer(new SmtpProtocolHandler());
        server.run();

Does it walk like a duck?

When using scripting languages such as Python, Ruby, or Groovy you don't have to define an interface like the one shown in Listing 4. Variables can hold references to any object type. When you send a message to a target object, the language's runtime checks whether a matching method exists and calls it. Otherwise, an exception is thrown. There is no need for the target object to implement specific interfaces or extending classes. If the method is there, the runtime will call it. This behavior is also called duck typing, which means "If it looks like a duck and quacks like a duck, it must be a duck."

Listing 5. A callback handler written in Groovy

// the call back implementation (which doesn't implement any interface)
        class SmtpProtocolHandler {
                def onData(nbc) {
                     def session = nbc.attachment
                     //... 
                }
        }

        def server = new MultithreadedServer(new SmtpProtocolHandler())
        server.run()

Quality and performance

When you compare the two small pieces of the handler code shown in Listing 4 and Listing 5, it becomes obvious that scripting languages tend to produce code that is more compact and readable than Java code. This is mainly because you don't have to write all the type declarations in Groovy or JRuby that you do in Java, which is a static, strongly typed language. On the other hand, the missing type information has some disadvantages. Supporters of static typing argue that static, strongly typed languages ensure the robustness of the program, by ensuring that type errors will be detected at compile time. Advocates of dynamic languages argue, conversely, that development techniques like test driven development compensate for the advantages of compile-time checks.

In general, scripting languages tend to run slower than system programming languages , but runtime performance isn't the only issue under consideration. The question is always whether the implementation is fast enough for the target infrastructure. Besides the speed requirements, other quality requirements like reliability and changeability have to be considered. For instance, it is not unusual for maintenance cost to become the biggest part of the total software lifecycle cost. The key to reducing maintenance cost is high reliability and simplicity. Scripting languages, being less complex, often yield better results in these areas than system programming languages like C++ or Java.

Approaches to scripting on the Java platform

Today you do not have to choose between using Java and using a scripting language like Groovy, Ruby, or Python. Your applications can benefit from both the productivity and elegance of scripting languages and the reliability of the Java platform. The key to scripting on the Java platform is knowing where a scripting language is your best choice, and where you are better off using Java code.

In the past, scripting languages have often been seen as a thin layer of glue code to string Java components together. Many Java developers today use scripting languages for most of their work on the Java platform, only relying on the Java libraries for features not supported by their scripting language of choice. (For instance, the Java platform provides many enterprise-level features that are not supported by most scripting language environments, such as transaction management, remoting, or monitoring.)

Regardless of how you approach it, the seamless integration of scripting languages and the Java platform produces a richer development environment, where you are able to choose the right language for the right task.

There are two approaches to realizing what is sometimes called a "polyglot" development environment on the Java platform: You can either run your scripting language on the top of the Java virtual machine, or use the Java Native Interface/inter-process communication to execute the scripting language within a native scripting environment.

Most runtimes of popular scripting languages are C/C++-based. Using JNI allows you to connect your Java environment to the native scripting environment. For Ruby you might use a JNI-based solution like RJB, or an inter-process solution like YAJB. On the downside, most bridging solutions present undesirable restrictions. For instance, inter-process-based solutions use a remote protocol to connect the environments, which can result in performance bottlenecks.

Java-based scripting runtime implementations

Open source projects such as Jython or JRuby have been realized as pure Java-based scripting runtime implementations, enabling you to execute Python or Ruby scripts on the top of the Java virtual machine. That means scripts written in Python or Ruby run on all platforms where a Java SE runtime exists. Java-based scripting runtimes represent the first step toward integrating the scripting languages into the Java platform, and do not offer as much as you might hope for.

Because the original Ruby or Python scripting runtimes are currently faster then the Java-based ones, using the native runtime would still be the first choice to process such scripts. The real value of languages like JRuby and Jython is that they can call Java classes and vice versa. This opens the whole world of Java to scripting languages. The scripting languages have access to everything that is implemented in Java.

From the infrastructure view JRuby or Jython can been seen as Java-based scripting runtimes to execute regular Ruby or Python scripts on top of the Java platform. From the developer's view JRuby or Jython scripts can been seen as enriched scripts that use Java classes and require additional capabilities of the scripting runtime. As consequence, these "J scripts" can not be executed on the native scripting runtime.

Embedding scripts into Java

To run Python, Ruby, or Groovy on the top of the Java virtual machine, you only need to add the jars of the Java-based scripting runtimes to your Java classpath. After this small setup the scripting engine can be instantiated and you can execute scripts within your Java environment. In most cases, the scripting implementation will provide simple engine classes to do this, as shown in Listing 6.

Listing 6. Runtime engines for Ruby, Groovy, and Python

// run a Ruby scripting  (JRuby V1.1b1)
Ruby runtime = Ruby.getDefaultInstance();
runtime.executeScript(scripting, filename);

// run a Groovy scripting (Groovy V1.1)
GroovyShell gs = new GroovyShell();
gs.evaluate(scripting);

// run a Python scripting (jython V2.2)
PythonInterpreter interp = new PythonInterpreter();
interp.exec(scripting)

Most scripting engines also allow you to bind host variables of the Java environment to your script, as well as invoking specific scripting functions. Please note that some scripting engines require additional settings to use extended features. For example system properties like jruby.home and jruby.lib have to be set within JRuby to call gems.

With the release of Java SE 6, a standard interface to host scripting engines has been established as integral part of the Java runtime. JSR 223: Scripting for the Java Platform has standardized functionality like script-engine discovery, binding for Java host variables, and script invocation. The JSR 223 interface requires a JSR 223-compatible scripting implementation. Implementations of major scripting languages can be downloaded from the scripting project homepage.

Calling Java classes from scripts

Referencing Java classes within a script requires importing the Java classes before using them. For instance, JRuby defines a special statement include Java to access the built-in Java classes of the Java runtime. Other, non-bundled classes have to be prefixed with Java::. As you can see in Listing 7 the first statements of the JRuby script activate the Java support and define constants to shorten the path of the SSLContext class and the BlockingConnection class.

In contrast to JRuby, scripting runtimes like Jython use existing, native script statements to import Java classes. Like Python modules, Java packages can be imported by using the ordinary Python statement import . Jython also supports the different variants of Python's import statement, such as from import . Groovy, which is explicitly designed to run on the Java platform, uses Java's import syntax.

Compact syntax means less code

The syntax of scripting languages is often more compact than the syntax of Java. For instance, neither Groovy, Ruby, nor Python requires you to write verbose getters and setters. To simplify JavaBeans property handling, the Java runtime of such scripting languages allows you to address the property in a direct, script-like way.

A Java method like can also be addressed by calling . Additionally, the JRuby runtime automatically maps Java's CamelCase naming convention to Ruby's convention. For instance, you could address the Java method within a JRuby script by calling .

Based on such features, Java classes looks like ordinary script classes, as you can see in Listing 7. In this example a Java network library is used to implement a rudimentary SMTP client in JRuby.

Listing 7. Using Java classes within JRuby (rudimentary SMTP client)

include Java

# set up constants to shorten the paths
SSLContext = javax.net.ssl.SSLContext
BlockingConnection = Java::org.xsocket.stream.BlockingConnection


#performs new BlockingConnection(String, int, SSLContext.getDefault(), boolean)
bc = BlockingConnection.new('smtp.web.de', 25, SSLContext::default, false) 
 
bc.receive_timeout_millis = 60 * 1000 # performs setReceiveTimeoutMillis(long)
puts bc.read_string_by_delimiter("rn")  # performs readStringByDelimiter(String)

bc.write("STARTTLSrn")  
puts bc.read_string_by_delimiter("rn")
bc.activate_secured_mode()

bc.write('AUTH PLAIN ' + ["00" + 'from.me' + "00" + 'myPassword'].pack('m'))
puts bc.read_string_by_delimiter("rn")

bc.write("HELO Serverrn")
puts bc.read_string_by_delimiter("rn")

bc.write("MAIL FROM: from.me@web.dern")
puts bc.read_string_by_delimiter("rn")

bc.write("RCPT TO: itsyou@gmx.netrn")
puts bc.read_string_by_delimiter("rn")

bc.write("datarn")
puts bc.read_string_by_delimiter("rn")

mail =
'Message-ID: 46D957AF.2020804
Date: Sat, 01 Sep 2007 14:14:39 +0200
From: from.me@web.de
User-Agent: my mail client
MIME-Version: 1.0
To: itsyou@gmx.net
Subject: what I have to say
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit

combining scripting languages with Java is great'

bc.write("#{mail}rn.rn")
puts bc.read_string_by_delimiter("rn")

bc.write("quitrn")
puts bc.read_string_by_delimiter("rn")
bc.close()

Running your scripting language on the top of the Java virtual machine allows you to seamlessly integrate Java code and scripts. By calling a method of an embedded Java object, the scripting runtime looks for a proper method signature of the Java class. By identifying a matching Java method, the runtime performs the method call. Script-specific data types of argument parameters are automatically converted to corresponding Java types and vice versa.

Calling overloaded Java methods

As previously mentioned, dynamically typed languages don't require type declarations. The downside of this convenience is the hidden trap of calling overloaded Java methods. When calling an overloaded Java method, the scripting runtime has to choose the proper one. Under some circumstances the selected method implementation isn't the expected one, as shown in the next two examples.

Listing 8 shows an overloaded Java class.

Listing 8. An overloaded Java class

package test;

public class NonBlockingConnection  {
        public void write(int i)  {
                System.out.println("writing int " + i);
        }
        public void write(long l)  {
                System.out.println("writing long " + l);
        }
}

Listing 9 shows how Jython would call these overloaded methods.

Listing 9. Jython used for calling overloaded methods

from java.lang import Integer
from test import NonBlockingConnection

# call 1 
NonBlockingConnection().write<wbr>(214748364700) # call method based on python built-in data type
# prints out:
# writing long 214748364700

# call 2
NonBlockingConnection().write<wbr>(55)  # call method based on python built-in data type
# prints out:
# writing long 55
# ups, not int?

# call 3
NonBlockingConnection().write<wbr>(Integer(55))  # pass over a Java object instead of python data type
# prints out:
# writing int 55

Call 2 of the Jython script doesn't perform the expected int-typed Java method. A practical solution to determine the overloaded method is to instantiate the desired Java data type object within the script and to pass it over instead of the scripting-internal type. When you use Groovy, method overloading is a non-issue because Groovy supports both static and dynamic typing.

Calling scripting classes from Java

You can execute scripts within the Java environment by using a script engine. Typically, such script engines also allow you to perform dedicated scripting methods or functions. JSR 223 defines a standard interface Invokable to perform such methods and functions. As you can see in Listing 10, a dedicated function write of the Ruby script will be called.

Listing 10. Invoke a specific function of a Ruby script

String rubyScript = "def write(msg) rn" +
                        "      puts msg rn" +
                        "end rn" + 
                        "rn" +
                        " def anotherFunction() rn" +
                        "     # do something rn" +
                        "end rn";


    ScriptEngineManager manager = new ScriptEngineManager();
    ScriptEngine engine = manager.getEngineByName("jruby");

    engine.eval(rubyScript);
    
    Invocable inv = (Invocable) engine;
    inv.invokeFunction("write", new Object[] { "hello" });
    // prints out:
    // hello

Some scripting environments such as Jython also support a compiler to produce Java classes. In this case a Jython class can be called like a normal Java class. Only the introspection capability of the generated Java class is limited.

Overriding overloaded Java methods

JRuby and Jython both have support for implementing Java interfaces as well as extending Java classes. But overwriting overloaded Java method brings in new issues for most popular scripting languages. How to define which overloaded Java method should be overridden? Languages such as Python 2.x or Ruby don't support method overloading based on type signatures. This requires handling all overloaded cases of the overridden method. The method implementation has to dispatch all overload cases by introspecting the argument types. Listing 11 shows how to do this using Jython.

Listing 11. Using Jython to call overloaded methods

from test import NonBlockingConnection
from java.lang import Integer
from java.lang import Long

class ExtendedNonBlockingConnection(NonBlockingConnection):
    def write(self, *args):
       # the signature to handle?
       if len(args) == 1 and isinstance(args[0], Integer):
           print 'writing overriden int ' + args[0].toString()
       # no, let the super method do the stuff
       else:
          NonBlockingConnection.write(self, *args)


ExtendedNonBlockingConnection().write(Integer(777))
# prints out:
# writing overridden int 777


ExtendedNonBlockingConnection().write(Long(777))
# prints out:
# writing long 777

Because Groovy also supports static type declaration the overloaded method can be declared by the argument types in the override method signature, as shown in Listing 12.

Listing 12. Using Groovy to call overloaded methods

import test.NonBlockingConnection

class ExtendedNonBlockingConnection extends NonBlockingConnection {
    // overwrite the int method
    def write(int i) {
        println 'writing overridden int ' + i
     }
}

new ExtendedNonBlockingConnection().write((int) 777)
// prints out:
// writing overridden int 777

new ExtendedNonBlockingConnection().write((long) 777)
// prints out:
// writing long 777
A bidirectional integrated application (SMTP server)
A closer integration of Java and scripting languages often requires that Java classes be called by scripting elements and scripting elements be called by Java classes. Thus, the application will consist of a seamless, bidirectional integration of scripting-based classes and Java-based classes. 
Listing 13 presents an example of  an integrated script-based application that call Java classes, and vice verse. It is a rudimentary SMTP server written in JRuby that uses a non-blocking Java network library. 
Listing 13. Rudimentary SMTP server written in JRuby
include Java

RandomAccessFile = java.io.RandomAccessFile
DataConverter = Java::org.xsocket.DataConverter 
MultithreadedServer = Java::org.xsocket.stream.MultithreadedServer
IConnection = Java::org.xsocket.stream.IConnection
IConnectHandler = Java::org.xsocket.stream.IConnectHandler
IDataHandler = Java::org.xsocket.stream.IDataHandler


class TestMessageSinkManager
   def new_message_sink_channel()
      file = java.io.File.create_temp_file('smtptest', 'mail')
      return RandomAccessFile.new(file, 'rw').channel
   end
end
 

class SmtpProtocolHandler
   include IConnectHandler
   include IDataHandler

   def initialize(domain, message_sink_manager)
      @domain = domain
      @msg_sink_mgr = message_sink_manager
      @helo_pattern = Regexp.compile(/HELO.*/, Regexp::IGNORECASE)
      @mail_from_pattern = Regexp.compile(/MAIL FROM:.*/, Regexp::IGNORECASE)
      @rcpt_to_pattern = Regexp.compile(/RCPT TO:.*/, Regexp::IGNORECASE)
      @data_pattern = Regexp.compile(/DATA.*/, Regexp::IGNORECASE)
      @quit_pattern = Regexp.compile(/QUIT.*/, Regexp::IGNORECASE)
   end
  
   # new incoming (non blocking) connection
   def onConnect(nbc)
      nbc.flushmode = IConnection::FlushMode::ASYNC
      nbc.attachment = { 'state' => 'CMD', 'msg_num' => 0 }
      nbc.write("220 #{@domain} SMTP ready rn")
      return true
   end
  
   # data received for the (non blocking) connection
   def onData(nbc)
   
      case nbc.attachment['state']
         # message receiving mode: non-blocking streaming of the msg data 
         when 'MESSAGE'
            # some validations have to be performed by the data sink
            delimiter_found = nbc.read_available_by_delimiter("rn.rn", nbc.attachment['message_channel'])          
            if delimiter_found
               nbc.attachment['message_channel'].close()
               nbc.attachment['state'] = 'CMD'
               nbc.write("250 OK #{nbc.get_id()}.#{nbc.attachment['msg_num']} rn")
            end

         # smtp-command mode: perform command 
         else
            # a BufferUnderflowException will been thrown, if delimiter not found
            smtp_cmd_line = nbc.read_string_by_delimiter("rn")
         
            case smtp_cmd_line
               when @helo_pattern
                  nbc.write("250 #{@domain} SMTP Service rn")
          
               when @mail_from_pattern
                  originator = smtp_cmd_line[10,9999].strip()
                  # ...here some validations should be performed (valid address, ...) 
                  nbc.attachment['originator'] = originator
                  nbc.attachment['recipients'] = []
                  nbc.write("250 #{@originator} is syntactically correctrn")

               when @rcpt_to_pattern
                  rec = smtp_cmd_line[8,9999].strip()
                  # ...here some validations should be performed (max recipients, ...)
                  nbc.attachment['recipients'] = nbc.attachment['recipients'] << rec
                  nbc.write("250 #{rec} verified rn")  

               when @data_pattern
                  # ...here some validation should be performed (recipients set, ...)
                  nbc.attachment['state'] = 'MESSAGE'
                  nbc.attachment['msg_num'] = nbc.attachment['msg_num'] + 1
                  nbc.attachment['message_channel'] = @msg_sink_mgr.new_message_sink_channel()
                  time_stamp = "Received: FROM #{nbc.remote_address.canonical_host_name} BY #{@domain}rn" + 
                                  "id #{nbc.get_id()}.#{nbc.attachment['msg_num']}; " + Time.new.to_s() + "rn"
                  nbc.attachment['message_channel'].write(DataConverter.to_byte_buffer(time_stamp, 'US-ASCII'))                  
                  nbc.write("354 Enter message, ending with "."  rn")

               when @quit_pattern
                  nbc.write("221 SMTP service closing connection rn")
                  nbc.close()

               else 
                  nbc.write("500 Unrecognized command rn")
         
            end         
      end
      return true
   end
end

server = MultithreadedServer.new(25, SmtpProtocolHandler.new('mSrv', TestMessageSinkManager.new))
server.run()
In this application a Java-based server is instantiated, which listens for incoming SMTP network connections. The network events are handled by a JRuby-based handler. To do this, the JRuby-based handler has to implement a Java callback interface defined by the Java network library. 
To implement a Java interface, the JRuby class has to declare all supported interfaces using the include statement. Unlike a Java-based interface implementation, the return type or exceptions don't have to be defined by the JRuby-based method implementation. By performing a callback method of the handler, a Java object is passed over (as a INonBlockingConnection instance) to the JRuby script. 
Access to this Java object is intercepted by the scripting environment. Therefore it can be handled within the JRuby method implementation like an ordinary Ruby artifact. Primitive data types like Java Integer or Long are mapped into the corresponding Ruby type. 
If a network event occurs, the server performs the proper callback method of the handler. This works, because the Ruby-based handler looks like a regular Java class to the server. The JRuby runtime automatically wraps the JRuby-based handler by passing it over to the Java server. Instead of getting the native JRuby handler, the Java server gets a proxy that supports all the methods of the Java interfaces that are implemented by the handler. 
In conclusion
Java-based scripting runtimes strive to integrate the Java platform with the scripting language of your choice. At this early stage, actual mileage with the various scripting runtime engines will vary. In current versions of JRuby or Jython, for instance, you will find some of the newer features of the Java platform missing, such as annotations support. It is also a challenge to bridge the semantic gap between the Java language and a scripting language, sometimes requiring ugly solutions. That said, in most cases the supported features are sufficient to write enterprise-level applications using the Java platform, Java code, and the scripting language you like. 
The bidirectional integrated application example in Listing 13 is a non-blocking, multithreaded SMTP server written using scripting language classes and Java classes. An existing Java network library has been used to handle low-level, performance-critical, and network-specific tasks like threading or connection management. The controlling task has been implemented using a scripting language. The emerging synergy between the Java platform and scripting languages makes it possible to write high-performance, scalable applications in a very productive and elegant way. The challenge is to choose the right language for the right task, in order to get the best of both. 
On the script side you can choose between Java ports of existing scripting languages such as JRuby or Jython, and a scripting language that is designed to run on the Java platform, like Groovy. The first group adapts Java classes to look like regular scripting artifacts. Groovy uses a syntax very similar to Java code but more evolved, and each Groovy class is a full-fledged Java class. Groovy is easier for Java developers to learn than most other scripting languages and can seamlessly use the Java libraries without the need for adapters. 
See the Resources section to learn more about scripting on the Java platform, polyglot programming, and the languages discussed in this article.

Scripting on the Java platform