Multicore processing for client-side Java applications

It's a well-known fact that hardware companies are abandoning the race for single-CPU speed and instead are focusing on multicore processors. Despite the fact that many algorithms can be easily parallelized, most client-side Java code is still written for single-CPU systems. In this article Kirill Grouchnikov shows you how to fine-tune a core JDK array-sorting algorithm for improved processing speed of as much as 35%.

The consistent increase in single-core CPU speed that programmers could rely on for years is no longer available. This has been true for the server side for at least a decade, and now it's also the reality for client-side programming. Quite a few tasks easily lend themselves to being parallelized using new features in JDK 6.0, letting client-side applications take advantage of newer multicore hardware. Using a core JDK array sorting algorithm as an example, this article walks through the implementation details, highlighting points that are specific to parallel algorithms. Even a simple implementation results in a significant speed-up -- around 35 % -- on a dual-core machine.

The demise of Moore's Law

Up until a year or two ago, API providers could simply rely on an empirical observation made in 1965 by Gordon Moore, the co-founder of Intel. Moore's Law states that the number of transistors on an integrated circuit at minimum cost doubles every 18 to 24 months. For software developers, this meant that you could write a program, run it on the simplest CPU available at that time, and be pretty sure that the same program would run twice as fast on the simplest CPU available two years later. As long as the operating system was backward compatible, you didn't even need to recompile the program.

However, in the last couple of years, hardware manufacturers have started hitting production barriers that make cramming more computing power into a single chip very costly. The solution most of them have adopted -- first for the server market and now for the consumer (client) market -- is to put multiple cores on the same chip, without increasing each core's processing speed. Translated into the software world, this means that you don't get "free rides" any more. If your program has a simple sequential flow, it no longer can enjoy advances in the underlying hardware. This is true for both the programs that you're writing and the core language libraries.

Collection sorting in the single-core world

Existing Java collection-sorting routines are no exception. They perform no faster on newer multicore machines than on single-core machines. This might seem acceptable on smaller inputs, but the input domains for most real-world problems don't stand still. Moreover, developers and users rightly expect their programs to run faster on newer hardware.

A look at the generic algorithms that operate on collections, and their existing Java implementations, will clarify the problem -- and light the way to a solution.

Algorithm complexity

Most of the generic core algorithms that operate on collections are available in the Arrays and Collections classes. Using the APIs exposed by these two classes, you can sort, search, and fill lists and arrays. Because most of the APIs operate on the contents of the entire collection, the running time is proportional to the collection size.

For some methods (such as sorting), the running time is even longer. You can't sort an arbitrary collection in a number of steps that is proportional to the collection's size. (This is known as linear complexity.) If the collection size is N, the best algorithm can sort this collection in a number of steps that is proportional to N*log(N); this lower bound has been theoretically proven. (As a side note, the quadratic complexity of the straightforward bubble sort means that the number of steps is proportional to N*N, which is highly undesirable for large collections.)

What does this mean to users of these core APIs? When you call Arrays.sort or Collections.sort, the running time doesn't grow linearly with the collection size. On my machine, calling Arrays.sort on an array of 200,000 strings takes 490ms on average. An array of 400,000 strings is sorted in 1290ms -- an increase in running time by factor of about 2.6.

Java implementations of sorting algorithms

A look at the implementation of the various sorting algorithms in the core Arrays and Collections classes is especially easy now that they are licensed as open source. All the sort routines in the Arrays class that operate on primitive types (such as byte, char, and double) use quicksort, which is generally recognized as the fastest sorting algorithm for most inputs. (In some corner cases the performance might degrade to quadratic.) The sort API that operates on an array of Objects is a mergesort, which is simpler to implement and provides N*log(N) performance as well. Finally, Collections.sort(List) delegates the actual sorting to the Arrays.sort(Object[]).

Taking a closer look at the mergesort in the Arrays class, you can see that it is indeed the classic mergesort with a few optimization tweaks for corner cases. If the array size is small (seven or fewer with the current implementation), it reverts to the bubble sort. Otherwise, the array is split in half, and the same method is called (recursively) on both halves. After both halves are sorted, the code "merges" them (hence the name).

Going back to the hardware advances, you can easily see that increasing the CPU speed (number of operations per second) by a factor of two results in a matching improvement in the algorithm running time. This is the case because the mergesort is a sequential recursive algorithm that does exactly the same sequence of steps, provided the same input.

Core sorting in the multicore world

What's the problem with the current mergesort implementation when it runs on a multicore machine? The answer is simple: it doesn't explore the inherent concurrency of the recursive implementation. When the array is split in half, the sorting of the second half begins only after the sorting of the first half is done. But new concurrency utilities in JDK 6.0 come to the rescue, letting you perform these subtasks in parallel without any communication between them. At the end, when both tasks are done, the sorted halves still need to be merged together. This still results in N*log(N) performance, but this time with a lower constant factor. To illustrate how much lower, I'll show you a simple implementation of a mergesort that takes advantage of a multicore environment. This takes you into the world of parallel or concurrent programming.

Parallel programming to the rescue

Here is a guided walkthrough for the concurrent implementation of this alternative implementation of Arrays.mergeSort. The code examples I'll show are based on the core JDK implementation (and as such are licensed under GPL). The first step, shown in Listing 1, is to check the number of available processors (cores). If you have only one processor, parallelizing the implementation gains you nothing; you will only pay for the overhead of creating threads and the thread-context switch while the array halves are being sorted.

Listing 1. Special case for small input and one processor

if ((a.length < 7) || (Runtime.getRuntime().availableProcessors() == 1)) {
   mergeSort(aux, a, 0, a.length, 0);
   return;
}

Now you need to decide how to split up the work among all the available processors. This decision is specific to the task at hand. In order to simplify the implementation, I'll split the work between only two processors. It's easy to see that the final merge stage can begin only when both halves are sorted. Listing 2 uses the new concurrency utilities available in JDK 6.0:

• The CountDownLatch class lets you wait for a specific number of tasks to be completed (two in this case).
• Executors.newFixedThreadPool launches two threads that sort the array halves.

Listing 2. Splitting the sort into two independently and concurrently processed halves

final CountDownLatch doneSignal = new CountDownLatch(2);
ExecutorService e = Executors.newFixedThreadPool(2);

class WorkerRunnable implements Runnable {
   int start;
   int end;

   WorkerRunnable(int start, int end) {
      this.start = start;
      this.end = end;
   }

   public void run() {
      mergeSort(aux, a, start, end, 0);
      doneSignal.countDown();
   }
}

int mid = a.length >> 1;
e.execute(new WorkerRunnable(0, mid));
e.execute(new WorkerRunnable(mid, a.length));
try {
   doneSignal.await(); // wait for all to finish
} catch (InterruptedException ie) {
}
e.shutdown();

After CountDownLatch.await returns, you know that both subtasks have been completed. At this point, you can merge the sorted halves, as shown in Listing 3. (This code is taken from Arrays.mergeSort with relevant variable renaming.)

Listing 3. Merging the two sorted halves

System.arraycopy(a, 0, aux, 0, a.length);
// merge two halves
for (int i = 0, p = 0, q = mid; i < a.length; i++) {
   if (q >= a.length || p < mid
         && ((Comparable) aux[p]).compareTo(aux[q]) <= 0)
      a[i] = aux[p++];
   else
      a[i] = aux[q++];
}
Performance results on multicore machine
The source archive accompanying this article contains the following Java classes: