Eclipse OpenJ9
Eclipse Incubation

Performance

Benchmark testing with the DayTrader 3 application demonstrates significantly better performance for OpenJDK v9 with OpenJ9 than OpenJDK v9 with Hotspot.

Many different metrics exist to measure the performance of an application including startup time, ramp up time, footprint, response time, as well as throughput. At Eclipse OpenJ9, we keep a watchful eye on all of these metrics, making sensible tradeoffs and providing tuning options that allow the JVM to be optimized for different workloads.

To showcase our performance pedigree we set about measuring the strengths of OpenJDK with Eclipse OpenJ9 compared to an OpenJDK with Hotspot. We chose the Daytrader3 application because it is meaningful to measure different performance metrics, unlike many microbenchmarks that focus almost exclusively on throughput. The metrics we focused on include startup time, the JVM footprint size during the experiment, and of course, throughput.

Take a look at these charts:

While the specific levels of performance shown in these tests might change from benchmark to benchmark or even from machine to machine, the same trends are expected to hold for a large class of server type applications.

CPU-constrained environments

In a further experiment we tuned Eclipse OpenJ9 for an environment with constrained resources, much like you'd find in a standard cloud deployment.

In this scenario, OpenJ9 ramped up 4× faster than Hotspot, performing far more work during the first 8 minutes. An excellent option for short-lived applications in the cloud that are sensitive to response times!

You can read the full details about how the data was generated at DayTrader 3 benchmark testing.

Memory footprint after startup

OpenJ9 is highly optimized for cloud workloads. The memory footprint (Resident Set Size) is around half the size of other JVM implementations.

Fig 1. DayTrader 3 footprint size after start up with a maximum Java heap size set to 1 GB.

×

Fig 1. DayTrader 3 footprint size after start up with a maximum Java heap size set to 1 GB.

Immediately after startup OpenJDK 9 with OpenJ9 showed a footprint size about 60% less than OpenJDK 9 with Hotspot.

Memory footprint during ramp up

Fig 2. DayTrader 3 footprint size during ramp up with a maximum Java heap size set to 1 GB.

×

Fig 2. DayTrader 3 footprint size during ramp up with a maximum Java heap size set to 1 GB.

Memory footprint settled after about 3 minutes from when load was applied to the system. OpenJDK with OpenJ9 was found to use approximately 45% less physical memory than OpenJDK with HotSpot.

Startup time

Shared classes and Ahead-of-Time (AOT) technologies typically reduce startup time while improving the overall ramp-up time of applications. As a developer you can achieve even faster startup times by using -Xquickstart mode as well.

Fig 3. DayTrader 3 startup time with a maximum Java heap size set to 1 GB.

×

Fig 3. DayTrader 3 startup time with a maximum Java heap size set to 1 GB.

With AOT enabled, startup time for OpenJDK 9 with OpenJ9 is almost 40% lower than OpenJDK 9 with Hotspot. Moreover, with AOT enabled and -Xquickstart, startup time for OpenJ9 is almost 50% lower.

Throughput

Fig 4. DayTrader 3 throughput during ramp up with a maximum Java heap size set to 1 GB.

×

Fig 4. DayTrader 3 throughput during ramp up with a maximum Java heap size set to 1 GB.

Eventually, both OpenJDK 9 with OpenJ9 and OpenJDK 9 with Hotspot reach a similar peak throughput. But OpenJDK 9 with OpenJ9 reaches that peak 5× faster for this particular benchmark - in under 2 minutes, compared with 9 minutes for OpenJDK 9 with Hotspot.

Ramp up time in CPU-constrained environments

Virtualization is heavily used in the cloud to split big computing machines into many smaller VM guests. These guests are typically provisioned with a small number of virtual CPUs in order to achieve a high application density and to control the resources dedicated to applications. One side-effect is that Java applications can take longer to ramp-up because JIT compilation threads compete with application threads over the more limited computing resources.

Fig 5. DayTrader 3 throughput during ramp up with a maximum Java heap size set to 1 GB and the JVM pinned to 1 core.

×

Fig 5. DayTrader 3 throughput during ramp up with a maximum Java heap size set to 1 GB and the JVM pinned to 1 core.

Without any additional configuration set, OpenJ9 already ramps up much better than HotSpot. Although we haven't shown it on the graph, HotSpot needs about 30 minutes to reach its peak throughput; in contrast, OpenJ9 needs only 7.5 minutes.

However, ramp-up throughput is improved even more by using a large shared class cache and dynamic Ahead-of-Time (AOT) compilation and also by using -Xtune:virtualized on the command line.

With our recommended configuration for the cloud, the ramp-up curve for OpenJ9 improves substantially compared to its default settings. Whilst OpenJ9 with default settings obtains a higher throughput in the end, it takes a full 3 minutes to equal the peak throughput of our configured OpenJ9. The lower peak throughput seen in the chart results from the use of a lot of AOT-compiled code, which is less optimized, and the presence of -Xtune:virtualized prevents most recompilations.

The total work done by the JVM since load was applied is represented by the area under the curve. The results show that our cloud configuration performs more work than the default OpenJ9 configuration and much more work than Hotspot in the first 8 minutes of the run. Therefore, configuring OpenJ9 to use AOT and -Xtune:virtualized is an excellent solution for short lived JVMs running in a CPU-constrained environment.

You can read the full details about how the data was generated at DayTrader 3 benchmark testing.