Benchmark testing demonstrates significantly better performance for both OpenJDK 8 and OpenJDK 9 when using Eclipse OpenJ9, than with Hotspot.
Many different metrics exist to measure the performance of an application including startup time, ramp up time, footprint, response time, as well as throughput. At Eclipse OpenJ9, we keep a watchful eye on all of these metrics, making sensible tradeoffs and providing tuning options that allow the virtual machine (VM) to be optimized for different workloads.
Our original tests were carried out with OpenJDK version 9, but as most users are more interested in version 8, we have taken the opportunity to re-evaluate the performance of OpenJ9 when running OpenJDK version 8 using the same metrics as before, namely footprint after start up and during ramp up, start-up time, throughput, and ramp-up time.
The results for OpenJDK 8 with OpenJ9 are very much in line with our previous tests using OpenJDK 9, and demonstrate a significantly better performance than OpenJDK 8 with Hotspot.
Take a look at these results:
- 66% smaller footprint after startup
- 63% smaller footprint during ramp up
- 42% faster startup time
- Comparable throughput
- Significantly faster ramp up time in a CPU constrained environment
While the specific levels of performance shown in these tests might change from benchmark to benchmark or even from machine to machine, the same trends are expected to hold for a large class of server-type applications.
These results show that OpenJ9 achieves a good balance between (often conflicting) performance metrics: it offers excellent footprint savings and faster start-up time with the help of AOT technology, while also delivering throughput performance that is competitive with Hotspot.
Due to its low memory footprint, OpenJ9 is particularly well suited for cloud computing environments where memory savings translate into cost savings for cloud users and providers alike.
Memory footprint after startup
OpenJ9 is highly optimized for cloud workloads. The memory footprint (Resident Set Size) is around a third the size of other VM implementations.
Immediately after startup OpenJDK 8 with OpenJ9 showed a footprint size about 66% less than OpenJDK 8 with Hotspot.
When OpenJ9 was tested with AOT enabled, and with AOT and
-Xquickstart enabled, the footprint size remained at around 66% less than Hotspot.
Memory footprint during ramp up
In all three scenarios, memory footprint (Resident Set Size) increased abruptly when load was applied to the system (time = 0). But at steady state, OpenJDK 8 with OpenJ9 was found to use approximately 63% less physical memory than OpenJDK 8 with HotSpot. This difference is even more striking than in the experiments with OpenJDK 9.
Enabling AOT on the OpenJ9 VM made almost no difference to these results.
Shared classes and Ahead-of-Time (AOT) technologies typically reduce startup time while improving the overall ramp-up time of applications. As a developer you can achieve even faster startup times by using
-Xquickstart mode as well.
With AOT enabled, startup time for OpenJDK 8 with OpenJ9 is 36% lower than Hotspot. Moreover, with AOT enabled and using the
-Xquickstart option, start-up time for OpenJ9 is almost 42% lower than Hotspot.
Ramp up time in CPU-constrained environments
Virtualization is heavily used in the cloud to split big computing machines into many smaller VM guests. These guests are typically provisioned with a small number of virtual CPUs in order to achieve a high application density and to control the resources dedicated to applications. One side-effect is that Java applications can take longer to ramp-up because JIT compilation threads compete with application threads over the more limited computing resources.
Without any additional configuration set, OpenJ9 already ramps up much better than HotSpot. Although we haven't shown it on the graph, HotSpot needs about 30 minutes to reach its peak throughput; in contrast, OpenJ9 needs only 8.5 minutes!
With our recommended configuration for the cloud, the ramp-up curve for OpenJ9 improves substantially compared to its default settings. While OpenJ9 with default settings obtains a higher throughput in the end, it takes a full 5 minutes to equal the peak throughput of our configured OpenJ9. The lower peak throughput seen in the chart results from the use of a lot of AOT-compiled code, which is less optimized, and the presence of
-Xtune:virtualized, which prevents most recompilations.
The total work done by the VM after load is applied, is represented by the area under the curve. The results show that our cloud configuration performs more work than the default OpenJ9 configuration and much more work than Hotspot in the first 12 minutes. Therefore, configuring OpenJ9 to use AOT and
-Xtune:virtualized is an excellent solution for short lived VMs running in a CPU-constrained environment.
You can read the full details about how the data was generated at OpenJDK 8 performance with OpenJ9.