OpenJ9 produces various types of diagnostic information for analysis when different events occur, such as a general protection fault. The dumps produced are controlled by dump agents, which are initialized when the OpenJ9 virtual machine (VM) starts. The default settings for the dump agents are sufficient for most cases. However, you can use the
-Xdump option on the command line to fine tune the dump agent settings. For example, you can use the
-Xdump option to add and remove dump agents for various VM events, update default dump settings, and limit the number of dumps that are produced.
A large set of options and suboptions are available for controlling dumps, which provides a lot of flexibility.
Xdump Option Builder
Use the Xdump Option Builder tool to help you specify the correct options and avoid incompatibilities.
The following table lists the help options for
-Xdump, which provide usage and configuration information:
||Displays general dump help.|
||Lists available trigger events.|
||Lists additional VM requests.|
||Lists recognized label tokens.|
||Shows registered agents on startup.|
||Displays dump agent usage information.|
The following options can be used to control the production of diagnostic data:
||Removes all default dump agents and any preceding dump options.|
||Enable support for pluggable agents|
||Discards dumps when the default or specified dump location is full.|
||Specifies a directory for all dump types to be written to. This directory path is prefixed to the path of all non-absolute dump file names, including the file names for the default dump agents.|
||Modifies the signal that is used to suspend VM threads while a dump file is being written. Use
||Provides detailed suboptions per dump agent that provide more granular control.|
Dump agents can be configured at a very granular level by specifying suboptions. The
<events> suboption is the prime trigger mechanism. The full set of suboptions are listed in the following table:
|Dump agent suboptions||Result|
||Removes the dump agent.|
||Prints the default options for the dump agent.|
||Triggers a dump agent when a specific event occurs.|
||Starts an external application for the dump agent.|
||Specifies where to write the dump for the dump agent.|
||Filters dumps by wildcards or events.|
||Filters on text strings within an exception message.|
||Used by specific dump agents to select the type of dump file to produce.|
||Specifies the priority that the dump agents run in.|
||Starts and stops a dump agent on a particular occurrence of a VM.|
||Asks the VM to prepare the state before starting the dump agent.|
You can have multiple
-Xdump options on the command line. You can also have multiple dump types triggered by multiple events. For example, the following command turns off the creation of heap dump files, and creates a dump agent that produces a heap dump file and a Java™ dump file when either a vmstart or vmstop event occurs:
java -Xdump:heap:none -Xdump:heap+java:events=vmstart+vmstop -mp . -m <class> [args...]
Note: Multiple suboptions that follow an
Xdump suboption must be split with a comma (,), for example:
java -Xdump:java:events=vmstart,file=/STDERR/ -version
For more detailed information on the these parameters and suboptions, including examples, see Parameters.
A dump agent performs diagnostic tasks when triggered. Most dump agents save information on the state of the VM in some form of dump or trace file for later analysis. An exception is the "tool" agent, which can be used to trigger external processes when specific events occur.
|stack||Stack dumps are very basic dumps in which the status and Java stack of the thread is written to stderr.|
|console||Console dumps are very basic dumps, in which the status of every Java thread is written to stderr.|
|system||System dumps capture the raw process image or address space of an application.|
|tool||The tool option allows external processes to be started when an event occurs.|
|java||Java dumps are an internally generated and formatted analysis of the VM, giving information that includes the Java threads present, the classes loaded, and heap statistics.|
|heap||Heap dumps capture all object instances in the heap, including each object address, type or class name, size, and references to other objects.|
|snap||Take a snap of the trace buffers, which contain tracepoint data.|
|ceedump||LE CEEDUMP dumps are z/OS® formatted summary system dumps that show stack traces for each thread that is in the VM process, together with register information and a short dump of storage for each register.|
|jit||JIT compiler dumps contain diagnostic data in a binary format.|
Default dump agents
During VM initialization a set of dump agents are added by default. You can override this set of dump agents using
-Xdump on the command line. To show the registered dump agents, user the
Xdump:what option on the command line. The following sample output shows the default dump agents that are in place on a Linux system:
java -Xdump:what Registered dump agents ---------------------- -Xdump:system: events=gpf+abort+traceassert+corruptcache, label=/home/user/core.%Y%m%d.%H%M%S.%pid.%seq.dmp, range=1..0, priority=999, request=serial ---------------------- -Xdump:system: events=systhrow, filter=java/lang/OutOfMemoryError, label=/home/user/core.%Y%m%d.%H%M%S.%pid.%seq.dmp, range=1..1, priority=999, request=exclusive+compact+prepwalk ---------------------- -Xdump:heap: events=systhrow, filter=java/lang/OutOfMemoryError, label=/home/user/heapdump.%Y%m%d.%H%M%S.%pid.%seq.phd, range=1..4, priority=500, request=exclusive+compact+prepwalk, opts=PHD ---------------------- -Xdump:java: events=gpf+user+abort+traceassert+corruptcache, label=/home/user/javacore.%Y%m%d.%H%M%S.%pid.%seq.txt, range=1..0, priority=400, request=exclusive+preempt ---------------------- -Xdump:java: events=systhrow, filter=java/lang/OutOfMemoryError, label=/home/user/javacore.%Y%m%d.%H%M%S.%pid.%seq.txt, range=1..4, priority=400, request=exclusive+preempt ---------------------- -Xdump:snap: events=gpf+abort+traceassert+corruptcache, label=/home/user/Snap.%Y%m%d.%H%M%S.%pid.%seq.trc, range=1..0, priority=300, request=serial ---------------------- -Xdump:snap: events=systhrow, filter=java/lang/OutOfMemoryError, label=/home/user/Snap.%Y%m%d.%H%M%S.%pid.%seq.trc, range=1..4, priority=300, request=serial ---------------------- -Xdump:jit: events=gpf+abort, label=/home/user/jitdump.%Y%m%d.%H%M%S.%pid.%seq.dmp, range=1..0, priority=200, request=serial ----------------------
Dump agent tokens
You can use tokens to add context to dump file names and directories, and to pass command-line arguments to the tool agent. The tokens available are listed in the following table:
|%Y||Year (4 digits)|
|%y||Year (2 digits)|
|%m||Month (2 digits)|
|%d||Day of the month (2 digits)|
|%H||Hour ( 2 digits)|
|%M||Minute (2 digits)|
|%S||Second (2 digits)|
|%home||Java home directory|
|%job||Job name (z/OS only)|
|%jobid||Job ID (z/OS only)|
|%asid||Address space ID (z/OS only)|
|&DS||Dump Section. An incrementing sequence number used for splitting TDUMP files to be less than 2 GB in size. (z/OS 64-bit only)|
Merging -Xdump agents
If you configure more than one dump agent, each responds to events according to its configuration. However, the internal structures representing the dump agent configuration might not match the command line, because dump agents are merged for efficiency. Two sets of options can be merged as long as none of the agent settings conflict. This means that the list of installed dump agents and their parameters produced by
-Xdump:what might not be grouped in the same way as the original
-Xdump options that configured them.
For example, you can use the following command to specify that a dump agent creates a Java dump file on class unload:
java -Xdump:java:events=unload -Xdump:what
This command does not create a new agent, as can be seen in the results from the
... ---------------------- -Xdump:java: events=gpf+user+abort+unload+traceassert+corruptcache, label=/home/user/javacore.%Y%m%d.%H%M%S.%pid.%seq.txt, range=1..0, priority=400, request=exclusive+preempt ----------------------
The configuration is merged with the existing Java dump agent for events gpf, user, abort, traceassert, and corruptcache, because none of the specified options for the new unload agent conflict with those for the existing agent.
In the previous example, if one of the parameters for the unload agent is changed so that it conflicts with the existing agent, then it cannot be merged. For example, the following command specifies a different priority, forcing a separate agent to be created:
java -Xdump:java:events=unload,priority=100 -Xdump:what
The results of the
-Xdump:what option in the command are as follows.
... ---------------------- -Xdump:java: events=unload, label=/home/user/javacore.%Y%m%d.%H%M%S.%pid.%seq.txt, range=1..0, priority=100, request=exclusive+preempt ---------------------- -Xdump:java: events=gpf+user+abort+traceassert+corruptcache, label=/home/user/javacore.%Y%m%d.%H%M%S.%pid.%seq.txt, range=1..0, priority=400, request=exclusive+preempt ----------------------
To merge dump agents, the request, filter, opts, label, and range parameters must match exactly. If you specify multiple agents that filter on the same string, but keep all other parameters the same, the agents are merged. For example:
java -Xdump:none -Xdump:java:events=uncaught,filter=java/lang/NullPointerException -Xdump:java:events=unload,filter=java/lang/NullPointerException -Xdump:what
The results of this command are as follows:
Registered dump agents ---------------------- -Xdump:java: events=unload+uncaught, filter=java/lang/NullPointerException, label=/home/user/javacore.%Y%m%d.%H%M%S.%pid.%seq.txt, range=1..0, priority=400, request=exclusive+preempt ----------------------
Dump agents are triggered by events occurring during operation of the OpenJ9 VM. Some events can be filtered to improve the relevance of the output.
The following table shows the events that are available as dump agent triggers:
|Event||Triggered when....||Filters on ....|
|gpf||A General Protection Fault (GPF) occurs.||Not applicable|
|user||The VM receives the SIGQUIT (Linux, macOS®, AIX®, z/OS) or SIGBREAK (Windows™) signal from the operating system.||Not applicable|
|abort||The VM receives the SIGABRT signal from the operating system.||Not applicable|
|vmstart||The virtual machine is started.||Not applicable|
|vmstop||The virtual machine stops.||Exit code; for example,
|load||A class is loaded.||Class name; for example,
|unload||A class us unloaded.||Not applicable|
|throw||An exception is thrown.||Exception class name; for example,
|catch||An exception is caught.||Exception class name; for example,
|uncaught||A Java exception is not caught by the application.||Exception class name; for example,
|systhrow||A Java exception is about to be thrown by the VM. This is different from the 'throw' event because it is only triggered for error conditions detected internally in the VM.||Exception class name; for example,
|thrstart||A new thread is started.||Not applicable|
|blocked||A thread becomes blocked.||Not applicable|
|thrstop||A thread stops.||Not applicable|
|fullgc||A garbage collection cycle is started.||Not applicable|
|slow||A thread takes longer than 50ms to respond to an internal VM request.||Time taken; for example, filter=#300ms will trigger when a thread takes longer than 300ms to respond to an internal VM request.|
|allocation||A Java object is allocated with a size matching the given filter specification.||Object size; a filter must be supplied. For example, filter=#5m will trigger on objects larger than 5 Mb. Ranges are also supported; for example, filter=#256k..512k will trigger on objects between 256 Kb and 512 Kb in size.|
|traceassert||An internal error occurs in the VM.||Not applicable|
|corruptcache||The VM finds that the shared class cache is corrupt.||Not applicable|
|excessivegc||An excessive amount of time is being spent in the garbage collector.||Not applicable|
Note: The gpf, traceassert, and abort events cannot trigger a heap dump, prepare the heap (request=prepwalk), or compact the heap (request=compact).
-Xdump:<agent>:<suboptions> descriptions and examples.
To print usage information for a specific dump agent, use
-Xdump:none option to add and remove dump agents for various VM events, update default dump settings (such as the dump name), and limit the number of dumps that are produced.
The option can be used to affect all agents by specifying
-Xdump:none:<options> or specific agents by specifying
<suboptions> is one of the following control types:
Explanations for these suboptions are provided elsewhere in this topic.
To remove all default dump agents and any preceding dump options, use
-Xdump:none. Use this option so that you can subsequently specify a completely new dump configuration.
You can also remove dump agents of a particular type. Here are some examples:
To turn off all heap dumps (including default agents) but leave Java dumps enabled, use the following option:
To turn off all dump agents for corruptcache events:
To turn off just system dumps for corruptcache events:
To turn off all dumps when a
java/lang/OutOfMemory error is thrown:
To turn off just system dumps when a
java/lang/OutOfMemory error is thrown:
If you remove all dump agents by using
-Xdump:none with no further
-Xdump options, the VM still provides these basic diagnostic outputs:
- If a user signal (kill -QUIT) is sent to the VM, a brief listing of the Java threads including their stacks, status, and monitor information is written to stderr.
- If a crash occurs, information about the location of the crash, VM options, and native and Java stack traces are written to stderr. A system dump file is also written to the user's home directory.
Note: Removing dump agents and specifying a new dump configuration can require a long set of command-line options. To reuse command-line options, save the new dump configuration in a file and use the
-Xoptionsfile option. For more information, see -Xoptionsfile.
Each dump type has default options. To view the default options for a particular dump type, use
You can change the default options at run time. For example, you can direct Java dump files into a separate directory for each process, and guarantee unique files by adding a sequence number to the file name using:
Or, for example, on z/OS, you can add the jobname to the Java dump file name using:
This option does not add a Java dump agent; it updates the default settings for Java dump agents. Further Java dump agents will then create dump files using this specification for filenames, unless overridden.
Note: Changing the defaults for a dump type will also affect the default agents for that dump type added by the VM during initialization. For example if you change the default file name for Java dump files, that will change the file name used by the default Java dump agents. However, changing the default range option will not change the range used by the default Java dump agents, because those agents override the range option with specific values.
To trigger a dump as a result of an event, use the
-Xdump:<agent>:events=<event> suboption. For a list of possible events, see Dump events.
For example, the following command instructs the VM to create a dump agent at startup that produces a Heap dump whenever the vmstop event happens:
The exec suboption is used by the tool dump agent to specify an external application to start. You can set a specific command to run for a particular dump agent with the following command:
The file suboption specifies where the diagnostics information is written for the specified dump type. The syntax is
For example, to create a Heap dump called
my.dmp when a vmstop event is received, use:
When producing system dump files on z/OS platforms, use the
dsn option instead of the
file option. For example:
Add one of the following options to write a Java dump file to STDOUT or STDERR respectively:
- The keywords
/STDERR/are not case sensitive;
- By common convention, you can use a dash (
-) to refer to STDOUT:
You can use tokens to add context to dump file names. For a list of tokens, see Dump agent tokens.
The location for the dump file is selected from the following options, in this order:
- The location specified by the
-Xdump:<agent>:filesuboption on the command line (if that location includes a path). This location applies to the specified dump agent type only.
- The location specified by the
-Xdump:directoryoption on the command line. This location applies to all dump agent types.
The location specified by the relevant environment variable:
Dump agent type z/OS operating systems Other operating systems Java dumps
The current working directory of the OpenJ9 VM process.
If the directory does not exist, it is created.
If the dump file cannot be written to the selected location, the VM reverts to using the following locations, in this order:
- On Windows platforms only, the system default location is
- The location specified by the
C:\Tempon Windows operating systems, or the
/tmpdirectory on other operating systems.
This VM action does not apply to system dumps on z/OS operating systems that use the
You can prevent the VM reverting to different dump locations by using the
Some VM events occur thousands of times during the lifetime of an application. Dump agents can use filters and ranges to avoid producing an excessive number of dump files. The following syntax must be used:
You can use a wildcard in your exception event filter by placing an asterisk only at the beginning or end of the filter. The following command does not work because the second asterisk is not at the end:
To fix the problem, change this filter to the following string:
Class loading and exception events
You can filter class loading (load) and exception (throw, catch, uncaught, systhrow) events by the name of the class that is being loaded, thrown or caught. For example:
-Xdump:java:events=load,filter=java/lang/String -Xdump:java:events=throw,filter=java/lang/ArrayStoreException -Xdump:java:events=catch,filter=java/lang/NullPointerException
In addition, you can filter throw, uncaught, and systhrow exception events by the name of the method that throws the exception. The name of the parent class must include the full package name, using the forward slash (/) as a separator. Use a dot (.) to separate the method name from the class name. You can use an asterisk (*) as a wildcard character, to include all methods (optional portions are shown in brackets). For example:
For example, to trigger a Java dump when method
MyApplication.myMethod() throws a
NullPointerException exception, use the following syntax:
The stack frame offset allows you to filter on the name of a method that calls the throwing method. This option is useful if the exception is being thrown from a general purpose or utility class. For example, to trigger a Java dump when a method called by
MyApplication.main() throws a
NullPointerException, use the following syntax:
The default value of the stack frame offset is zero.
You can filter the catch exception events by Java method name (optional portions are shown in brackets). For example:
You can filter throw, uncaught, and systhrowexception events by Java method name (optional portions are shown in brackets):
You can filter the catch exception events by Java method name (optional portions are shown in brackets):
Note: The filters apply to the stacktrace and fire every time the same exception is rethrown, which might result in multiple Java core files.
You can filter the VM shut down event (vmstop) by using one or more exit codes:
You can filter the slow event to change the time threshold from the default of 50 ms:
You must filter the allocation event to specify the size of objects that cause a trigger. You can set the filter size from zero up to the maximum value of a 32-bit pointer on 32-bit platforms, or the maximum value of a 64-bit pointer on 64-bit platforms. Setting the lower filter value to zero triggers a dump on all allocations.
For example, to trigger dumps on allocations greater than 5 Mb in size, use:
To trigger dumps on allocations between 256Kb and 512Kb in size, use:
If you apply a filter to an event that does not support filtering, the filter is ignored.
You can use the msg_filter suboption to filter on text strings within an exception message, allowing you to reduce the number of dump files produced. This option is supported only for the following events: throw, catch, systhrow, and uncaught.
Use the following syntax to include message filtering in your dump output:
<filter> is a text string from the exceptions that you want to include in the dump file. This suboption supports asterisks as wild cards.
The following example filters
java/lang/VerifyError exceptions that contains the text string class format:
The full syntax is
The heap dump agent uses this suboption to specify the type of file to produce. On z/OS, the system dump agent uses this suboption to specify the type of dump to produce.
You can specify a PHD heap dump file (PHD), a classic text heap dump file (CLASSIC), or both. The default is a PHD file. For example:
-Xdump:heap:opts=PHD -Xdump:heap:opts=CLASSIC -Xdump:heap:opts=PHD+CLASSIC
z/OS system dumps
You can specify a system transaction dump (IEATDUMP), an LE dump (CEEDUMP), or both. The default is an IEADUMP file. For example:
-Xdump:system:opts=IEATDUMP -Xdump:system:opts=CEEDUMP -Xdump:system:opts=IEATDUMP+CEEDUMP
The ceedump agent is the preferred way to specify LE dumps, for example:
The tool dump agent supports two suboptions that can be specified using the
opts subption. You can run the external process asynchronously with opts=ASYNC. You can also specify a delay in milliseconds that produces a pause after starting the command. These two options can be used independently or together. The following examples show different options for starting a new process that runs
opts suboption, the tool dump agent starts the process, and waits for the process to end before continuing.
opts=ASYNC is specified, the tool dump agent starts the process, and continues without waiting for the new process to end.
This option starts the process, waits for the process to end, and then waits a further 1 second (1000 milliseconds) before continuing.
Finally the last example starts the process and waits for 10 seconds before continuing, whether the process is still running or not. This last form is useful if you are starting a process that does not end, but requires time to initialize properly.
One event can generate multiple dump files. The agents that produce each dump file run sequentially and their order is determined by the priority keyword set for each agent. The full syntax for this command is
Examination of the output from
-Xdump:what shows that a gpf event produces a snap trace, a Java dump file, and a system dump file. In this example, the system dump runs first, with priority 999. The snap dump runs second, with priority 500. The Java dump runs last, with priority 10:
The maximum value allowed for priority is 999. Higher priority dump agents are started first.
If you do not specifically set a priority, default values are taken based on the dump type. The default priority and the other default values for a particular type of dump, can be displayed by using
-Xdump:<type>:defaults. For example:
java -Xdump:heap:defaults -version
events=gpf+user filter= file=/home/user/heapdump.%Y%m%d.%H%M%S.%pid.phd range=1..0 priority=500 request=exclusive+compact+prepwalk opts=PHD
You can start and stop dump agents on a particular occurrence of a VM event by using the
Note: range=1..0 against an event means "on every occurrence".
The VM default dump agents have the range suboption set to 1..0 for all events except systhrow. Most systhrow events with
filter=java/lang/OutOfMemoryError have the range suboption set to 1..4, which limits the number of dump files produced on
OutOfMemory conditions to a maximum of 4. For more information, see Default dump agents.
If you add a new dump agent and do not specify the range, a default of 1..0 is used.
Use the request suboption to ask the VM to prepare the state before starting the dump agent:
The available suboptions are listed in the following table:
|exclusive||Request exclusive access to the VM.|
|compact||Run garbage collection. This option removes all unreachable objects from the heap before the dump file is generated.|
|prepwalk||Prepare the heap for walking. You must also specify exclusive when you use this option.|
|serial||Suspend other dumps until this dump is finished.|
|preempt||Applies to the Java dump agent and controls whether native threads in the process are forcibly pre-empted in order to collect stack traces. If this option is not specified, only Java stack traces are collected in the Java dump.|
You can specify more than one request option by using +. For example:
The VM exclusive access mechanism allows a VM thread to halt the activity of other VM threads in a controlled way by using internal VM locks. When the
request=exclusive option is specified for a dump agent, the VM thread that is producing the dump waits for threads that are running Java code to halt, and for garbage collection operations to complete, before the dump file is written. This process helps ensure that the dump has consistent data. When the dump is complete, the mechanism allows the other threads to resume.
By default, only system dumps for
OutOfMemoryError exceptions request exclusive access. Other system dump events typically result from a crash. In these cases, exclusive access is not requested because acquiring locks during a crash can be problematic.
If system dumps are requested by using the
com.ibm.jvm.Dump.SystemDump() API, the default system dump agent settings are used, and exclusive access is not requested. However, if you intend to use the system dump file for Java heap memory analysis, use the following option to request exclusive access when the dump is taken:
These settings avoid capturing a dump file with in-flight data during garbage collection.
As an alternative, you can use the
com.ibm.jvm.Dump.triggerDump() API and specify
request=exclusive+compact+prepwalk on the API call.
For more information about the
com.ibm.jvm.Dump API, see the API reference information.
The default setting of the
request suboption for Java dump files is
request=exclusive+preempt. To change the settings so that Java dump files are produced without pre-empting threads to collect native stack traces, use the following option:
In general, the default request options are sufficient.
Dump output is written to different files, depending on the type of dump and the platform. File names include a time stamp.
|Dump type||File name (AIX, Linux, macOS, Windows)||File name (z/OS)|
|System dump||core.%Y%m%d.%H%M%S.%pid.dmp||%uid.JVM.TDUMP.%job.D%Y%m%d.T%H%M%S (31-bit), %uid.JVM.%job.D%y%m%d.T%H%M%S.X&DS (64-bit) See Note|
|LE CEEDUMP||-||CEEDUMP.%Y%m%d.%H%M%S.%pid See Note|
The tokens used in this table, for example
%Y, are described in Dump agent tokens.
Note: On z/OS, the system dump file name can be set with the
JAVA_DUMP_TDUMP_PATTERN environment variable. The CEEDUMP, which is not produced by default, is stored in the directory specified by
_CEE_DMPTARG or the current directory if
_CEE_DMPTARG is not specified.
System dumps on Linux
Linux does not provide an operating system API for generating a system dump from a running process. The VM produces system dumps on Linux by using the fork() API to start an identical process to the parent VM process. The VM then generates a SIGSEGV signal in the child process. The SIGSEGV signal causes Linux to create a system dump for the child process. The parent VM processes and renames the system dump, as required, by the
-Xdump options, and might add additional data into the dump file.
The system dump file for the child process contains an exact copy of the memory areas used in the parent. The dump viewer can obtain information about the Java threads, classes, and heap from the system dump. However, the dump viewer, and other system dump debuggers show only the single native thread that was running in the child process.
You can use the Linux
kernel.core_pattern setting to specify the name and path for system dumps. The VM dump agents override the Linux system dump name and path by renaming the dump as specified in the
-Xdump options. If the
kernel.core_pattern setting specifies a different file system to the
-Xdump options, the VM dump agents might be unable to change the file path. In this case the VM renames the dump file, but leaves the file path unchanged. You can find the dump file name and location in the JVMDUMP010I message.
Note: If you use the
%t specifier in the
kernel.core_pattern setting, the VM does not rename the dump. The VM cannot determine the exact time that Linux generated the core file, and therefore cannot be certain which Linux dump file is the correct one to rename.