UPDATE for TPTP AG: JVMTI status, decision points

·        September 9, 2005

·        Allan Pratt

Why JVMTI now?

·        It’s the right time for TPTP to enter the Java5 / Java6 fray

·        Consuming products want JVMTI benefits without requiring 4.3+

·        Consuming products that target Java6 runtimes need JVMTI

·        Can be faster / more performant

o       Enabling method events in JVMPI makes the JVM run slow, regardless of how narrowly you target your filters

o       With TI, the parts of the target program that you don’t instrument run full-speed

o       TI Allows better JITs, GCs, and memory managers

o       A TI agent can have less internal cruft than PI agents did

§         Example: not tracking “object move” events just to keep id’s straight

“The Perfect Storm”

·        (A “perfect storm” is when three or more new factors come together to create a large-scale change in the landscape, making independent and isolated decision making inadequate.)

·        Lots of new features and technologies are colliding in 4.2

o       The New RAC, with features like HTTP tunneling

o       Dynamic Probekit / dynamic BCI

o       JVMTI (which requires dynamic BCI)

o       Launch configuration UI for multiple agents

§         Today’s Profile tab is really a piAgent config tab

§         Some implementation strategies for JVMTI require more

o       Aggregating data collection feature

§         Can be done with JVMPI (slow) or using BCI (faster, but requires dynamic BCI, thus depends on BCI strategy)

·        The decisions on these topics can not be made separately

·        This is why it’s the right time to make decisions about this

Wait a minute, isn’t a JVMTI agent easy?

·        No, only the easy part is easy: launching a J2SE app from the Workbench and collecting an execution trace from it.

o       Allan has already shown a prototype of this using Probekit

·        Everything else is harder

o       Memory analysis modes

o       Thread and monitor/lock usage analysis

o       Heap dump

o       WAS startup and “Profile on Server” use cases

·        Attach mode is hardest

o       Agent starts dormant, ready to do anything you ask

o       You attach and send configuration and filters

o       You can detach, then reattach with a different configuration and filters

o       This is a major use case for consuming products

o       JVMPI made this pretty easy: just enable and disable events

o       It’s hard with JVMTI

§         You have to reinstrument classes (if you can).

§         Enabling some capabilities makes the JVM run slower, even if you don’t use them.

§         This motivates a desire to tell the agent what you want to be able to ask it to do later, even as it starts up not doing anything.

Can’t we just make a piAgent that speaks JVMTI?

·        This strategy means making something that is 100% compatible with piAgent’s capabilities, options and command set

·        Call this “Strategy One.”

·        Populates existing models and views

·        A direct drop-in replacement for piAgent on the target machine

·        Requires no UI changes

·        This would be an Isolated project, without dependencies on UI changes and the new RAC

·        User would have to select TI vs. PI agent at RAC config time

·        This is probably possible but has some problems

o       Can’t implement any new features

o       Must implement all piAgent features

§         Some piAgent features are hard or inefficient to replicate, and lack a compelling ROI story: they would not be justified on their own.

§         Example: making object IDs from an instance-level trace match those from “collect references,” for model navigability from trace to heap dump.

·        But mainly, it leaves us in the past

o       Old RAC interface

o       piAgent feature set, with nothing new and no growth path

What’s the alternative?

·        Create a modified piAgent that speaks JVMTI

o       And possibly JVMPI as well! See below.

·        Call this “Strategy Two.”

·        Uses similar options, capabilities, and command set

·        Also populates existing models and views

·        Permits new features

·        Doesn’t force carryover of old ones where it doesn’t make sense

·        piAgent still exists, to satisfy compatibility needs

·        A new agent can use the new RAC and get its benefits

o       Pluggable transports, including HTML

o       Security, including per-user authentication

o       Better agent and communications management

·        Forces our hand regarding the “agent selection” feature

o       Requires launch UI and framework changes

o       Lets the user select PI vs. TI at launch time

o       User sees options appropriate to selected agent

·        Future-looking, not past: something to build on

Intel’s original proposed implementation strategy

·        The people @intel.com:

o       Guru.Nagarajan, Asaf.Yaffe, Israel.Hirsh, Chris.L.Elford

·        Based on Strategy One (full 100% direct replacement)

·        A direct piAgent replacement with an agent that can target JVMPI or JVMTI

·        Intel has system they call MPI which abstracts differences between JVMPI and JVMTI (and also MS.Net)

·        An agent written to MPI will work against any of those systems

·        Today, MPI is limited to method trace and object alloc/free events

·        Intel could port piAgent to use MPI instead of JVMPI, enhancing MPI as necessary to achieve this

·        The result would be an agent with piAgent’s capabilities, but which works with JVMTI also: a direct piAgent replacement

·        Another result of this is Intel contributing MPI to TPTP, so it’s available for other agent writers to use

o       Makes agent writing much less troublesome

§         MPI not only abstracts JVMPI vs. JVMTI, it also deals with picky little JVMPI implementation differences. Even in an all-JVMTI world (2006-7 and beyond), it has value is in isolating picky little JVMTI differences too, making tool vendors’ jobs easier.’

o       This API would start out in TPTP as “provisional.”

o       Having tools vendors actually write agents using MPI will put us back in the “agent selection” and “multiple agents” world

·        Allan feels this strategy “starts in a hole and finishes in the past”:

o       The “hole” is that MPI doesn’t support what piAgent needs yet, would need extensive enhancements.

o       The “finish in the past” is a direct piAgent replacement that uses the old RAC and has no new features or growth path.

Modified implementation strategy

·        Based on Strategy Two above

·        Create a new agent similar to piAgent

·        Use the MPI system

·        Start with piAgent and keep appropriate parts

o       Don’t really start from scratch

·        Make it a “new RAC” agent

·        It need not be 100% piAgent compatible

·        It can have new features and discard some old ones

·        It still populates existing TPTP models and views compatibly

·        Because it’s based on MPI, it’ll target JVMPI too

·        TPTP still gets Intel’s contribution of MPI as an agent implementation foundation

·        This strategy requires the multiple-agent UI changes

o       So you can choose to use piAgent for compatibility

·        This strategy has a dependency on the new RAC

·        Because of the new RAC and launch UI dependencies, this carries risk: if either of those doesn’t deliver in 4.2, neither will JVMTI

o       Well, the new RAC part could be optional. It’s not central to the success of the new agent.

o       Regardless, the entanglement will add schedule risk

·        The payoff is having a forward-looking data collection system we can grow from and build on

So: where does that leave us?

·        TPTP planning and AG people need to make some decisions

o       Commit to a JVMTI capability in 4.2?

o       If so, choose Strategy One (direct piAgent replacement)

§         Tactically seductive, lower risk

§         Result is largely maintaining status quo

o       …or Strategy Two (a new agent)?

§         Commit to UI changes and new RAC ready for 4.2

§         Accept risks

§         Result is a higher-function, forward-looking system

o       …or ask us to go back to the drawing board.

o       Also: Consider / approve contribution of MPI as a public, supported TPTP interface?

§         Intel can contribute it and use it for JVMTI agent implementation whether or not we document it as a public, supported API.

§         It will start out “provisional” as new APIs should.

 

End of main section

 

Supporting material: JVMTI overview

·        JVMTI is a new profiling interface for Java, a replacement for JVMPI.

·        It was created because supporting JVMPI forces JVM vendors to make some unhappy tradeoffs.

o       Sending method entry/exit events adds complexity to the JITter, exception handlers, etc. When enabled, they are global (for every method), not just for those you are interested in.

o       Sending GC start/finish events and object alloc, move, and free events limits your memory manager and GC design.

o       Getting a heap dump meant creating a single big array, limiting the size and performance of taking heap dumps.

o       Being able to enable any event at any time in the future means you can’t optimize your JVM for the things the agent wants vs. those things it’ll never be asking for.

o       And more…

·        How is JVMTI different from JVMPI?

o       It does not, in general, generate method entry/exit events

§         Instead, you’re expected to use BCI (bytecode instrumentation) to call out to a data collector if you want.

o       It doesn’t generate object alloc events, either

§         You have to instrument the “new” bytecode, or constructors, or something.

o       Heap dumps are handled in a completely new way.

o       And more…

·        What’s the migration path?

o       Java5 JVMs (now) support both JVMPI and JVMTI.

o       Java6 JVMs (2006) support only JVMTI.

Supporting material: Attach mode

·        Why “attach mode” is possible:

o       Most JVMs support “retransform classes.” This means you don’t need to do BCI on all methods of all classes from the beginning “just in case” somebody asks for a certain data collection mode in the future.

o       This makes supporting “attach mode” harder than it is in JVMPI, but we’re not afraid of a little work.

·        However: “attach mode” is harder than it looks.

o       With JVMTI, you reserve “capabilities” at startup time for those things you’ll want to do later.

o       Some capabilities are expensive to enable, even when you aren’t using them: they cause the JVM, JIT, or GC to run slower.

o       It’s a shame to enable them without knowing they’ll be wanted.

o       The ability to reinstrument classes is critical for efficient attach mode, but there are limits.

§         You can’t add fields to a class, something your BCI runtime is likely to want.

o       So you add your fields to all classes on their initial load, “just in case.” Greater footprint.

§         It won’t be available in all JVMs.

§         It might be expensive to enable: it might cause all code in the JVM to run more slowly, with extra levels of indirection on each method call.

§         It’s potentially risky: it might be a less-tested part of the JVM’s execution path. Just ask the debugger people about hot-swap, which is a similar JVM feature.

Supporting material: piAgent use cases not to forget

·        piAgent supports use cases beyond the launch dialog.

·        Profile on Server is a RAD feature that knows the incantation “-XrunpiAgent:server=enabled” …

·        Users start Java processes outside the Workbench

o       They have been taught they can put –XrunpiAgent in their scripts.

o       Need a migration plan and document for them.

·        Standalone mode

o       piAgent-specific options are on the command line and in INI files.

o       Output goes to a file, can be imported into the workbench.

Supporting material: features that JVMTI enables

·        Examples of features that JVMTI enables:

o       With JVMTI, you can get a filtered heap dump: one that only dumps certain objects that were deemed “interesting” at some time in their lives. Say you identify a particular collection as “interesting.” You can tag the objects that are in that collection, and get a heap dump that only includes those objects, not the whole heap.

o       You can get better performance by limiting object free notification to those objects you deem “Interesting.” This can be done not just by the object’s class, but by its behavior during its life. By tagging objects, you can get “free” notifications for those Strings added to a certain collection, not all String objects.

Supporting material: The launch UI

·        Today’s Profiling launch UI is really a piAgent configuration UI.

·        The RAC today has no UI for selecting which JVM to start: it’s set at install time in the RAC configuration file.

·        The launch configuration system has no way to select from among multiple agents to start as “bound agents” (that is, those started via options on the Java command line). It always uses –XrunpiAgent.

·        (You actually want to be able to select multiple agents in a JVMTI world: it’s more modular and extensible than the monolith that piAgent is.)

·        If there were a way to pick an agent, you’d like to see only those profiling types and options that apply to it.

·        When you launch for attach mode, there’s no way to pick an agent, or to tell the agent you’ve picked the scope of what you will want it to do for you, and thus let it preconfigure itself and the JVM for those capabilities and not others.

·        When you attach to a running agent (however it was launched), you want to see just the profiling types and options available from that one agent.

·        A possible solution is to have a tab per agent, and introduce an agent selection step into the launch sequence.

o       Defaults and “Don’t ask me again” will be important here.

·        But wait…

o       The launch UI isn’t the only way piAgent gets started.

o       Consider RAD’s “Profile on Server” or batch files that set up the command line of a target process like a server.

o       These assume piAgent, would have to grow more flavors.

o       They’d also have to grow mechanisms to tell the agent (at startup time) what they are going to want it to do when they attach, so it can preconfigure itself.

Supporting material: Existing cases of multiple agents

·        IBM’s IPOT agent

o       The “IPOT” (IBM Performance Optimization Toolkit, on devworks) is an agent that already suffers from the limitations of the launch UI.

o       When you start with IPOT’s agent and later attach from the workbench, IPOT shows up as one of the agents you can attach to.

o       But then in the Profiling tab you are presented with the usual piAgent profiling types and options, even though you haven’t attached to piAgent.

o       You are also presented with the IPOT profiling type and configuration options, since IPOT plugged its type in to the Profiling tab.

o       But you are also shown the IPOT profiling type and options during a “launch” (not “attach”), even though you can’t launch it that way.

o       IPOT addressed some of this by creating a new launch configuration type, which is not the most desirable outcome: it limits the user’s ability to configure multiple agents simultaneously.