Testers

Committers: Darin Wright, Michael Rennie, Curtis Windatt, Samantha Chan (Kevin Jiang will substitute for Sam for RC1), Pawel Piech

Community: Kevin Jiang, Anyone who wants to help

JRE Grid

The following JREs should be considered when testing. Committers have been assigned a JRE and platform to test on. The community can chose any JRE and platform.

Test Grid

Feature Testing

Java 6.0 Features

• All Instances
  • Should not be available when not debugging with 1.6 VM
  • Variables view popup
  • Java Outline view popup (type and constructors only)
  • Java editor popup (type name or constructor name selected)
  • Change number of instances to show on Heap walking preference page
• All References
  • Should not be available when not debugging with 1.6 VM
  • Variables view popup
  • In-line as variable in the variables view (change in view preferences and preference page)
  • Change number of references to show on Heap walking preference page
• Force Early Return
  • Should not be available when not debugging with 1.6 VM
  • Works as an evaluation (from display and details pane)
  • Run menu and key binding
  • Java editor context menu

Back to top

---

Evaluations

• Switch statements should be supported in evaluations
• Java 5.0 features
  • Generic types
  • enhanced for loop syntax
  • varargs
  • autoboxing
  • enumerations
• From various contexts
  • Java Compilation Unit editor
  • Java Classfile editor
  • Display view
  • Details area
  • Snippet editor
  • in class files without attached source
  • for arrays as the receiving 'this' context
• Use key bindings and context menus
• Use code assist (key binding and context menu)
• Inspect vs. Display
  • Note that in the Display view and Snippet editor, "Display" prints the result in-line
  • Everywhere else, popups are used to display/inspect the result
• Persisting the result - i.e. move the result to the Display view or Expressions view
• In a clean workspace, popups should size reasonable on the first use, after which the popup sizes are persisted and reused

Back to top

---

Context Launching

• Resource properties page
  • Available on launchable resources
  • Only show listing of launch configurations that apply to selected resource
  • Create/delete/edit configuration
  • Changes to configurations are cancelable
• Launching
  • Launch button(s) tooltip(s) should track selection changes (update as selections change)
  • Non-launchable resource should prompt to run the parent (if set to do so, toggle pref on Launching preference page)

  • Running and debugging applications has been simplified to run or debug the selected file or active editor. Use the "Run/Debug > Launching > Launch the selected resource or active editor" preference to enable this behavior. When the selected resource (or active editor) is not executable, you can opt to launch the associated project by selecting "Always launch project if selected resource cannot be launched".

    Pressing the Run or Debug toolbar button will launch the selected resource (as will the Run or Debug action in the top level Run menu). When no launch configuration exists for the selected resource, you will be prompted to select how to run or debug an application if there is more than one way to launch the selection. For example, when running a JUnit test you can launch it as a Java application or JUnit test. This will create a launch configuration and entry in the launch history for the application.

    When the same resource is launched again, the most recent configuration in the launch history associated with that resource is re-launched.

Back to top

---

Launch Options

• Build before launch
  • build is scoped to relevant projects
  • Standard job progress dialog is displayed for the following cases. The "Run in Background" button is never present.
    • Launching in foreground (un-check "Launch in background" option on common tab)
      • Waiting for ongoing build to complete before launching
      • Building before launching
  • The progress dialog never appears when launching in the background
    • When waiting for an ongoing build to complete before launching a node appears in the debug view in italics showing "waiting for build"
• Prompt to launch in face of compilation errors
  • always/prompt
  • prompt should tell you which project(s) contain errors
  • irrelevant errors should not cause prompt (i.e. Ant errors should not prevent Java launch)
• Prompt to launch in debug mode when breakpoints present
  • always/never/prompt
• Save dirty editors
• Launch in foreground/background (controlled via common tab on launch config)

Back to top

---

Launch Modes

• For this test, you require the use of the launch modes test plugin available here.
• Preferences Dialog
  • Ensure launch tool preferences page is available, even if there is no conflicting tooling. Page should be disabled with an explanation if no conflicting tooling
  • Ensure page can be searched for with tree viewer searching
  • Ensure there is a help topic for the page
  • With duplicate tooling select a variety of items in both viewers
  • Ensure once an item is selected in the preferred tooling viewer it cannot be unselected (checked)
  • Changing selected tooling without saving changes should not modify saved choices
  • Selecting preferred tooling from a quick launch or the launch configuration dialog should be reflected in the pref page (and changeable)
• Launch Configuration Dialog
  • Ensure incompatible modes are flagged correctly (check the widget on the Coverage Tab) with an error message and link
  • Link should present complete list of support mode combinations
  • Selection of mode set should update dialog and remove error
  • Duplicate delegates should be flagged correctly with error message and link
  • Link should present complete set of contributed launch tooling
  • Selection of tooling should remove error from dialog, and be reflected in the preference page
• Launch History
  • Resolution dialog should be presented if duplicate tooling found and no choice has been made
  • Choice of tooling should allow launch to continue and be reflected in the preferences page
  • Canceling either choice from the resolution dialog should cancel the launch as well

Back to top

---

Launch Configurations

• Launch History / Favourites (toolbar drop down)
• New launched configurations added to top of history
• Launching from history changes order of history
• Test the 'organize favorites' dialog
• Configurations should not be in both history and favorites
• Ctrl-clicking a configuration should open to launch configuration dialog
• Ctrl-Shift-clicking a configuration should offer to remove it from the history/favorites
• Importing and exporting of launch configurations through File > Import/Export
• Using java search to find launch configurations that launch a class
• Launch Configuration Dialog
  • Creating/Deleting configurations of different types
  • Deleting multiple launch configurations, ensure selection is preserved/updated
  • Filtering configurations from tree
  • Test error handling, apply/revert buttons, and launching
  • Check if tabs make configuration dirty even if no changes are made

Back to top

---

Debug View Management

• Switch to Debug perspective
  • When launching (always/never/prompt)
  • When suspending (always/never/prompt)
• Activate the Debug view when program suspends
  • Debug action set should also open automatically when a debug view is opened
  • Relevant debug views should automatically open when stack frame is selected
    • In debug perspective by default
    • In other perspectives as configured on the "View Management" pref page (also available in Debug View drop down menu)
    • Use PDA debugger to try debugging different languages in same perspective (hope to have this available - example from draft debug article)
    • Automatically opened views should close when debugging ends
• Activate the workbench when a breakpoint is hit

Back to top

---

Breakpoints

• Breakpoint groups
  • show breakpoints, breakpoint types, breakpoint working sets, files, projects, resource working sets, advanced
  • auto-add to default breakpoint working set
  • drag & drop between breakpoint working sets (move)
  • copy & paste between breakpoint working sets
  • stratum breakpoints should group by stratum (eg. JSP breakpoints).
• Creation of different types of breakpoints (line, class load, method, exception, and watchpoint).
  • No breakpoints can be set in interfaces
  • Breakpoints can be set in external source file (file from OS that does not belong to a project)
  • Toggle Line breakpoints - double click in ruler, Run menu action (menu & keybinding), context menu in outline view
  • Toggle Method breakpoints - double click in ruler, Run menu action (menu & keybinding), context menu in outline view
  • Class Load Breakpoint - double click in ruler, Run menu action (menu & keybinding), context menu in outline view
  • Toggle Watchpoints - double click in ruler, Run menu action (menu & keybinding), context menu in outline view
    • watchpoint cannot appear on 'final' field
    • watchpoint cannot appear on 'static final' field
  • Exception Breakpoints - Run menu action, hyperlink from stack trace, breakpoints view
• Refactorings
• breakpoints should be updated if enclosing type is refactored
• method breakpoints should be updated after refactoring method
• line breakpoints should be updated if enclosing method refactored
• watchpoint should be updated if field refactored
• Skip all breakpoints option (manual use of the toggle)
  • groups and breakpoints should be rendered with "skip" (crossed out) line
• Import/Export
  • export wizard is properly initialized based on breakpoint selection in Breakpoints view
  • wizard settings are persisted for subsequent import/export operations
  • breakpoint working set information is exported and are re-created on import
  • existing breakpoints are updated with imported information (for example, hit count, etc.)
  • breakpoint working sets are recreated properly
• Option to open inspector on exception when an exception breakpoint is hit
  • Java > Debug: "open popup when suspended on exception"
• Option to delete breakpoints vs. working set when "delete" pressed on a working set
  • choices are persisted
• Preference for suspending Thread by default on newly created breakpoints

Back to top

---

Stepping Tests

• Run to Line
  • Context menu
  • Run menu action and keybinding
  • Classfile and Compilation unit editor
  • Preference to "skip breakpoints during run to line" (on main Run/Debug pref page)
• Step Into Selection
  • Run menu action and keybinding
  • Hyperlink action using Ctrl+Alt+Click
• Step Filters
  • Step filters on/off should affect stepping (no stepping within filtered types)
  • Toggle step filters action
  • Edit step filters via context menu, preference page
  • Persistence of step filters, view open/closure, workbench restart
• Drop To Frame

Back to top

---

Logical Structures

• View variables with logical structures on/off
• Stepping with logical structures on/off
• Works for
  • java.util.Collection (flat list)
  • java.util.Map (list of map entries)
  • java.util.Map (key/value)
• Selectively turn off logical structures via context menu "Show As > ..."
• User defined logical structures
  • more than one structure for a type
    • all appear in "Show As > .." menu
    • can choose between structures properly
  • structures persist restore properly across workbench invocations
  • structures with compilation errors
  • structures with runtime errors
  • when two different types (A,B) share the same possible logical structures, their enabled (selected) structure is shared (i.e. if you select "show A as X" then B will also show as X.
• Import/Export logical structures

Back to top

---

Watch Items

• Create
  • From context menu "Add Watch Expression"
  • From variables view context menu "Watch"
  • Dragging variables from variables view to expressions view
  • Dragging text to expressions view
• Persist
• Reordering of expressions view by dragging and dropping
• Enable/disable
• Update while stepping
• Update when selecting a different stack frame
• Errors appear as children of a watch expression (compilation errors, runtime errors)

Back to top

---

Context Menu Launching

• Run/Debug from the context and top level Run/Debug cascade menus. Note that before a shortcut's property tester is loaded, it will appear in the context menu by default. For example, the Java application shortcut requires the "hasMain" tester from the Java debug plug-in, and thus, until the plug-in is loaded, the shortcut will appear on all Java files whether they have a main method or not.
  • Java Applications (with main, without main, compilation unit, type, method, classfile, package, project)
  • Java Applets
  • JUnit tests
  • Ant build, Ant build... (build file, targets in outline and Ant view)
• The context menu appears in
  • Resource navigator
  • Package explorer
  • Outline view
  • Editor
• Use keybindings to launch (ALT-Shift-D/X)
  • an active editor
  • a selection in an outline/navigator
• Ctrl+click on a launch shortcut (not all shortcuts have support)
  • Should select the most recently used configuration for the launchable class
  • If no recently used configurations exist, the type should be selected

Back to top

---

Runtime Classpaths

• The default classpath should track a project's buildpath
• Using a non-default bootpath
  • Prepending and appending to the default bootpath
  • Using an explicit bootpath (i.e. without prepend/append)
• Using a non-default classpath
  • different types of entries
  • new entry type - String Variables
• Option to only include exported items in runtime classpath (by default, all items are included).

Back to top

---

Source Lookup

• The default source lookup path should track changes in the default buildpath (when using a default runtime classpath). If a non-default runtime classpath is used, the default source lookup path should track the runtime classpath.
• Searching for duplicate source files
• Modifying the source lookup path when a source file cannot be found
• Types of entries on the source lookup path
  • Internal archives
  • External archives
  • Java projects
  • Java classpath variables
  • Directory (external)
  • Folder (in the workspace)
  • Java Library (JRE, required plug-ins, etc).
• Multiple source roots in archives

Back to top

---

Threads & Monitors & Thread Groups

• Monitor information is shown in the debug view
  • No information/errors for VMs that do not support monitor information
  • Monitor information appears for suspended threads
  • Monitor information is not displayed for running threads
  • Ensure valid deadlocks are detected
    • threads are rendered red (preference color), and have deadlock overlay
  • Locks support resume/terminate
• Thread groups
  • Thread groups can be displayed in debug view by turning on the "Show Thread Groups" option in the view menu
  • Thread groups support terminate

Back to top

---

Hot Code Replace

• Valid HCR
  • method changes (existing)
  • changes to statements inside method work on SUN and IBM vms
  • HCR fails adding/removing/changing signature of method on SUN vm
  • HCR allows methods to be added for IBM J9 vm
  • changes in existing instance variable initializer
  • affected frames are automatically popped (unless top frame or native method on the stack)
  • change method in declared inner type
  • change method in anonymous inner type
  • default and non-default packages
• Manually drop to frame
  • works in all methods except last method on stack
  • a native method cannot be dropped (and thus a method above a native cannot be dropped/re-entered)

Back to top

---

Installed JREs

• Changing the default JRE triggers a build if the default JRE is referenced by a project.
• JREs allow a javadoc location to be set per library in the JRE
  • Changing the javadoc location on a JRE updates the buildpath javadoc location
  • Changing the javadoc location on a buildpath (JRE library) updates the JRE
• Changing a javadoc location or source attachment does not causes an explicit bootpath to be used when launching with that JRE.
• Copy a JRE definition
• Import/export of JRE preferences

Back to top

---

Execution Environments

• Specifying a build path via EE
  • Changing the default JRE for an EE updates buildpath
• Specifying a runtime JRE via EE
  • Changing the default JRE for an EE updates the default and/or selected EE runtime JRE
• Modifying Access Rules
  • Adding an access rule should cause a rebuild (if auto-build is enabled)
  • Ensure that the access rules defined in a system library are not editable/removable
  • Add a new forbidden rule, ensure accesses to forbidden types is flagged as an error
  • Add a new discouraged rule, ensure access to discouraged types is flagged as a warning
  • Ensure access rules can be removed
• Over-riding Access Rules
  • Create a rule that over-rides a system library rule, and ensure it works

Back to top

---

String Variables

• Use String Variables in launch configs
  • VM args, program args, main type name
• String Variable selection dialog

Back to top

---

Console

• Show when output to standard out/err
• Console title becomes bold when content changes (but not if on top/auto activated)
• Console input - typing, pasting, pasting multiple lines
• CTRL-Z/D for EOF when doing console input
• Pin a console - pin action is specific to console view. When pinned, console should not change in that view
• Scroll lock - specific to console view. When locked, the console in that view should not scroll
• Switching between consoles - use the drop down to toggle between consoles. Clicking on the drop-down should toggle between the last two consoles shown
• Open new console - CVS and Java Stack Trace Console
• Stack trace console
  • Pasting and navigating stack traces
  • hyperlinks are colored by hyperlink color preferences
  • editing stack traces/clearing console
  • contents are persisted across workbench invocations and close/re-open
  • formatting
• Hyperlinks for stack traces, link to source code
• Hyperlinks for exception names allow creation/modification of exception breakpoint settings
• Use alternate console encoding to display DBCS character set
  • remember to use "-Dfile.encoding=UTF-8" (or whatever encoding) on the VM args
• Open more than one console view
  • pin a view selectively
  • scroll lock a view selectively
  • key-bindings work in views - FORMAT in Java stack trace console/CTRL-D/Z in process console
• Multiplex output via common tab
  • a file
  • the console
  • both
  • none
• Link to console preferences from context menu
• Next / Previous page switcher
  • Create a keybinding for Next / Previous page for the Console view
  • Try the new key binding from every kind of console available; should be available for all console kinds

Back to top

---

Variables

• Find Variable support and key-binding
• Changed variables appear red (preference color) when stepping
• Variables remain expanded/selected as stepping
• When switching between stack frames (without resuming), the variables expanded/selected are maintained (i.e. expand and select for frame 1, then select frame 2 and expand/select. when re-selecting frame 1, the original state is restored).
• toString() in-line in the tree viewer
  • for all variables
  • for variables with detail formatters
  • only in the details area
• Debug hover works for symbols with the same name and different values (inherited fields), showing correct value
• Changing variable values
  • via "Change Value..." context action
    • dialogs are different for primitives & objects
    • evaluations are supported
    • editing a String value via evaluation and direct editing
  • via CTRL-S in the details area
    • evaluations for objects
    • direct editing for primitives
• Column support
  • Turn columns off/on
  • Colum size automatically until user changes them
  • Sizes are persisted
  • Select & persist columns to display
  • background "changed" color highlighting

Back to top

---

Refactoring

• launch configuration main type is updated after type rename, type move, package rename
  
• launch configuration project is updated after project rename, package move, type move
  
• Java breakpoints are updated after type rename, type move, package rename, package move, project rename
  
  • VM suspend at the new location
    
  • breakpoint attributes are preserved
    
  • label is updated
    
  • marker location is updated (text selected when double-clicking on a breakpoint in the breakpoint view)

Back to top

---

Applets

• Main tab
  • specify a project via typing/browsing
    
  • specify a nonexistent project (should see error if does not exist)
    
  • specify an applet class via typing/searching (no validation is performed on type name in the tab - error occurs on launch if an invalid type is specified)
    
  • specify a class that is not an applet (no validation is performed - error occurs on launch if invalid)
    
  • specify default applet viewer class
    
  • specify an explicit applet viewer class
• Parameters
  • specify various width & height
    
  • specify parameters (available via Applet#getParameter(...))
    
  • specify applet name - allows Applets to communicate with each other via AppletContext#getApplet(...)

Back to top

---

Remote Debugging

• connect to a remote VM
  • try "suspend=y" and "suspend =n" options
  • try different sockets
  • try connecting to a socket that has no VM
  • try "localhost" and "127.0.0.1"
• have remote VM connect to waiting debug session
  • try different sockets
  • try terminating waiting debug process
  • try stopping waiting job
• terminate should only be enabled when the config specifies that terminate is supported

Back to top

---

JSP Example

• Try JSP debugging example, as documented in the "org.eclipse.jdt.ui.examples.javafamily" plug-in ReleaseNotes.html. The instructions are in the section titled "Debugging a JSP"
  • Tomcat launch configuration
  • Breakpoints in JSP files
  • Enable/Disable installed JSP breakpoints
  • Enable/Disable un-installed breakpoints when a target/server is running
  • Changing the stratum displayed via context menu in debug view
  • Stepping in a Java file from a JSP file and stepping back (source lookup should work)
  • Evaluation in the context of a JSP file (requires adding jsp-api.jar, jasper-runtime.jar and servlet-api.jar from /common/lib to the project's build path)

Back to top

---

Stress Tests

• Stepping with many threads, while breakpoints are hit on other threads
  • Expansion and selection are maintained properly in the Debug View
  • Othter suspended threads should not interfere with the current stepping thread
  • Selection should remain on the stackframe that is being stepped
• Stepping with many variables - Debug a testcase with many variables. Expand many variables and set selection at the bottom of the Variables View. Do a few Step Over. Test with logical structures turned on / off.
  • The Variables View should properly maintain expansion and selection
  • The Variables View should not scroll to the top unexpectedly
• Test maintaining variables expansion and selection when switching stackframes. Debug a testcase with many variables. Expand many variables and set selection. Switch between different stackframes.
  • Verify that expansion and selection are maintained properly in the Variables View
  • The Variables View should not scroll to the top unexpectedly
• Test stepping quickly by holding down the step over key. (F6) Test with logical structures turned on / off. Test with thread filters turned on / off.
  • Verify that expansion and selection are maintained properly in the Debug View
  • Verify that expansion and selection are maintained properly in the Variables View
  • Verify that source lookup is performed properly.
  • Verify that debug actions are properly enabled/disabled
• Toggle logical structures quickly - Debug a testcase with many variables. Some of these variables can be displayed differently when logical structures is turned on. (e.g. Hashmap) Expand many of these variables and then toggle the logical structures action quickly.
  • Verify that expansion and selection are maintained properly in the Variables View
  • Verify that there is no blank lables in the Variables View

Back to top

Non-Java Testing

Memory View

To test the Memory View, you will need the example.debug.memoryview sample. You can get the sample from the "Advanced Tutorial" from EclipseCon 2007.

• Add memory block
  • Try adding multiple memory blocks by using a comma delimited list.
  • Try adding a memory block with empty expression
• Remove memory block
  • Remove single memory block
  • Remove multiple memory blocks
  • Remove all
• Add Memory Rendering
  • Default renderings get added automatically when a memory block is added.
  • Add rendering by clicking the "+" button
  • Add rendering from the Add Memory Rendering tab
  • Add multiple renderings
• Remove Memory Rendering
  • Default renderings get added automatically when a memory block is added.
  • Add rendering by clicking the "+" button
  • Add rendering from the Add Memory Rendering tab
  • Add multiple renderings
• Renderings
  • Make sure all the these renderings can be created: Hex, Signed Integer, Unsigned Integer, ASCII, Hex Integer
  • Bring different renderings to the top. Make sure that the tab label shows the address of the memory block when it is brought to the top.
  • Content of the renderings get updated after each step.
  • Renderings should mark memory that has changed.
  • Memory shows up as grey if the rendering does not have cache to compute changes. e.g hidden when the user is stepping
  • Scroll to end of buffer and make sure memory is reloaded. Try with scrollbar and cursor.
  • Top index and curor in rendering are maintined after rendering update.
• Synchronization
  • Toggle synchronization on and off
  • Scroll Bar
  • Table Cursor
  • Format...
• Reset to Base Address
• Go to Address: Context Menu and CTRL+G. Try different modes.
• Auto vs Manual Loading
  • Try auto loading
  • Try manual loading
  • Try different buffer sizes
  • Try page up and page down actions during manual loading. Try shortcut keys: CTRL+sHIFT+> and CTRL+SHIFT+<
  • Make sure buffer size and loading preferences are preserved after workbench restart.
• Try swtiching between memory blocks: Tree viewer and Switch Memory Monitor action
• Memory View Layout
  • Show and Hide Memory Monitors Pane
  • Show and Hide Renderings Pane
  • Try with having multiple memory views/li>
  • Preferences are persisted across workbench restart
  • Vertical vs Horizontal Orientation

Back to top

---

Modules View

• Modules View can be populated with flexible hierarchy
• Find...
• Details pane can be popuated

Back to top

---

Registers View

• The Registers View can be populated with Register Group and Registers
• Register values get updated when their values change.
• Modify register values
• Find a register
• Copy registers
• Turn column on and off
• Show and hide columns
• Details pane shows details of registers
• Top index and expansion of registers are maintained after each step or switching stackframe.

Back to top

---