HTTP Connection Tracer Diagnostic Utility

NOTE: this page is out of date, and should be considered deprecated. See the HTTP Tracer wiki page for current inforamtion.

Problem

In the past we have encountered many different 'going off to the internet problems' in various parts of our tools (on the development side and the server side). As you know these problems have proven to be a real pain for support and development .... they're difficult to track down and take a great deal of effort to reproduce. I'm sure these are also a real pain for customers who are nagged by many different variations of the 'going off to the internet problem'.

The solution (for diagnosis)

I've finally found a few hours to write an "HTTP Connection Tracer" tool to help us manage these 'going off to the internet problems' Really the tool has two purposes:

I should emphasize that this tool is not intended to be a 'solution' ... just a useful short term aid until a time that we can provide 'proper' fixes. Below I've described how to install \ and configure the tool and how to use it. Please help test this out! If you agree that it proves useful I would encourage you to direct users to utilize this when helping address these sorts of problems.

Please address any questions, comments or suggestions in bugs or via the wtp-dev@eclipse.org mailing list.

Installation

Download the HTTP connection tracer diagnostic utility jar file.

The httphandler.jar file must be added to the JRE's lib/ext directory. This must be done for each JRE that you're using in your environment. So if you're developing your code with eclipse and deploying to a server (and you want to get trace information for both) you'll need to make the JAR available for both the eclipse JRE and the server's JRE.

Configuration

When you launch eclipse or run your sever, you'll need to specify some vmargs to activate the "HTTP Connection Tracer" tool. Here's an example of how you'd do this on the command line for Eclipse.

eclipse.exe -vmargs -Djava.protocol.handler.pkgs=com.ibm.etools.www.protocol -DurlMapLocation=C:\map.txt -DurlLogLocation=C:\log.txt

Here's a description of what each argument does ....

name example value description
java.protocol.handler.pkgs com.ibm.etools.www.protocol Tells the JVM to utilize the "HTTP Connection Tracer" tool.
urlMapLocation C:\map.txt Specifies the location of a 'properties' file where URL mapping information can be provided. Note that if no map location is specified then URL redirection never occurs.
urlLogLocation C:\log.txt Specifies the location of a log file where logging information will be written. Note that if no log location is specified the logging information will be written to System.out. Note also that if no HTTP requests are made, then no log or output is produced. If there is no output, you may want to confirm your setup by running a small test program that explicitly does a URL connection. See bug 216524 for an example.

The Map File

Here's an example of what the map file looks like....

http://www.hello.com/bar.xsd =file:///D:/my-schema-cache/bar.xsd
http://www.ibm.com/foo.xsd=file:///D:/my-schema-cache/foo.xsd

Notice its just a list of mapping pairs (separated by an '=' sign) that specifies how a web address should be redirected to a local file system address. By editing this file, a customer can workaround nagging 'going off to the internet problems' until a proper fix is delivered.

The Log File

Each time the JVM attempts to create an HTTP connection, an entry is added to the log file. Below I've shown an example log file entry. It consists of three interesting pieces of information...

URL request - provides the URL address for the connection
URL mapped - provides the 'mapped' address (as specified in the 'map.txt' file) to help the customer see that the URL has been succesfully redirected
STACK TRACE - dumps of stack to help the support and development teams understand the code paths involved in creating the connection request

Note that the presense of an entry in the log does not neccessarily imply an defect in the product. Some attempts to create an HTTP connections are expected. So I'd encourage you to scrutinize the entries in the log file. At the very least the logs will provide you some useful stack trace information that will help developers understand the nature of the HTTP connections.

Example Output


URL requested : http://www.ibm.com/foo.xsd
URL mapped : file:///D:/workspaces/corona-test/XMLExamples/substitutionGroup/Catalogue4.xsd
STACK TRACE :
java.lang.Exception: dumpTheStack
   at com.ibm.etools.www.protocol.http.Handler.openConnection(Handler.java:156) 
   at java.net.URL.openConnection(URL.java:924) 
   at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source) 
   at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown Source) 
   at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) 
   at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) 
   at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) 
   at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source) 
   at javax.xml.parsers.SAXParser.parse(Unknown Source) 
   at org.eclipse.xsd.util.XSDParser.parse(XSDParser.java:175) 
   at com.ibm.xsdeditor.internal.wizards.XSDSelectIncludeFileWizard.doLoadExternalModel(XSDSelectIncludeFileWizard.java:183) 
   at com.ibm.xsdeditor.internal.wizards.XSDSelectIncludeFileWizard$XSDURLPage.openExternalSchema(XSDSelectIncludeFileWizard.java:293)
   at com.ibm.xsdeditor.internal.wizards.XSDSelectIncludeFileWizard$XSDURLPage.access$0(XSDSelectIncludeFileWizard.java:271) 
   at com.ibm.xsdeditor.internal.wizards.XSDSelectIncludeFileWizard$1.run(XSDSelectIncludeFileWizard.java:318) 
   at org.eclipse.jface.operation.ModalContext.runInCurrentThread(ModalContext.java:303) 
   at org.eclipse.jface.operation.ModalContext.run(ModalContext.java:253) 
   at org.eclipse.jface.wizard.WizardDialog.run(WizardDialog.java:752) 
   at com.ibm.xsdeditor.internal.wizards.XSDSelectIncludeFileWizard.performFinish(XSDSelectIncludeFileWizard.java:130) 
   at org.eclipse.jface.wizard.WizardDialog.finishPressed(WizardDialog.java:609) 
   at org.eclipse.jface.wizard.WizardDialog.buttonPressed(WizardDialog.java:305) 
   at org.eclipse.jface.dialogs.Dialog$2.widgetSelected(Dialog.java:506) 
   at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:89) 
   at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java(Compiled Code)) 
   at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:796) 
   at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:2772) 
   at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java(Compiled Code)) 
   at org.eclipse.jface.window.Window.runEventLoop(Window.java:668) 
   at org.eclipse.jface.window.Window.open(Window.java:648) 
   at com.ibm.xsdeditor.internal.properties.section.NamespaceAndSchemaLocationSection.widgetSelected(NamespaceAndSchemaLocationSection.java:159)
   at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:89) 
   at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java(Compiled Code)) 
   at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:796) 
   at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:2772) 
   at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:2431) 
   at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:1377) 
   at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:1348) 
   at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:254) 
   at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:141) 
   at org.eclipse.ui.internal.ide.IDEApplication.run(IDEApplication.java:96) 
   at org.eclipse.core.internal.runtime.PlatformActivator$1.run(PlatformActivator.java:335) 
   at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:273) 
   at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:129) 
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:85) 
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:58) 
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:60) 
   at java.lang.reflect.Method.invoke(Method.java:391) 
   at org.eclipse.core.launcher.Main.basicRun(Main.java:185) 
   at org.eclipse.core.launcher.Main.run(Main.java:704) 
   at org.eclipse.core.launcher.Main.main(Main.java:688) 

The solution (for code)

While there is no one solution for all situations where you want to avoid or minimize network access, here we will mention a few things to consider.