Imagine: You deliver broken software but no one tells you…

Automated Error Reporting and UI Freeze Detection for Eclipse Mars Milestones

Christmas is just around the corner. Besides being a contemplative time, this is also an unmistakable sign that the Eclipse community is already halfway there on its journey towards the next annual simultaneous release. As in the years before, there will (likely) be more than 70 participating projects, with more than 700 people contributing more than 60 million lines of code. And somewhere, hidden within these 60 million lines, there will be bugs. Unavoidably.

Perspective switch: Imagine it’s June and you’ve just downloaded the latest annual Eclipse release. You are curious about all the new features that have been implemented in the last year and maybe it even feels a bit like unwrapping a Christmas present in June. But as you start working with it, you notice that some things do not always work as expected. Nothing severe yet, but it starts getting annoying. Then you open the error log and see that slowly, but steadily, it starts to fill up...


Some of these errors may be severe, while others are minor annoyances only, and yet others may go by entirely unnoticed in the error log as the IDE handles them gracefully enough. But many of them are bugs that Eclipse committers would like to know about sooner rather than later.

Like Christmas, the Eclipse simultaneous release comes but once a year. It is thus paramount that the milestones released prior to the annual release in June are tested – and that bugs (big and small) found therein are reported. Unfortunately, reporting those bugs is a tedious job often done only for the most severe errors; for minor annoyances we just hope that someone else will make the effort to report them to the development team and thus help to get them fixed in time for the release.But obviously this assumption is too optimistic, otherwise there wouldn’t be so many error reports for major version and service releases.

One part of the problem is that committers do not get notified as soon as possible about such errors. Committers can only fix what they know is broken. Thus, to give the development team a chance to fix a problem, someone has to tell them about it. Alas, reporting every tiny malfunction is tedious and sometimes makes you feel like a moaner - which of course you aren’t.

This looks like a stalemate: Committers need (and want) to know about every error that occurs and users want every error to be fixed but don’t have the resources (or the will)? to report every issue they experience. So what can we do about it?

We go and automate error reporting:

Starting with Eclipse Mars M3 the Eclipse Committer Package and the Eclipse Modelling Tools Package feature an automated error reporting tool that automatically sends every error logged in the Eclipse IDE to a server hosted at eclipse.org.

These automated error reports contain common information like the stack trace where the error occured, which bundles were involved, which Java version was used and the like.

Of course, before it sends anything, the error reporting tool asks you whether you want to report the error. In addition you have the opportunity to provide further information such as steps to reproduce the error or an email address useful for follow-up discussions:


As soon as the error report is sent, Eclipse committers will be notified and can start analyzing the data you provided to find a fix for your problem.

Of course, error reports are not just simply forwarded to committers. Behind the scenes all incoming errors are analyzed, checked for duplicates, counters are increased, and finally a new bugzilla entry is created for every unique error report. From this point, committers can take care of your report.

Sending error reports is great. But how about providing feedback to the reporter?

Interesting point. While reporting an error is the first step to solving the issue, it would be great if you as a reporter could also access the information about the current state of an error you just submitted.

With Mars M4, the error reporter now offers several means to let reporters know about the current state of a report. For instance, whenever you report an error, the reporting tool will make the URL of the bugzilla entry associated with your report available in your progress view:


Moreover, in case your problem has, in the meantime, already been fixed, you will get notified straight-away with information on how to continue from your current point:


Additionally, committers now also have the means to get in touch with reporters. Whenever committers flag a bug with the “needinfo” keyword in Bugzilla, users reporting that exact problem will see the request to provide more details.

There is likely more to say, but you certainly get the idea.

How about issues that do not produce an error log entry? Like UI freezes?

With Mars M4, even for that, there is a solution. The platform now contains a UI monitoring tool that detects UI freezes and reports them as errors to the Eclipse error log - from where they are picked up by the automated error reporting and reported to Bugzilla as well. There are quite a few reports already, let’s see how many can be addressed during the Mars Milestones.

Credits to UI freeze detector and automated error reporting

The UI responsiveness monitoring was used at Google for a number of years and was contributed to the platform recently. The Eclipse 4.x version of this plug-in was written by Steve Foreman, Markus Eng and is currently maintained by Sergey Prigogin. Thank you very much for this contribution!

The automated error reporting is under active development by Daniel Haftstein and Marcel Bruch from Codetrails which initiated this project for the Eclipse Mars Milestones to help make Mars a great next version of the Eclipse IDE.

How to continue from here?

The UI freeze detector is here to stay. The automated error reporting is an experimental feature which is currently planned to be part of the Mars milestones only. Whether or not it will eventually become integral part of the Eclipse IDE depends on how users and committers embrace it.

There are also a couple of interesting ideas floating around like making it part of the Eclipse Long-Term-Support Program to help vendors of commercial tools to improve even older version of Eclipse, or to enable plugin-vendors to access error reports that affect their products to improve the Eclipse Ecosystem as a whole.

But for now let’s make Eclipse Mars release in June a great (early) Christmas present for all of us...

Marcel

Links

About the Authors

marcel bruch

Marcel Bruch
Codetrails