[cross-project-issues-dev] Update on Error Log Reporting for Mars Milest

[Marcel Bruch <marcel.bruch@xxxxxxxxxxxxxx>]

- I apologize for cross-posting to several lists - 


Hello Eclipse Committers and Enthusiasts,


In the last meeting the architecture-council requested little more details about the current state of the error log reporting for Mars Milestone Builds, which I present in this email. 

The error log reporting got approval by EMO and can go live soon. Legal aspects in code are addressed; some administrative tasks need to be done but will be fixed in the next days hopefully. All in all, we are making good progress. For a quick summary of previous state please visit the slides available at [1].



Server Side
===========

The error log reporter's web front-end is accessible via http://dev.eclipse.org/recommenders/. Please note that it requires you to authenticate with your Bugzilla ID and password and is only accessible for Eclipse *committers*.

The web front-end features, among other things, an error log dashboard which provides some top-level information like 
* how many reports have been processed in the past days, 
* which plugins participated often in error reports, 
* lists common exception messages,
* etc. 
It also offers some basic data query functionality to build your own personal dashboard. Feel free to use this to build your personal dashboard for plugins you’d like to monitor but keep in mind that the dashboard is still subject to change without prior notice.

Besides the dashboard, the error log reporting now reports new errors directly to Bugzilla. What’s „new“ is decided based on a hash computed from all stack trace elements contained in an error report. From the way how the hash is computed you quickly get that this groups similar stack traces, i.e., stack traces that have exactly the same stack trace elements and order but ignores the error code, exact error message, and source code line numbers. This may or may not be a good decision. I initially planned to use error codes. But they are usually ‚0‘ (29%), ‚2‘ (42%), or ‚4‘ (25%) (see [4]) which effectively renders them quite useless. We’ll see what happens…

Anyways, as said, bugs are created for every new error that arrives. See http://eclip.se/2A for a list of example bug reports; see http://eclip.se/2B for a particular example. Please note that these bug reports may become visible for Eclipse committers only in the near future since these reports may contain sensitive information.

When looking at the description of http://eclip.se/2B more closely, you’ll notice that there are two links at the bottom. The first link sends you to the full error report that contains user names, unabbreviated messages, all child status objects with their stack traces etc. Thus, if the summary presented in the bug report may be missing something, please make sure to visit the details link.

The second link is an „update“ link which stores the current processing status (close, fixed, invalid, won’t fix etc.) in the error log reporter’s data base. This information is used to inform clients sending new reports about the current status of their issue. This feature is new and not yet really used on client side. But we have a couple of ideas how to make use of this in the future (e.g., the system could inform the user about reasons why this problem appears, committers may request for additional information from senders like a comprehensive installation log etc.) In case you have a good idea or feature request, let me know.

In addition to the stack trace, the bug report also contains a list of bundles (and their versions) that where present the exception's stack trace. Please note that this is a rather simple heuristic based on the package names present in the stack trace: For every package it checks whether there is an activated bundle with a similar name. If so, it’s added to the list. If the bundle is available in several versions, or we use split packages this may fail brilliantly. Thus, the list of present bundles may or may not be accurate. But it’s better than not guessing at all I guess…



Joining the review efforts
--------------------------

With reporting new errors to Bugzilla enabled, I’d like to encourage all committers to help reviewing error reports for their corresponding plugins. At the moment you have to define your own Bugzilla queries. http://eclip.se/2C gives and example how to register for error report containing „org.eclipse.m2e“. Please note that you can even specify arbitrary regular expressions for matching reports you are interested in. Bugzilla has a lot advanced query features to offer here.

In the near future we may be able to automatically add committers to a bug report’s cc list based on the bundles present in the stack trace, regex, or other metrics. This, however, would require some manual registration of the committer beforehand and is yet not available. At the moment I’ve to wait for a library to implement some missing features. I’ll give you a heads up as soon as such a feature is available.

Until then, please consider registering yourself via a Bugzilla query as illustrated above. If you have suggestions how automatic assignment/cc list addition could be implemented based on an error report - or are interested to be put on the cc list of all or a subset of reports - please let me know.




Client Side
===========

The client side basically looks the same as before (see [1] for screenshots). The log listener now features a cache, that keeps the information alive which errors were observed in the last 10 minutes, and thus, skips (by default) duplicated or very frequently reoccurring error log entries. However, if an error occurs after 10 minutes, it’s reported again - if only to make us aware that our users face that issue more than once.

As said above, the server response is not processed yet. But in future we may send the error’s fingerprint first to see whether the error is known on the server side and get the information whether there is a bug open for this and whats it’s state, or what’s the proposed workaround to fix that issue on the client. The last one is particular interesting IMHO but not yet in this project scope.




Please feel invited to test the current version and provide feedback. I know, this rarely happens but I thought I should at least ask :-)
It’s available on the Code Recommenders Milestones update site [2] (see [1] for details). In case you have some feature requests, please report them to bugzilla directly [3].



EPP Package Maintainers
=======================

The slides contain some information which dependencies the error reporting tool has. Due to some changes this morning, there are a few more dependencies to add: org.eclipse.emf.ecore.xcore.lib and org.eclipse.xtext.xbase.lib. The reporting feature is available on the simrel update site and will be updated before every milestone build.




Good luck, thanks for listening, and thanks for your help making Eclipse a little bit more stable.
Marcel





[1] https://drive.google.com/a/codetrails.com/#folders/0B9Gf8BD0WA1SVlQwOXZNd0dfZmc
[2] http://download.eclipse.org/recommenders/updates/milestones/
[3] https://bugs.eclipse.org/bugs/enter_bug.cgi?product=Recommenders.Incubator&component=Stacktraces
[4] http://dev.eclipse.org/recommenders/