Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[eclipse-dev] Top 10 (slowest) Unit Test Suites ... plus proposal to run only subset of tests for Nightly (Head) builds

As briefly discussed on status call this week, here are the most time consuming unit test suites.  It is not necessarily bad to be on this list ... just drawing attention to it, in case something looks suspicious.  It could mean your component is doing a really good job of testing your code!

More details in bug 377718

  1.  org.eclipse.jdt.ui.tests.refactoring              54.8 (m)
  2.  org.eclipse.jdt.ui.tests                          40.7 (m)
  3.  org.eclipse.ui.tests                              37.4 (m)
  4.  org.eclipse.core.tests.resources                  33.4 (m)
  5.  org.eclipse.equinox.p2.tests                      30.9 (m)
  6.                   30.3 (m)
  7.                       29.7 (m)
  8.  org.eclipse.osgi.tests                            24.2 (m)
  9.  org.eclipse.jdt.core.tests.model                  23.3 (m)
 10.  org.eclipse.jdt.core.tests.compiler               22.6 (m)

These are "overall times" ... including time to "setup" and "teardown" tests (each of those methods is called for each unit tests; not once per suite, as I'm sure most of you know). These overall times are useful, since usually the tests themselves are very fast (just a few minutes, or so, at most) so much time is spent in "setup" and "teardown", presumably.  

As far as I know, these are all required and reasonable for what is being tested ... but, how would I know!?  In other words, if any of this lists surprises a component's committers they may want to take a look if there's something that could be done better.

Plus, more significantly, I propose that for nightly builds, we run (only) a subset of tests ... a set of the quick tests, and run the full set (still) for the more official I- and M-builds. The set of "quickTests" should take only about 25% of the current time, so I think is a better use of our shared resources. To put in perspective, we could do an extra N-build with that same time to do the long running tests!

If concerns, objections, or suggestions, please comment in bug 377718.


Back to the top