Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [wtp-dev] M4 and testing (as M3 and testing)

There's an possible technical answer to this issue ... and that's just to keep the non-implemented API tests temporarily in a separate plugin that's not included in I-builds and Milestone builds, just nightly builds, where they would fail. But I can't help but think this would be a very rare case. After all, to be API is, in part, to have clients, so to have clients, there must be an implementation. And, its my understanding, an API without an implementation is not really an API yet, as it would surely change some as its implemented and change some as its used by clients.

More importantly, I think there's a lot of value in having the JUnit tests serve as a BVT, "Build Verification Test" ... that is, confirms that the build would be expected to basically run ok, even if not perfectly ... so would hate to set a precedent that its ok to have red X's on build page.

While I fully agree with the principles of "API First", we can't ignore that we already have lots of implementation, and lots of undocumented APIs, and I suspect 80% of our efforts is documenting the APIs we think we have, fine-tuning and finalizing them, not, in other words, coming up with new ones.

I don't mean any of this as disagreement with the goals having platform quality APIs, just to further this discussion as to how to get there.

Perhaps its a more clear concrete goal to state that API are to be defined, documented, and have working JUnit tests by M4. Anything that's not in that category would have to go through the API approval process, as has been suggested on this list by another post from Arthur?

Suggestions/comments welcome.


Arthur Ryman <ryman@xxxxxxxxxx>
Sent by: wtp-dev-admin@xxxxxxxxxxx

02/24/2005 04:09 PM

Please respond to

Re: [wtp-dev] M3 and testing

Looking forward to M4, we should be developing API First and being Test-Driven. This means that it is OK for some JUnit tests to fail at M4 exit, i.e. those that test API implementation. In fact, I expect the implementation to only be complete by M5.

We need to identify the subset of JUnit tests that need to pass for M4 and beyond, i.e. not let just any JUnit failure make the build fail and not have to pull test cases from the build.

Arthur Ryman,
Rational Desktop Tools Development

phone: +1-905-413-3077, TL 969-3077
assistant: +1-905-413-2411, TL 969-2411
fax: +1-905-413-4920, TL 969-4920
mobile: +1-416-939-5063, text: 4169395063@xxxxxxx

Chuck Bridgham <cbridgha@xxxxxxxxxx>
Sent by: wtp-dev-admin@xxxxxxxxxxx

02/24/2005 08:46 AM

Please respond to

Re: [wtp-dev] M3 and testing

Hi Naci,

We have been making incremental improvements to the junit's success, but all problems remaining are setup/junit issues, not blocking functionality.

I would like to spend a little more time investigating these, and if not successful, we will pull these test cases from the build.

I do not think this should delay our declaration of M3.   We will be smoke testing the latest build this morning.

Thanks - Chuck

Rational J2EE Tooling Team Lead
IBM Software Lab - Research Triangle Park, NC
Email:  cbridgha@xxxxxxxxxx
Phone: 919-254-1848 (T/L: 444)

Naci Dai <naci.dai@xxxxxxxxxxxxx>
Sent by: wtp-dev-admin@xxxxxxxxxxx

02/24/2005 06:22 AM

Please respond to

[wtp-dev] M3 and testing

The last build still has test errors in jst.jee, jst.servlet and
jst.validation.  I have tried to run these tests on my local machine and
they have failed in a different way.

-At this point if we think these tests pass but are not running due to a
configuration problem, we should pull the tests back and respin another
build.  All other criteria for decalring M3 are still valid.

-  If they are failing due to a valid problem in WTP then I guess we will
not declare M3 until after eclipsecon

Naci Dai,
Managing Director

eteration a.s.
Inonu cad. Sumer sok. Zitas D1-15
Kozyatagi, Istanbul 81090
+90 (532) 573 7783 (cell)
+90 (216) 361 5434 (phone)
+90 (216) 361 2034 (fax)

wtp-dev mailing list

Back to the top