Re: [platform-dev] Mass changes again

[Lars Vogel <lars.vogel@xxxxxxxxxxx>]

Friends of Eclipse,

To summarize the analysis of various people: Looks like neither of the
three reported issues were caused by recent changes.

The following two issue seems to be have been "just" flaky tests, we
did not see the failure happen again.

https://bugs.eclipse.org/bugs/show_bug.cgi?id=559352
https://bugs.eclipse.org/bugs/show_bug.cgi?id=559353

The last bug could also not be traced back to any recent changes and
resulted in deactivation of the tests on Windows.

https://bugs.eclipse.org/bugs/show_bug.cgi?id=559355

Thanks to Andrey for watching the tests.

Thanks to Kaylan, Dani, Noopur and Paul for looking into the related tests.

Especially a big thanks to Paul who fixed various test issues over the
last few days which makes our tests less flaky. For an example, see
his fix and impressive analysis in
https://bugs.eclipse.org/bugs/show_bug.cgi?id=541448#c5.

Best regards, Lars





On Wed, Jan 22, 2020 at 1:48 PM Aleksandar Kurtakov <akurtako@xxxxxxxxxx> wrote:
>
>
>
> On Wed, Jan 22, 2020 at 1:29 PM Lakshmi P Shanmugam <lshanmug@xxxxxxxxxx> wrote:
>>
>> > Mac and Windows builds are unstable for probably a year now (or even more!).
>> The Mac and Windows builds are stable. But, the tests in e4.ui.tests and ui.tests.* have been failing, many of them for several releases. It should be investigated if the tests are unstable.
>
>
> We can go into semantic analysis of what stable means of course :) but the simple fact is I go to https://download.eclipse.org/eclipse/downloads/drops4/I20200121-2225/ and I claim all unstable (until looking into the tests themselves). And I know many would not do that.
>
>>
>>
>> Kalyan has been opening bugs for the failing tests and investigating many of them. Bug numbers for all failing tests in 4.15M1 are in comment https://bugs.eclipse.org/bugs/show_bug.cgi?id=558953#c7. Component owners or people familiar with the affected areas need to investigate the failing tests to see if they are real problems in the code.
>>
>> Since these long failing ones affect developers from seeing new failures, they could disable the tests for the failing platform and open a bug for tracking and investigation.
>
>
> If this is the best path forward I recommend anyone that cares about these platforms to step up and do it.
>
>>
>> We did this in 4.14 for the 2 SWT tests that were failing for a long time on the Mac test machine.
>>
>> Thanks & Regards,
>> Lakshmi P Shanmugam,
>> Eclipse Platform Co-lead,
>> India Software Lab, Bangalore
>>
>>
>>
>> ----- Original message -----
>> From: Aleksandar Kurtakov <akurtako@xxxxxxxxxx>
>> Sent by: platform-dev-bounces@xxxxxxxxxxx
>> To: "Eclipse platform general developers list." <platform-dev@xxxxxxxxxxx>
>> Cc:
>> Subject: [EXTERNAL] Re: [platform-dev] Mass changes again
>> Date: Tue, Jan 21, 2020 2:42 PM
>>
>>
>>
>> On Tue, Jan 21, 2020 at 10:46 AM Andrey Loskutov <loskutov@xxxxxx> wrote:
>>
>> Hi,
>>
>> we had numerous regressions in two last builds, I've opened
>>
>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=559352
>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=559353
>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=559355
>>
>> And I guess there are more, we just don't see them because our test coverage is not the best.
>>
>> I don't know why should we continue this practice of blind "mass changes" for no good reason, that caused so many regressions so far.
>>
>> My best example of such regression, on which I've spent a full work week of my time, was https://bugs.eclipse.org/bugs/show_bug.cgi?id=551147.
>>
>> I'm tired to spend my time to do house keeping for others, and I don't see anyone else doing this work. I don't think this is fair.
>>
>> I would propose that committers that merge "mass changes" *must* do the work I do:
>>
>> 1) Check SDK build results after integration of mass changes and identify new failures
>> 2) Report bugs for new failures
>> 3) Identify offending commits and notify authors
>>
>> If this sounds as too much work, I would propose to re-think the "benefit" of mass changes.
>> If we continue in the same way as today, at some point in time the code is "fully optimized" but Eclipse is not usable anymore.
>>
>>
>> This actually brings one very significant problem - Mac and Windows builds are unstable for probably a year now (or even more!). This is long enough period for contributors to gain the habbit of just ignoring test results on Mac and Windows. I can't blame anyone for that (thanks Andrey for still checking them!).
>> IMHO is current failing tests on Mac and Windows tests can't/won't be fixed ASAP - these should be run only on Linux so seeing test failure finally means there is something to be looked at. As it should have always been.
>> Lakshmi, Niraj, as you're respective SWT port maintainers and the long failing tests are UI related: What is your opinion on this?
>>
>>
>>
>>
>> Kind regards,
>> Andrey Loskutov
>>
>> Спасение утопающих - дело рук самих утопающих
>>
>> https://www.eclipse.org/user/aloskutov
>>
>> _______________________________________________
>> platform-dev mailing list
>> platform-dev@xxxxxxxxxxx
>> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
>> https://www.eclipse.org/mailman/listinfo/platform-dev
>>
>>
>>
>> --
>> Alexander Kurtakov
>> Red Hat Eclipse Team
>> _______________________________________________
>> platform-dev mailing list
>> platform-dev@xxxxxxxxxxx
>> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
>> https://www.eclipse.org/mailman/listinfo/platform-dev
>>
>>
>>
>> _______________________________________________
>> platform-dev mailing list
>> platform-dev@xxxxxxxxxxx
>> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
>> https://www.eclipse.org/mailman/listinfo/platform-dev
>
>
>
> --
> Alexander Kurtakov
> Red Hat Eclipse Team
> _______________________________________________
> platform-dev mailing list
> platform-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> https://www.eclipse.org/mailman/listinfo/platform-dev



--
Eclipse Platform project co-lead
CEO vogella GmbH

Haindaalwisch 17a, 22395 Hamburg
Amtsgericht Hamburg: HRB 127058
Geschäftsführer: Lars Vogel, Jennifer Nerlich de Vogel
USt-IdNr.: DE284122352
Fax (040) 5247 6322, Email: lars.vogel@xxxxxxxxxxx, Web: http://www.vogella.com