Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [epp-dev] 2020-12 - Retrospective


On Wed, 16 Dec 2020 at 02:53, Mickael Istria <mistria@xxxxxxxxxx> wrote:
The thing about the above problem is that builds have always been green, despite there being problems since 2020-06 release. The first step on my release process is "Ensure that the CI build is green. [...]

That echoes my question above: could we have checks to detect the issue and make builds red?

Yes - we can launch each EPP package and error the package if there are any errors in the output. We can also install the previous R build and do a check for updates and install updates. I don't know how hard it is to automate any of that, but it seems feasible enough.

However that wouldn't have detected the problem in 2020-06 as that was done to a bad MANIFEST.MF allowing for illegal OSGi wiring which eventually leads to a ClassCastException or NoClassDefFoundError because of the wiring, but those don't show up until that particular piece of code is exercised, which is not feasible to automate checks for. (In this specific case it only happened once a user tried to connect to a Jenkins server with the Mylyn Builds view.)
 
 
This basically means that someone (me for now) has to own making sure that the packages build. 

Should we have a policy in EPP that mandates the package maintainers to look at those issue when their packages are affected?
SimRel usually disables contributions that are causing trouble. In EPP, we could do something similar: when there is a known issue with a package, we comment out its lines in the pom.xml so its excluded from the future builds and warn on the mailing-list about package being disabled, then the package maintainers have to deal with the resolution, it's not in your hands any more ;)

I think that is reasonable for cases that I can't track down quickly enough. So far the cases have been either easy (like recurring EMF Parsley problem) or affect all packages anyway.
 

 
- Updating the N&N links in the epp.website.xml files. This activity ends up serving two functions, first is I collect all the relevant N&N links into epp.website.xml, but secondly I end up reviewing the state of simrel. There are currently 18 unique N&N links, but I had to send emails out to 4 of the projects because their N&N links were somehow broken (e.g. simply 404, or fully missing PMI entry). This is improved in one of two ways - 1. all projects have a generic N&N entry, like wwd[1] or 2. use the Eclipse web API to automate this step[2], but that would require projects to still have up to date N&N links.

Do you think we should keep those N&N info for individual packages? There is a N&N aggregated for all SimRel projects, isn't it enough? Do you need this "intermediary" grain between SimRel and projects? Would it hurt of we get rid of package specific N&N links?

The N&N entries are the ones from the simrel that are relevant, but I am skeptical of how useful they are. If we replaced the link to the global simrel one that may be more than enough. For reference I am referring to the links in the red box in this screenshot. This page is linked to from the "details" link in the Eclipse Installer too:

image.png
 

- The build itself takes a long time. In particular the promotion job. While I can theoretically press run and then come back to it, it is annoying that the promotion job spends 20 minutes downloading the artifacts from the build being promoted, does some massaging on them, and then spends another 20 minutes copying the results to download.eclipse.org. (I also imagine that every EPP Thursday the webmasters are cursing the network spike - but maybe it doesn't register?). 

Do I get it right that
1. we could do the "massaging" on every EPP build so built artifacts area always ready for promotion
2. we could try to publish the "snapshots" to download.eclipse.org on every build? (that would probablt make the snapshots build 20 minutes longer)
3. promoting would then be sending a `ssh cp ...` command to download.eclipse.org and would then become very fast
? Would that save time overall? Is it OK to have snapshots taking 20+ more minutes if it saves 20 minutes on promotion?

If we published snapshots to download.eclipse.org directly it would save 30+ minutes. The build already has to archive the artifacts which takes time and that time could be removed and the promotion would be simply a "mv" instead. I have had looking into this on my todo list for a while. Until quite recently (2020-12 M1) the mac notarization was the biggest bottleneck, for 2020-12 M2 I fixed that so it runs in ~20 minutes instead of the nearly 2 hours it used to take. I fear once we get moved to JIRO that all these times will get worse due to the containerized structure, so if I haven't fixed it by then I will be fixing it once we are on JIRO.

HTH,
Jonah

 
_______________________________________________
epp-dev mailing list
epp-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/epp-dev

Back to the top