Re: [cross-project-issues-dev] SimRel JIPP migration to new infrastructu

[Ed Merks <ed.merks@xxxxxxxxx>]

Fred,

Yes, people don't look at reports.  That's why no one noticed that bad licenses and unsigned content are present for 2021-03 M1; they've been present in staging for a while:

  https://download.eclipse.org/oomph/archive/reports/download.eclipse.org/staging/2021-03/index.html

Note that I've opened https://bugs.eclipse.org/bugs/show_bug.cgi?id=570380 because https://git.eclipse.org/c is completely unreliable these days, which is why the above report has a bunch of error icons due to of random failures loading the icons during generation.

This "gitc" problem also makes reporting on SimRel broken:

  https://ci.eclipse.org/oomph/job/simrel-analyzer/

And it makes it next to impossible for me to generate the product catalog for M1.

Broken processes are frustrating and unreliable infrastructure is both annoying and time consuming.

    Regards,
        Ed

On 14.01.2021 15:38, Frederic Gurr wrote:

> Hi,
>
> As you might know, the SimRel JIPP has been migrated to our new
> cluster-based infrastructure (see
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=568118).
>
> This has some implications on the SimRel aggregation build:
> On the old infra, the SimRel JIPP could utilize up to 16 CPUs of the
> host machine (hipp8) and had file system access (via NFS mounts) to
> download.eclipse.org. On the new infra, resource restrictions are in
> place and there is no direct access to download.eclipse.org (only via
> SCP/SSH).
>
> Therefore we are using a dedicated "promotion" build agent with 4vCPUs
> that provides filesystem access. Due to the CPU resource limits of the
> build agent, an aggregation build (simrel.runaggregator.pipeline) now
> takes approximately 2 hours on the new infra (vs ~1h hour on the old
> infra). One hour of this build was spent on "repo reports" alone. AFAIK
> nobody reads the repo reports on a regular basis...if ever. Hence I've
> split out the repo report generation from the aggregation build.
> I'd suggest to run the separate repo reports job
> (https://ci-staging.eclipse.org/simrel/job/simrel.reporeports.pipeline/)
> at least every day/night, but not after every aggregation build anymore.
>
> Please also note, more changes (e.g. native Tycho build) will come soon,
> but the focus is on the migration for now.
>
> If you have any questions or concerns, please let me know.
>
> Regards,
>
> Fred
>
>
>