Re: [cross-project-issues-dev] [WARNING] SimRel Headed Off the Tracks

[Sebastian Zarnekow <sebastian.zarnekow@xxxxxxxxx>]

Everyone,

> his proposal is the only one we got so far with a potential to improve the situation

I don't think that's true. Ed W also brought the option to the table, that we could reassess the benefit of 4 release trains per year. Does this even carry its own weight?

I'm fully aware of the fact that there are projects that want to deliver more often than once or twice a year. But in general, the current release cadence puts too much of a burden on the shoulders of all maintainers.

Platform and JDT used to be rock solid with the annual releases. Now we see more releases but also more bugs and complains from users as far as I can tell. Most of the people that I'm talking too are no longer confident in the quality of the release. For them it's nowadays a tradeoff between the bugs the suffer from and know of vs the bugs they will suffer from but don't know yet.

The proposal to drop the idea of release trains in their current shape appears to be striking at a first glance (less work is better, who does not like that idea!!). But to me its seems to based on false assumptions and also based on oblivion.

We used to be at a place called plugin-hell living next jar-hell and being the little sister of dll-hell. Introducing the train back then was such a huge step forwards when it comes to quality and robustness and customer satisfaction, I always want to shed tears of joy when thinking about this. Of course we suffered from the curse of good intentions since the train became too crowded, but at least we had a certain level of quality. When going for four release per anno, we suffered again, but even more so. Four times the work for participant, no maintenance releases - which used to be a great value back in the days, and people moving on (no offence, such is life).

The valid question after this rambling remains. What are our options? And how do we find a feasible path forward - not daring to say the best path here.

There seems to be consensus that we want to reduce the workload. But certainly dropping the train entirely has to be carefully weighted against a reliable schedule with fewer outages.

My gut feeling is that 2 releases per year would already greatly help to allow a little room the breath for those project maintainers, that are responsive and try to remain compatible to the other projects.

> For stable components there is an option to keep the same version contributed for a number of releases - this should be sufficient to support "annual release" experience.

> For new and incubation components "annual" may mean mostly "next life".

Nothing prevents these to ship more often against the released bundles of other projects. Why does one need to ship the same bits 4 times a year if there is no observable progress?

> For eclipse-based vendors - annual release is the invitation to do a fork and think about switching to another platform. Why? Because "another platform" with growing popularity has monthly releases.

I don't know a single vendor that will immediately jump the most recent train. See rantings about quality above.

> Also annual releases will resurrect a number of "service releases" with all the effort required,

And with the quality gains. Exactly.

> Right, but at least, if it's in EPP, then we're sure the projects that are integrated have at least 1 person (the EPP maintainer for this package) caring about those issues and dealing with them. While with current push model in SimRel, some project push outdated stuff and aren't reachable.

Assuming you depend on these bits. How would it improve the situation. The poor maintainer of the package cannot reach the unreachable people either. No obvious gain to me.

And to put myself into the shoes of the responsive maintainer of a project. If you want to / have to contribute your project to multiple EPPs, do you coordinate with everyone individually when you do changes? How do you make sure, that you do not break these packages? How can be ensured, that your project remains a good citizen. Also in that regard, no obvious gain to me.

It's a little unfortunate that I don't have a great idea either and can only express my concerns about the current direction of Eclipse, the quality and the proposed solutions.

But it may be worthwhile to revisit the train schedule and have fewer big releases per anno and reconsider service releases with a more curated list of changes.

Best

Sebastian

On Wed, Jan 29, 2020 at 12:09 PM Aleksandar Kurtakov <akurtako@xxxxxxxxxx> wrote:

On Wed, Jan 29, 2020 at 11:49 AM Mickael Istria <mistria@xxxxxxxxxx> wrote:

It seems mostly based on the the principle/assumption that moving the problem will simplify the problem or make existing problems disappear by magic.

Indeed, I believe that moving the problem to a better functioning project according to EDP practices, and by the way getting rid of a lot of questionable effort, will make many projects disappear.

Currently the train repo is a prerequisite for producing the packages and it composes a large set of repositories into a single aggregate at which point a high level of consistency is checked and ensured. In the end, ensuring that all the artifacts that comprise each package are coherent (and stable) does not go away even if somehow the packages were produced by directly pulling content from something other than the train repository. 

With EPP, we ensure that the packages are consistent and of good enough quality to be released. I don't think that would change much, and EPP could also add some tests verifying all features can install in the same IDE.

 

Nothing changes with regard to ensuring consistent licenses, signed content, proper inter-operation, stable repositories, and mutual instability.

Right, but at least, if it's in EPP, then we're sure the projects that are integrated have at least 1 person (the EPP maintainer for this package) caring about those issues and dealing with them. While with current push model in SimRel, some project push outdated stuff and aren't reachable.

In the end, I'm not even sure if you're suggesting that there needs to be no aggregation at all, but simply a very large set of direct references to various project repositories.  But I can assure you that loading 50 repositories instead of 2 when doing an install will not improve the experience, and that getting n projects to maintain long-term stable sites is a new problem that will also turn into yet another cat herding exercise and when it fails (as all things do on occasion), the users will notice immediately.

I suggest EPP builds the aggregated and categorized p2 repository, containing only stuff that are included in packages.

To build. EPP references different source p2 repositories, just like SimRel reference other repositories.

It feel as if you've joined the discussion years after all the reasons for having a train the first place were had, and that you assume there really are no good reasons because you were not part of those discussions.  So all the reasons need to be reiterated, at which point you are highly inclined to try to shoot each one down because they don't fit you current conclusion.

I know the reasons, and participated in integration of a project in SimRel 11 years ago and was very excited about SimRel and invested in it. I think it was interesting back then, but times have changed: many project are almost inactive in SimRel, SimRel build has lost maintenance workforce and there doesn't seem to be much hope for more, here is Marketplace, there are EPP packages, there is an installer... So I think it's just time to question again whether we can collectively afford SimRel as it is now, and even whether this is still the appropriate solution for past problems.

Those who need more need to invest more; but if no-one wishes to invest more despite the urge, then it's that there is actually no strong enough need to keep things as they are now.

In any case, no matter exactly all the concrete details of what you are suggesting, the question of who does that work remains the same one.

If we keep separated SimRel and EPP and keep projects that are irrelevant to EPP in a push mode, it's a lot of work and no-one will want to do this work.

If we make things simpler and better address current issues instead of sticking to older ones, then it's less work and it's more interesting, some people will continue it.

My reasons to support that is that:

* Marketplace would still be available -> no loss for users

I also pointed out that you could fix your marketplace entry:  https://www.eclipse.org/setups/marketplace/?id=3394048

This is far from being top-priority, I have more valuable work in the backlog (like lobbying for the end of SimRel as we know it ;)

That hasn't happened and the fully 1/3 of the marketplace entries are completely broken or somewhat broken.  Consistent/correct marketplace listings is yet another exercise of cat herding.

There are stats about installation issues in Marketplace already, and IIRC there is even a system to notify owner in case of too many install failures. That seems far enough to me, and my current usage of Marketplace is pretty pleasant and leads to good experience. (while I never use the SimRel site...)

 

* packages would still be available -> no loss for users

So at least we agree that packages are needed.  Unfortunately there's no one to produce them.

On the epp-dev mailing-lists, some questions are pending for others to evaluate how much the can invest.

But EPP packages are a trivial-ish Tycho project, that builds itself on CI on top of active technologies and reactive, communities. It's not a too big deal (compared to SimRel), and once the actual maintenance steps are clarified, there will be some people to do the work.

 

* installer would still be available -? no loss for users

Here the question is:  Which repositories will contain all the artifacts?

I guess the same download.eclipse.org/2020-06 repo, expect that it's built by EPP, not by SimRel.

 

How much work will I personally (Oomph) have because of a complete change in design in EPP structure?

Hopefully none.

* SimRel and its strange governance and all the discussions that have emerged with it disappear -> time saved

It will only disappear from view, but the identical technical problems will simply migrate somewhere else.   Somewhere decentralized?  Somewhere with no central oversight?  This doesn't not sound like the problems will go away, but rather will become invisible to most of us, but not for the users.

Indeed, some checks need to remain; but most of them can be automated, and in EPP, much is already done by package testers.

* EPP starts handing everything, and EPP governance is working well -> EDP used efficiently.

The person who does not exist will restructure everything and will manage everything personally and none of us will have to do anything at all anymore to help that person.  That sounds great in principle, except for that person.  And I'm often that person, and I can assure you it's not great at all; I often cannot solve problems that come from elsewhere.

That's fair, however, I'm convinced the proposed change is profitable. But it indeed requires investment; but it's always the same, if we can convince people there is a return on investment (and IMO there is as SimRel day-to-day maintenance will be drastically reduced), then we'll have them more likely to help.

* Active contributors like you stop spending effort on projects that are not worth it (to you): many projects are in SimRel just by the force of habit, but the value for the user community is arguable and the cost for maintainers is present (more things to check, more bugs to open, more files to watch, longer build time....).

Removing projects from the train will be somewhat helpful in reducing the overhead.  We could start with EMF and finish with Oomph; that would save me personally one hell of a lot of work.  Or did you have specific projects in mind that are not worth the effort?

Every project that is not part of an EPP package and not investing in testing the EPP package is IMO not worth the effort.

With this proposal, the maintenance cost would be drastically reduced and the process be made more streamlined with typical EDP. Hopefully this will become simple enough for the few active contributors on SimRel (build & infra, not contributions) and EPP to be able to cope with this for some years.

Are you volunteering to step up to prototype and demonstrate all the new infrastructure that would be involved in your proposal so that we may concretely assess how that alternative would work in detail rather than in the abstract?

:)

Yes and no. I may try it at some point, but cannot guarantee it nor estimate when. But as I'm convinced of the ROI and my suggestion is mostly about adding a .target file into EPP build that happens to be a simple Tycho build, then I may find some time for it when I'm bored with other things.

Let's add that I fully stand by Mickael here and his proposal is the only one we got so far with a potential to improve the situation. We have to just admit that Simrel and EPP can't continue in the way they are and look out for changes that will improve the situation.

From Ed's proposal:

* Choice 1 - let's be realistic and admit that this would not happen. No one will step up and do things the way they used to be just because someone is used to it being that way E.g. If I (or anyone from my team) step up - we will effectively implement the proposal. Of course anyone is free to jump in and keep things the way they are - I'll be more than happy to be proven wrong here :)

* Choice 2 - speak to representatives. From all the Planning/Architecture council meetings there used to be a lot of wishful thinking over the years and pretty much no one there spent the time/resources to make it happen. Read this as - we don't need talks, we need actions.

* Choice 3 - do nothing . I understand this is meant to be provocative and I fully support some stress over the community. We should start questioning every single piece of our process and if it has resource issues consider it ineffective or not needed before trying to solve it. For many of the existing plugins that are part of the Simrel that would be the best to do - nothing.

Well actually do single step - remove them.

To use DLTK project  - we did exactly that - dropped the python (Pydev is better offering!), ruby (Solargraph is better offering!), shell script (ShellWas is better offering), js (WildWebDeveloper is better offering)  from the December release.

To use CDT project  - launchbar and templates repo are merged into main one to reduce cycles. Terminal is getting moved to CDT so RSE can finally be left to rot there. Ancient parsers are getting dropped and so on.

I'm not even going to mention the amount of work and automation that went on Platform and Tycho side .

Aka active projects are already actively engaged into streamlining the developer experience so burden is manageable. In my eyes there is no other way but to do that for Simrel+EPP

 

About enforcing or checking SimRel rules, then they are not really SimRel rules and checking that or declaring compatibility should be handled by projects, as part of their releases; not by a downstream consumption.

I've seen that projects are so very very responsive in addressing the issues raised for them, not!

Then it becomes the responsibility and choice of the ones who "pull" the project: if i maintain an EPP and see a dependency in a bad state (let's say Mylyn), it becomes my duty to get it fixed or get it removed from the EPP packages/aggregation. But as long as Mylyn doesn't react and comply with the necessary rules, they'll just be removed from EPP and aggregated site.

_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cross-project-issues-dev

--

Alexander Kurtakov

Red Hat Eclipse Team

_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cross-project-issues-dev