The opt-in process for the simultaneous release is, IMHO, broken. We've taken several runs at this, and have not yet (again, IMHO) come to a satisfying solution.
I'll start by saying that I believe that some amount of rigor is desired. Specifically, I believe that shipping broken content is bad and that some amount of rigor is required to avoid shipping broken content.
Bugs happen. With this discussion, I'm not focusing on bugs. Rather, I'm focusing on the features provided by participating projects and whether or not those features are kept current.
Historically, the opt-in process required that all project teams send an email to cross-project-issues-dev to let everybody participating in the simultaneous release know what projects were participating, what version of their content was being contributed, and at what offset. This is still the requirement today for projects that are new to the simultaneous release. This actually works pretty well (IMHO); but only because the volume is pretty low. Coordinating eighty opt-in statements by email (and identifying those project teams that had neglected to opt-in) was pretty time consuming.
Projects that participated in the previous release, signal their opt-in by making any change to their aggrcon file. For the past two releases, maybe half of the projects actually did this. It's worth noting that even if every project did actually follow this process, identifying those projects that neglect to to opt-in is still pretty time consuming.
It's completely reasonable for a project team to decide to let one particular version of their bits stand for one, two, three, or even all future releases. If a project team is paying attention, letting an existing version's bits stand is fine; if they're paying attention and properly testing aggregation builds, then we at least have a fighting chance that they'll be prepared to fix problems should they arise. The concern, then, is that without an explicit opt-in, we have no idea whether a project team is leaving a particular version of their bits stand, or that they've stopped paying attention.
Frankly, the most time consuming part of managing opt-in is hunting down those projects that do not explicitly signal their intention. As I stated, maybe half of the projects actually followed the opt-in instructions in the last release. I spent a lot of time hunting down projects (and identified two that actually had just walked away from the simultaneous release without telling us), but eventually just gave up and just added all of those projects that we hadn't heard from to the list.
It's worth noting that we do have some very strong participants who take their participation very seriously. We need to take care to not add undue burden on these project teams.
So... again, the opt-in process is IMHO broken. To date, this hasn't bitten us too hard, but I'm concerned that it will. My sense is that we're riding a wave of Eclipse IDE Love right now, and that we should do everything that we can to improve and enhance that.
I'm going to complicate matters and restate that I cannot continue to manage this. There may be some opportunities for automation, but somebody is still going to have to step up and own this.
A first step for that somebody might be to open a bug to track discussion (and open it up for community discussion). I thought that we had one open already, but I can't find it...
Director of Open Source Projects | Eclipse Foundation, Inc.