[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
[embed-cdt-dev] Eclipse Common Build Infrastructure
|
(since I'm not sure Alexander is on the list, I added his address as Cc:)
The current status of the project is that the source code for release v5.1.1 is ready, awaiting for the Jenkins scripts to be implemented, in order to build and publish the plug-ins and the packages.
Below is a detailed analyse of what needs to be done, and some suggestion on how to do it.
Please feel free to suggest improvements.
# SimRel
For the moment the project is not ready to join SimRel, but I think we should do our best to prepare for it.
From https://wiki.eclipse.org/SimRel/Simultaneous_Release_Requirements_PROPOSED (a document dated 2013, still work in progress!), I understand that
- projects must announce at M4 the intention to participate
- projects should provide changes to the xml used in epp builds
- project may support their own set of platforms/target environments
- Projects must use 4-part version numbers (although https://wiki.eclipse.org/Version_Numbering focuses on 3-part versions)
- projects must be signed (macOS, Windows)
- Projects must provide their own project p2 repository
- Projects must optimize their p2 repositories (I don't know what this is, the documentation page points to a different content)
At first sight, these requirements can be met, but, in practical terms I need to see how much work they demand.
# Custom epp scripts
Until a full SimRel integration, the workaround seems to be to fork the epp project, and customise it, as it is done at the moment.
This will probably require to transfer the content, including history, of the gnu-mcu-eclipse/org.eclipse.epp.packages to the eclipse-embed-cdt organization, and continue to maintain it for a while.
# The documentation pages
From https://wiki.eclipse.org/Jenkins#How_to_build_my_project.27s_website_with_Jenkins.3F I understand that
- the mandatory web url is http://www.eclipse.org/embed-cdt
- at the moment this redirects to https://projects.eclipse.org/projects/iot.embed-cdt
- for content to appear there, the site html files should be present in the http://git.eclipse.org/www.eclipse.org/embed-cdt.git project, and an automated process will publish it.
- Jenkins offers support for Hugo
- the Embed CDT web site will follow the GitHub Pages specs, which use Jekyll, a Ruby tool
- apparently Jenkins does not support Ruby, so the site should be generated on GitHub
This results in the following configuration
- a GitHub project like eclipse-embed-cdt/web-jekyll.git, with the Jekyll source code
- a GitHub project like eclipse-embed-cdt/web-html.git, with the generated html code
- a Travis job to run the conversion
- a Jenkins script to copy the web-html.git content to www.eclipse.org/embed-cdt.git
- a GitHub project preview.git, with GitHub Pages associated to it, which will be visible as http://eclipse-embed-cdt.github.io/preview, where the web content can be previewed during development, and only when ready the content is copied to web-html.git
It seems a bit convoluted, but I guess it can be functional.
# The p2 update sites
According to the Handbook (https://www.eclipse.org/projects/handbook/#resources-downloads) the project download page should be
https://download.eclipse.org/embed-cdt/
Since this is the project root download, where other things will reside too, most probably the p2 update site(s) should be grouped in a subfolder here.
As of now, there is a Jenkins script contributed by Jonah that runs the maven build and leaves the result in some standard folder like
.../repositories/ilg.gnumcueclipse.repository/target/ilg.gnumcueclipse.repository-*.zip
I could not test it, since I do not have write access to Jenkins yet, but I guess it is functional.
There is also a builds-upload.sh script that copies the p2 ilg.gnumcueclipse.repository.zip and unzips it to:
/home/data/httpd/download.eclipse.org/embed-cdt/builds/${BRANCH_NAME}
which gives
/home/data/httpd/download.eclipse.org/embed-cdt/builds/master
/home/data/httpd/download.eclipse.org/embed-cdt/builds/develop
The current URLs for the p2 sites are:
http://gnu-mcu-eclipse.netlify.com/v4-neon-updates/
http://gnu-mcu-eclipse.netlify.com/v4-neon-updates-test/
http://gnu-mcu-eclipse.netlify.com/v4-neon-updates-experimental/
I don't say that the structure with builds/master and builds/develop are not a valid proposal, but I think that the original 'v4-neon-updates*' names also have some value, and I would not dismiss them without good reasons.
The first reason is that the current plug-ins are expected to run on Eclipse Neon and up, so keeping the word 'neon' in the url might make some sense.
When, due to incompatible APIs, we'll have to update, I guess we'll have to use something like 202N-NN, freeze the 'neon' folders and continue to publish to the new folders.
Also the 'builds/master|develop' names seem more oriented to developers than users, since users should not have to do with internal branch names.
Thus, since we have to share the parent folder with packages (see below), I suggest something like
https://download.eclipse.org/embed-cdt/updates/neon
https://download.eclipse.org/embed-cdt/updates/neon-test
https://download.eclipse.org/embed-cdt/updates/neon-experimental
As a comparison, the CDT p2 update sites use URLs like
http://download.eclipse.org/tools/cdt/releases/9.11
I'm not particularly happy with such a structure, since it mentions the actual version; I prefer it to mention the base version (like neon), such that old installs are still able to get updates from the same URL.
In terms of Jenkins, once we decide on the folder names, I guess the current 'builds.Jenkinsfile' and 'scripts/builds-upload.sh' will need to be reworked.
We'll probably need three scripts, that are manually initiated, and that will run the build and publish the results in the three specific folders.
I'm not sure if I can do it alone, I might need help to write the three scripts.
For CI tests, I would continue to use Travis, in my opinion it is more convenient than Jenkins.
# Archive of releases
Currently the archive of all releases is kept in the GitHub Releases, which is very elegant:
https://github.com/eclipse-embed-cdt/eclipse-plugins/releases
We can move it to Eclipse downloads, for example as:
https://download.eclipse.org/embed-cdt/releases/5.1.1/ilg.gnumcueclipse.repository-5.1.1-202007201244.zip
There was a request to keep an archive of all releases in p2 format, such that CI scripts to be able to install a specific version of the plug-ins.
The docs (https://help.eclipse.org/2020-06/index.jsp?topic=%2Forg.eclipse.platform.doc.isv%2Fguide%2Fp2_director.html) mention
• -repository: a comma separated list of repository URL where each entry denotes colocated meta-data and artifact repositories.
It does not mention URI and there is no mention on how to install from a local archive, so this request might make sense.
In this case the folders /releases/x.y.z/ may contain not only the archive, but the extracted archive content too, as a p2 site.
In terms of Jenkins, I guess this can be automated too.
# The EPP download site
Given the requirement to use the Eclipse download infrastructure, I guess we can use a system of folders like
https://download.eclipse.org/embed-cdt/packages/
https://download.eclipse.org/embed-cdt/packages/2020-03
https://download.eclipse.org/embed-cdt/packages/2020-06
and inside each folder have archives like
eclipse-embed-cdt-2020-03-R-linux-gtk-x86_64.tar.gz
eclipse-embed-cdt-2020-03-R-macosx-cocoa-x86_64.dmg
eclipse-embed-cdt-2020-03-R-win32-x86_64.zip
or whatever naming convention is recommended by SimRel.
From what I see, CDT uses a redirector
https://www.eclipse.org/downloads/download.php?file=/technology/epp/downloads/release/2020-06/R/eclipse-cpp-2020-06-R-linux-gtk-x86_64.tar.gz
I have no idea where it goes.
In terms of Jenkins, this will probably require a file similar to builds.Jenkinsfile, or additions to it.
It should describe a job that is initiated manually, to download the epp fork repository, run the build, and store the archives in a folder.
I might need help to write such a file.
# Building from custom repo forks which include the Arm new support files
I took a look at some of the projects, and to the way they can be integrated, but I currently did not understood the details, so I don't know how realistic this can be.
The idea is to first be able to build the Eclipse CDT packages from forks of the original git projects. Once this is possible, I can start changing here and there, to add Arm support.
Alexander, I would really appreciate your help in understanding how to approach this.
---
Any comments/suggestions will be highly appreciated.
Liviu
