Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [gemoc-dev] better automatisation of the multibranch pipeline

Awesome! Thanks Didier for the great work. 

Developers' feedback is more than welcome for the continuous improvement. 
Please, all, report them your experience with this new integration pipeline. 

Thanks, 
benoit

Le 15 mai 2019 à 17:27, Didier Vojtisek <didier.vojtisek@xxxxxxxx> a écrit :

Hi all

just to let you know that I recently worked on a better automatisation of GEMOC multibranch pipeline (cf. https://github.com/eclipse/gemoc-studio/issues/142).

  • previous situation:
    •  branches were taken into account only if a branch with the same name exist in the repo eclipse/gemoc-studio for ci.eclipse.org/gemoc (or in gemoc/gemoc-studio-jenkins for ci.inria.fr/gemoc) the branch was required even if there were no commit in it.
      • Additionnaly, as the eclipse and the gemoc fork doesn't use the same ci and configuration the build architecture  was slightly different (the additional gemoc/gemoc-studio-jenkins repo)
    • In many situation, a manual triggering was required:
      • push in branches of branched repo didn't trigger the build of the branches
      • push in master branch didn't trigger the build of branches where the repo to consider remains "master"  (ex. branch "feature-x" in repo moccml, branch master for all the other repo, a commit in master in one of these other repo did not trigger the build of pipeline "feature-x")

So the new jobs try to have the same behavior as before while fixing these issues, without additional task for developpers.

Basically, it improves the "scan multibranch pipeline" action by adding a preliminary phase that automates branch collection into git modules in a dedicated repository.

Internally how it works:

  • Gemoc maintainers @Eclipse organization, use the repo gemoc-studio-eclipse-integration , in the integration branch (master) they define a list of tracked git modules pointing to all the gemoc repositories to consider
  • based on dvojtise/git-sync , a job periodically (5 min)  scans these modules in order to detect branches in any of the git modules, for each branch name, it creates one in the eclipse-integration repo and make sure that its git modules follow the given branch or master branch. It also makes sure to update the git ref to point to the latest commit of all branches. (Obviously, it takes care of branch addition and deletion)
  •  a gemoc-studio-integration job (replacing the old gemoc-studio job) is then a simple multibranch pipeline that is triggered correctly

So, once pushed on github, your branches are supposed to be available automatically (after the polling delays)

In order to have the same features for GEMOC organization, maintainers simply have to duplicate the very same, they only have to adapt the jenkinsfile (but this was allready done in some way)


Some impact:

  • as branches are collected automatically, it is even more important to delete old and deprecated branches on the remote (github) in order to save disk space.

Steps not done yet:

  • due to the fact I'm not admin neither on ci.eclipse.org nor in github.com/eclipse ,  I've set the integration repository and the scan branches job on GEMOC organization and Inria CI.
    • If the behavior is OK, then we need to move them to Eclipse organization
    • admins should also be able to speed up the reactivity of the process by triggering the "branch scan" instead of having to do it periodically.
  • I haven't duplicated the repo+jobs for the gemoc organization yet. (please raise your hand if you need it in short term)


Please let me know if you have any question or if you observe an incorrect behavior.

cheers

Didier

-- 
Didier Vojtisek
SED Rennes - DiverSE Team - LogicA Team
Inria, Univ Rennes, CNRS, IRISA
Campus de beaulieu
35042 Rennes
02 99 84 75 07
_______________________________________________
gemoc-dev mailing list
gemoc-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/gemoc-dev


Back to the top