Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [tycho-user] How to prepare the p2-site for an offline target platform to be used in a tycho build 
Hi,

Why tycho does not store the whole updatesite is simple.
Tycho would need to download the whole content of the update site (what can be gigabytes in case of sim-rel for example).
Also a has over target content would not be suitable as it don't mean that content has not changed (just think about update composite repos).
If you need reproduceable target resolution your best is to mirror the target and deploy it as a zipped artifact into some kind of maven repository. The next tycho release will be able to use such targets and as long as you don't clear the maven cache will reuse the local stored content. But as noted be aware such "snapshots" of a repo can become quite large.
[1] https://wiki.eclipse.org/Tycho/Release_Notes/2.2#Support_for_consuming_maven_artifacts_made_of_zipped_P2_update_sites 
[2] https://github.com/eclipse-m2e/m2e-core/pull/37

With current approach (used) artifacts are already cached.

Am 07.12.20 um 11:36 schrieb Johan Compagner:

Hi,
If i read this subject of this email then i rather have a way better fix right into Tycho. 

I use a "target" site in my pom/tycho configuration
This is not local but the real thing
Why is it if it is already builded with that on jenkins for many times already it can fail suddenly if one of the sites has a hiccup? 
Why does it want to continuously check and download the stuff?

Why can't tycho not make a local site right from my target ?
For example generate a sha hash from the contents of the target file
and cache based on that hash on local disk the full resolved site.
(or it fills up in that dir the stuff it needs or it encounters that is not there)
So when a full build is done everything is in that dir based on the sha hash of the contents of that target file if that file changes it just generates a new one (i guess we need to be able to clean up stuff later ) But as long as the sha hash doesn't change tycho doesn't try to contact any site it always gets everything locally This should speed up the builds, and we shouldn't suddenly have breaking builds because of a network problem to one of the things inside the target.
This is kind of currently how we work with target files in our Eclipse environments 
But then more manually (a user checks the hash ;) )
We have our main target file with the urls, and everybody just exports that to a local site and use that local.target as the real target in eclipse Because eclipse has the same problems, if there is somehow suddenly an io exception it could even be that you can't really work anymore because the target is just bombing out...
Maybe tycho only needs to cache the contents.jar/artifacts.jar of  the sites? because the plugins/features are already cached in the mvn repo. So what we cache based on a sha hash is very little. 

Johan
On Thu, 3 Dec 2020 at 11:14, Christoph Läubrich <laeubi@xxxxxxxxxxxxxx <mailto:laeubi@xxxxxxxxxxxxxx>> wrote: 

    I can't tell why it does not work with exporting the feature from
    within
    eclipse, most probably one needs to have a special target (including
    all
    binaries) and proper export settings.

    I can only tell that if you create a self-contained updatesite via
    tycho
    category.xml build (this can be an independet build you onyl use local
    to generate the site) using org.eclipse.equinox.executable (!) as a
    feature produces an update-site that works when used in another build.

    It is not enough to have the feature/jars you also need the appropriate
    binary jars for your product build.


    Am 28.11.20 um 18:40 schrieb gm:
     > Hi,
     >
     > I redescribe my problem:
     > I cannot get a working tycho Jenkins build because it says that
    it can’t find org.eclipse.equinox.executable. This happens in the
    build product phase of the build.
     > I first thought that the p2-site was the problem because I could
    not add the org.eclipse.equinox.executable there. But that problem
    is solved thanks to the p2-dev - people. I appended this feature to
    the site.
     > So if I now look into my site, I see an entry for
    org.eclipse.equinox feature in the artefacts.xml and I see an entry
    for the org.eclipse.equionox.feature.jar in the content.xml but
    still the problem persists.
     > What could be the reason?
     >
     > Kind regards,
     > Günther Mahr
     >
     >> Am 24.11.2020 um 13:57 schrieb Christoph Läubrich
    <laeubi@xxxxxxxxxxxxxx <mailto:laeubi@xxxxxxxxxxxxxx>>:
     >>
     >> You could take a look at
     >>
     >>
    https://wiki.eclipse.org/Tycho/eclipse-repository#Creating_a_self-contained_p2_repository
    <https://wiki.eclipse.org/Tycho/eclipse-repository#Creating_a_self-contained_p2_repository>
     >>
     >>
     >>
     >> Am 24.11.20 um 13:51 schrieb gm:
     >>> Hi,
     >>> currently I’m creating the offlineSite by using the export
    function of my  feature.xml which in Eclipse terms is a „Feature
    manifest file“.
     >>> I would like to try the approach you mention, but is there any
    documentation on how to do that?
     >>> Kind regards,
     >>> Günther
     >>>> Am 24.11.2020 um 10:56 schrieb Christoph Läubrich
    <laeubi@xxxxxxxxxxxxxx <mailto:laeubi@xxxxxxxxxxxxxx>>:
     >>>>
     >>>> As you are exporting it vie Eclipse IDE maybe there is some
    option missing.
     >>>>
     >>>> Anyway I would recommend that you create a tycho project that
    uses a category.xml to build the "offline-site", if you enable
    "includeAllDependecies" this will produce a standalone P2 repository
    that can be consumed by tycho afterwards.
     >>>>
     >>>>
     >>>> Am 24.11.20 um 10:15 schrieb gm:
     >>>>> Hi Christoph,
     >>>>> thank you for your quick response!
     >>>>> I created a GitHub project under
    https://github.com/guenthermahr/offlineTargetPlatform/
    <https://github.com/guenthermahr/offlineTargetPlatform/>
     >>>>> with some files in it which should make my problem clearer,
    although I cannot upload the whole p2-site, it’s too big.
     >>>>> If you need more information, please say so!
     >>>>> Thanks again for your help!
     >>>>> Günther
     >>>>>> Am 24.11.2020 um 07:19 schrieb Christoph Läubrich
    <laeubi@xxxxxxxxxxxxxx <mailto:laeubi@xxxxxxxxxxxxxx>>:
     >>>>>>
     >>>>>> I think it would be more interesting to see how you build
    your update-site, maybe you can create a small example on github?
     >>>>>>
     >>>>>> I have already created custom update-sites including the
    launchers and haven't had problem in the past.
     >>>>>>
     >>>>>> Am 23.11.20 um 23:00 schrieb gm:
     >>>>>>>> Anfang der weitergeleiteten Nachricht:
     >>>>>>>>
     >>>>>>>> *Von: *"Mahr, Guenther" <gmahr_extern@xxxxxxxxxx
    <mailto:gmahr_extern@xxxxxxxxxx> <mailto:gmahr_extern@xxxxxxxxxx
    <mailto:gmahr_extern@xxxxxxxxxx>>>
     >>>>>>>> *Betreff: **WG: How to prepare the p2-site for an offline
    target platform to be used in a tycho build*
     >>>>>>>> *Datum: *23. November 2020 um 22:57:57 MEZ
     >>>>>>>> *An: *"guenther.mahr@xxxxxxx
    <mailto:guenther.mahr@xxxxxxx> <mailto:guenther.mahr@xxxxxxx
    <mailto:guenther.mahr@xxxxxxx>>" <guenther.mahr@xxxxxxx
    <mailto:guenther.mahr@xxxxxxx> <mailto:guenther.mahr@xxxxxxx
    <mailto:guenther.mahr@xxxxxxx>>>
     >>>>>>>>
     >>>>>>>> *Von:*Mahr, Guenther
     >>>>>>>> *Gesendet:*Montag, 23. November 2020 22:54
     >>>>>>>> *An:*'tycho-user@xxxxxxxxxxx
    <mailto:tycho-user@xxxxxxxxxxx> <mailto:tycho-user@xxxxxxxxxxx
    <mailto:tycho-user@xxxxxxxxxxx>>' <tycho-user@xxxxxxxxxxx
    <mailto:tycho-user@xxxxxxxxxxx> <mailto:tycho-user@xxxxxxxxxxx
    <mailto:tycho-user@xxxxxxxxxxx>>>
     >>>>>>>> *Betreff:*How to prepare the p2-site for an offline target
    platform to be used in a tycho build
     >>>>>>>> Hi,
     >>>>>>>> I have a local p2-site and have tried various things to
    make the tycho build work and produce the desired result. The
    problem is always the executable feature. To build my offline target
    platform (the p2-site) that I give to tycho I use a feature manifest
    file. If I add the feature org.eclipse.equinox.executable there - in
    included features - the export function creates entries for the
    launchers in the artifacts.jar but does not create an entry for the
    feature itself and also does not output a
    org.eclipse.equinox.executable.jar. Then during the tyoho build this
    jar is missing which leads to an error.
     >>>>>>>> I tried also to create a separate additional p2-site just
    for the executable by using the FeaturesAndBundlesPublisher. (I
    added the the feature folder and a packed version of it there). This
    works fine and if I add this site to the tycho build, the build
    succeeds - but with the "disadvantage" that there is no launcher in
    the product (no eclipse.exe). I understand that this comes from the
    fact that the FeaturesAndBundlesPublisher works different then the
    feature.xml export function: it does not care for binaries, so the
    binaries in which the launchers are, are missing.
     >>>>>>>> So my simple question is what can I do to make a tycho
    build work just with offline links.
     >>>>>>>> I attach my the target profile file used in the tycho build.
     >>>>>>>> Thank you for your help!
     >>>>>>>> Günther Mahr
     >>>>>>> _______________________________________________
     >>>>>>> tycho-user mailing list
     >>>>>>> tycho-user@xxxxxxxxxxx <mailto:tycho-user@xxxxxxxxxxx>
     >>>>>>> To unsubscribe from this list, visit
    https://www.eclipse.org/mailman/listinfo/tycho-user
    <https://www.eclipse.org/mailman/listinfo/tycho-user>
     >>>>>> _______________________________________________
     >>>>>> tycho-user mailing list
     >>>>>> tycho-user@xxxxxxxxxxx <mailto:tycho-user@xxxxxxxxxxx>
     >>>>>> To unsubscribe from this list, visit
    https://www.eclipse.org/mailman/listinfo/tycho-user
    <https://www.eclipse.org/mailman/listinfo/tycho-user>
     >>>>> _______________________________________________
     >>>>> tycho-user mailing list
     >>>>> tycho-user@xxxxxxxxxxx <mailto:tycho-user@xxxxxxxxxxx>
     >>>>> To unsubscribe from this list, visit
    https://www.eclipse.org/mailman/listinfo/tycho-user
    <https://www.eclipse.org/mailman/listinfo/tycho-user>
     >>>> _______________________________________________
     >>>> tycho-user mailing list
     >>>> tycho-user@xxxxxxxxxxx <mailto:tycho-user@xxxxxxxxxxx>
     >>>> To unsubscribe from this list, visit
    https://www.eclipse.org/mailman/listinfo/tycho-user
    <https://www.eclipse.org/mailman/listinfo/tycho-user>
     >>> _______________________________________________
     >>> tycho-user mailing list
     >>> tycho-user@xxxxxxxxxxx <mailto:tycho-user@xxxxxxxxxxx>
     >>> To unsubscribe from this list, visit
    https://www.eclipse.org/mailman/listinfo/tycho-user
    <https://www.eclipse.org/mailman/listinfo/tycho-user>
     >> _______________________________________________
     >> tycho-user mailing list
     >> tycho-user@xxxxxxxxxxx <mailto:tycho-user@xxxxxxxxxxx>
     >> To unsubscribe from this list, visit
    https://www.eclipse.org/mailman/listinfo/tycho-user
    <https://www.eclipse.org/mailman/listinfo/tycho-user>
     >
     > _______________________________________________
     > tycho-user mailing list
     > tycho-user@xxxxxxxxxxx <mailto:tycho-user@xxxxxxxxxxx>
     > To unsubscribe from this list, visit
    https://www.eclipse.org/mailman/listinfo/tycho-user
    <https://www.eclipse.org/mailman/listinfo/tycho-user>
     >
    _______________________________________________
    tycho-user mailing list
    tycho-user@xxxxxxxxxxx <mailto:tycho-user@xxxxxxxxxxx>
    To unsubscribe from this list, visit
    https://www.eclipse.org/mailman/listinfo/tycho-user
    <https://www.eclipse.org/mailman/listinfo/tycho-user>



--
Johan Compagner
Servoy

_______________________________________________
tycho-user mailing list
tycho-user@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/tycho-user

Back to the top