Re: [cbi-dev] Questions about Python and publishing to another Git repo

[Mikael Barbero <mikael.barbero@xxxxxxxxxxxxxxxxxxxxxx>]

Jim,

See my answers inline.

On 17 Mar 2021, at 17:14, Jim Hughes <jhughes@xxxxxxxx> wrote:

Hi Mikael,

Replies inlined.

On 3/17/2021 7:35 AM, Mikael Barbero wrote:

Hi Jim, 

See my answers below

On 17 Mar 2021, at 00:53, Jim Hughes <jhughes@xxxxxxxx> wrote:

Hi all,

I have two things I'm trying to iron out for Eclipse GeoMesa's CI jobs.

First, GeoMesa is mostly a typical Maven with Java 8 build; there is a PySpark integration which is hidden behind a profile and building with that profile requires Python and installing a few pip packages (setuptools and pyspark).  Also, GeoMesa's documentation is built with Sphinx.

Is there a recommended way to add the necessary Python packages to a build?  (In an internal Jenkins, we have used virtualenv to install the packages in an ad hoc manner.  I tried that on the GeoMesa CI, but it wasn't available.)

You can run your build in a custom docker image that you provide (and as such, with the tools you need). See https://wiki.eclipse.org/Jenkins#How_do_I_run_my_build_in_a_custom_container.3F

Yeah, I was afraid that was the answer.  I can see multiple sides here.  Since I had perfectly working Jenkins jobs, I'm in the camp that has to do more work to get things to work as they once did.;)  *shrugs*

I get that initially it's more work, but think also about the flexibility you gain by controlling entirely the environment you will be building in.

Second, GeoMesa's documentation source is in the main GeoMesa repo and the docs are hosted in a separate one hooked up to GitHub pages.  For historical reasons, the GitHub pages repo is in a separate GitHub organization.

Is there an obvious way to sort out the necessary pieces for that to work?  (Admittedly, this is kinda like https://wiki.eclipse.org/Jenkins#How_to_build_my_project.27s_website_with_Jenkins.3F where Hugo is replaced with Sphinx, things have already moved to github, and separate GitHub organizations are in play.)

First, I would work toward moving the GH pages repo in the proper organization. What you are doing may not be compliant with the Eclipse Development Process.

If the GitHub pages repo were in the LocationTech organization, would that make it easier?

Slightly.

I don't think there is an obvious way to sort out this thing though. Please open a ticket at https://bugs.eclipse.org/bugs/enter_bug.cgi?product=Community&component=CI-Jenkins with as many details as possible with what you want to achieve so that we can help you with this.

Is there a general procedure for adding an access secret to an Eclipse CI job?  (I did see something on the CI Wiki page and I kinda tried, but it didn't look like I had access to add a secret readily.) 

Open a ticket https://bugs.eclipse.org/bugs/enter_bug.cgi?product=Community&component=CI-Jenkins and we will set this up for you (the actual secret will be exchanged by mail or any other channel you find secure enough).

Cheers,

Mikaël Barbero 

Manager — Release Engineering and Technology | Eclipse Foundation

🐦 @mikbarbero

Eclipse Foundation: The Platform for Open Innovation and Collaboration

Attachment: signature.asc
Description: Message signed with OpenPGP