Preparing the AWS system for installing Che

The following sections describe how to set up Kubernetes for Eclipse Che on Amazon Elastic Compute Cloud (Amazon EC2).

Prerequisites
  • A running instance of Kubernetes, version 1.9 or higher, and Ingress

  • The kubectl tool installed

  • The chectl tool installed

Installing Kubernetes on Amazon EC2

  1. Set up the AWS Command Line Interface (AWS CLI). For detailed installation instructions, see Installing the AWS CLI.

  2. Check the PATH environment variable if the aws tool is not available.

  3. Install Kubernetes on EC2. There are several ways to have a running Kubernetes instance on EC2. Here, the kops tool is used to install Kubernetes. For details, see Installing Kubernetes with kops.

  4. Set up IAM user for kops. For detailed instructions, see Set up IAM user.

This document assumes that Eclipse Che must be configured to run at Che AWS Cloud.

Setting up DNS

One way to set up Domain Name System (DNS) is to create the Amazon Route53 to manage the aws.my-ide.cloud domain.

To set up DNS:

  1. Create the zone on AWS:

    $ aws route53 create-hosted-zone --name aws.my-ide.cloud --caller-reference 1
    
    \{
     "Location": "https://route53.amazonaws.com/2013-04-01/hostedzone/<ZONE-ID>",
     "HostedZone": \{
     "Id": "/hostedzone/<ZONE-ID>",
     "Name": "aws.my-ide.cloud.",
     "CallerReference": "1",
     "Config": \{
     "PrivateZone": false
     },
     "ResourceRecordSetCount": 2
     },
     "ChangeInfo": \{
     "Id": "/change/C1ZNLBU45DJUJL",
     "Status": "PENDING",
     "SubmittedAt": "2019-07-08T08:14:39.772Z"
     },
     "DelegationSet": \{
     "NameServers": [
     "ns-1693.awsdns-19.co.uk",
     "ns-1133.awsdns-13.org",
     "ns-150.awsdns-18.com",
     "ns-965.awsdns-56.net"
     ]
     }
    }
  2. Configure the four DNS nameservers on the my-ide.cloud DNS. Note that when a custom DNS provider, updating the record takes a few hours.

    dns nameservers
  3. Create the Simple Storage Service (s3) storage to store the kops configuration.

    $ aws s3 mb s3://clusters.aws.my-ide.cloud
    make_bucket: clusters.aws.my-ide.cloud
  4. Inform kops of this new service:

    $ export KOPS_STATE_STORE=s3://clusters.aws.my-ide.cloud
  5. Create the kops cluster by providing the cluster zone. For example, for Europe, the zone is eu-west-1c.

    $ kops create cluster --zones=eu-west-1c eu.aws.my-ide.cloud
  6. Create the cluster:

    $ kops update cluster eu.aws.my-ide.cloud --yes
  7. After the cluster is ready, validate it:

    $ kops validate cluster
    
    Using cluster from kubectl context: eu.aws.my-ide.cloud
    
    Validating cluster eu.aws.my-ide.cloud
    
    INSTANCE GROUPS
    NAME                ROLE    MACHINETYPE  MIN  MAX  SUBNETS
    master-eu-west-1c   Master  m3.medium    1    1    eu-west-1c
    nodes               Node    t2.medium    2    2    eu-west-1c
    
    NODE STATUS
    NAME                                         ROLE    READY
    ip-172-20-38-26.eu-west-1.compute.internal   node    True
    ip-172-20-43-198.eu-west-1.compute.internal  node    True
    ip-172-20-60-129.eu-west-1.compute.internal  master  True
    
    Your cluster eu.aws.my-ide.cloud is ready
  8. Check the cluster using the kubectl command. The kubectl context is also configured automatically by the kops tool:

    $ kubectl config current-context
    eu.aws.my-ide.cloud
    
    $ kubectl get pods --all-namespaces
    
    All the pods in the running state are displayed.

Installing Ingress-nginx

To install Ingress-nginx:

  1. Install the default configuration.

    $ kubectl apply \
      -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml
  2. Install the configuration for AWS.

    $ kubectl apply \
      -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/aws/service-l4.yaml
    $ kubectl apply \
      -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/aws/patch-configmap-l4.yaml

    The following output confirms that the Ingress controller is running.

    $ kubectl get pods --namespace ingress-nginx
    NAME                                        READY   STATUS    RESTARTS   AGE
    nginx-ingress-controller-76c86d76c4-gswmg   1/1     Running   0          9m3s
  3. Find the external IP of ingress-nginx.

    $ kubectl get services --namespace ingress-nginx -o jsonpath='{.items[].status.loadBalancer.ingress[0].hostname}'
    Ade9c9f48b2cd11e9a28c0611bc28f24-1591254057.eu-west-1.elb.amazonaws.com

    Troubleshooting: If the output is empty, it implies that the cluster has configuration issues. Use the following command to find the cause of the issue:

    $ kubectl describe service -n ingress-nginx ingress-nginx

    Output similar to the following means a needed role must be created manually:

    arn:aws:sts::269287474311:assumed-role...4bff is not authorized to perform: iam:CreateServiceLinkedRole on resource: arn:aws:iam::269287474311:role/aws-service-role/elasticloadbalancing.amazonaws.com/AWSServiceRoleForElasticLoadBalancing

    Run the following command to create the role:

    $ aws iam create-service-linked-role --aws-service-name "elasticloadbalancing.amazonaws.com"
  4. Add hosts on route 53 with this given hostname https://console.aws.amazon.com/route53/home?region=eu-west-1#hosted-zones:. Ensure that you include the colon (:) at the end of this URL.

  5. Create the wildcard DNS (for .aws-my-ide.cloud) with the previous hostname and ensure to add the dot (.) at the end of the hostname. In the Type drop-down list, select CNAME.

    create record set

    The following is an example of the resulting window after adding all the values.

    create record set all values

    The che.aws.my-ide.cloud address must resolve to an IP address.

    $ host che.aws.my-ide.cloud
    che.aws.my-ide.cloud is an alias for ade9c9f48b2cd11e9a28c0611bc28f24-1591254057.eu-west-1.elb.amazonaws.com.
    ade9c9f48b2cd11e9a28c0611bc28f24-1591254057.eu-west-1.elb.amazonaws.com has address 54.77.155.195

It is now possible to install Eclipse Che on this existing Kubernetes instance.

Enabling the TLS and DNS challenge

To use the Cloud DNS and TLS, some service accounts must be enabled to have cert-manager managing the DNS challenge for the Let’s Encrypt service.

  1. Create a new permission file.

  2. Use the following command to obtain the zone ID:

    $ aws route53 list-hosted-zones
    {
        "HostedZones": [
            {
                "Id": "/hostedzone/ABCDEFGH",
                "Name": "aws.my-ide.cloud.",
                "CallerReference": "1",
                "Config": {
                    "PrivateZone": false
                },
                "ResourceRecordSetCount": 5
            }
        ]
    }
  3. Copy the following content and replace INSERT_ZONE_ID with the route53 zone ID:

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "route53:GetChange",
                    "route53:ListHostedZonesByName"
                ],
                "Resource": [
                    "*"
                ]
            },
            {
                "Effect": "Allow",
                "Action": [
                    "route53:ChangeResourceRecordSets"
                ],
                "Resource": [
                    "arn:aws:route53:::hostedzone/<INSERT_ZONE_ID>"
                ]
            }
        ]
    }
  4. In the EC2 Dashboard, identify the IAM role used by the master node.

    aws lauch instance

    It is located under the Description tab, in the IAM role field.

    describtion tab iam role
  5. Click the IAM role link (masters.eu.aws.my-ide.cloud, in this case).

  6. Click the Add inline policy link at the bottom of the window.

    aws summary iam role
  7. In the Create policy window, on the JSON tab, paste the content of the JSON file created earlier and click the Review policy button.

    aws create policy
  8. In the Name field, type eclipse-che-route53 and click Create Policy.

    create policy review policy

Installing cert-manager

  1. To install cert-manager, run the following commands (for details, see Installing Cert on Kubernetes):

    $ kubectl create namespace cert-manager
    namespace/cert-manager created
    $ kubectl label namespace cert-manager certmanager.k8s.io/disable-validation=true
    namespace/cert-manager labeled
  2. Set validate=false. If set to true, it will only work with the latest Kubernetes:

    $ kubectl apply \
      -f https://github.com/jetstack/cert-manager/releases/download/v0.8.1/cert-manager.yaml \
      --validate=false
  3. Create the Che namespace if it does not already exist:

    $ kubectl create namespace che
    namespace/che created
  4. Create the cert-manager user:

    $ aws iam create-user --user-name cert-manager
    {
        "User": {
            "Path": "/",
            "UserName": "cert-manager",
            "UserId": "ABCDEF",
            "Arn": "arn:aws:iam::1234:user/cert-manager",
            "CreateDate": "2019-07-30T13:50:48Z"
        }
    }
  5. Create the access key:

    $ aws iam create-access-key --user-name cert-manager
    {
        "AccessKey": {
            "UserName": "cert-manager",
            "AccessKeyId": "ABCDEF",
            "Status": "Active",
            "SecretAccessKey": "mySecret",
            "CreateDate": "2019-07-30T13:52:59Z"
        }
    }
    Remember the access key for later use.
  6. Create a secret from the SecretAccessKey content.

    $ kubectl create secret generic aws-cert-manager-access-key \
      --from-literal=CLIENT_SECRET=<REPLACE WITH SecretAccessKey content> -n cert-manager
  7. Use the Add inline policy link to add the inline policy to AWS Cert-Manager.

    aws summary iam role
  8. Paste the following inline policy in the JSON tab:

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": "route53:GetChange",
                "Resource": "arn:aws:route53:::change/*"
            },
            {
                "Effect": "Allow",
                "Action": "route53:ChangeResourceRecordSets",
                "Resource": "arn:aws:route53:::hostedzone/*"
            },
            {
                "Effect": "Allow",
                "Action": "route53:ListHostedZonesByName",
                "Resource": "*"
            }
        ]
    }
    json review policy
  9. Click Review policy.

    create policy review
  10. In the Name field, type route53, and click Create policy.

  11. To create the certificate issuer, change the email address and specify the accessKeyID:

    $ cat <<EOF | kubectl apply -f -
    apiVersion: certmanager.k8s.io/v1alpha1
    kind: ClusterIssuer
    metadata:
      name: che-certificate-issuer
    spec:
      acme:
        dns01:
          providers:
          - route53:
              region: eu-west-1
              accessKeyID: <USE ACCESS_KEY_ID_CREATED_BEFORE>
              secretAccessKeySecretRef:
                name: aws-cert-manager-access-key
                key: CLIENT_SECRET
            name: route53
        email: florent@example.com
        privateKeySecretRef:
          name: letsencrypt
        server: https://acme-v02.api.letsencrypt.org/directory
    EOF
  12. Add the certificate by editing the domain name value (aws.my-ide.cloud, in this case) and the dnsName value:

    $ cat <<EOF | kubectl apply -f -
    apiVersion: certmanager.k8s.io/v1alpha1
    kind: Certificate
    metadata:
     name: che-tls
     namespace: che
    spec:
     secretName: che-tls
     issuerRef:
       name: che-certificate-issuer
       kind: ClusterIssuer
     dnsNames:
       - '*.aws.my-ide.cloud'
     acme:
       config:
         - dns01:
             provider: route53
           domains:
             - '*.aws.my-ide.cloud'
    EOF
  13. Check if the issuerRef name is the same as the ClusterIssuer. A new DNS challenge is being added to the DNS zone for Let’s encrypt.

    aws hosted zones dns

    The cert-manager logs should contain information about the DNS challenge.

  14. Obtain the name of the pods:

    $ kubectl get pods --namespace cert-manager
    NAME                                       READY   STATUS    RESTARTS   AGE
    cert-manager-6587688cb8-wj68p              1/1     Running   0          6h
    cert-manager-cainjector-76d56f7f55-zsqjp   1/1     Running   0          6h
    cert-manager-webhook-7485dd47b6-88m6l      1/1     Running   0          6h
  15. Obtain the logs using the following command (here, cert-manager-8d478bb45-sdfmz is the name of the cert-manager pod):

    $ kubectl logs -f cert-manager-8d478bb45-sdfmz -n cert-manager
    I0730 14:46:25.382385       1 sync.go:274] Need to create 0 challenges
    I0730 14:46:25.382401       1 sync.go:319] Waiting for all challenges for order "che-tls-3365293372" to enter 'valid' state
    I0730 14:46:25.382431       1 controller.go:204] cert-manager/controller/orders "level"=0 "msg"="finished processing work item" "key"="che/che-tls-3365293372"
    I0730 14:46:25.382813       1 controller.go:219] cert-manager/controller/challenges "level"=0 "msg"="finished processing work item" "key"="che/che-tls-3365293372-0"
    I0730 14:46:25.382843       1 controller.go:213] cert-manager/controller/challenges "level"=0 "msg"="syncing resource" "key"="che/che-tls-3365293372-0"
    I0730 14:46:25.383037       1 dns.go:101] Presenting DNS01 challenge for domain "aws.my-ide.cloud"
    I0730 14:47:03.061546       1 dns.go:112] Checking DNS propagation for "aws.my-ide.cloud" using name servers: [100.64.0.10:53]
    I0730 14:47:03.220952       1 dns.go:124] Waiting DNS record TTL (60s) to allow propagation of DNS record for domain "_acme-challenge.aws.my-ide.cloud.”
  16. Ensure that the certificate is ready:

    $ kubectl describe certificate/che-tls -n che
    Status:
      Conditions:
        Last Transition Time:  2019-07-30T14:46:23Z
        Message:               Certificate issuance in progress. Temporary certificate issued.
        Reason:                TemporaryCertificate
        Status:                False
        Type:                  Ready
    Events:
      Type    Reason        Age   From          Message
      ----    ------        ----  ----          -------
      Normal  OrderCreated  50s   cert-manager  Created Order resource "che-tls-3365293372"
  17. Wait for the status to become OK and ensure that the log contains the following entry:

    I0729 13:56:26.140886       1 conditions.go:143] Found status change for Certificate "che-tls" condition "Ready": "False" -> "True"; setting lastTransitionTime to 2019-07-29 13:56:26.140866531 +0000 UTC m=+4557.134131468
  18. Ensure that the status is up-to-date using the following command:

    $ kubectl describe certificate/che-tls -n che
    
    Status:
      Conditions:
        Last Transition Time:  2019-07-30T14:48:07Z
        Message:               Certificate is up to date and has not expired
        Reason:                Ready
        Status:                True
        Type:                  Ready
      Not After:               2019-10-28T13:48:05Z
    Events:
      Type    Reason         Age    From          Message
      ----    ------         ----   ----          -------
      Normal  OrderCreated   5m29s  cert-manager  Created Order resource "che-tls-3365293372"
      Normal  OrderComplete  3m46s  cert-manager  Order "che-tls-3365293372" completed successfully
      Normal  CertIssued     3m45s  cert-manager  Certificate issued successfully

Installing Che on Kubernetes using the chectl command

Prerequisites
Procedure

To install Che:

  1. Run the following command:

    $ chectl server:start --platform=k8s --installer=helm --domain=aws.my-ide.cloud --multiuser --tls
      ✔ ✈️  Kubernetes preflight checklist
        ✔ Verify if kubectl is installed
        ✔ Verify remote kubernetes status...done.
        ✔ Verify domain is set...set to aws.my-ide.cloud.
      ✔ 🏃‍  Running Helm to install Che
        ✔ Verify if helm is installed
        ✔ Check for TLS secret prerequisites...che-tls secret found.
        ✔ Create Tiller Role Binding...it already exist.
        ✔ Create Tiller Service Account...it already exist.
        ✔ Create Tiller RBAC
        ✔ Create Tiller Service...it already exist.
        ✔ Preparing Che Helm Chart...done.
        ✔ Updating Helm Chart dependencies...done.
        ✔ Deploying Che Helm Chart...done.
      ✔ ✅  Post installation checklist
        ✔ PostgreSQL pod bootstrap
          ✔ scheduling...done.
          ✔ downloading images...done.
          ✔ starting...done.
        ✔ Keycloak pod bootstrap
          ✔ scheduling...done.
          ✔ downloading images...done.
          ✔ starting...done.
        ✔ Che pod bootstrap
          ✔ scheduling...done.
          ✔ downloading images...done.
          ✔ starting...done.
        ✔ Retrieving Che Server URL...https://che-che.aws.my-ide.cloud
        ✔ Che status check
    Command server:start has completed successfully.
  2. The certificate generated by Let’s Encrypt is a valid certificate.

    certificate generate lets encrypt
    eclipse che welcome to your workspace
Tags: