Pre-Requisites
Getting you started with a common set of tools, used by all packages.
Kubernetes Client
First, you will need a command tool named kubectl
. This application allows you to interact with
your Kubernetes cluster from the command line. While Kubernetes also comes with a Web UI, it is much simpler
to document the installation procedures using command line tool. Also does the Web UI change over time, and
with different Kubernetes variants. However, the kubectl
tool works with all variations of Kubernetes, as it
uses the standardized API in the background.
You can find more information in the Kubernetes documentation: Install and Set Up kubectl.
Kubernetes Cluster
For the cloud side environment, you will need an installation of Kubernetes. Kubernetes comes in different forms, and we try to document a few of them for you. There are other variants as well, and you are welcome to try all of them. All packages should be able to run on any Kubernetes you provide.
Generally, you can either use a cloud provider, such as Azure (AKS) or AWS (EKS) to provide a Kubernetes cluster for you, or you can set up one by yourself. Here you can find a list of Kubernetes options. As the Kubernetes API is standardized both ways will work. However, setting up Kubernetes yourself is not trivial and requires some more effort. As part of this tutorial, we present two different K8s distributions (and how to deploy Eclipse IoT Packages with them), namely MicroK8s and Minikube.
Packages are encouraged to give you an estimate of what resources they require. The following is an example of what this may look like. You will need to translate this into the specific Kubernetes environment you have. Also may the package declare on which Kubernetes platform it was tested. This doesn’t mean that other Kubernetes versions don’t work, but sets some expectations of what was tested at some point.
Requirements
-
- Kubernetes
- 1.17.x
-
- CPUs
- 2
-
- Memory
- 1024 MiB
-
- Disk
- 40 GiB
For each documentation Kubernetes environment on this page, you will find a section that explains how to do this.
MicroK8s
MicroK8s is a Kubernetes distribution, maintained by Canonical and enables developers to run a fully-fledged Kubernetes cluster on their own infrastructure. In contrast to minikube it is not only intended for testing purposes but also for production scenarios, e.g. in cases where a cloud setup is not possible or desirable.
All you need is a virtual or physical machine with root access and the snap package manager.
Install MicroK8s, executing sudo snap install microk8s --classic
Check the cluster status, by running microk8s status --wait-ready
Once the cluster is up and running you will need to enable a few modules that are required for installing an Eclipse IoT Packages package such as Cloud2Edge.
LOCAL_ADDRESS=$(ip addr show eth0 | grep "inet\b" | awk '{print $2}' | cut -d/ -f1)
microk8s enable metrics-server
microk8s enable storage
microk8s enable dns
microk8s enable metallb:$LOCAL_ADDRESS-$LOCAL_ADDRESS
These commands enable
- the metrics server, which is helpful for further investigating your cluster usage
- the storage module, which enables persistent volumes
- the cluster dns server
- and the load balancing module, which enables external access to your cluster
In order to control the cluster using the Kubernetes CLI (kubectl
) you can either use integrated option which ships
with MicroK8s. Alternatively, e.g. if you want to control your cluster from another machine you can extract the cluster
config file and use it with a native install of kubectl
.
Integrated option
Use microk8s kubectl <command>
Native option
Execute microk8s kubectl config view --raw > $HOME/.kube/config
Now, use kubectl <command>
In case you are not running the cluster on a your local machine you can also enable remote access by following these steps:
- stop your cluster by running
microk8s stop
- edit the file
/var/snap/microk8s/current/certs/csr.conf.template
and add your domain/ip address:
DNS.6 = <domain_of_microk8s_host>
IP.7 = <public_ip_of_microk8s_host>
Restart your cluster by running microk8s start
and extract the new Kubernetes configuration by executing
microk8s kubectl config view --raw
.
Now copy the configuration description to your local machine under ~/.kube/config
.
Loadbalancing and Ingress
Once you have a Kubernetes cluster available and installed the Kubernetes and Helm CLI (see below), you are now ready to setup an Eclipse IoT Packages deployment such as Cloud2Edge. By default the containers can communicate within the cluster. But to access a container and its respective services from outside the cluster (e.g. from another machine) you need to perform some extra steps. Depending on your setup, you have 3 options available how to make your services externally available.
NodePort
The simplest way to expose your services is by using a Kubernetes Service with type NodePort. The Cloud2Edge Helm chart uses this option by default. Using this in production mode has some drawbacks and is not recommended as it is very static, enables just one Service per port and only allows you to use ports in the range 30000–32767.
LoadBalancer
Another option is the LoadBalancer
Service type, which requires your Kubernetes provider to supply an external load balancing module. The provider is
also responsible for provisioning external IP addresses.
The Cloud2Edge enables this option by setting the flags hono.useLoadBalancer
to true
and ditto.nginx.service.type
to LoadBalancer
.
The downside of this approach is that each of your Services requires its own publicly available IP address, which
might either not be desirable (due to higher costs) or possible at all e.g. in case you are running your deployment
on a custom Kubernetes setup (such as MicroK8s).
Ingress Loadbalancing
Ingress controllers employ a single LoadBalancer service, which then routes all incoming traffic to the actual controller in charge of distributing it to the right endpoints. This allows not only differentiating by port, but also routing by path (HTTP) or subdomain. We are using the Ambassador controller, as it is capable of routing not only HTTP ingress but also on the TCP level.
- Deploy the Cloud2Edge Helm chart setting the aforementioned flags to not use the LoadBalancer type
- Create a namespace for Ambassador:
kubectl create namespace ambassador
- Add the Helm repository:
helm repo add datawire https://www.getambassador.io
- Create a file named
override.yaml
, that contains all necessary Ambassador configurations:enableAES: false image: repository: docker.io/datawire/ambassador replicaCount: 1 service: ports: - name: mqtt-adapter port: 1883 targetPort: 1883 - name: http-adapter port: 18080 targetPort: 18080 - name: device-registry port: 28080 targetPort: 28080 - name: dispatch-router port: 5671 targetPort: 15671 - name: ditto port: 38080 targetPort: 38080
- Start Ambassador by running
helm install ambassador -n ambassador -f override.yaml datawire/ambassador
- Then we create the necessary mappings, such that the controller knows where to route incoming traffic.
These mappings are specifically adjusted to the Cloud2Edge deployment, so you will have to create your own
mappings for other packages. Create a new file named
ambassador-mappings.yaml
and replace the release name and the namespace of your deployment:apiVersion: getambassador.io/v2 kind: TCPMapping metadata: name: ambassador-http-adapter namespace: ambassador spec: port: 18080 service: {name-of-helm-release}-adapter-http.{kubernetes-namespace}:8080 --- apiVersion: getambassador.io/v2 kind: TCPMapping metadata: name: ambassador-mqtt-adapter namespace: ambassador spec: port: 1883 service: {name-of-helm-release}-adapter-mqtt.{kubernetes-namespace}:1883 --- apiVersion: getambassador.io/v2 kind: TCPMapping metadata: name: ambassador-device-registry namespace: ambassador spec: port: 28080 service: {name-of-helm-release}-service-device-registry-ext.{kubernetes-namespace}:28080 --- apiVersion: getambassador.io/v2 kind: TCPMapping metadata: name: ambassador-dispatch-router namespace: ambassador spec: port: 15671 service: {name-of-helm-release}-service-device-registry-ext.{kubernetes-namespace}:15671 --- apiVersion: getambassador.io/v2 kind: TCPMapping metadata: name: ambassador-ditto namespace: ambassador spec: port: 38080 service: {name-of-helm-release}-ditto-nginx.{kubernetes-namespace}:8080
- Now add the mappings to your cluster:
kubectl apply -f ambassador-mappings.yaml
You are now able to access the deployed Cloud2Edge services externally. Check what IP address you are using by
running kubectl get service -n ambassador
.
Navigate to http://
Minikube
Minikube is Kubernetes in a bottle.
Instead of provisioning a full-blown cluster, it will create a virtual machine on your local system, and provision a small, single-node cluster inside it. As it puts the operating system in a VM, Minikube itself can run on all major operating systems, including Windows and macOS.
Instead of duplicating the effort, documenting how to get Minikube up and running, we leave this to the excellent documentation of Minikube itself.
Getting started
Once you have everything installed, you should be able to start a new cluster by executing:
minikube start
And you can switch kubectl
to the context minikube
, and interact with your cluster:
kubectl config use-context minikube
For example, get the current version of the client and server:
kubectl version
Which should show a proper version for the client and the server:
Client Version: version.Info{Major:"1", Minor:"11+", GitVersion:"v1.11.0+d4cacc0", GitCommit:"d4cacc0", GitTreeState:"clean", BuildDate:"2018-10-10T16:38:01Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.4+c2a5caf", GitCommit:"c2a5caf", GitTreeState:"clean", BuildDate:"2019-09-21T02:12:52Z", GoVersion:"go1.11.13", Compiler:"gc", Platform:"linux/amd64"}
Starting and stopping
When you no longer need your cluster, you can stop it using:
minikube stop
This will suspend the VM so that you can, later on, resume it by executing:
minikube start
Or delete it using:
minikube delete
Resources
You can translate the package resources requirements into arguments for the start
command like this:
minikube start --cpus <cpus> --disk-size <size> --memory <memory> --kubernetes-version <version>
Using the following arguments:
- cpus
- The number of CPUs you allocate for Minikube (e.g.
--cpus 2
). - size
-
The size of the disk available for the cluster and persistent volumes. In the format
<number><unit>
, where unit can be eitherk
,m
, org
(e.g. 20GiB means--disk-size 20g
). - memory
-
The amount of RAM allocated to the virtual machine. This is the amount in MiB (e.g. for 8GiB means
--memory 8192
. - version
-
The Kubernetes version deployed into the virtual machine (e.g.
--kubernetes-version v1.20.2
).
Addons
Some packages may require additional addons, like the ingress addon for providing a default ingress controllers.
Such addons can be enabled when starting the Minikube instance, using the following flag:
minikube start ... --addons ingress
Helm
You will need an installation of Helm on the machine which is used to deploy the packages. You can find installation instructions for Helm in the Helm documentation under Installing Helm.
The required Helm version is 3.4 or later.
Repository
The Eclipse IoT Packages project hosts a Helm chart repository for Eclipse IoT projects. Adding the repository can be done on your local machine be executing:
helm repo add eclipse-iot https://eclipse.org/packages/charts
Read more: Helm chart repository.
Command line tools
Some tutorials might require some common command line tool. The installation depends on the operating system you are using.
It is required to have the following tools installed:
Bash
Bash is available on Windows, Linux and Mac OS X. True platform independence. So all commands which you are supposed to execute can be executed in Bash, version 3 or newer.
curl
For downloading files and execution API calls the tool curl
will be used.
Mosquitto CLI
Mosquitto command line tools: e.g. mosquitto_pub