Che workspaces architecture

A Che deployment on the cluster consists of the Che server component, a database for storing user profile and preferences, and a number of additional deployments hosting workspaces. The Che server orchestrates the creation of workspaces, which consist of a deployment containing the workspace containers and enabled plug-ins, plus related components, such as:

  • ConfigMaps

  • services

  • endpoints

  • ingresses/routes

  • secrets

  • PVs

The Che workspace is a web application. It is composed of microservices running in containers that provide all the services of a modern IDE (an editor, language auto-completion, debugging tools). The IDE services are deployed with the development tools, packaged in containers and user runtime applications, which are defined as Kubernetes resources.

The source code of the projects of a Che workspace is persisted in a Kubernetes PersistentVolume. Microservices run in containers that have read-write access to the source code (IDE services, development tools), and runtime applications have read-write access to this shared directory.

The following diagram shows the detailed components of a Che workspace.

che workspaces
Figure 1. Che workspace components

In the diagram, there are three running workspaces: two belonging to User A and one to User C. A fourth workspace is getting provisioned where the plug-in broker is verifying and completing the workspace configuration.

Use the devfile format to specify the tools and runtime applications of a Che workspace.

Che workspace components

This section describes the components of a Che workspace.

Che Plugin plug-ins

Che Plugin plug-ins are special services that extend Che workspace capabilities. Che Plugin plug-ins are packaged as containers. Packaging plug-ins into a container has the following benefits:

  • It isolates the plug-ins from the main IDE, therefore limiting the resources that a plug-in has access to.

  • It uses the consolidated standard of container registries to publish and distribute plug-ins (as with any container image).

The containers that plug-ins are packaged into run as sidecars of the Che workspace editor and augment its capabilities.

Visual Studio Code extensions packaged in containers are Che plug-ins for the Che-Theia editor.

Multiple Che plug-ins can run in the same container (for better resource use), or a Che Plugin can run in its dedicated container (for better isolation).

Che Editor plug-in

A Che Editor plug-in is a Che workspace plug-in. It defines the web application that is used as an editor in a workspace. The default Che workspace editor is Che-Theia.

The Che-Theia source-code repository is at Che-Theia Github. It is based on the Eclipse Theia open-source project.

Che-Theia is written in TypeScript and is built on the Microsoft Monaco editor. It is a web-based source-code editor similar to Visual Studio Code (VS Code). It has a plug-in system that supports VS Code extensions.

Source code

Che-Theia

Container image

eclipse/che-theia

Endpoints

theia, webviews, theia-dev, theia-redirect-1, theia-redirect-2, theia-redirect-3

Che user runtimes

Use any non-terminating user container as a user runtime. An application that can be defined as a container image or as a set of Kubernetes or OpenShift resources can be included in a Che workspace. This makes it easy to test applications in the Che workspace.

To test an application in the Che workspace, include the application YAML definition used in stage or production in the workspace specification. It is a 12-factor app dev/prod parity.

Examples of user runtimes are Node.js, SpringBoot or MongoDB, and MySQL.

Che workspace JWT proxy

The JWT proxy is responsible for securing the communication of the Che workspace services. The Che workspace JWT proxy is included in a Che workspace only if the Che server is configured in multi-user mode.

An HTTP proxy is used to sign outgoing requests from a workspace service to the Che server and to authenticate incoming requests from the IDE client running on a browser.

Source code

JWT proxy

Container image

eclipse/che-jwtproxy

Che plug-ins broker

Plug-in brokers are special services that, given a plug-in meta.yaml file:

  • Gather all the information to provide a plug-in definition that the Che server knows.

  • Perform preparation actions in the workspace namespace (download, unpack files, process configuration).

The main goal of the plug-in broker is to decouple the Che plug-ins definitions from the actual plug-ins that Che can support. With brokers, Che can support different plug-ins without updating the Che server.

The Che server starts the plug-in broker. The plug-in broker runs in the same Kubernetes namespace as the workspace. It has access to the plug-ins and project persistent volumes.

A plug-ins broker is defined as a container image (for example, eclipse/che-plugin-broker). The plug-in type determines the type of the broker that is started. Two types of plug-ins are supported: Che Plugin and Che Editor.

Source code

Che Plug-in broker

Container image

quay.io/eclipse/che-plugin-artifacts-broker
eclipse/che-plugin-metadata-broker

Che workspace configuration

This section describes the properties of the Che server that affect the provisioning of a Che workspace.

Storage strategies for che workspaces

Workspace Pods use Persistent Volume Claims (PVCs), which are bound to the physical Persistent Volumes (PVs) with ReadWriteOnce access mode. It is possible to configure how the Che server uses PVCs for workspaces. The individual methods for this configuration are called PVC strategies:

Strategy Details Pros Cons

common (default)

One PVC for all workspaces in one Kubernetes namespace

Easy to manage and control storage

If PV does not support ReadWriteMany (RWX) access mode, the configuration will work only if one or both of the following apply:

* Workspaces are in separate Kubernetes namespaces

* Only one workspace is running per Kubernetes namespace at the same time. See Configuring namespace strategies.

per-workspace

One PVC for one workspace

Easier to manage and control storage compared to unique strategy

PV count is not known and depends on workspaces number

unique

One PVC per workspace volume or user-defined PVC

Storage isolation

An undefined number of PVs is required

Eclipse Che uses the common PVC strategy in combination with the "one namespace per user" namespace strategy when all Che workspaces operate in the user’s namespace, sharing one PVC.

The common PVC strategy

All workspaces inside a Kubernetes-native namespace use the same Persistent Volume Claim (PVC) as the default data storage when storing data such as the following in their declared volumes:

  • projects

  • workspace logs

  • additional Volumes defined by a use

When the common PVC strategy is in use, user-defined PVCs are ignored and volumes that refer to these user-defined PVCs are replaced with a volume that refers to the common PVC. In this strategy, all Che workspaces use the same PVC. When the user runs one workspace, it only binds to one node in the cluster at a time.

The corresponding containers volume mounts link to a common volume, and sub-paths are prefixed with <workspace-ID> or <original-PVC-name>. For more details, see How subpaths are used in PVCs.

The Che Volume name is identical to the name of the user-defined PVC. It means that if a machine is configured to use a Che volume with the same name as the user-defined PVC has, they will use the same shared folder in the common PVC.

When a workspace is deleted, a corresponding subdirectory (${ws-id}) is deleted in the PV directory.

Restrictions on using the common PVC strategy

When the common strategy is used and a workspace PVC access mode is ReadWriteOnce (RWO), only one node can simultaneously use the PVC.

If there are several nodes, you can use the common strategy, but:

  • The workspace PVC access mode must be reconfigured to ReadWriteMany (RWM), so multiple nodes can use this PVC simultaneously.

  • Only one workspace in the same namespace may be running. See Configuring namespace strategies.

The common PVC strategy is not suitable for large multi-node clusters. Therefore, it is best to use it in single-node clusters. However, in combination with per-workspace namespace strategy, the common PVC strategy is usable for clusters with around 75 nodes. The PVC used in this strategy must be large enough to accommodate all projects since there is a risk of the event, in which one project depletes the resources of others.

The per-workspace PVC strategy

The per-workspace strategy is similar to the common PVC strategy. The only difference is that all workspace Volumes, but not all the workspaces, use the same PVC as the default data storage for:

  • projects

  • workspace logs

  • additional Volumes defined by a user

It’s a strategy when Che keeps its workspace data in assigned PVs that are allocated by a single PVC.

The per-workspace PVC strategy is the most universal strategy out of the PVC strategies available and acts as a proper option for large multi-node clusters with a higher amount of users. Using the per-workspace PVC strategy, users can run multiple workspaces simultaneously, results in more PVCs being created.

The unique PVC strategy

When using the `unique `PVC strategy, every Che Volume of a workspace has its own PVC. This means that workspace PVCs are:

Created when a workspace starts for the first time. Deleted when a corresponding workspace is deleted.

User-defined PVCs are created with the following specifics:

  • They are provisioned with generated names to prevent naming conflicts with other PVCs in a namespace.

  • Subpaths of the mounted Physical persistent volumes that reference user-defined PVCs are prefixed with <workspace-ID> or <PVC-name>. This ensures that the same PV data structure is set up with different PVC strategies. For details, see How subpaths are used in PVCs.

The unique PVC strategy is suitable for larger multi-node clusters with a lesser amount of users. Since this strategy operates with separate PVCs for each volume in a workspace, vastly more PVCs are created.

How subpaths are used in PVCs

Subpaths illustrate the folder hierarchy in the Persistent Volumes (PV).

/pv0001
  /workspaceID1
  /workspaceID2
  /workspaceIDn
    /che-logs
    /projects
    /<volume1>
    /<volume2>
    /<User-defined PVC name 1 | volume 3>
    ...

When a user defines volumes for components in the devfile, all components that define the volume of the same name will be backed by the same directory in the PV as <PV-name>, <workspace-ID>, or `<original-PVC-name>. Each component can have this location mounted on a different path in its containers.

Example

Using the common PVC strategy, user-defined PVCs are replaced with subpaths on the common PVC. When the user references a volume as my-volume, it is mounted in the common-pvc with the /workspace-id/my-volume subpath.

Configuring a Che workspace with a persistent volume strategy

A persistent volume (PV) acts as a virtual storage instance that adds a volume to a cluster.

A persistent volume claim (PVC) is a request to provision persistent storage of a specific type and configuration, available in the following Che storage configuration strategies:

  • Common

  • Per-workspace

  • Unique

The mounted PVC is displayed as a folder in a container file system.

Configuring a PVC strategy using the Helm chart

The following section describes how to configure workspace persistent volume claim (PVC) strategies of a Che server using the Helm chart.

It is not recommended to reconfigure PVC strategies on an existing Che cluster with existing workspaces. Doing so causes data loss.
Prerequisites
Procedure

When deploying Che using Helm Chart, configure the workspace PVC strategy by setting values for the global.pvcStrategy option.

  • For a new installation, use the helm install command with the global.pvcStrategy option:

    $ helm install --set global.pvcStrategy=<per-workspace>

  • For an already installed instance, use the helm upgrade command with the global.pvcStrategy option:

    $ helm upgrade --set global.pvcStrategy=<per-workspace>

Depending on the strategy used, replace the <per-workspace> option in the above examples with unique or common.

Configuring a PVC strategy strategy by editing a configMap

Based on the Che installation method, configMaps can be used to customize the working environment. A configMap is provided as an editable file that lists options to customize the Che environment. This method of configuring a persistent volume claim (PVC) strategy for a Che workspace is available only for the Helm installation.

Changes to a configMap created during Operator installation are not permanent because the Operator overwrites them back to default.

Prerequisites

  • The helm tool method was used to deploy Che.

  • The kubectl tool is available.

Procedure

  1. Set the configMap variable to reflect the requested PVC strategy:

    CHE_INFRA_KUBERNETES_PVC_STRATEGY=<per-workspace>

    Depending on the strategy used, replace the <per-workspace> option in the above example with unique or common.

  2. Restart Che by scaling the deployment to zero and then back to one again:

    $ oc scale --replicas=0 deployment {prod-deployment}
$ oc scale --replicas=1 deployment {prod-deployment}

  3. Restart the workspace for the changes to take effect.

Configuring a PVC strategy using the Operator

The following section describes how to configure workspace persistent volume claim (PVC) strategies of a Che server using the Operator.

It is not recommended to reconfigure PVC strategies on an existing Che cluster with existing workspaces. Doing so causes data loss.

Operators are software extensions to Kubernetes that use Custom Resources to manage applications and their components.

When deploying Che using the Operator, configure the intended strategy by modifying the spec.storage.pvcStrategy property of the CheCluster Custom Resource object YAML file.

Prerequisites

  • The kubectl tool is available.

Procedure

The following procedure steps are available for:

  • OpenShift command-line tool, oc

  • Kubernetes clusters controlling command-line tool, kubectl

To do changes to the CheCluster YAML file, choose one of the following:

  • Create a new cluster by executing the kubectl apply command. For example:

    $ kubectl apply -f <my-cluster.yaml>

  • Update the YAML file properties of an already running cluster by executing the kubectl patch command. For example:

    $ kubectl patch checluster eclipse-che --type=json \
  -p '[{"op": "replace", "path": "/spec/storage/pvcStrategy", "value": "<per-workspace>"}]'

Depending on the strategy used, replace the <per-workspace> option in the above example with unique or common.

Workspace namespaces configuration

The Kubernetes namespace where a new workspace Pod is deployed depends on the Che server configuration. By default, every workspace is deployed in a distinct namespace, but the user can configure the Che server to deploy all workspaces in one specific namespace. The name of a namespace must be provided as a Che server configuration property and cannot be changed at runtime.

Che workspace creation flow

che workspace creation flow

The following is a Che workspace creation flow:

  1. A user starts a Che workspace defined by:

    • An editor (the default is Che-Theia)

    • A list of plug-ins (for example, Java and Kubernetes tools)

    • A list of runtime applications

  2. Che server retrieves the editor and plug-in metadata from the plug-in registry.

  3. For every plug-in type, Che server starts a specific plug-in broker.

  4. The Che plug-ins broker transforms the plug-in metadata into a Che Plugin definition. It executes the following steps:

    1. Downloads a plug-in and extracts its content.

    2. Processes the plug-in meta.yaml file and sends it back to Che server in the format of a Che Plugin.

  5. Che server starts the editor and the plug-in sidecars.

  6. The editor loads the plug-ins from the plug-in persistent volume.