Creating A Telemetry Plugin

This section shows how to create an AnalyticsManager class that extends AbstractAnalyticsManager and implements the following methods:

isEnabled() - determines whether or not the telemetry back-end is functioning correctly. This could mean always returning true, or have more complex checks, for example, returning false when a connection property is missing.
destroy() - cleanup method that is run before shutting down the telemetry back-end. This method sends the WORKSPACE_STOPPED event.
onActivity() - notifies that some activity is still happening for a given user. This is mainly used to send WORKSPACE_INACTIVE events.
onEvent() - submits telemetry events to the telemetry server, such as WORKSPACE_USED or WORKSPACE_STARTED.
increaseDuration() - increases the duration of a current event rather than sending multiple events in a small frame of time.

Getting Started

This document describes the steps required to extend the Che telemetry system to connect to a custom back-end:

Creating a server process that receives events
Extending Che libraries to create a back-end that send events to the server
Packaging the telemetry back-end in a container and deploying it to an image registry
Adding a plug-in for your back-end and instructing Che to load the plug-in in your workspaces

Optional: creating a server that receives events

This example shows how to create a server that receives events from Che and writes them to standard output.

For production use cases, consider integrating with a third-party telemetry system (for example, Segment, Woopra) rather than creating your own telemetry server. In this case, use your provider’s APIs to send events from your custom back-end to their system.

The following Go code starts a server on port 8080 and writes events to standard output:

main.go

package main

import (
	"io/ioutil"
	"net/http"

	"go.uber.org/zap"
)

var logger *zap.SugaredLogger

func event(w http.ResponseWriter, req *http.Request) {
	switch req.Method {
	case "GET":
		logger.Info("GET /event")
	case "POST":
		logger.Info("POST /event")
	}
	body, err := req.GetBody()
	if err != nil {
		logger.With("err", err).Info("error getting body")
		return
	}
	responseBody, err := ioutil.ReadAll(body)
	if err != nil {
		logger.With("error", err).Info("error reading response body")
		return
	}
	logger.With("body", string(responseBody)).Info("got event")
}

func activity(w http.ResponseWriter, req *http.Request) {
	switch req.Method {
	case "GET":
		logger.Info("GET /activity, doing nothing")
	case "POST":
		logger.Info("POST /activity")
		body, err := req.GetBody()
		if err != nil {
			logger.With("error", err).Info("error getting body")
			return
		}
		responseBody, err := ioutil.ReadAll(body)
		if err != nil {
			logger.With("error", err).Info("error reading response body")
			return
		}
		logger.With("body", string(responseBody)).Info("got activity")
	}
}

func main() {

	log, _ := zap.NewProduction()
	logger = log.Sugar()

	http.HandleFunc("/event", event)
	http.HandleFunc("/activity", activity)
	logger.Info("Added Handlers")

	logger.Info("Starting to serve")
	http.ListenAndServe(":8080", nil)
}

Create a container image based on this code and expose it as a deployment in OpenShift in the che namespace. The code for the example telemetry server is available at che-workspace-telemetry-example. To deploy the telemetry server, clone the repository and build the container:

$ git clone https://github.com/che-incubator/che-workspace-telemetry-example
$ cd che-workspace-telemetry-example
$ docker build -t registry/organization/che-workspace-telemetry-example:latest
$ docker push registry/organization/che-workspace-telemetry-example:latest

In manifest.yaml, replace the image and host fields to match the image you pushed, and the public hostname of your Kubernetes or OpenShift cluster. Then run:

$ kubectl apply -f manifest.yaml -n <namespace to deploy>

Creating a new Maven project

For fast feedback when developing, it is recommended to do development inside a Che workspace. This way, you can run the application in a cluster and connect to the workspaces front-end telemetry plug-in to send events to your custom back-end.

Create a new Maven Quarkus project scaffolding:

$ mvn io.quarkus:quarkus-maven-plugin:1.2.1.Final:create \
  -DprojectGroupId=mygroup -DprojectArtifactId=telemetryback-end \
  -DprojectVersion=my-version -DclassName="org.my.group.MyResource"

Add a dependency to org.eclipse.che.incubator.workspace-telemetry.back-end-base in your pom.xml:

pom.xml

<dependency>
    <groupId>org.eclipse.che.incubator.workspace-telemetry</groupId>
    <artifactId>backend-base</artifactId>
    <version>0.0.11</version>
</dependency>
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.12</version>
</dependency>

Add the Apache HTTP components library to send HTTP requests.
Consult the GitHub packages for the latest version and Maven coordinates of back-end-base. GitHub packages require a personal access token with read:packages permissions to download the Che telemetry libraries. Create a personal access token and copy the token value.

Create a settings.xml file in the repository root and add the coordinates and token to the che-incubator packages:

settings.xml

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
http://maven.apache.org/xsd/settings-1.0.0.xsd">
   <servers>
      <server>
         <id>che-incubator</id>
         <username>${env.GITHUB_USERNAME}</username>
         <password>${env.GITHUB_TOKEN}</password>
      </server>
   </servers>

   <profiles>
      <profile>
         <id>github</id>
         <activation>
            <activeByDefault>true</activeByDefault>
         </activation>
         <repositories>
            <repository>
               <id>central</id>
               <url>https://repo1.maven.org/maven2</url>
                <releases><enabled>true</enabled></releases>
                <snapshots><enabled>false</enabled></snapshots>
                </repository>
                <repository>
                <id>che-incubator</id>
                <name>GitHub navikt Apache Maven Packages</name>
                <url>https://maven.pkg.github.com/che-incubator/che-workspace-telemetry-client</url>
                </repository>
            </repositories>
        </profile>
    </profiles>
    </settings>

This file is used when packaging the application in a container. When running locally, add the information to your personal settings.xml file.

Running the application

Run and test the application is in a Che workspace:

$ mvn quarkus:dev -Dquarkus.http.port=${CHE_WORKSPACE_TELEMETRY_BACKEND_PORT}

If Che is secured using a self-signed certificate, add the certificate to a trust store and mount it into the workspace. Also add the Java system property, -Djavax.net.ssl.trustStore=/path/to/trustStore, to the mvn command. For example, assuming the trust store is located in $JAVA_HOME/jre/lib/security/cacerts:

$ keytool -import -alias self-signed-certificate \
  -file <path/to/self-signed-certificate> -keystore $JAVA_HOME/jre/lib/security/cacerts

Followed by:

$ mvn quarkus:dev -Dquarkus.http.port=${CHE_WORKSPACE_TELEMETRY_BACKEND_PORT} \
  -Djavax.net.ssl.trustStore=$JAVA_HOME/jre/lib/security/cacerts

Creating a concrete implementation of AnalyticsManager and adding specialized logic

Create two new files in your project:

AnalyticsManager.java contains the logic specific to our telemetry system.
MainConfiguration.java is the main entrypoint that creates an instance of AnalyticsManager and starts listening for events.

AnalyticsManager.java

package org.my.group;

import java.util.Map;

import org.eclipse.che.api.core.rest.HttpJsonRequestFactory;
import org.eclipse.che.incubator.workspace.telemetry.base.AbstractAnalyticsManager;
import org.eclipse.che.incubator.workspace.telemetry.base.AnalyticsEvent;

public class AnalyticsManager extends AbstractAnalyticsManager {

    public AnalyticsManager(String apiEndpoint, String workspaceId, String machineToken,
            HttpJsonRequestFactory requestFactory) {
        super(apiEndpoint, workspaceId, machineToken, requestFactory);
    }

    @Override
    public boolean isEnabled() {
        // TODO Auto-generated method stub
        return true;
    }

    @Override
    public void destroy() {
        // TODO Auto-generated method stub
    }

    @Override
    public void onEvent(AnalyticsEvent event, String ownerId, String ip, String userAgent, String resolution,
            Map<String, Object> properties) {
        // TODO Auto-generated method stub
    }

    @Override
    public void increaseDuration(AnalyticsEvent event, Map<String, Object> properties) {
        // TODO Auto-generated method stub
    }

    @Override
    public void onActivity() {
        // TODO Auto-generated method stub
    }
}

MainConfiguration.java

package org.my.group;

import javax.enterprise.context.Dependent;
import javax.enterprise.inject.Produces;

import org.eclipse.che.incubator.workspace.telemetry.base.AbstractAnalyticsManager;
import org.eclipse.che.incubator.workspace.telemetry.base.BaseConfiguration;

@Dependent
public class MainConfiguration extends BaseConfiguration {
    @Produces
    public AbstractAnalyticsManager analyticsManager() {
      return new AnalyticsManager(apiEndpoint, workspaceId, machineToken, requestFactory());

    }
}

Implementing `isEnabled()`

For the purposes of the example, this method just returns true whenever it is called. Whenever the server is running, it is enabled and operational.

AnalyticsManager.java

@Override
public boolean isEnabled() {
    return true;
}

It is possible to put more a complex login in isEnabled(). For example, the service should not be considered operational in certain cases. The hosted Che woopra back-end checks that a configuration property exists before determining if the back-end is enabled.

Implementing `onEvent()`

onEvent() sends the event passed to the back-end to the telemetry system. For the example application, it sends an HTTP POST payload to our server. The example telemetry server application is deployed to OpenShift at the following URL: http://little-telemetry-back-end-che.apps-crc.testing.

AnalyticsManager.java

@Override
public void onEvent(AnalyticsEvent event, String ownerId, String ip, String userAgent, String resolution, Map<String, Object> properties) {
    HttpClient httpClient = HttpClients.createDefault();
    HttpPost httpPost = new HttpPost("http://little-telemetry-backend-che.apps-crc.testing/event");
    HashMap<String, Object> eventPayload = new HashMap<String, Object>(properties);
    eventPayload.put("event", event);
    StringEntity requestEntity = new StringEntity(new JsonObject(eventPayload).toString(),
            ContentType.APPLICATION_JSON);
    httpPost.setEntity(requestEntity);
    try {
        HttpResponse response = httpClient.execute(httpPost);
    } catch (IOException e) {
        e.printStackTrace();
    }
}

This sends an HTTP request to the telemetry server and automatically debounces identical events in a small time period (the default is 1500 milliseconds, and it can be changed by subclasses.

Implementing `increaseDuration()`

Many telemetry systems recognize event duration. The AbstractAnalyticsManager merges similar events that happen in the same frame of time into one event, so that you do not get several identical events sent to the server in a small frame of time. This implementation of increaseDuration() is a no-op. This method uses the APIs of your telemetry provider to alter the event or event properties to reflect an increased duration of the event.

AnalyticsManager.java

@Override
public void increaseDuration(AnalyticsEvent event, Map<String, Object> properties) {}

Implementing `onActivity()`

Set an inactive timeout limit, and use onActivity() to send a WORKSPACE_INACTIVE event if the last event time is longer than the inactivity timeout.

AnalyticsManager.java

public class AnalyticsManager extends AbstractAnalyticsManager {

    ...

    private long inactiveTimeLimt = 60000 * 3;

    ...


    @Override
    public void onActivity() {
        if (System.currentTimeMillis() - lastEventTime >= inactiveTimeLimt) {
            onEvent(WORKSPACE_INACTIVE, lastOwnerId, lastIp, lastUserAgent, lastResolution, commonProperties);
        }
    }

Implementing `destroy()`

When destroy() is called, send a WORKSPACE_STOPPED event and shutdown any resources, such as connection pools.

AnalyticsManager.java

@Override
public void destroy() {
    onEvent(WORKSPACE_STOPPED, lastOwnerId, lastIp, lastUserAgent, lastResolution, commonProperties);
}

Now when you run mvn quarkus:dev as described in Running the application, you should see a WORKSPACE_STOPPED event sent to the server when you kill the Quarkus application.

Packaging the Quarkus application

See the quarkus documentation for the best instructions to package the application in a container. Build and push the container to a container registry of your choice.

Creating a `meta.yaml` for your plug-in.

Create a meta.yaml definition representing a Che plug-in that runs your custom back-end in a workspace Pod. For more information on meta.yaml, see What is a Che-Theia plug-in.

meta.yaml

apiVersion: v2
publisher: demo-publisher
name: little-telemetry-backend
version: 0.0.1
type: Che Plugin
displayName: Little Telemetry Backend
description: A Demo telemetry backend
title: Little Telemetry Backend
category: Other
spec:
  workspaceEnv:
    - name: CHE_WORKSPACE_TELEMETRY_BACKEND_PORT
      value: '4167'
  containers:
  - name: YOUR BACKEND NAME
    image: YOUR IMAGE NAME
    env:
      - name: CHE_API
        value: $(CHE_API_INTERNAL)

Ordinarily, you would deploy this file to a corporate web server. For this guide, we create an Apache web server on OpenShift and host the plug-in there. Create a configMap referencing your new meta.yaml file.

$ oc create configmap --from-file=meta.yaml -n che telemetry-plugin-meta

Then create a deployment, a service, and a route to expose the web server. The deployment references this configMap and places it in the /var/www/html directory.

manifests.yaml

kind: Deployment
apiVersion: apps/v1
metadata:
  name: apache
  namespace: <che>
spec:
  replicas: 1
  selector:
    matchLabels:
      app: apache
  template:
    metadata:
      labels:
        app: apache
    spec:
      volumes:
        - name: plugin-meta-yaml
          configMap:
            name: telemetry-plugin-meta
            defaultMode: 420
      containers:
        - name: apache
          image: 'registry.redhat.io/rhscl/httpd-24-rhel7:latest'
          ports:
            - containerPort: 8080
              protocol: TCP
          resources: {}
          volumeMounts:
            - name: plugin-meta-yaml
              mountPath: /var/www/html
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 25%
      maxSurge: 25%
  revisionHistoryLimit: 10
  progressDeadlineSeconds: 600
---
kind: Service
apiVersion: v1
metadata:
  name: apache
  namespace: <che>
spec:
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080
  selector:
    app: apache
  type: ClusterIP
---
kind: Route
apiVersion: route.openshift.io/v1
metadata:
  name: apache
  namespace: <che>
spec:
  host: apache-che.apps-crc.testing
  to:
    kind: Service
    name: apache
    weight: 100
  port:
    targetPort: 8080
  wildcardPolicy: None

$ oc apply -f manifests.yaml

Wait a few minutes for the image to pull and the deployment to start, and then confirm that meta.yaml is available in the web server:

$ curl apache-che.apps-crc.testing/meta.yaml

This command should return the meta.yaml file.

Updating Che to reference your telemetry plug-in

Update the CheCluster Custom Resource, and add the CHE_WORKSPACE_DEVFILE_DEFAULT__EDITOR_PLUGINS environment variable to spec.server.customCheProperties. The value of the environment variable should be the URL of the location of the meta.yaml file on your web server. This can be accomplished by running oc edit checluster -n che and typing in the change at the terminal, or by editing the CR in the OpenShift console (Installed Operators → Eclipse Che → Eclipse Che Cluster → eclipse-che → YAML).

apiVersion: org.eclipse.che/v1
kind: CheCluster
metadata:
  creationTimestamp: '2020-05-14T13:21:51Z'
  finalizers:
    - oauthclients.finalizers.che.eclipse.org
  generation: 18
  name: eclipse-che
  namespace: <che>
  resourceVersion: '5108404'
  selfLink: /apis/org.eclipse.che/v1/namespaces/che/checlusters/eclipse-che
  uid: bae08db2-104d-4e44-a001-c9affc07528d
spec:
  auth:
    identityProviderURL: 'https://keycloak-che.apps-crc.testing'
    identityProviderRealm: che
    updateAdminPassword: false
    oAuthSecret: ZMmNPRbgOJJQ
    oAuthClientName: eclipse-che-openshift-identity-provider-yrlcxs
    identityProviderClientId: che-public
    identityProviderPostgresSecret: che-identity-postgres-secret
    externalIdentityProvider: false
    identityProviderSecret: che-identity-secret
    openShiftoAuth: true
  database:
    chePostgresDb: dbche
    chePostgresHostName: postgres
    chePostgresPort: '5432'
    chePostgresSecret: che-postgres-secret
    externalDb: false
  k8s: {}
  metrics:
    enable: false
  server:
    cheLogLevel: INFO
    customCheProperties:
      CHE_WORKSPACE_DEVFILE_DEFAULT__EDITOR_PLUGINS: 'http://apache-che.apps-crc.testing/meta.yaml'
    externalDevfileRegistry: false
    cheHost: che-che.apps-crc.testing
    selfSignedCert: true
    cheDebug: 'false'
    tlsSupport: true
    allowUserDefinedWorkspaceNamespaces: false
    externalPluginRegistry: false
    gitSelfSignedCert: false
    cheFlavor: che
  storage:
    preCreateSubPaths: true
    pvcClaimSize: 1Gi
    pvcStrategy: per-workspace
status:
  devfileRegistryURL: 'https://devfile-registry-che.apps-crc.testing'
  keycloakProvisioned: true
  cheClusterRunning: Available
  cheURL: 'https://che-che.apps-crc.testing'
  openShiftoAuthProvisioned: true
  dbProvisioned: true
  cheVersion: 7.13.1
  keycloakURL: 'https://keycloak-che.apps-crc.testing'
  pluginRegistryURL: 'https://plugin-registry-che.apps-crc.testing/v3'

Wait for the Che server to restart, and create a new workspace. You should see a new message saying that your plug-in is being installed into the workspace.

Perform a some operations in the workspace you started. You should see those events in the logs of the example telemetry server.

Summary

In this guide, you:

Created a telemetry server to echo events to standard out.
Extended the Che telemetry client and implemented your own custom back-end.
Created a meta.yaml file to represent a Che workspace plug-in for your custom back-end.
Told Che where to find your custom plug-in by changing the CHE_WORKSPACE_DEVFILE_DEFAULT__EDITOR_PLUGINS environment variable.

Creating A Telemetry Plugin

Getting Started

Optional: creating a server that receives events

Creating a new Maven project

Running the application

Creating a concrete implementation of AnalyticsManager and adding specialized logic

Implementing isEnabled()

Implementing onEvent()

Implementing increaseDuration()

Implementing onActivity()

Implementing destroy()

Packaging the Quarkus application

Creating a meta.yaml for your plug-in.

Updating Che to reference your telemetry plug-in

Summary

Implementing `isEnabled()`

Implementing `onEvent()`

Implementing `increaseDuration()`

Implementing `onActivity()`

Implementing `destroy()`

Creating a `meta.yaml` for your plug-in.