Re: [che-dev] Single Host on OpenShift

[Lukas Krejci <lkrejci@xxxxxxxxxx>]

Hi all,

I am following up on the topic of enabling single-host on OpenShift.

We have concluded the performance tests and I would like to present to you the 
results that we have found.

tl;dr There is no clear winning solution.

In our testing we concentrated on 3 areas. The performance of routing of the 
HTTP traffic, performance of Websocket communication and correct handling of 
cookies under path rewriting.

We were trying to choose between 3 candidates for the HTTP gateway that we 
identified in the prior POCs:

* HAProxy
* Nginx
* Traefik

Unfortunately, none of them came out of our testing with flying colors.

Generally, HTTP and websockets have somewhat unsurprisingly very similar 
performance profile in each of the solutions so I won't be discussing them 
separately.

== HAProxy
Pros: 
* fast and hardware-efficient even under high load
Cons:
* Some issues with live reconfiguration
** The slowest to establish a new route within the gateway
** Rare routing errors

== Nginx
Pros:
* fast and hardware-efficient under moderate load
* Stable under live reconfiguration
Cons:
* "Flappy" performance under high load - high variance in response times
* Rare routing errors under high load

== Traefik
Pros:
* Performant
* Best support for live reconfiguration
* Support for OAuth and other "modern" features that we could take advantage 
of in the future
Cons:
* BLOCKER - incorrect handling of cookies defined on a specific path. Such 
cookie paths are not rewritten along with requests. This is essentially a 
security issue because it would enable auth cookie overwriting/stealing.
* Higher hardware requirements (especially CPU under higher load)

Our current favorite is Traefik despite the blocking issue of incorrect cookie 
handling. We think it might be worth trying to fix that and get a solution 
that seems to be the most stable of the 3. If fixing Traefik proves too 
difficult, our second choice would probably be nginx but that would require 
further testing.

We will present our findings with all the fancy graphs and discussion on the 
next community call.

We have now concluded our performance testing though and are moving forward 
with the actual implementation (and will soon pick the gateway solution).

We'd appreciate your feedback and advice on any of the above detailed pros or 
cons.

You can check our progress on this epic at
https://github.com/eclipse/che/issues/12914.

Thanks,

Lukas