Hyperfoil
Previous: Beginner's Guide to Hyperfoil: part 2

Beginner's Guide to Hyperfoil: part 3

In this article we’ll show how to run Hyperfoil inside an Openshift cluster, benchmarking workload within the same cluster. This assumes that you have sufficient privileges to install operators and create new namespaces (projects).

Demo workload

We will install Vehicle Market, our demo application we’ve used in the first article using console and Openshift Template.

Login into your cluster and run this:

# Fetch the default domain your cluster uses
export DOMAIN=$(oc get ingresscontrollers.operator.openshift.io \
    -n openshift-ingress-operator default -o jsonpath='{ .status.domain }')
oc new-project vehicle-market
curl -Ls http://vehicle-market-template.hyperfoil.io | \
    oc process -f - -p DOMAIN=vehicle-market.$DOMAIN | oc apply -f -
echo http://vehicle-market.$DOMAIN

This will deploy 5 pods, 5 services and 4 routes in the vehicle-market namespace, and load some example data. You can open the URL printed as the last line in your browser and explore the application.

Installing Hyperfoil

You can install Hyperfoil Operator either through web-console, or on command-line using oc. For the web-console installation switch to the ‘Administrator view’ and go to Operators/OperatorHub. Filter ‘Hyperfoil’ and you should see the operator:

Click on Hyperfoil and go through the wizard with all settings on default values (install to all namespaces with automatic approval).

Alternatively you can create the subscription by defining this subscription:

apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: hyperfoil-operator
  namespace: openshift-operators
spec:
  channel: alpha
  name: hyperfoil-bundle
  source: community-operators
  sourceNamespace: openshift-marketplace

In any case you should end up with Hyperfoil listed amongst the installed operators:

Now you can create a new namespace for Hyperfoil and add the custom resource. In web-console open the dropdown on top and create a new namespace hyperfoil. When the page reloads with all operators (that can take a couple of seconds) click on ‘Hyperfoil’ in the Provided APIs column (highlighted with red ellipse above). Press the ‘Create Hyperfoil’ button on the right side and switch to YAML View, filling the definition below and push ‘Create’ button below the editor.

apiVersion: hyperfoil.io/v1alpha2
kind: Hyperfoil
metadata:
  name: hyperfoil
spec:
  route:
    host: hyperfoil.apps.your.domain.com

The only non-default value in this example is the hostname for the route; default hostname would be probably a bit longer. By default the route uses edge termination (the TLS connection is terminated at the router, the communication inside cluster is not encrypted). Therefore we will be connecting to port 443.

On command line just use commands below, passing the CR above as hyperfoil.yaml.

$ oc new-project hyperfoil
$ oc apply -f hyperfoil.yaml
$ oc get hf
NAME        VERSION   ROUTE                            PVC   STATUS
hyperfoil             hyperfoil.apps.your.domain.com         Ready

Using Hyperfoil

Now it’s time to open CLI as we’ve done in the previous articles:

export VMB=$HOME/vehicle-market/benchmarks
podman run -it --rm -v $VMB:/benchmarks:Z -v /tmp/reports:/tmp/reports:Z \
    --network=host quay.io/hyperfoil/hyperfoil cli



[hyperfoil]$ connect hyperfoil.apps.your.domain.com:443
ERROR: javax.net.ssl.SSLHandshakeException: Failed to create SSL connection: Failed to create SSL connection: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target: unable to find valid certification path to requested target
Hint: TLS certificate verification might have failed. Use --insecure to disable validation.
Failed connecting to hyperfoil.apps.your.domain.com:443

We are connecting to the route on port 443 (you could also use connect https://hyperfoil.apps.your.domain.com or connect hyperfoi.apps.your.domain.com -p 443) but the certificate issued by the router might not be valid if this is not a production cluster. Use --insecure (or -k) in that case:


[hyperfoil]$ connect hyperfoil.apps.ocp.scalelab:443 -k
WARNING: Hyperfoil TLS certificate validity is not checked. Your credentials might get compromised.
Connected!
[hyperfoil@hyperfoil]$ upload /benchmarks/first-benchmark.hf.yaml
Loaded benchmark first-benchmark, uploading...
... done.
[hyperfoil@hyperfoil]$ run
ERROR: Server responded with unexpected code: 403, Forbidden:
Server is started in clustered mode; benchmarks must define agents.
Failed to start benchmark first-benchmark

Oops, what happened now? We have tried to execute the benchmark from Part 1 that worked just fine when executed from start-local. The controller in Openshift is started in a clustered mode, though, and single-vm runs are prohibited. We need to tell Hyperfoil to run this benchmark using an agent - and the controller will spin up another pod that will fire the load.

name: clustered-benchmark
agents:
  agent-one:
  agent-two:
http:
  host: http://vehicle-market.apps.your.domain.com
  sharedConnections: 10
usersPerSec: 10
duration: 120s
scenario:
- fetchIndex:
  - httpRequest:
      GET: /

Besides adding the agents (with empty definition) we have corrected the endpoint hostname and increased duration to 2 minutes. While the benchmark is running you can verify that controller created another pod:

$ oc get pods -n hyperfoil
NAME                   READY   STATUS    RESTARTS   AGE
agent-0001-agent-one   1/1     Running   0          13s
agent-0001-agent-two   1/1     Running   0          13s
hyperfoil-controller   1/1     Running   0          17m

Both the sharedConnections and usersPerSec have been evenly distributed between the two agents; each will use 5 connections and run 5 users per second.

See the agents section documentation for further customization of the agents.

Configuration options such as securing the controller with password, persistence, logging, hooks etc. can be found in the installation guide.

Tips and tricks

http://vehicle-market.apps.your.domain.com is probably resolved to the load balancer that routes the traffic even before Openshift Ingresses. You might want to bypass this component, hitting the Ingresses directly. You can do that by setting addresses for the endpoints manually:

$ oc get po -n openshift-ingress -o wide
NAME                              READY   STATUS    RESTARTS   AGE   IP               NODE        NOMINATED NODE   READINESS GATES
router-default-56b959876c-94x2f   1/1     Running   1          60d   192.168.222.19   worker006   <none>           <none>
router-default-56b959876c-9tqvb   1/1     Running   0          60d   192.168.222.22   worker009   <none>           <none>
http:
  host: http://vehicle-market.apps.your.domain.com
  sharedConnections: 10
  addresses:
  - 192.168.222.19
  - 192.168.222.22

The connections will be randomly assigned to one of those IPs. You can also use bypass ingress completely and target the service directly - addresses can use hostnames and customize the actual port as well:

http:
  host: http://vehicle-market.apps.your.domain.com
  sharedConnections: 10
  addresses:
  - frontend.vehicle-market.svc.cluster.local:8080
Previous: Beginner's Guide to Hyperfoil: part 2