There’s plenty of web benchmark tools around and it might be hard to pick one, and investing time into not-established tools is risky. Hyperfoil was not created for the pure joy of coding but as a solution to set of problems which could not be solved all by single other existing tool.

Free software

Free software allows you to take your benchmark and publish it for everyone to verify. With proprietary licenses that wouldn’t be so easy.

Hyperfoil is licensed under Apache License 2.0.

Distribution

Generating load with single node stops scaling at certain point and you need to orchestrate the benchmark across a cluster of nodes. Simple command-line tools usually ignore this completely (you’re supposed to start them, gather and merge the data in a bash scripts). Other tools use open-core model with the clustering part being a paid/proprietary option. There are frameworks that have clustering built in as well.

Hyperfoil uses leader-follower model with Vert.x Event bus as the clustering middleware. While running from single VM is possible (and quite easy) as well, the design is distributed by default.

Accuracy

The point of web benchmark is usually finding out what happens when your system is accessed by thousands of concurrent users - each doing a page load every few seconds. However many traditional load drivers simplify the scenario to few dozen of virtual users (VUs) that execute one request after another or with very short delays in between - this is referred to as the Closed System Model (as the set of VUs is finite). This discrepancy leads to problem known as coordinated omission and results in significantly skewed latency results and pathological conditions (queues overflow…) not being triggered.

Hyperfoil embraces Open System Model by default - virtual users are completely independent until it runs out of resources, recording that situation in consequence. Hyperfoil runs a state-machine for each VU and all requests are executed asynchronously.

Versatility

While you can design your benchmark to just hit single endpoint (URL) with desired rate this is likely not what the users would be doing. Randomizing some parts of the query or looping through a list of endpoint might be better but the resulting scenario might be still too simplified.

Hyperfoil introduces a DSL expressed in YAML with which you can sketch the scenario in a fine detail, including timing, concurrent requests, processing responses and so forth. We’re not trying to invent a new programming language, though, so if the DSL gets too complex you can write the logic in Java or any other JVM language.

If you’re eager to try out Hyperfoil go to the first quickstart. Otherwise let’s have a deeper look into the terms and concepts.

1.1 - Concepts

Hyperfoil key terms and concepts

This document explains some core terms used throughout the documentation.

Controller and agents

While it is possible to run benchmarks directly from CLI, in its nature Hyperfoil is a distributed tool with leader-follower architecture. Controller has the leader role; this is a Vert.x-based server with REST API. When a benchmark is started controller deploys agents (according to the benchmark definition), pushes the benchmark definition to these agents and orchestrates benchmark phases. Agents execute the benchmark, periodically sending statistics to the controller. This way the controller can combine and evaluate statistics from all agents on the fly. When the benchmark is completed all agents terminate.

All communication between the controller and agents happens over Vert.x eventbus - therefore it is independent on the deployment type.

Phases

Conceptually the benchmark consists of several phases. Phases can run independently of each other; these simulate certain load executed by a group of users (e.g. visitors vs. admins). Within one phase all users execute the same scenario (e.g. logging into the system, selling all their stock and then logging off).

Phases are also using for scaling the load during the benchmark; when looking for maximum throughput you schedule several iterations of given phase, gradually increasing the number of users that run the scenario.

A phase can be in one of these states:

not running (scheduled)
running
finished: users won’t start new scenarios but we’ll let already-started users complete the scenario
terminated: all users are done, all stats are collected and no further requests will be made

The state of phase on every agent is managed by Controller; this is also the finest grained unit of work it understands (controller has no information about state of each user).

Sessions

The state of each user’s scenario is saved in the session; sometimes we speak about (re)starting sessions instead of starting new users. Hyperfoil tries to keep allocations during benchmark as low as possible and therefore it pre-allocates all memory for the scenario execution ahead. This is why all resources the benchmark uses are capped - it needs to know the sizes of pools.

It is also necessary to know how many sessions we should preallocate - maximum concurrency of the system. If this threshold is exceeded Hyperfoil allocates further session as needed, but this is not the optimal mode of operation. It means that either you’ve underestimated the resources need or you’ve put a load on the system that it can’t handle anymore, requests are not being completed and scenarios are not finished - which means that session objects cannot be recycled for reuse by next user.

Scenario

Scenario consists of one or more sequences that are composed of steps. Steps are similar to statements in programming language and sequences are an equivalent of blocks of code.

While most of the time the scenario will consist of sequential operations as the user is not multi-tasking, the browser (or other system you’re simulating) actually executes some operations in parallel - e.g. during page load it loads images concurrently. Therefore at any time the session contains one or more active sequence instances; when all sequence instances are done, the session has finished and can be recycled for a new user. Most of the time the scenario will start with only one active instance and as it progresses, it might create instances of other sequences (e.g after evaluating a condition it creates a sequence instance according to the branching logic).

1.2 - FAQ

Frequently Asked Questions

Why is Hyperfoil written in Java?

People are often concerned about JVM performance or predictability. While nowadays JVM is very good in the sense of throughput, dealing with jitter can be challenging. We are Java engineers, though, and we believe that these issues can be mitigated with a right design. That’s why we try to be very careful on the execution hot-path.

We could achieve even better properties with C/C++, but the development effectivity would suffer. We could be succesful in Go, but we’re not as intimate with its internals. Other languages and frameworks would pose its own challenges. So far, the choice of Java was not found to be a limiting factor.

Why are you inventing your own YAML-based DSL instead of using Javascipt/Lua/…?

While some people might be more comfortable with describing their complex scenarios in a familiar language, running a script execution engine would have impact on performance and could put us out of control. We are not trying to invent a new language, written in YAML structure. Instead we propose a set of components common to many scenarios that could be recombined as it suits you. If you ever feel that the YAML is becoming cumbersome, try to move your complex benchmark logic to Java code and use it that way, instead.

I just want to know what is the maximum throughput!

Maximum throughput is a single number which makes comparison very easy. Was my code change for better or worse? However finding the sweet spot is not as simple as throwing in few hundred concurrent threads running one request after another and taking the readings. With too high concurrency you can get worse results due to contention and longer queues, so you need to try different concurrency levels anyway.

There’s nothing wrong with this type of test as long as you know what you’re doing, and that the response latencies might be far off reality. It’s actually very good test when you look only for regressions - and Hyperfoil supports that, too.

Hyperfoil is so hard to set up, I’ll just use …

Some tools can be run from the shell, with everything set just through options and arguments. That is quite handy for a quick test - and if the tool is sufficient for the job, use it. Or you can try runnning the same through Hyperfoil - e.g. for wrk/wrk2 we offer a facade (CLI command wrk or bin/wrk.sh) that creates a benchmark with the same behaviour but you also get all the detailed results as from any other run - see the migration guide. Once that your requirements outgrow what’s possible in these simple tools, you can embrace the full power of benchmark composition.

What does that ‘Exceeded session limit’ error mean?

With open-model phase types (constantRate, increasingRate, decreasingRate) the concurrency should not be limited. However as Hyperfoil tries not to allocate any memory during the benchmark we need to reserve space ahead for all sessions that could run concurrently - we call this the session limit. By default this limit is equal to number of users per second (assuming that the scenario won’t take more than 1 second).

When you get the ‘Exceeded session limit’ error this means that some of the requests took a long time (or you have delays as part of the scenario) and Hyperfoil ran out of session pool. In that case you can change the limit using maxSessions property on the phase to the expected maximum concurrency. E.g. if you expect that the scenario will take 3 seconds and you’re running at usersPerSec: 100 you should set maxSessions: 300 (or rather more to give it a buffer for unexpected jitter).

If increasing the limit doesn’t help it usually means that the load at the tested system is too high and the responses are not arriving as fast as you fire the requests. In that case you should lower the load.

2 - Getting Started

Collection of quickstarts aiming to make the Hyperfoil adoption a lot easier.

Embark on this journey with our collection of 8 quickstarts, guiding you through a wide range of potential use cases that you might encounter in daily operations.

Info

Do not hesitate to suggest additional use cases that you believe would be valuable for this section.

2.1 - First benchmark

Download, set up, and run your first Hyperfoil benchmark

1. Download latest release and unpack it

wget {{ site.last_release.url }} \
    && unzip {{ site.last_release.zip }} \
    && cd {{ site.last_release.dir }}

2. Start Hyperfoil in interactive mode (CLI)

bin/cli.sh

For our first benchmark we’ll start an embedded server (controller) within the CLI:

[hyperfoil]$ start-local
Starting controller in default directory (/tmp/hyperfoil)
Controller started, listening on 127.0.0.1:41621
Connecting to the controller...
Connected!

3. Upload the minimalistic benchmark and run it

As you can see below, the benchmark is really minimalistic as it is doing only single request to http://hyperfoil.io.

# This is the name of the benchmark. It's recommended to keep this in sync with
# name of this file, adding extension `.hf.yaml`.
name: single-request
# We must define at least one HTTP target, in this case it becomes a default
# for all HTTP requests.
http:
  host: http://hyperfoil.io
# Simulation consists of phases - potentially independent workloads.
# We'll discuss phases more in detail in next quickstarts.
phases:
# `example` is the name of the single phase in this benchmark.
- example:
    # `atOnce` with `users: 1` results in running the scenario below just once
    atOnce:
      users: 1
      scenario:
      # The only sequence in this scenario is called `test`.
      - test:
        # In the only step in this sequence we'll do a HTTP GET request
        # to `http://hyperfoil.io/`
        - httpRequest:
            GET: /
            # Inject helpers to make this request synchronous, i.e. keep
            # the sequence blocked until Hyperfoil processes the response.
            sync: true

Create the same benchmark in your local environment or download it. After that, upload it using the upload command as follow:

[hyperfoil@in-vm]$ upload .../single-request.hf.yaml
Loaded benchmark single-request, uploading...
... done.
[hyperfoil@in-vm]$ run single-request
Started run 0001
Run 0001, benchmark single-request
Agents: in-vm[STARTING]
Started: 2019/11/15 16:11:43.725    Terminated: 2019/11/15 16:11:43.899
<span class="hfcaption">NAME     STATUS      STARTED       REMAINING  COMPLETED     TOTAL DURATION               DESCRIPTION
example  TERMINATED  16:11:43.725             16:11:43.899  174 ms (exceeded by 174 ms)  1 users at once

4. Check out performance results:

[hyperfoil@in-vm]$ stats
Total stats from run 000A
<span class="hfcaption">PHASE    METRIC  REQUESTS  MEAN       p50        p90        p99        p99.9      p99.99     2xx  3xx  4xx  5xx  CACHE  TIMEOUTS  ERRORS  BLOCKED
example  test           1  172.49 ms  173.02 ms  173.02 ms  173.02 ms  173.02 ms  173.02 ms    0    1    0    0      0         0       0       0 ns

Doing one request is not much of a benchmark and the statistics above are moot, but hey, this is a quickstart.

In the future you might find editing with schema useful but at this point any editor with YAML syntax highlighting will do the job.

Ready? Let’s continue with something a bit more realistic…

2.2 - Steps and statistics

Learn how to create more steps and how to gather run statistics

In previous quickstart you created a benchmark that fires only one HTTP request. Our next example is going to hit random URLs at this server with 10 requests per second. We’ll see how to generate random data and collect statistics for different URLs.

Let’s start a container that will serve the requests:

podman run --rm -p 8080:8083 quay.io/hyperfoil/hyperfoil-examples

If you prefer running this in Docker just replace podman with docker. You can explore the handling of requests from this example on GitHub.

Here is the benchmark we’re going to run:

name: random-urls
http:
  host: http://localhost:8080
  sharedConnections: 10
# 10 users will be starting the scenario every second
usersPerSec: 10
duration: 5s
scenario:
- test:
  # Step `randomItem` randomly picks one item from the list below...
  - randomItem:
      list:
      - index.html
      - foo.png
      - bar.png
      - this-returns-404.png
      # ... and stores it in users's session under key `my-random-path`
      toVar: my-random-path
  - httpRequest:
      # HTTP request will read the variable from the session and format
      # the path for the GET request
      GET: /quickstarts/random-urls/${my-random-path}
      # We'll use different statistics metric for webpages and images
      metric:
      - .*\.html -> pages
      - .*\.png -> images
      - -> other
      # Handler processes the response
      handler:
        # We'll check that the response was successful (status 200-299)
        status:
          range: 2xx
        # When the response is fully processed we'll set variable `completed`
        # in the session.
        onCompletion:
          set: completed <- yes
      # For demonstration purposes we will set `sync: false`.
      # Next step is executed immediately after we fire the request, not
      # waiting for the response.
      sync: false
  # We'll wait for the `completed` var to be set in this step, though.
  - awaitVar: completed

So let’s run this through CLI:

[hyperfoil]$ start-local
...
[hyperfoil@in-vm]$ upload .../random-urls.hf.yaml
...
[hyperfoil@in-vm]$ run
Started run 0002
Run 0002, benchmark random-urls
Agents: in-vm[STARTING]
Started: 2019/11/15 17:49:45.859    Terminated: 2019/11/15 17:49:50.904
NAME  STATUS      STARTED       REMAINING  COMPLETED     TOTAL DURATION               DESCRIPTION
main  TERMINATED  17:49:45.859             17:49:50.903  5044 ms (exceeded by 44 ms)  10.00 users per second

[hyperfoil@in-vm]$ stats
Total stats from run 0002
PHASE  METRIC  REQUESTS  MEAN       p50        p90        p99        p99.9      p99.99     2xx  3xx  4xx  5xx  CACHE  TIMEOUTS  ERRORS  BLOCKED
main   images        34    3.25 ms    3.39 ms    4.39 ms   12.58 ms   12.58 ms   12.58 ms   12   13   12    0      0         0       0    1.11 ms
main   pages         13    2.89 ms    3.19 ms    4.15 ms    4.33 ms    4.33 ms    4.33 ms   13    0    0    0      0         0       0       0 ns

main/images: Progress was blocked waiting for a free connection. Hint: increase http.sharedConnections.

There are several things worth mentioning in this example:

The command run does not have any argument. In this case, the benchmark name random-urls is optional as you’ve just uploaded it and CLI knows that you are going to work with it. The same holds for stats - you don’t have to write down run ID 0002 when displaying statistics as the implicit run ID is set automatically in the run/status command.
The test did only 47 requests in 5 seconds, instead of 50. It does not execute one request every 100 ms sharp, it randomizes the times of requests as well; this simulates the Poisson point process. Longer runs would have lower variance in the total numbers.
In metric images the test reports 1.11 ms being blocked and there’s SLA failure below the stats. This is happening because in the default configuration Hyperfoil opens only one connection to the target server. All (possibly concurrent) requests have to share the common pool of 1 connection and if some request cannot be executed immediatelly we report this as blocked time. All practical benchmarks should increase the pool size to a value that reflects simulated load and prevent this situation.
The test took 44 ms longer than the configured 5 seconds. We terminate the test only after all responses for sent requests arrive (or time out).

In the next quickstart you’ll see a more complex scenario

2.3 - Complex workflow

Start creating a more complex workflow

The previous example was the first ‘real’ benchmark, but it didn’t do anything different from what you could run through wrk, ab, siege or similar tools.

Of course, the results were not suffering from the coordinated omission problem, but Hyperfoil can do more. Let’s try a more complex scenario:

name: choose-movie
http:
  host: http://localhost:8080
  # Use 80 concurrent HTTP connections to the server. Default is 1,
  # therefore we couldn't issue two concurrent requests (as HTTP pipelining
  # is disabled by default and we use HTTP 1.1 connections).
  sharedConnections: 80
usersPerSec: 10
duration: 5s
# Each session will take at least 3 seconds (see the sleep time below),
# and we'll be running ~10 per second. That makes 30, let's give it
# some margin and set this to 40.
maxSessions: 40
scenario:
  # In previous scenarios we have used only single sequence and we could
  # define the list of sequences right away. In this scenario, we're going
  # to be using 3 different sequences.
  # Initial sequences are scheduled at session start and are not linked
  # to the other sessions.
  initialSequences:
  - home:
    # Pick a random username from a file
    - randomItem:
        file: usernames.txt
        toVar: username
    # The page would load a profile, e.g. to display full name.
    - httpRequest:
        GET: /quickstarts/choose-movie/profile?user=${username}
        sync: false
        metric: profile
    # Fetch movies user could watch
    - httpRequest:
        GET: /quickstarts/choose-movie/movies
        sync: false
        metric: movies
        handler:
          body:
            # Parse the returned JSON that is an array and for each
            # element fire the processor.
            json:
              query: .[]
              processor:
                # Store each element in a collection `movies`
                array:
                  toVar: movies
                  # Store as byte[] to avoid encoding UTF-8 into String
                  format: BYTES
                  # Every data structure in session has maximum size.
                  # This space is pre-allocated.
                  maxSize: 10
    # This step waits until responses for all sent requests are received and processed.
    - awaitAllResponses
    # Wait 3 seconds to simulate user-interaction
    - thinkTime:
        duration: 3s
    # Set variable `quality` and `movieNames` to an uninitialized array
    # of 10 elements. We will use them later on.
    - set:
        var: quality
        objectArray:
          size: 10
    - set:
        var: movieNames
        objectArray:
          size: 10
    # For each element in variable `movies` schedule one (new) instance
    # of sequence `movies`, defined below. These instances differ in
    # one intrinsic "variable" - their index.
    - foreach:
        fromVar: movies
        sequence: addComment
    # Schedule one more sequence
    - newSequence: watchMovie
  # These sequences are defined but don't get scheduled at session start. We activate
  # them explicitly (and multiple times in parallel) in foreach step above.
  sequences:
  # Sequences that can run multiple instances concurrently must declare the maximum
  # concurrency level explicitly using the brackets.
  - addComment[10]:
    # Variables `movies` hosts an array, and in the foreach step
    # we've created one sequence for each element. We'll access
    # the element through the '[.]' notation below.
    - json:
        fromVar: movies[.]
        query: .quality
        # We'll extract quality to another collection under
        # this sequence's index. We shouldn't use global variable
        # as the execution of sequences may interleave.
        toVar: quality[.]
    # For high-quality movies we won't post insults (we haven't seen
    # the movie yet anyway). Therefore, we'll stop executing
    # the sequence prematurely.
    - breakSequence:
        intCondition:
          fromVar: quality[.]
          # Note: ideally we could filter the JSON directly using query
          #     .[] | select(.quality >= 80)
          # but this feature is not implemented yet.
          greaterOrEqualTo: 80
    - json:
        fromVar: movies[.]
        query: .name
        toVar: movieNames[.]
    - httpRequest:
        # URLs with spaces and other characters don't work well;
        # let's encode it (e.g. space -> %20)
        POST: /quickstarts/choose-movie/movie/${urlencode:movieNames[.]}/comments
        body:
          text: This movie sucks.
        # The sync shortcut actually sets up a bit in the session state
        # cleared before the request and set when the request is complete,
        # automatically waiting it after this step.
        # You can write your own handlers (using sequence-scoped vars)
        # to change this behaviour.
        sync: true
    # Set value to variable `commented`. The actual value does not matter.
    - set: commented <- true
  - watchMovie:
    # This sequence is blocked in its first step until the variable gets
    # set. Therefore we could define it in `initialSequences` and omit
    # the `newSequence` step at the end of `home` sequence.
    - awaitVar: commented
    # Choose one of the movies (including the bad ones, for simplicity)
    - randomItem: selectedMovie <- movies
    - json:
        fromVar: selectedMovie
        query: .name
        # This sequence is executed only once so we can use global var.
        toVar: movieName
    # Finally, go watch the movie!
    - httpRequest:
        GET: /quickstarts/choose-movie/movie/${urlencode:movieName}/watch
        sync: true

Start the server and fire the scenario the usual way:

# start the server to interact with
podman run --rm -d -p 8080:8083 quay.io/hyperfoil/hyperfoil-examples

# start Hyperfoil CLI
bin/cli.sh

[hyperfoil]$ start-local
...
[hyperfoil@in-vm]$ upload .../choose-movie.hf.yaml
...
[hyperfoil@in-vm]$ run
...

Is this scenario too simplistic? Let’s define phases…

2.4 - Phases - basics

Deep dive into the basics of phases

So far the benchmark contained only one type of load; certain number of users hitting the system, doing always the same (though data could be randomized). In practice you might want to simulate several types of workloads at once: in an eshop users would come browsing or buying products, and operators would restock the virtual warehouse.

Also, driving constant load may not be the best way to run the benchmark: often you want to slowly ramp the load up to let the system adjust (scale up, perform JIT, fill pools) and push the full load only after that. When trying to find system limits, you do the same repetitevely - ramp up the load, measure latencies and if the system meets SLAs (latencies below limits) continue ramping up the load until it breaks.

In Hyperfoil, this all is expressed through phases. We’ve already seen phases in the first quickstart as we wanted to execute a non-default type of load - running the workload only once. Let’s take a look on the “eshop” case first:

# This benchmark simulates operations in an eshop, with browsing/shopping users
# and operators restocking the warehouse.
name: eshop
http:
  host: http://localhost:8080
  sharedConnections: 80
phases:
# This defines a workload where users just look through the pages.
- browsingUser:
    # This is the default type of workload, starting constant number of users
    # each second. Note that we don't speak about 'requests per second' as
    # the scenario may issue any number of requests.
    constantRate:
      duration: 10s
      usersPerSec: 10
      scenario:
      # Browse is the name of our only sequence. We will avoid steps generating
      # random data for browsing for the sake of brevity.
      - browse:
        - httpRequest:
            GET: /quickstarts/eshop/items
# Workload simulating users that are going to buy something
- buyingUser:
    constantRate:
      # The length of this phase is not synchronized with other phases.
      # You might think that this is too flexible at first.
      duration: 10s
      usersPerSec: 5
      scenario:
      - browse:
        - httpRequest:
            GET: /quickstarts/eshop/items
            handler:
              body:
                json:
                  query: .[].id
                  # This is a shortcut to store in array-typed variable
                  # `itemIds` holding at most 10 elements.
                  toArray: itemIds[10]
      - buy:
        # Pick id for a random item
        - randomItem: itemId <- itemIds
        - httpRequest:
            POST: /quickstarts/eshop/items/${itemId}/buy
- operator:
    # This is a different type of phase, running fixed number of users.
    # It is what most benchmarks do when you set number of threads; here we use
    # that as we know that we have fixed number of employees (operators) who
    # are restocking the warehouse.
    always:
      users: 5
      duration: 10s
      scenario:
      - restock:
        # Select an id for random item to restock
        # Variables in different scenarios are completely unrelated.
        - randomInt: itemId <- 1 .. 999
        - randomInt: units <- 1 .. 10
        - httpRequest:
            POST: /quickstarts/eshop/items/${itemId}/restock
            body:
              # We are using url-encoded form data
              form:
              - name: addUnits
                fromVar: units
        # Operators need some pauses - otherwise we would start another
        # scenario execution (and fire another request) right away.
        - thinkTime:
            duration: 2s

Start the same server as you did in the previous quickstarts:

podman run --rm -p 8080:8083 quay.io/hyperfoil/hyperfoil-examples

In next quickstart you’ll learn how to repeat and link the phases.

2.5 - Phases - advanced

Delve into more advanced phase configuration

Previous quickstart presented a benchmark with three phases that all started at the same moment (when the benchmark was started) and had the same duration - different phases represented different workflows (types of user). In this example we will adjust the benchmark to scale the load gradually up.

At this point it would be useful to mention the lifecycle of phases; phase is in one of these states:

not started: As the name clearly says, the phase is not yet started.
running: The agent started running the phase, i.e., performing the configured load.
finished: When the duration elapses, no more new users are started. However, some might be still executing their scenarios.
terminated: When all users complete their scenarios the phase becomes terminated. Users may be forcibly interrupted by setting maxDuration on the phase.
cancelled If the benchmark cannot continue further, all remaining stages are cancelled.

Let’s take a look into the example, where we’ll slowly (over 5 seconds) increase load to 10+5 users/sec, run with this load for 10 seconds, again increase it by another 10+5 users/sec and so forth until we reach 100+50 users per second. As we define maxIterations for these phases the benchmark will actually contain phases browsingUserRampUp/0, browsingUserRampUp/1, browsingUserRampUp/2 and so forth.

name: eshop-scale
http:
  host: http://localhost:8080
  sharedConnections: 80
phases:
- browsingUserRampUp:
    # This type of phase is similar to constantRate in the way how new users
    # are started but gradually increases the rate from `initialUsersPerSec`
    # to `targetUsersPerSec`.
    increasingRate:
      duration: 5s
      # In Hyperfoil, everything is pre-allocated = limited in size. Here we'll
      # set that we won't run more than 10 iterations of this phase.
      maxIterations: 10
      # Number of started users per sec increases with the iteration; in first
      # iteration we'll go from 0 to 10 users/second, in second from 10 to 20
      # and in last (10th) we'll reach 100 users/second.
      initialUsersPerSec:
        base: 0
        increment: 10
      targetUsersPerSec:
        base: 10
        increment: 10
      # Nth iteration of this phase will start when (N-1)th iteration of other
      # steady-state phases are finished. First iteration can start
      # immediatelly, of course.
      startAfter:
      - phase: browsingUserSteady
        iteration: previous
      - phase: buyingUserSteady
        iteration: previous
      # The &browsingUser syntax below creates YAML alias: we can later
      # reference this scenario and it will be used verbatim in another phase.
      # It is possible to use aliases for both scenarios and sequences.
      scenario: &browsingUser
      # We'll use the same scenario as in eshop.hf.yaml
      - browse:
        - httpRequest:
            GET: /quickstarts/eshop/items
- browsingUserSteady:
    constantRate:
      duration: 10s
      maxIterations: 10
      usersPerSec:
        base: 10
        increment: 10
      # Nth iteration of this phase will start when Nth iteration of ramp-up
      # phases is finished.
      # Note that there's implicit rule that Nth iteration of given phase will
      # start only after (N-1)th iteration terminates.
      startAfter:
      - phase: browsingUserRampUp
        iteration: same
      - phase: buyingUserRampUp
        iteration: same
      # This refers to the alias created above; in steady state we'll use the
      # same scenario.
      scenario: *browsingUser
# These two phases will be very similar to browsingUserSteady and RampUp
- buyingUserRampUp:
    increasingRate:
      duration: 5s
      maxIterations: 10
      initialUsersPerSec:
        base: 0
        increment: 5
      targetUsersPerSec:
        base: 5
        increment: 5
      startAfter:
      - phase: browsingUserSteady
        iteration: previous
      - phase: buyingUserSteady
        iteration: previous
      # Again we'll use the same scenario as in eshop.hf.yaml
      scenario: &buyingUser
      - browse:
        - httpRequest:
            GET: /quickstarts/eshop/items
            handler:
              body:
                json:
                  query: .[].id
                  toArray: itemIds[10]
      - buy:
        - randomItem: itemId <- itemIds
        - httpRequest:
            POST: /quickstarts/eshop/items/${itemId}/buy
- buyingUserSteady:
    constantRate:
      duration: 10s
      maxIterations: 10
      usersPerSec:
        base: 5
        increment: 5
      startAfter:
      - phase: browsingUserRampUp
        iteration: same
      - phase: buyingUserRampUp
        iteration: same
      scenario: *buyingUser
# Operator phase is omitted for brevity as we wouldn't scale that up

Don’t forget to start the mock server as we’ve used in the previous quickstart.

podman run --rm -p 8080:8083 quay.io/hyperfoil/hyperfoil-examples

Synchronizing multiple workloads across iteration can become a bit cumbersome. That’s why we can keep similar types of workflow together, and split the phase into forks. In fact forks will become different phases, but these will be linked together so that you can refer to all of them as to a single phase. Take a look at the benchmark rewritten to use forks:

name: eshop-forks
http:
  host: http://localhost:8080
  sharedConnections: 80
phases:
- rampUp:
    increasingRate:
      duration: 5s
      maxIterations: 10
      # Note that we have increased both the base and increment from 10 and 5
      # to 15. This value is split between the forks based on their weight.
      initialUsersPerSec:
        base: 0
        increment: 15
      targetUsersPerSec:
        base: 15
        increment: 15
      startAfter:
        phase: steadyState
        iteration: previous
      forks:
        browsingUser:
          weight: 2
          scenario: &browsingUser
          - browse:
            - httpRequest:
                GET: /quickstarts/eshop/items
        buyingUser:
          weight: 1
          scenario: &buyingUser
          - browse:
            - httpRequest:
                GET: /quickstarts/eshop/items
                handler:
                  body:
                    json:
                      query: .[].id
                      toArray: itemIds[10]
          - buy:
            - randomItem: itemId <- itemIds
            - httpRequest:
                POST: /quickstarts/eshop/items/${itemId}/buy
- steadyState:
    constantRate:
      duration: 10s
      maxIterations: 10
      usersPerSec:
        base: 15
        increment: 15
      startAfter:
        phase: rampUp
        iteration: same
      forks:
        browsingUser:
          weight: 2
          scenario: *browsingUser
        buyingUser:
          weight: 1
          scenario: *buyingUser
# Operator phase is omitted for brevity as we wouldn't scale that up

This definition will create phases rampUp/0/browsingUser, rampUp/0/buyingUser, rampUp/1/browsingUser etc. - you’ll see them in statistics.

You could orchestrate the phases as it suits you, using startAfter, startAfterStrict (this requires the referenced phase to me terminated instead of finished as with startAfter) or startTime with relative time since benchmark start.

This sums up basic principles, in next quickstart you’ll see how to start and use Hyperfoil in distributed mode.

2.6 - Running the server

Learn how to start the Hyperfoil server in standalone mode

Until now we have always started our benchmarks using an embedded controller in the CLI, using the start-local command. This spawns a server in the CLI JVM. CLI communicates with it using standard REST API, though the server port is randomized and listens on localhost only. All the benchmarks and run results are also stored in /tmp/hyperfoil/ - you can change the directory as an argument to the start-local command. While the embedded controller might be convenient for a quick test or when developing the scenario it’s not something that you’d use for a full-fledged benchmark.

When testing a reasonably performing system you need multiple nodes driving the load - we call them agents. These agents sync up, receive commands and report statistics to a master node, the controller. This node exposes a RESTful API to upload & start the benchmark, watch its progress and download results.

There are two other scripts in the bin/ directory:

standalone.sh starts both the controller and (one) agent in a single JVM. This is not too different from the controller embedded in CLI.
controller.sh starts clustered Vert.x and deploys the controller. Agents are started as needed in different nodes. You’ll see this in the next quickstart.

Also note that it is possible to run Hyperfoil in Openshift.

Open two terminals; in one terminal start the standalone server and in second terminal start the CLI.

bin/standalone.sh

and

bin/cli.sh

Then, let’s try to connect to the server (by default running on http://localhost:8090) and upload the single-request benchmark:

# This is the name of the benchmark. It's recommended to keep this in sync with
# name of this file, adding extension `.hf.yaml`.
name: single-request
# We must define at least one HTTP target, in this case it becomes a default
# for all HTTP requests.
http:
  host: http://hyperfoil.io
# Simulation consists of phases - potentially independent workloads.
# We'll discuss phases more in detail in next quickstarts.
phases:
# `example` is the name of the single phase in this benchmark.
- example:
    # `atOnce` with `users: 1` results in running the scenario below just once
    atOnce:
      users: 1
      scenario:
      # The only sequence in this scenario is called `test`.
      - test:
        # In the only step in this sequence we'll do a HTTP GET request
        # to `http://hyperfoil.io/`
        - httpRequest:
            GET: /
            # Inject helpers to make this request synchronous, i.e. keep
            # the sequence blocked until Hyperfoil processes the response.
            sync: true

From the second terminal, the one running the Hyperfoil CLI, issue the following commands:

[hyperfoil@localhost]$ connect
Connected! Server has these agents connected:
* localhost[REGISTERED]

[hyperfoil@localhost]$ upload .../single-request.hf.yaml
Loaded benchmark single-request, uploading...
... done.

[hyperfoil@localhost]$ run single-request
Started run 0001

When you switch to the first terminal (the one running the controller), you can see in the logs that the benchmark definition was stored on the server, the benchmark has been executed and its results have been stored to disk. Hyperfoil by default stores benchmarks in directory /tmp/hyperfoil/benchmark and data about runs in /tmp/hyperfoil/run; check it out:

column -t -s , /tmp/hyperfoil/run/0001/stats/total.csv

Phase    Name  Requests  Responses  Mean       Min        p50.0      p90.0      p99.0      p99.9      p99.99     Max        MeanSendTime  ConnFailure  Reset  Timeouts  2xx  3xx  4xx  5xx  Other  Invalid  BlockedCount  BlockedTime  MinSessions  MaxSessions
example  test  1         1          267911168  267386880  268435455  268435455  268435455  268435455  268435455  268435455  2655879       0            0      0         0    1    0    0    0      0        0             0

Reading CSV/JSON files directly is not too comfortable; you can check the details through CLI as well:

[hyperfoil@localhost]$ stats
Total stats from run 002D
Phase   Sequence  Requests      Mean       p50       p90       p99     p99.9    p99.99    2xx    3xx    4xx    5xx Timeouts Errors
example:
	test:            1 267.91 ms 268.44 ms 268.44 ms 268.44 ms 268.44 ms 268.44 ms      0      1      0      0        0      0

By the time you type the stats command into CLI the benchmark is already completed and the CLI shows stats for the whole run. Let’s try running the {% include example_link.md src=‘eshop-scale.hf.yaml’ %} we’ve seen in previous quickstart; this will give us some time to observe on-line statistics as the benchmark is progressing:

podman run --rm -p 8080:8083 quay.io/hyperfoil/hyperfoil-examples

[hyperfoil@localhost]$ upload .../eshop-scale.hf.yaml
Loaded benchmark eshop-scale, uploading...
... done.
[hyperfoil@localhost]$ run eshop-scale
Started run 0002
Run 0002, benchmark eshop-scale
...

Here the console would automatically jump into the status command, displaying the progress of the benchmark online. Press Ctrl+C to cancel that (it won’t stop the benchmark run) and run the stats command:

[hyperfoil@localhost]$ stats
Recent stats from run 0002
Phase   Sequence  Requests      Mean       p50       p90       p99     p99.9    p99.99    2xx    3xx    4xx    5xx Timeouts Errors
buyingUserSteady/000:
        buy:             8   1.64 ms   1.91 ms   3.05 ms   3.05 ms   3.05 ms   3.05 ms      8      0      0      0        0      0
        browse:          8   2.13 ms   2.65 ms   3.00 ms   3.00 ms   3.00 ms   3.00 ms      8      0      0      0        0      0
browsingUserSteady/000:
        browse:          8   2.74 ms   2.69 ms   2.97 ms   2.97 ms   2.97 ms   2.97 ms      8      0      0      0        0      0
Press Ctr+C to stop watching...

You can go back to the run progress using the status command (hint: use status --all to display all phases, including those not started or already terminated):

[hyperfoil@localhost]$ status
Run 0002, benchmark eshop-scale
Agents: localhost[INITIALIZED]
Started: 2019/04/15 16:27:24.526
NAME                    STATUS   STARTED       REMAINING  FINISHED  TOTAL DURATION
browsingUserRampUp/006  RUNNING  16:28:54.565  2477 ms
buyingUserRampUp/006    RUNNING  16:28:54.565  2477 ms
Press Ctrl+C to stop watching...

Since we are showing this quickstart running the controller and CLI on the same machine it’s easy to fetch results locally from /tmp/hyperfoil/run/XXXX/.... To save you SSHing into the controller host and finding the directories in a ’true remote’ case there’s the export command; This fetches statistics to your computer where you’re running CLI. You can chose between default JSON format (e.g. export 0002 -f json -d /path/to/dir) and CSV format (export 0002 -f csv -d /path/to/dir) - the latter packs all CSV files into single ZIP file for your convenience.

When you find out that the benchmark is not going well, you can terminate it prematurely:

[hyperfoil@localhost]$ kill
Kill run 0002, benchmark eshop-scale(phases: 2 running, 0 finished, 40 terminated) [y/N]: y
Killed.

In the next quickstart we will deal with starting clustered Hyperfoil.

2.7 - Clustered mode

Learn how to start the Hyperfoil server in clustered mode

Previously we’ve learned to start Hyperfoil in standalone server mode, and to do some runs through CLI. In this quickstart we’ll see how to run your benchmark distributed to several agent nodes.

Hyperfoil operates as a cluster of Vert.x. When the benchmark is started, it deploys agents on other nodes according to the benchmark configuration - these are Vert.x nodes, too. Together controller and agents form a cluster and communicate over the event bus.

In this quickstart we’ll use the SSH deployer; make sure your machine has SSH server running on port 22 and you can login using your pubkey ~/.ssh/id_rsa. The SSH deployer copies the necessary JARs to /tmp/hyperfoil/agentlib/ and starts the agent there. For instructions to run Hyperfoil in Kubernetes or Openshift please consult the Installation docs.

When we were running in the standalone or local mode we did not have to set any agents in the benchmark definition. That changes now as we need to inform the controller where the agents should be deployed. Let’s see a benchmark - two-agents.hf.yaml that has those agents defined.

name: two-agents
# List of agents the Controller should deploy
agents:
  # This defines the agent using SSH connection to localhost, port 22
  agent-one: localhost:22
  # Another agent on localhost, this time defined using properties
  agent-two:
    host: localhost
    port: 22
http:
  host: http://localhost:8080
usersPerSec: 10
duration: 10s
scenario:
- test:
  - httpRequest:
      GET: /

The load the benchmark generates is evenly split among the agents, so if you want to use another agent, you don’t need to do any calculations - just add the agent and you’re good to go.

Open three terminals; in the first start the controller using bin/controller.sh, in second one open the CLI with bin/cli.sh and in the third one start the example workload server:

podman run --rm -p 8080:8083 quay.io/hyperfoil/hyperfoil-examples

Connect, upload, start and check out the benchmark using CLI exactly the same way as we did in the previous quickstart:

[hyperfoil@localhost]$ connect
Connected!

[hyperfoil@localhost]$ upload .../two-agents.hf.yaml
Loaded benchmark two-agents, uploading...
... done.

[hyperfoil@localhost]$ run two-agents
Started run 004A

[hyperfoil@localhost]$ status
Run 004A, benchmark two-agents
Agents: agent-one[STARTING], agent-two[STARTING]
Started: 2019/04/17 17:08:19.703    Terminated: 2019/04/17 17:08:29.729
NAME  STATUS      STARTED       REMAINING  FINISHED      TOTAL DURATION
main  TERMINATED  17:08:19.708             17:08:29.729  10021 ms (exceeded by 21 ms)

[hyperfoil@localhost]$ stats
Total stats from run 004A
Phase   Sequence  Requests      Mean       p50       p90       p99     p99.9    p99.99    2xx    3xx    4xx    5xx Timeouts Errors
main:
	test:          106   3.12 ms   2.83 ms   3.23 ms  19.53 ms  25.30 ms  25.30 ms    106      0      0      0        0      0

You see that we did 106 requests which fits the assumption about running 10 user sessions per second over 10 seconds, while we have used 2 agents.

Vert.x clustering is using Infinispan and JGroups; depending on your networking setup it might not work out-of-the-box. If you experience any trouble, check out the FAQ.

Next quickstart will get back to the scenario definition; we’ll show you how to extend Hyperfoil with custom steps and handlers.

2.8 - Custom components

Hyperfoil offers some basic steps to do HTTP requests, generate data, alter control flow in the scenario etc., but your needs may surpass the features implemented so far. Also, it might be just easier to express your logic in Java code than combining steps in the YAML. The downside is reduced ability to reuse and more tight dependency on Hyperfoil APIs.

This quickstart will show you how to extend Hyperfoil with custom steps and handlers. As we use the standard Java ServiceLoader approach, after you build the module you should drop it into extensions directory. (Note: if you upload the benchmarks through CLI you need to put it to both the machine where you run the CLI and to the controller.)

Each extension will consist of two classes:

Builder, is loaded as service and creates the immutable extension instance
extension (Step, Action or handler)

Let’s start with a io.hyperfoil.api.config.Step implementation. The interface has single method invoke(Session) that should return true if the step was executed and false if its execution has been blocked and should be retried later. In case that the execution is blocked the invocation must not have any side effects - e.g. if the step is fetching objects from some pools and one of the pools is depleted, it should release the already acquired objects back to the pool.

We’ll create a step that will divide variable from a session by a (configurable) constant and store the result in another variable.

Java


public class DivideStep implements Step {
   // All fields in a step are immutable, any state must be stored in the Session
   private final ReadAccess fromVar;
   private final IntAccess toVar;
   private final int divisor;

   public DivideStep(ReadAccess fromVar, IntAccess toVar, int divisor) {
      // Variables in session are not accessed directly using map lookup but
      // through the Access objects. This is necessary as the scenario can use
      // some simple expressions that are parsed when the scenario is built
      // (in this constructor), not at runtime.
      this.fromVar = fromVar;
      this.toVar = toVar;
      this.divisor = divisor;
   }

   @Override
   public boolean invoke(Session session) {
      // This step will block until the variable is set, rather than
      // throwing an error or defaulting the value.
      if (!fromVar.isSet(session)) {
         return false;
      }
      // Session can store either objects or integers. Using int variables is
      // more efficient as it prevents repeated boxing and unboxing.
      int value = fromVar.getInt(session);
      toVar.setInt(session, value / divisor);
      return true;
   }

  ...

Then we need a builder class that will allow us to configure the step. To keep related classes together we will define it as inner static class:

Java


public class DivideStep implements Step {
  ...

     // Make this builder loadable as service
   @MetaInfServices(StepBuilder.class)
   // This is the step name that will be used in the YAML
   @Name("divide")
   public static class Builder extends BaseStepBuilder<Builder> implements InitFromParam<Builder> {
      // Contrary to the step fields in builder are mutable
      private String fromVar;
      private String toVar;
      private int divisor;

      // Let's permit a short-form definition that will store the result
      // in the same variable. Note that the javadoc @param is used to generate external documentation.

      /**
       * @param param Use myVar /= constant
       */
      @Override
      public Builder init(String param) {
         int divIndex = param.indexOf("/=");
         if (divIndex < 0) {
            throw new BenchmarkDefinitionException("Invalid inline definition: " + param);
         }
         try {
            divisor(Integer.parseInt(param.substring(divIndex + 2).trim()));
         } catch (NumberFormatException e) {
            throw new BenchmarkDefinitionException("Invalid inline definition: " + param, e);
         }
         String var = param.substring(0, divIndex).trim();
         return fromVar(var).toVar(var);
      }

      // All fields are set in fluent setters - this helps when the scenario
      // is defined through programmatic configuration.
      // When parsing YAML the methods are invoked through reflection;
      // the attribute name is used for the method lookup.
      public Builder fromVar(String fromVar) {
         this.fromVar = fromVar;
         return this;
      }

      public Builder toVar(String toVar) {
         this.toVar = toVar;
         return this;
      }

      // The parser can automatically convert primitive types and enums.
      public Builder divisor(int divisor) {
         this.divisor = divisor;
         return this;
      }

      @Override
      public List<Step> build() {
         // You can ignore the sequence parameter; this is used only in steps
         // that require access to the parent sequence at runtime.
         if (fromVar == null || toVar == null || divisor == 0) {
            // Here is a good place to check that the attributes are sane.
            throw new BenchmarkDefinitionException("Missing one of the required attributes!");
         }
         // The builder has a bit more flexibility and it can create more than
         // one step at once.
         return Collections.singletonList(new DivideStep(
               SessionFactory.readAccess(fromVar), SessionFactory.intAccess(toVar), divisor));
      }
   }

  ...

As the comments say, the builder is using fluent setter syntax to set the attributes. When you want to nest attributes under another builder, you can just add parameter-less method FooBuilder foo() the returns an instance of FooBuilder; the parser will fill this instance as well. There are some interfaces your builder can implement to accept lists or different structures, but the description is out of scope of this quickstart.

The builder class has two annotations: @Name which specifies the name we’ll use in YAML as step name, and @MetaInfServices with StepBuilder.class as the parameter. If you were to implement other type of extension, this would be Action.Builder.class, Request.ProcessorBuilder.class etc. In order to record the service in META-INF directory in the jar you must also add this dependency to your module:

<dependency>
    <groupId>org.kohsuke.metainf-services</groupId>
    <artifactId>metainf-services</artifactId>
    <optional>true</optional>
</dependency>

The whole class can be inspected here and it is already included in the extensions directory. You can try running bin/standalone.sh, upload and run divide.hf.yaml. You should see about 5 log messages in the server log.

# This benchmark demonstrates custom steps
name: divide
http:
  host: http://localhost:8080
usersPerSec: 1
duration: 5s
scenario:
- test:
  - setInt: foo <- 33
  - divide: foo /= 3
  - log:
      message: Foo is {}
      vars:
      - foo

There are several other integration points but Step:

io.hyperfoil.api.session.Action is very similar to step, but it does not allow blocking. Implement Action.BuilderFactory to define new actions.
StatusHandler, HeaderHandler and BodyHandler in io.hyperfoil.api.http package process different stages of HTTP response parsing. All these have BuilderFactory inner interface for you to implement.
io.hyperfoil.api.connection.Processor performs later generic stages of response processing. As this interface is generic, there are two factories that you could use: i.h.a.c.Request.ProcessorBuilderFactory and i.h.a.c.HttpRequest.ProcessorBuilderFactory.

There is quite some boilerplate code in the process of creating a new component; that’s why you can use Hyperfoil Codegen Maven plugin to scaffold the basic outline for you. Go to the module where you want the component generated and run:

mvn io.hyperfoil:hyperfoil-codegen-maven-plugin:skeleton

The plugin will ask you for the package name, component name and type and write down the source code skeleton. You can provide the parameters right on commandline like

mvn io.hyperfoil:hyperfoil-codegen-maven-plugin:skeleton \
    -Dskeleton.package=foo.bar -Dskeleton.name=myComponent -Dskeleton.type=step

Note

The generator does not add the dependency to org.kohsuke.metainf-services:metainf-services to this module’s pom.xml, you need to do that manually.

If you add io.hyperfoil as a plugin group to your $HOME/.m2/settings.xml like this:

<settings>
  <pluginGroups>
    <pluginGroup>io.hyperfoil</pluginGroup>
  </pluginGroups>
  ...
</settings>

you can use the short syntax for the generator:

mvn hyperfoil-codegen:skeleton -Dskeleton.name=....

See also further information about custom extensions development.

This is the last quickstart in this series; if you seek more info check out the documentation or talk to us on GitHub Discussions.

3 - User Guide

Comprehensive set of resourcs for everything you need to get started with Hyperfoil

Welcome to the Hyperfoil User Guide, your comprehensive resource for everything you need to get started. This section covers installation, detailed instructions on defining benchmarks, and troubleshooting tips to help you resolve common issues. Whether you’re a beginner or an advanced user, you’ll find valuable information to enhance your performance testing with Hyperfoil.

3.1 - Installation

Detailed instructions for installing Hyperfoil manually, on Kubernetes/Openshift, or with Ansible.

In this section, you’ll find detailed instructions for installing and setting up Hyperfoil using various methods, including manual setup, Ansible, and Kubernetes/Openshift. Follow these guides to choose the best installation procedure for your environment.

3.1.1 - Manual startup

Explore manual startup options for the Hyperfoil controller.

Hyperfoil controller is started with

bin/controller.sh

Any arguments passed to the scripts will be passed as-is to the java process.

By default io.hyperfoil.deployer is set to ssh which means that the controller will deploy agents over SSH, based on the agents configurion. This requires that the user account running the controller must have public-key SSH authorization set up using key $HOME/.ssh/id_rsa. The user also has to be able to copy files to the directory set in agent definition (by default /tmp/hyperfoil) using SCP - Hyperfoil automatically synchronizes library files in this folder with the currently running instance and then executes the agent.

When you don’t intend to run distributed benchmarks you can start the controller in standalone mode:

bin/standalone.sh

This variant won’t deploy any agents remotely and therefore it does not need any agents: section in the benchmark definition; instead it will use single agent started in the same JVM.

Below is the comprehensive list of all the properties Hyperfoil recognizes. All system properties can be replaced by environment variables, uppercasing the letters and replacing dots and dashes with underscores: e.g. io.hyperfoil.controller.host becomes IO_HYPERFOIL_CONTROLLER_HOST.

Property	Default	Description
io.hyperfoil.controller.host	0.0.0.0	Host for Controller REST server
io.hyperfoil.controller.port	8090	Port for Controller REST server
io.hyperfoil.rootdir	/tmp/hyperfoil	Root directory for stored files
io.hyperfoil.benchmarkdir	root/benchmark	Benchmark files (YAML and serialized)
io.hyperfoil.rundir	root/run	Run result files (configs, stats…)
io.hyperfoil.deployer	ssh	Implementation for agents deployment
io.hyperfoil.deployer.timeout	15000 ms	Timeout for agents to start
io.hyperfoil.agent.debug.port		If set, agent will be started with JVM debug port open
io.hyperfoil.agent.debug.suspend	n	Suspend parameter for the debug port
io.hyperfoil.controller.cluster.ip	first non-loopback	Hostname/IP used for clustering with agents
io.hyperfoil.controller.cluster.port	7800	Default JGroups clustering port
io.hyperfoil.controller.external.uri		Externally advertised URI of REST server
io.hyperfoil.controller.keystore.path		File path to Java Keystore
io.hyperfoil.controller.keystore.password		Java Keystore password
io.hyperfoil.controller.pem.keys		File path(s) to private TLS key(s) in PEM format
io.hyperfoil.controller.pem.certs		File path(s) to server TLS certificate(s) in PEM format
io.hyperfoil.controller.password		Password used for Basic authentication
io.hyperfoil.controller.secured.via.proxy		This must be set to `true` for Basic auth without TLS encryption
io.hyperfoil.trigger.url		See below

If io.hyperfoi.trigger.url is set the controller does not start benchmark run right away after hitting /benchmark/my-benchmark/start ; instead it responds with status 301 and header Location set to concatenation of this string and BENCHMARK=my-benchmark&RUN_ID=xxxx. CLI interprets that response as a request to hit CI instance on this URL, assuming that CI will trigger a new job that will eventually call /benchmark/my-benchmark/start?runId=xxxx with header x-trigger-job. This is useful if the the CI has to synchronize Hyperfoil to other benchmarks that don’t use this controller instance.

Security

Since Hyperfoil accepts and invoked any serialized Java objects you must not run it exposed to public to prevent a very simple remote code execution. Even if using HTTPS and password protection (see below) we recommend to limit access and privileges of the process to absolute minimum.

You can get confidential access to the server using TLS encryption, providing the certificate and keys either using Java Keystore mechanism (properties above starting with io.hyperfoil.controller.keystore) or via PEM files (properties starting with io.hyperfoil.controller.pem). These options are mutually exclusive. In the latter case it is possible to use multiple certificate/key files, separated by comma (,).

Authentication uses Basic authentication scheme accepting any string as username. The password is set using io.hyperfoil.controller.password or respective environment variable. If you’re exposing the server using plaintext HTTP you must set -Dio.hyperfoil.controller.secured.via.proxy=true to confirm that this is a desired configuration (e.g. if the TLS is terminated at proxy and the connection from proxy does not require confidentiality).

3.1.2 - K8s/Openshift deployment

Deploy Hyperfoil in Kubernetes or Openshift environment using the out-of-the-box Hyperfoil operator

A convenient alternative to running Hyperfoil on hosts with SSH access is deploying it in Kubernetes or Openshift environment. The recommended way to install it using an operator in your Openshift console - just go to Operators - OperatorHub and search for ‘hyperfoil’, and follow the installation wizard. Alternatively you can deploy the controller manually.

In order to start a Hyperfoil Controller instance in your cluster, create a new namespace hyperfoil: Go to Operators - Installed Operators and open Hyperfoil. In upper left corner select ‘Project: ’ - Create project and fill out the details. Then click on the ‘Hyperfoil’ tab and find the button ‘Create Hyperfoil’.

You should see a YAML definition like this:

apiVersion: hyperfoil.io/v1alpha2
kind: Hyperfoil
metadata:
  name: example-hyperfoil
  namespace: hyperfoil
spec:
  agentDeployTimeout: 60000
  log: myConfigMap/log4j2-superverbose.xml
  route:
    host: hyperfoil.apps.mycloud.example.com
  version: latest

Change the name to just hyperfoil (or whatever you prefer) and delete all the content from the spec section:

apiVersion: hyperfoil.io/v1alpha2
kind: Hyperfoil
metadata:
  name: hyperfoil
  namespace: hyperfoil
spec:

This is a perfectly valid Hyperfoil resource with everything set to default values. You can customize some properties in the spec section further - see the reference.

The operator deploys only the controller; each agent is then started when the run starts as a pod in the same namespace and stopped when the run completes.

When the resource becomes ready (you can check it out through Openshift CLI using oc get hf) the controller pod should be up and running. Now you can open Hyperfoil CLI and connect to the controller. While default Hyperfoil port is 8090, default route setting uses TLS (edge) and therefore Openshift router will expose the service on port 443. If your cluster’s certificate is not recognized (such as when using self-signed certificates) you need to use --insecure (or -k) option.

bin/cli.sh

[hyperfoil]$ connect hyperfoil-hyperfoil.apps.my.cluster.domain:443 --insecure
WARNING: Hyperfoil TLS certificate validity is not checked. Your credentials might get compromised.
Connected!
WARNING: Server time seems to be off by 12124 ms

Now you can upload & run benchmarks as usual - we’re using {% include example_link.md src=‘k8s-hello-world.hf.yaml’ %} in this example. Note that it can take several seconds to spin up containers with agents.

[hyperfoil@hyperfoil-hyperfoil]$ upload examples/k8s-hello-world.hf.yaml
Loaded benchmark k8s-hello-world, uploading...
... done.

[hyperfoil@hyperfoil-hyperfoil]$ run k8s-hello-world
Started run 0000
Run 0000, benchmark k8s-hello-world
Agents: agent-one[STARTING]
Started: 2019/11/18 19:07:36.752    Terminated: 2019/11/18 19:07:41.778
NAME  STATUS      STARTED       REMAINING  COMPLETED     TOTAL DURATION               DESCRIPTION
main  TERMINATED  19:07:36.753             19:07:41.778  5025 ms (exceeded by 25 ms)  5.00 users per second

[hyperfoil@hyperfoil-hyperfoil]$

You can find more details about adjusting the agents in the benchmark format reference.

Running Hyperfoil inside the cluster you are trying to test might skew results due to different network topology compared to driving the load from ‘outside’ (as real users would do). It is your responsibility to validate if your setup and separation between load driver and SUT (system under test) is correct. You have been warned.

Reference

Property	Description
version	Tag for controller image (e.g. `0.14` for a released version or `0.15-SNAPSHOT` for last build from main (master) branch). Defaults to `latest`.
image	Controller image. If ‘version’ is defined, too, the tag is replaced (or appended). Defaults to ‘quay.io/hyperfoil/hyperfoil’
route	Configuration for the route leading to Controller REST endpoint.
auth	Authorization configuration.
log	Name of the config map and optionally its entry (separated by ‘/’: e.g myconfigmap/log4j2-superverbose.xml) storing Log4j2 configuration file. By default the Controller uses its embedded configuration.
agentDeployTimeout	Deploy timeout for agents, in milliseconds.
triggerUrl	Value for `io.hyperfoil.trigger.url` - see above
preHooks	List of config map names holding hooks that run before the run starts.
postHooks	List of config map names holding hooks that run when the run finishes.
persistentVolumeClaim	Name of the PVC Hyperfoil should mount for its workdir.

route

Property	Description
host	Host for the route leading to Controller REST endpoint. Example: hyperfoil.apps.cloud.example.com
type	Either ‘http’ (for plain-text routes - not recommended), ’edge’, ‘reencrypt’ or ‘passthrough’
tls	Name of the secret hosting `tls.crt`, `tls.key` and optionally `ca.crt`. This is mandatory for `passthrough` routes and optional for edge and reencrypt routes

auth

Property	Description
secret	Name of secret used for basic authentication. Must contain key `password`; Hyperfoil accepts any username for login.

3.1.3 - Manual k8s/Openshift deployment

Manually deploy Hyperfoil in Kubernetes or Openshift environment

If you cannot use the operator or if you’re running vanilla Kubernetes you can define all the resource manually. You deploy only the controller; each agent is then started, when the run starts, as a pod in the same namespace and stopped when the run completes.

Following steps install Hyperfoil controller in Openshift, assuming that you have all the required priviledges. With vanilla Kubernetes you might have to replace the route with an appropriate ingress.

1. Create new namespace for hyperfoil

oc new-project hyperfoil

2. Create required resources

curl -s -L k8s.hyperfoil.io | oc apply -f -

role.rbac.authorization.k8s.io/controller created
serviceaccount/controller created
service/hyperfoil created
rolebinding.rbac.authorization.k8s.io/controller created
deploymentconfig.apps.openshift.io/controller created
route.route.openshift.io/hyperfoil created

The route will use hostname following the format hyperfoil-hyperfoil.apps.my.cluster.domain - feel free to customize the hostname as needed.

3. Wait until the image gets downloaded and the container starts

oc get po

NAME                  READY   STATUS              RESTARTS   AGE
controller-1-pqbvs    1/1     Running             0          57s
controller-1-deploy   0/1     Completed           0          72s

4. Open CLI and connect to the controller

While default Hyperfoil port is 8090, Openshift router will expose the service on port 80.

bin/cli.sh

[hyperfoil]$ connect hyperfoil-hyperfoil.apps.my.cluster.domain -p 80
Connected!
WARNING: Server time seems to be off by 12124 ms

5. Upload and run benchmarks as usual

We’re using k8s-hello-world.hf.yaml in this example.

name: k8s-hello-world
agents:
  agent-one:
http:
  host: https://kubernetes.default.svc.cluster.local
duration: 5s
usersPerSec: 5
scenario:
- test:
  - httpRequest:
      GET: /

Note that it can take several seconds to spin up containers with agents.

[hyperfoil@hyperfoil-hyperfoil]$ upload .../k8s-hello-world.hf.yaml
Loaded benchmark k8s-hello-world, uploading...
... done.

[hyperfoil@hyperfoil-hyperfoil]$ run k8s-hello-world
Started run 0000
Run 0000, benchmark k8s-hello-world
Agents: agent-one[STARTING]
Started: 2019/11/18 19:07:36.752    Terminated: 2019/11/18 19:07:41.778
NAME  STATUS      STARTED       REMAINING  COMPLETED     TOTAL DURATION               DESCRIPTION
main  TERMINATED  19:07:36.753             19:07:41.778  5025 ms (exceeded by 25 ms)  5.00 users per second

[hyperfoil@hyperfoil-hyperfoil]$

You can find more details about adjusting the agents in the benchmark format reference.

3.1.4 - Ansible startup

Deploy Hyperfoil using Ansible Galaxy scripts

You can fetch release, distribute and start the cluster using Ansible Galaxy scripts; setup, test, shutdown

First, get the scripts:

ansible-galaxy install hyperfoil.hyperfoil_setup,{{ site.last_release.galaxy_version }}
ansible-galaxy install hyperfoil.hyperfoil_shutdown,{{ site.last_release.galaxy_version }}
ansible-galaxy install hyperfoil.hyperfoil_test,{{ site.last_release.galaxy_version }}

Now, edit your hosts file, it could look like this:

[hyperfoil-controller]
controller ansible_host=localhost

[hyperfoil-agent]
agent-1 ansible_host=localhost

Note

You can add more agents by duplicating the last line with agent-2 etc.

Prepare your playbook; here is a short example that starts the controller, uploads and starts simple benchmark (the templating engine replaces the agents in benchmark script based on Ansible hosts) and waits for its completion. When it confirms number of requests executed it stops the controller.

- hosts: [ hyperfoil-agent, hyperfoil-controller ]
  tasks: [] # This will only gather facts about all nodes
- hosts: hyperfoil-controller
  roles:
  - hyperfoil.hyperfoil_setup
- hosts: 127.0.0.1
  connection: local
  roles:
  - hyperfoil.hyperfoil_test
  vars:
    test_name: example
# Note that due to the way Ansible lookups work this will work only if hyperfoil-controller == localhost
- hosts: 127.0.0.1
  connection: local
  tasks:
  - name: Find number of requests
    set_fact:
      test_requests: "{{ lookup('csvfile', 'example file=/tmp/hyperfoil/workspace/run/' + test_runid + '/stats/total.csv col=2 delimiter=,')}}"
  - name: Print number of requests
    debug:
      msg: "Executed {{ test_requests }} requests."
- hosts:
  - hyperfoil-controller
  roles:
  - hyperfoil.hyperfoil_shutdown

Finally, run the playbook:

ansible-playbook -i hosts example.yml

3.2 - Benchmark

Detailed breakdown of each component involved in defining a benchmark

In Hyperfoil, defining a benchmark involves structuring scenarios, phases, variables and other components to simulate realistic user behavior and workload patterns. This section provides a detailed breakdown of each component involved in defining a benchmark.

3.2.1 - Agents

Entities responsible for executing benchmark and collecting statistics

This section can be omitted in standalone mode.

Agents section forms either a list or map with arbitrary agent names and either an inline or properties-style definition:

agents:
  someAgent: "inline definition"
  otherAgent:
    foo: bar

The definition is passed to an instance of i.h.api.deployment.Deployer which will interpret the definition. Deployer implementation is registred using the java.util.ServiceLoader and selected through the io.hyperfoil.deployer system property. The default implementation is ssh.

Common properties

Property	Default	Description
threads	from benchmark	Number of threads used by the agent (overrides `threads` in benchmark root).
extras		Custom options passed to the JVM (system properties, JVM options…)

SSH deployer

The user account running Hyperfoil Controller must have a public-key authorization set up on agents’ hosts using key $HOME/.ssh/id_rsa. It also has to be able to copy files into the dir directory using SCP - all the required JARs will be copied there and you will find the logs there as well.

ssh deployer accepts either the [user@]host[:port] inline syntax or these properties:

Property	Default	Description
user	Current username
host		This property is mandatory.
port	22
sshKey	id_rsa	Optionally define a different named key in the `$HOME/.ssh` directory
dir	Directory set by system property `io.hyperfoil.rootdir` or `/tmp/hyperfoil`	Working directory for the agent. This directory can be shared by multiple agents running on the same physical machine.
cpu	(all cpus)	If set the CPUs where the agent can run is limited using `taskset -c <cpu>`. Example: `0-2,6`

See an example of ssh deployment configuration:

agents:
  agent1: testserver1:22
  agent2: testuser@testserver2
  agent3:
    host: testserver3
    port: 22
    dir: /some/other/path

Kubernetes/Openshift deployer

To activate the kubernetes deployer you should set -Dio.hyperfoil.deployer=k8s; the recommended installation does that automatically.

The agents are configured the same way as with SSH deployment, only the properties differ. Full reference is provided below.

Example:

agents:
  my-agent:
    node: my-worker-node

Property	Default	Description
node		Configures the labels for the `nodeSelector`. If the value does not contain equals sign (`=`) or comma (`,`) this sets the desired value of label `kubernetes.io/hostname`. You can also set multiple custom labels separated by commas, e.g. `foo=bar,kubernetes.io/os=linux`.
stop	true	By default the controller stops all agents immediatelly after the run terminates. In case of errors this is not too convenient as you might want to perform further analysis. To prevent automatic agent shutdown set this to false.
log		Name of config map (e.g. `my-config-map`) or config map and its entry (e.g. `my-config-map/log4j2.xml`) that contains the Log4j2 configuration file. Default entry from the config map is `log4j2.xml`. Hyperfoil will mount this configmap as a volume to this agent.
image	quay.io/hyperfoil/hyperfoil:controller-version	Different version of Hyperfoil in the agents
imagePullPolicy	Always	Image pull policy for agents
fetchLogs	true	Automatically watch agents’ logs and store them in the run directory.

3.2.2 - HTTP

This section defines servers that agents contact during benchmarks, allowing configurations for multiple targets with specific connection settings

All servers that Hyperfoil should contact must be declared in this section. Before the benchmark starts Hyperfoil agents will open connections to the target servers; if this connection fails the benchmark is terminated immediatelly.

You can either declare single target server (the default one) within this section or more of them:

http:
  host: http://example.com
  ...

http:
- host: http://example.com
  sharedConnections: 100
- host: http://example.com:8080
  sharedConnections: 50

HTTP configuration has these properties:

Property	Default	Description
protocol		Either `http` or `https`
host		Hostname of the server. For convenience you can use the `http[s]://host[:port]` inline syntax as shown above
port	`80` or `443`	Default is based on the `protocol`
sharedConnections	1	Number of connections to open. It is recommended to set this property to a non-default value.
connectionStrategy	SHARED_POOL	Connection pooling model (see details below)
addresses		Supply list of IPs or IP:port targets that will be used for the connections instead of resolving the `host` in DNS and using `port` as set - `host` and `port` will be used only for `Host` headers and SNI. If this list contains more addresses the connections will be split evenly.
requestTimeout	30 seconds	Default request timeout, this can be overridden in each `httpRequest`.
allowHttp1x	true	Allow HTTP 1.1 for connections (e.g. during ALPN).
allowHttp2x	true	Allow HTTP 2.0 for connections (e.g. during ALPN). If both 1.1 and 2.0 are allowed and `https` is not used (which would trigger ALPN) Hyperfoil will use HTTP 1.1. If only 2.0 is allowed Hyperfoil will start with HTTP 1.1 and perform protocol upgrade to 2.0.
directHttp2	false	Start with H2C HTTP 2.0 without protocol upgrade. Makes sense only for plain text (`http`) connections. Currently not implemented.
maxHttp2Streams	100	Maximum number of requests concurrently enqueued on single HTTP 2.0 connection.
pipeliningLimit	1	Maximum number of requests pipelined on single HTTP 1.1 connection.
rawBytesHandlers	true	Enable or disable using handlers that process HTTP response raw bytes.
keyManager		TLS key manager for setting up client certificates.
trustManager		TLS trust manager for setting up server certificates.

Shared connections

This number is split between all agents and executor threads evenly; if there are too many agents/executors each will get at least 1 connection.

When a scalar value is used for this property the connection pool has fixed size; Hyperfoil opens all connections when the benchmark starts and should a connection be closed throughout the benchmark, another connection is reopened instead. You can change this behaviour by composing the property of these sub-properties:

Property	Description
core	Number of connections that will be opened when the benchmark starts. Number of connections in the pool should never drop below this value (another connection will be opened instead).
max	Maximum number of connections in the pool.
buffer	Hyperfoil will try to keep at least `active + buffer` connections in the pool where `active` is the number of currently used connection (those with at least 1 in-flight request)
keepAliveTime	When a connection is not used for more than this value (in milliseconds) it will be closed. Non-positive value means that the connection is never closed because of being idle.

Example:

http:
  host: http://example.com
  sharedConnections:
    core: 10
    buffer: 10
    max: 10
    keepAliveTime: 30000

Connection strategies

This property describes the connection pooling model, you can choose from the options below:

Strategy	Description
SHARED_POOL	Connections are created in a pool and then borrowed by the session. When the request is complete the connection is returned to the shared pool.
SESSION_POOLS	Connections are created in a shared pool. When the request is completed it is not returned to the shared pool but to a session-local pool. Subsequent requests by this session first try to acquire the connection from this local pool. When the session completes all connections from the session-local pool are returned to the shared pool.
OPEN_ON_REQUEST	Connections are created before request or borrowed from a session-local pool. When the request is completed the connection is returned to this pool. When the session completes all connections from the session-local pool are closed.
ALWAYS_NEW	Always create the connection before the request and close it when it is complete. No pooling of connections.

KeyManager configuration

All files are loaded when the benchmark is constructed, e.g. on the machine running CLI. You don’t need to upload any files to controller or agent machines.

Property	Default	Description
storeType	JKS	Implementation of the store.
storeFile		Path to a file with the store.
password		Password for accessing the store file.
alias		Keystore alias.
certFile		Path to a file with the client certificate.
keyFile		Path to a file with client’s private key.

TrustManager configuration

All files are loaded when the benchmark is constructed, e.g. on the machine running CLI. You don’t need to upload any files to controller or agent machines.

Property	Default	Description
storeType	JKS	Implementation of the store.
storeFile		Path to a file with the store.
password		Password for accessing the store file.
certFile		Path to a file with the server certificate.

3.2.3 - Phases

Defines a unit of workload simulation within a benchmark, representing a specific load pattern or behavior

You might want to simulate several types of workloads at once: e.g. in an eshop users would come browsing or buying products, and operators would restock the virtual warehouse. Also, driving constant load may not be the best way to run the benchmark: often you want to slowly ramp the load up to let the system adjust (scale up, perform JIT, fill pools) and push the full load only after that. When trying to find system limits, you do the same repetitevely - ramp up the load, measure latencies and if the system meets SLAs (latencies below limits) continue ramping up the load until it breaks.

In Hyperfoil, this all is expressed through phases. Phases can run independently of each other; these simulate certain load execute by a group of users. Within one phase all users execute the same scenario (e.g. logging into the system, buying some goods and then logging off).

A phase can be in one of these states:

not running (scheduled): As the name clearly says, the phase is not yet getting executed.
running: The agent started running the phase, i.e., performing the configured load.
finished: Users won’t start new scenarios but we’ll let already-started users complete the scenario.
terminated: All users are done, all stats are collected and no further requests will be made.
cancelled: Same as terminated but this phase hasn’t been run at all.

There are different types of phases based on the mode of starting new users:

Type	Description
constantRate	The benchmark will start certain number of users according to a schedule regardless of previously started users completing the scenario. This is the open-model.
increasingRate	Similar to `constantRate` but ramps up the number of started users throughout the execution of the phase.
decreasingRate	The same as `increasingRate` but requires `initialUsersPerSec` > `targetUsersPerSec`.
atOnce	All users are be started when the phase starts running and once the scenario is completed the users won’t retry the scenario.
always	There is fixed number of users and once the scenario is completed the users will start executing the scenario from beginning. This is called a closed-model and is similar to the way many benchmarks with fixed number of threads work.
noop	This phase cannot have any scenario (or forks). It might be useful to add periods of inactivity into the benchmark.

See the example of phases configuration:

...
phases:
# Over one minute ramp the number of users started each second from 1 to 100
- rampUp:
    increasingRate:
      initialUsersPerSec: 1
      targetUsersPerSec: 100
      # We expect at most 200 users being active at one moment - see below
      maxSessions: 200
      duration: 1m
      scenario: ...
# After rampUp is finished, run for 5 minutes and start 100 new users each second
- steadyState:
    constantRate:
      usersPerSec: 100
      maxSessions: 200
      startAfter: rampUp
      duration: 5m
      # If some users get stuck, forcefully terminate them after 6 minutes from the phase start
      maxDuration: 6m
      scenario: ...
# 2 minutes after the benchmark has started spawn 5 users constantly doing something for 2 minutes
- outOfBand:
    always:
      users: 5
      startTime: 2m
      duration: 2m
      scenario: ...
- final:
    atOnce:
      users: 1
      # Do something at the end: make sure that both rampUp and steadyState are terminated
      startAfterStrict:
      - rampUp
      - steadyState
      scenario: ...

These properties are common for all types of phases:

Property	Description
startTime	Time relative to benchmark start when this phase should be scheduled. In other words, it’s the earliest moment when it could be scheduled, other conditions (below) may delay that even further.
startAfter	Phases that must be finished before this phase can start. You can use either single phase name, list of phases or a reference to certain iteration.
startAfterStrict	Phases that must be terminated before this phase can start. Use the same syntax as for `startAfter`.
duration	Intended duration for the phase (must be defined but for the `atOnce` type). After this time elapses no new sessions will be started; there might be some running sessions still executing operations, though.
maxDuration	After this time elapses all sessions are forcefully terminated.
isWarmup	This marker property is propagated to results JSON and allows the reporter to hide some phases by default.
maxUnfinishedSessions	Maximum number of session that are allowed to be open when the phase finishes. When there are more open sessions all the other sessions are cancelled and the benchmark is terminated. Unlimited by default.
maxIterations	Maximum number of iterations this phase will be scaled to. More about that below.
scenario	The scenario this phase should execute.
forks	See forks section below.

Below are properties specific for different phase types:

atOnce:
- users: Number of users started at the start of the phase.
always:
- users: Number of users started at the start of the phase. When a user finishes it is immediatelly restarted (any pause must be part of the scenario).
constantRate:
- usersPerSec: Number of users started each second.
- variance: Randomize delays between starting users following the exponential distribution. That way the starting users behave as the Poisson point process. If this is set to false users will be started with uniform delays. Default is true.
- maxSessions: Number of preallocated sessions. This number is split between all agents/executors evenly.
increasingRate / decreasingRate:
- initialUsersPerSec: Rate of started users at the beginning of the phase.
- targetUsersPerSec: Rate of started users at the end of the phase.
- variance: Same as in `constantRate
- maxSessions: Same as in constantRate.

Hyperfoil initializes all phases before the benchmark starts, pre-allocating memory for sessions. In the open-model phases it’s not possible to know how many users will be active at the same moment (if the server experiences a 3-second hiccup and we have 100 new users per second this should be at least 300 as all the users will be blocked). However we need to provide the estimate for memory pre-allocation. If the estimate gets exceeded the benchmark won’t fail nor block new users from starting, but new sessions will be allocated which might negatively impact results accuracy.

Properties users, usersPerSec, initialUsersPerSec and targetUsersPerSec can be either a scalar number or scale with iterations using the base and increment components. You’ll see an example below.

Forks

As mentioned earlier, users in each phase execute the same scenario. Often it’s convenient to define the ramp-up and steady-state phases just once: the builders allow to declare such ‘sub-phases’ called forks. For all purposes but the benchmark configuration these become regular phases of the same type, duration and dependencies (startAfter, startAfterStrict) as the ‘parent’ phase but slice the users according to their weight:

...
phases:
- steadyState:
    constantRate:
      usersPerSec: 30
      duration: 5m
      forks:
        sellShares:
          # This phase will start 10 users per second
          weight: 1
          scenario: ...
        buyShares:
          # This phase will start 20 users per second
          weight: 2
          scenario: ...

These phases will be later identified as steadyState/sellShares and steadyState/buyShares. Other phases can still reference steadyState (without suffix) as the dependency: there will be a no-op phase steadyState that starts (becomes running) as soon as both the forks finish, finish immediately and terminate once both the forks terminate.

Iterations

In some types of tests it’s useful to repeat given phase with increasing load - we call this concept iterations. In the example below you can see that *usersPerSec are not scalar values; in first iteration the actual value is set to the base value but in each subsequent iteration the value is increased by increment.

...
phases:
- rampUp:
    increasingRate:
      # Create phases rampUp/000, rampUp/001 and rampUp/002
      maxIterations: 3
      # rampUp/000 will go from 1 to 100 users, rampUp will go from 101 to 200 users...
      initialUsersPerSec:
        base: 1
        increment: 100
      targetUsersPerSec:
        base: 100
        increment: 100
      # rampUp/001 will start after steadyState/000 finishes
      startAfter:
        phase: steadyState
        iteration: previous
      duration: 1m
      scenario: ...
- steadyState:
    constantRate:
      maxIterations: 3
      usersPerSec:
        base: 100
        increment: 100
      # steadyState/000 will start after rampUp/000 finishes
      startAfter:
        phase: rampUp
        iteration: same
      duration: 5m

Similar to forks, there will be a no-op phase rampUp that will start after all rampUp/xxx phases finish and terminate after these terminate. Also there’s an implicit dependency between consecutive iterations: subsequent iteration won’t start until previous iteration terminates.

The startAfter property in this example uses a relative reference to iteration in another phase. Each reference has these properties:

Iterations can be combined with forks as well - the result name would be e.g. steadyState/000/sellShares.

Note that the maxSessions parameter is not scaling in iterations: all iterations execute the same scenario, the execution does not overlap and therefore it is possible to share the pool of sessions. Therefore you should provide an estimate for the iteration spawning the highest load.

Staircase

Hyperfoil tries to make opinionated decisions, simplifying common types of benchmark setups. That’s why it offers a simplified syntax for the scenario where you:

ramp the load to a certain level
execute steady state for a while
ramp it up further
execute another steady state
repeat previous two steps over and over

This is called a staircase as the load increases in a shape of tilted stairs. Phases such benchmark should consist of are automatically created and linked together, using the same scenario/forks.

staircase as a top-level element in the benchmark is mutually exclusive to scenario and phases elements.

Here is a minimalistic example of such configuration:

name: simple benchmark
http:
  host: http://localhost:8080
staircase:
  initialRampUpDuration: 10s
  steadyStateDuration: 10s
  rampUpDuration: 5s
  initialUsersPerSec: 5
  incrementUsersPerSec: 1
  maxIterations: 3
  scenario:
  - test:
    - httpRequest:
        GET: /foo

This element uses these properties:

Property	Description
initialRampUpDuration	Duration of the very first phase. Default is no initial ramp-up.
initialUsersPerSec	Rate of users starting at the end of the initial ramp-up.
steadyStateDuration	Duration of each steady-state phase.
rampUpDuration	Duration of each but first ramp-up. Default are no ramp-ups.
incrementUsersPerSec	Increase in the rate of started users in each iteration.
maxIterations	Maximum number of steady-state iterations.
scenario	The scenario to be executed.
forks	The forks with different scenarios.

3.2.4 - Scenario

Defines the behavior and sequence of actions that virtual users (VU) perform during a benchmark execution

Scenario

Scenario is a set of sequences. The sequence is a block of sequentially executed steps. Contrary to steps in a sequence the sequences within a scenario do not need to be executed sequentially.

The scenario defines one or more initialSequences that are enabled from the beginning and other sequences that must be enabled by any of the previously executed sequences. To be more precise it is not the sequence that is enabled but a sequence instance as we can run a sequence multiple times in parallel (on different data). The initialSequences enable one instance of each of the referenced sequence.

The session keeps a currently executed step for each of the enabled sequence instances. The step can be blocked (e.g. waiting for a response to come). The session is looping through current steps in each of the enabled sequence instances and if the step is not blocked, it is executed. There’s no guaranteed order in which non-blocked steps from multiple enabled sequence instances will be executed.

Here is an example of scenario:

scenario:
  initialSequences:
  - login:
    - httpRequest:
        POST: /login
    # Enable instance of sequence 'wait5seconds'
    - next: wait5seconds
  sequences:
  - wait5seconds:
    - thinkTime:
        duration: 5s
    - next: logout
  - logout:
    - httpRequest:
        POST: /logout

While this generic approach is useful for complex scenarios with branching logic, simple sequential scenarios can use orderedSequences short-cut enabling sequences in given order:

scenario:
  orderedSequences:
  - login:
    - httpRequest:
        POST: /login
  - wait5seconds:
    - thinkTime:
        duration: 5s
  - logout:
    - httpRequest:
        POST: /logout

This syntax makes the first sequence (login in this case) an initial sequence, adds the subsequent sequences and as the last step of each but the last sequence appends a next step scheduling a new instance of the following sequence.

To make configuration even more concise you can omit the orderedSequences level and start defining the list of sequences under scenario right away:

scenario:
- login:
  - httpRequest:
      POST: /login
- wait5seconds:
  - thinkTime:
      duration: 5s
- logout:
  - httpRequest:
      POST: /logout

An exhaustive list of steps can be found in the steps reference.

3.2.5 - Variables

Data placeholders within sessions that hold values throughout the execution of a benchmark scenario

All but the simplest scenarios will use session variables. Hyperfoil sports steps that generate values into these variables (randomInt, randomItem, …), processors that write data from other sources to variables (store, array) and many places that read variables and use the values to perform some operations (httpRequest.path ) or alter control flow.

Hyperfoil uses different types of variables (slots in the session) for integer variables and generic objects (commonly strings). When a numeric value is received as a string (e.g. when parsing response headers) and you want to use it in a step that expects exclusively integral values you have to convert it explicitly, e.g. using the stringToInt action. Steps that read values to form a string can usually consume both types of variables, without any need for conversion.

Besides user-defined variables there are some read-only pseudo-variables that can be used in the scenario as if these were regular variables:

Variable	Type	Description
hyperfoil.agent.id	integer	Zero-based index of the agent node
hyperfoil.agents	integer	Number of agent nodes or 1 when running in in-VM mode (standalone or CLI)
hyperfoil.agent.thread.id	integer	Zero-based index of current executor thread within this agent.
hyperfoil.agent.thread s	integer	Number of executor threads running in this agent.
hyperfoil.global.thread.id	integer	Zero-based index of current executor thread across all agents (unique).
hyperfoil.global.threads	integer	Total number of executor threads on all agents.
hyperfoil.phase.name	object	Full name of the currently executed phase (possibly including fork and iteration number).
hyperfoil.phase.id	integer	Index of the currently executed phase.
hyperfoil.phase.iteration	integer	Iteration number of the currently executed phase.
hyperfoil.run.id	object	Identifier of the current run, e.g. `0123`.
hyperfoil.session.id	integer	Unique index of this virtual user (session). Note that in benchmarks with multiple phases the indices might not be zero-based.

String interpolation

Components that accept string values usually allow you to use a pattern - parts of the string can be replaced in runtime with the value from a session variable. A simple example of pattern would be The quick brown ${wild-animal} jumps over the lazy ${domestic-animal} - variables wild-animal and domestic-animal would get replaced with their respective values.

When you really want to use ${wild-animal} in a value for such component you should escape it with one more dollar sign: This $${variable} won't be replaced will be rendered into This ${variable} won't be replaced.

There are a few transformations that you can perform with a variable value while interpolating the pattern:

${urlencode:my-variable} will replace characters in the my-variable using URLEncoder.encode (using UTF-8 encoding).
${{ '{%05d' }}:my-number} and other formatter strings ending with d, o, x or X will convert an integer variable using Formatter.
${replace/<regexp>/<replacement>/<flags>:my-variable} perform Java regexp replacement on my-variable contents. Note that you can use any character after replace, not just / - this becomes the separator between regexp, replacement and flags. The only flags currently supported is g - replacing all occurences of that string (by default only first occurence is replaced).

Sequence-scoped access

When an array or collection is stored in a session variable you can access the individual elements by appending [.] to the variable name, e.g. my-variable[.]. You can see that we don’t use the actual index into the array: instead we use current sequence instance index. You can read more about running multiple sequences concurrently in the Architecture/Scenario Execution.

3.2.6 - Templates

Templates in Hyperfoil allow for efficient benchmark parametrization, enabling users to customize benchmarks based on specific execution environments or intended loads

It is often useful to keep a single benchmark in version control but change parts of it depending on the infrastructure where it is executed or intended load. Since version 0.18 Hyperfoil supports parametrization of the benchmark through templates.

Inspired by other (more complex) YAML templating systems we decided to use YAML tags to pre-process the YAML. Templating happens even before applying the YAML nodes onto BenchmarkBuilder, therefore it is not possible to do that programmatically or with the serialized form.

If you are working with CLI or WebCLI there is little difference to regular benchmarks: you upload and edit the benchmark as usual. However it is not possible to auto-detect files before the benchmark is constructed from the template (the reference to a file could be a template, too!), therefore you need to pass all files using option -f to the upload/edit command.

When the benchmark template is uploaded, upon running it (run mybenchmark) you either pass the parameters using option -P or you are interactively asked to provide those params. The parameters are stored in CLI context and on subsequent invocations of run you don’t need to set these. If you want to remove the parameters from the context use option -r/--reset-params. To see both default and current parameters you can use the inspect command.

Param

You should use !param to replace single scalar value:

1name: example
2http:
3  host: http://localhost:8080
4usersPerSec: !param NUM_USERS
5duration: !param DURATION 60s
6scenario: # ...

In this simple constant-rate benchmark you can customize the number of users starting each second as well as the duration. There’s no default for NUM_USERS; you will be asked to provide it when you run the benchmark. On the other hand DURATION has a default value of 60s - anything after the space after parameter name counts as the default value.

Parameters don’t have to be upper-case. The identifier is case-sensitive, though.

run scalar-value-example -PNUM_USERS=5 -PDURATION=60s

Concat

Sometimes you need to replace only part of a string: !concat will let you do that:

1name: example
2http:
3  host: !concat [ "http://", !param SERVER localhost, ":8080" ]
4usersPerSec: 10
5duration: 60s
6scenario: # ...

In this example we will customize the host with the concatenation of http://, parameter SERVER with default localhost and :8080. This example uses inline-form of list, though you can use the regular list (one item per line), too.

Foreach

Chances are you need to generate a list based on a param: you can do this using the !foreach:

 1name: example
 2http: !foreach
 3  items: http://example.com,http://hyperfoil.io
 4  separator: "," # comma is the default separator
 5  param: ITEM   # ITEM is the default parameter name
 6  do:
 7    host: !param ITEM
 8usersPerSec: 10
 9duration: 60s
10scenario: # ...

This splits the items using the separator regexp and produces a list of values or mappings while the param ITEM is set to one of the values from items list. The example above would result in:

name: example
http:
- host: http://example.com
- host: http://hyperfoil.io
usersPerSec: 10
duration: 60s
scenario: # ...

You can also set items to a YAML list; in that case the separator is not used:

myList: !foreach
  items: ["A", !param B, "C"]
  param: FOO
  do: !param FOO

The last example with -PB=bar would result in:

myList:
- A
- bar
- C

Renaming the param used for iteration can be useful in nested loops: without renaming the inner foreach would shadow the outer one.

Anchors and aliases

YAML has a built-in concept for removing repetitive sections: anchors and aliases. With the templating system you can use that universally throughout the file (in versions before 0.18 the support was limited to forks, scenarios and sequences):

foo: &hello-world
  hello: world
anotherFoo:
  sayHi: *hello-world
  myList:
  - *hello-world
  - bar

is interpretted as

foo:
  hello: world
anotherFoo:
  sayHi:
    hello: world
  myList:
  - hello: world
  - bar

3.2.7 - Hooks

Mechanisms that allow users to run specific scripts or commands automatically before and after executing a benchmark run

It might be useful to run certain scripts before and after the run, e.g. starting some infrastructure, preloading database, gathering CPU stats during the test and so on. That’s why Hyperfoil introduces pre- and post-hooks to the run.

Some scripts are not specific to the test being run - these should be deployed on controller as files in *root*/hooks/pre/ and *root*/hooks/post directories where root is controller’s root directory, /tmp/hyperfoil/ by default. Each of these directories should contain executable scripts or binaries that will be run in alphabetic order. We strongly suggest using the format 00-my-script.sh to set the order using first two digits.

Kubernetes/Openshift deployments use the same strategy; the only difference is that the pre and post directories are mapped as volumes from a ConfigMap resource.

Other scripts may be specific to the benchmark executed and therefore you can define them directly in the YAML files. You can either use inline command that will be executed using sh -c your-command --your-options or create a Java class implementing io.hyperfoil.core.hooks.RunHook and register it to be loaded as other Hyperfoil extensions.

name: my-benchmark
pre:
  01-inline: curl http://example.com
  02-custom:
    my-hook:
      foo: bar
post:
  99-some-final-hook: ...
...

The lists of hooks from controller directories and benchmark are merged; if there’s a conflict between two hooks from these two sources the final execution order is not defined (but both get executed).

In case of inline command execution the stderr output will stay on stderr, stdout will be caputered by Hyperfoil and stored in *rundir*/*XXXX*/hooks.json. As the post-hooks are executed after info.json and all.json get written the output cannot be included inside those files. This order of execution was chosen because it’s likely that you will upload these files to a database - yes, using a post-hook.

3.2.8 - Ergonomics

Configuration options that enhance usability and automation of benchmarking sessions

This section hosts only single property at this moment:

Property	Default	Description
repeatCookies	true	Automatically parse cookies from HTTP responses, store them in session and resend them with subsequent requests.
userAgentFromSession	true	Add user-agent header to each request, holding the agent name and session id.
autoRangeCheck	true	Mark 4xx and 5xx responses as invalid. You can also turn this off in each step.
stopOnInvalid	true	When the session receives an invalid response it does not execute any further steps, cancelling all requests and stopping immediately.
followRedirect	NEVER	Default value for httpRequest.handler.followRedirect.

3.3 - Examples

Collection of benchmark examples

If you haven’t checked the Getting started guide we strongly recommend going there first.

Below you’ll see commented examples of configuration; contrary to the Getting started guide these don’t present scenarios but rather list the various configuration options by example.

httpRequest

You will most likely use step httpRequest in each of your scenarios, and there’s many ways to send a request.

# This example should demonstrate various ways to configure one of the most important
# steps - the httpRequest.
name: http-requests
http:
  host: http://example.com
ergonomics: # Disable stopping the scenario on 4xx or 5xx response
  autoRangeCheck: false
usersPerSec: 1
duration: 1
scenario:
- jsonBody:
  - httpRequest:
      POST: /foo/bar
      # Hyperfoil doesn't know what's the content of the body string, if the server
      # requires correct content-type header you have to provide it yourselves
      headers:
        content-type: application/json
      # Here we specify a multi-line string. For more info about multiline strings,
      # compacting/chopping of newlines etc. please check out https://yaml-multiline.info/
      body: |
        {
           "foo" : "bar"
        }        
- formBody:
  - set: myVar <- foobar
  - httpRequest:
      POST: /myform
      # Here we don't need to add any headers as the form: knows that you're sending
      # a HTML form and it can add 'content-type: application/x-www-form-urlencoded'
      # automatically.
      body:
        # This will generate body 'foo=bar&bar=foobar&goo=foofoobarbar'. Any non-ascii
        # or otherwise illegal characters are correctly URL-encoded.
        form:
        - name: foo
          value: bar
        - name: bar
          fromVar: myVar
        - name: goo
          pattern: foo${myVar}bar
- bodyFromFile:
  - httpRequest:
      POST: /foo/bar
      body:
        # This simply loads the file and sends it as the body without any conversion.
        # It does not add any headers nor make it a multipart upload as the browser would do.
        fromFile: usernames.txt
- customHeaders:
  - set: token <- dGhpcyBpcyBhIG5pY2UgYW5kIHNlY3VyZSB0b2tlbgo=
  - set: etag <- "ETag received in some previous request"
  - httpRequest:
      GET: /secured/page
      # Note that HTTP headers are case-insensitive (use your preferred capitalization)
      headers:
        # Headers can be set inline
        accept: text/html
        # Session variables are replaced using the pattern syntax
        authorization: Bearer ${token}
        # Values from session variables can be also loaded using fromVar
        if-match:
          fromVar: etag
- nonDefaultMetric:
  - httpRequest:
      GET: /cats
      # By default the metric name equals to name of the sequence ('nonDefaultMetric' here).
      # We can override that either with a constant value...
      metric: mammals
  - randomItem:
      toVar: animal
      list:
      - cats
      - dogs
      - locusts
  - httpRequest:
      GET: /foo/${animal}
      # ... or a regexp switch on the actual authority+path (e.g. example.com:8080/foo/cats).
      # If the benchmark uses single (default) HTTP target the authority is omitted.
      metric:
      - .*cats -> mammals
      - .*dogs -> mammals
      - -> insects
- toughSLAs:
  - httpRequest:
      GET: /index.html
      handler:
        status:
          # Any request that is not responded with status code will be marked as invalid.
          range: 200
      # When you need only one SLA you can use mapping without the list (just forget the dash).
      sla:
      # This first SLA is evaluated when the phase completes from all requests that happened
      # during the phase.
      # Errors are connection failures, timeouts, 4xx and 5xx responses
      - errorRatio: 0.1
        # You can set custom criteria for what is considered valid/invalid as with the status
        # handler above. By default any response with status that is not within 200-399
        # is deemed invalid (as well as error).
        invalidRatio: 0.2
        blockedRatio: 0.0
        meanResponseTime: 10 ms
        # 90% requests should be under 100 ms, only 1% can be over 1 second
        limits:
          0.9: 100 ms
          0.99: 1 s
      # Following SLA is evaluated when all the statistics for the past second arrive,
      # accumulating results from the window (last 10 seconds). Therefore it can detect shorter
      # peaks of degraded performance.
      - window: 10s
        meanResponseTime: 50 ms

Some scenarios need to access multiple HTTP endpoints; following example shows an example configuration for that:

# This example manifests running a benchmark against multiple domains
name: more-servers
http:
- host: http://example.com
  sharedConnections: 100
- host: https://hyperfoil.io
  # With HTTPS, most modern servers will negotiate HTTP 2.0 as the application protocol.
  # Since HTTP 2.0 uses multiple streams over single TCP (TLS in this case) connection
  # you can usually set lower number of connections.
  sharedConnections: 10
# You may want to route requests through a proxy/load balancer or simply use a domain
# that is not resolvable. Configuration below will actually send the requests to addresses
# set below, but the requests will use in the 'host: foobar.com' in the headers
# and in SNI if this goes over TLS (HTTPS).
- host: http://foobar.com
  # Hyperfoil will split the connections evenly to the defined addresses
  # (an entry is considered single address for this purpose even if DNS registers
  # multiple IP addresses for the hostname).
  sharedConnections: 30
  addresses:
  # Both hostnames and IP addresses are allowed
  - proxy.my-locally-defined-domain.test
  - 192.168.1.10
  # You can set a custom port as well
  - 192.168.1.11:8080
usersPerSec: 1
duration: 1
scenario:
- test:
  - httpRequest:
      # Authority is the combination of hostname and port.
      authority: hyperfoil.io:443
      GET: /docs
  - randomItem:
      toVar: hostname
      list:
      - example.com
      - foobar.com
  - httpRequest:
      # The target must be configured in the 'http' section above; the correctness
      # is usually validated when parsing/building the benchmark but sometimes it is
      # only possible at runtime, potentially resulting in errors during execution.
      authority: ${hostname}:80
      GET: /foo

3.4 - Troubleshooting

Common technical issues that you could hit during benchmark development

It doesn’t work. Can you help me?

The first step to identifying any issue is getting a verbose log - setting logging level to TRACE. How exactly you do that depends on the way you deploy Hyperfoil:

If you use CLI and the start-local command, just run it as start-local -l TRACE which sets the default logging level. You’ll find the log in /tmp/hyperfoil.local.log by default.
If you run Hyperfoil manually in standalone mode (non-clustered) the agent will run in the same JVM as the controller. You need to add -Dlog4j.configurationFile=file:///path/to/log4j2-trace.xml option when starting standalone.sh. If you start Hyperfoil through Ansible the same is set using hyperfoil_log_config variable.
If you run Hyperfoil in clustered mode, the failing code is probably in the agents. You need to pass the logging settings to agents using the deployer; with SSH deployer you need to add -Dlog4j.configurationFile=file:///path/to/log4j2-trace.xml to the extras property, in Kubernetes/Openshift there is the log option that lets you set the logging configuration through a config-map.

An example of Log4j2 configuration file with TRACE logging on is here:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
   <Appenders>
      <Console name="CONSOLE" target="SYSTEM_OUT">
         <PatternLayout pattern="%d{HH:mm:ss,SSS} %-5p [%c] (%t) %m%n"/>
      </Console>
   </Appenders>
   <Loggers>
      <Root level="TRACE">
         <AppenderRef ref="CONSOLE"/>
      </Root>
   </Loggers>
</Configuration>

TRACE-level logging can be very verbose to a point where it will pose a bottleneck. It’s recommended to isolate your problem at lower request rate if that’s possible.

If you need to print variable values for debugging, check out log step.

My phase fails with SLA failure ‘Progress was blocked waiting for a free connection. Hint: increase http.sharedConnections.’

By default Hyperfoil uses single connection to each HTTP(s) host; the default is set so low to force you thinking about connection limits early during test development. If you don’t override this value as in:

http:
  host: http://hyperfoil.io
  sharedConnections: 1000

you get the error above, as the default SLA does not allow a session (virtual user) to be blocked due to not being able to acquire a connection from the connection pool immediatelly. If you can’t increase number of connections (or use HTTP2 that allows multiple requests to multiplex within single connection), you can set

- httpRequest:
    sla:
    - blockedRatio: 1000 # any value big enough

on each request to drop the default SLA. The blockedRatio value is a threshold ratio between time spent waiting for a free connection and waiting for the response.

You could also wonder why the sessions are missing a connection when the scenario should guarantee there’s always a free connection e.g. when using always phase type with same number of users and connections. However this may not hold when the connection is closed (either explicitly or after receiving a 5xx response) - while Hyperfoil starts replacing that connection immediatelly it takes a moment. If you expects connections to be closed add a few (10%) extra connections. Another reason could be poor balancing of connections and sessions to threads (should be gone in version 0.8).

When I set ‘Host’ header for HTTP request I get warnings

Hyperfoil automatically inserts the ‘Host’ header to each request and when you try to override that for certain request it emits a warning:

Setting `host` header explicitly is not recommended. Use the HTTP host and adjust actual target using `addresses` property.

With this warning on we don’t inject the header as it might be intended, e.g. when the target server does not parse headers in a case-sensitive way (as it should!) and you have to use certain case. However, if you want to run your requests to a different IP than the host resolves to (e.g. hit 127.0.0.1:8080 with Host: example.com) you should rather use

http:
  host: http://hyperfoil.io
  addresses:
  - 127.0.0.1:8080

When I use a session variable I am seeing the error “Variable foo is not set!”

Errors:
in-vm: Variable foo is not set!

On occasion a scenario step has been seen to execute out of sequence. To ensure the variable is set beforehand use initialSequences with the step that populates the variable.

4 - How To

A collection of practical advices for common things in Hyperfoil

This section contains practical advices for common things you could want to use in a benchmark.

4.1 - Benchmark schema support

How to make your life easier while building your own benchmarks

For your convenience we recommend using editor with YAML validation against JSON schema; you can point your editor to docs/schema.json. We can recommend Visual Studio Code with redhat.vscode-yaml plugin.

You need to edit settings file to map benchmark configuration files (with .hf.yaml extension) to the schema, adding:

"yaml.schemas" : {
    "file:///path/to/hyperfoil-distribution/docs/schema.json" : "/*.hf.yaml"
},

Note that you can also directly point to the hosted JSON schema definition:

"yaml.schemas" : {
    "https://hyperfoil.io/schema.json" : "/*.hf.yaml"
},

4.2 - Using credentials

Hyperfoil benchmarks can refer to external files. When you use the upload command in CLI the files are automatically attached the the benchmark YAML (relative paths are resolved relative to this file). Later on when editing the file you can choose to re-upload some of these. Once the benchmark is built the files are loaded to the in-memory representation - Hyperfoil won’t access these files during runtime. With clustered benchmarks these files don’t need to be on the agents either - the controller sends serialized in-memory representation to the agents and that contains everything needed for the actual execution.

When testing a workload you will likely skip user registration and come with a list of username+password keys. A convenient way is to keep these in a CSV file that looks like

"johny","my-5ec12eT_pwd!"
"bob","superlongpassphrasethatnobodywillguess"

We will create a step that selects a random line from such file and stores it in session variables username and password:

- randomCsvRow:
    file: credentials.csv # Path relative to the benchmark
    columns:
      0: username
      1: password

The columns property has a mapping of zero-based indices of columns in the CSV file. This way you can use a file with some extra information.

In case you don’t need to split lines into separate variables you can use the randomItem:

- randomItem:
    file: usernames.txt
    toVar: username

We have selected the credentials, now it’s time to execute the login. When users submit a form on HTML page this usually results in a POST request with content-type: application/x-www-form-urlencoded header and the form contents encoded in the request body. Hyperfoil lets you specify this conveniently using the form body formatter:

- httpRequest:
    POST: /login
    body:
      form:
        - name: username
          fromVar: username
        - name: password
          fromVar: password
        - name: action
          value: log-me-in-please

Successful response from the server carries a token in most cases but the actual method can vary. If the server sets a cookie Hyperfoil automatically records this and sends it on subsequent requests (this can be switched off using ergonomics.repeatCookies). If the server sends the token in a response header, e.g. x-token you can store it using the header handler into the token session variable:

- httpRequest:
    # ...
    handler:
      header:
        filter:
          header: "x-token"
          processor:
            store: token

If the server returns this token as a part of JSON object - e.g { "token": "abc123" } you would process response body using the json processor in body handler:

- httpRequest:
    # ...
    handler:
      body:
        json:
          query: ".token"
          toVar: token

Runnable example

Below you can find a runnable example that we have prepared for you and that you can download here.

name: credentials
http:
- host: https://localhost:8084
phases:
- use-cookie:
    atOnce:
      users: 1
      scenario:
      # Make sure that without the cookie (before login) the request fails with 401
      - before-login:
        - httpRequest:
            GET: /howto/credentials/secure
            handler:
              autoRangeCheck: false # Don't fail with 4xx-5xx
              status:
                range: 401
      - login-with-form:
        - randomCsvRow:
            file: credentials.csv
            columns:
              0: username
              1: password
        - httpRequest:
            POST: /howto/credentials/login
            body:
              form:
              - name: username
                fromVar: username
              - name: password
                fromVar: password
      # Here we already have the cookie set to a httpRequest will succeed
      - after-login:
        - httpRequest:
            GET: /howto/credentials/secure
- use-bearer-token:
    atOnce:
      users: 1
      scenario:
      - request-login-page:
        # This first request will response with WWW-Authenticate header
        - httpRequest:
            GET: /howto/credentials/login
            handler:
              autoRangeCheck: false
              status:
                range: 401
      - login-with-basic-auth:
        - randomCsvRow:
            file: credentials.csv
            columns:
              0: username
              1: password
        - template:
            pattern: ${username}:${password}
            toVar: concatenated
        - httpRequest:
            GET: /howto/credentials/login
            headers:
              # Our example runs over HTTP2 and that mandates lower-case header names
              authorization: Basic ${base64encode:concatenated}
            # With Authorization header the server will reply with token
            # and redirect us to the secured page. `httpRequest` implements
            # automatic redirection through `handler.followRedirect` but that
            # wouldn't send the token, so we'll do that manually
            handler:
              header:
              - filter:
                  header: x-token
                  processor:
                  - store: token
              status:
                range: 30x
      - after-login:
        - httpRequest:
            GET: /howto/credentials/secure
            headers:
              authorization: Bearer ${token}

There are two scenarios: use-cookie performs the login using the POST from form as shown above and then keeps the secret token in a cookie; use-bearer-token performs HTTP Basic authentication, receives a token in headers and then uses that for HTTP Bearer authentication.

You should start with running a server with mocked responses in one console:

podman run --rm -p 8084:8084 quay.io/hyperfoil/hyperfoil-examples

In second console start the CLI with in-vm controller (or standalone and open the WebCLI in your browser). We are running in host-network mode to be able to reach localhost from within the container.

podman run --rm -it --net host quay.io/hyperfoil/hyperfoil cli

[hyperfoil]$ start-local
Starting controller in default directory (/tmp/hyperfoil)
Controller started, listening on 127.0.0.1:35243
Connecting to the controller...
Connected to 127.0.0.1:35243!

[hyperfoil@in-vm]$ upload https://hyperfoil.io/benchmarks/credentials.hf.yaml
Loaded benchmark credentials, uploading...
... done.

[hyperfoil@in-vm]$ run
Started run 0000
Run 0000, benchmark credentials
Agents: in-vm[STOPPED]
Started: 2022/10/27 15:48:39.271    Terminated: 2022/10/27 15:48:39.301
NAME              STATUS      STARTED       REMAINING  COMPLETED     TOTAL DURATION             DESCRIPTION</span>
use-bearer-token  TERMINATED  15:48:39.271             15:48:39.301  30 ms (exceeded by 31 ms)  1 users at once
use-cookie        TERMINATED  15:48:39.271             15:48:39.300  29 ms (exceeded by 30 ms)  1 users at once

[hyperfoil@in-vm]$ stats
Total stats from run 0000

PHASE             METRIC                 THROUGHPUT   REQUESTS  MEAN     p50      p90      p99      p99.9    p99.99   TIMEOUTS  ERRORS  BLOCKED  2xx  3xx  4xx  5xx  CACHE</span>
use-bearer-token  after-login            33.33 req/s         1  3.79 ms  3.80 ms  3.80 ms  3.80 ms  3.80 ms  3.80 ms         0       0     0 ns    1    0    0    0      0
use-bearer-token  login-with-basic-auth  33.33 req/s         1  4.57 ms  4.59 ms  4.59 ms  4.59 ms  4.59 ms  4.59 ms         0       0     0 ns    0    1    0    0      0
use-bearer-token  request-login-page     33.33 req/s         1  7.49 ms  7.50 ms  7.50 ms  7.50 ms  7.50 ms  7.50 ms         0       0     0 ns    0    0    1    0      0
use-cookie        after-login            34.48 req/s         1  3.69 ms  3.70 ms  3.70 ms  3.70 ms  3.70 ms  3.70 ms         0       0     0 ns    1    0    0    0      0
use-cookie        before-login           34.48 req/s         1  9.60 ms  9.63 ms  9.63 ms  9.63 ms  9.63 ms  9.63 ms         0       0     0 ns    0    0    1    0      0
use-cookie        login-with-form        34.48 req/s         1  4.96 ms  4.98 ms  4.98 ms  4.98 ms  4.98 ms  4.98 ms         0       0     0 ns    1    0    0    0      0

The list of possibilities is endless; if your use case does not fit any of the above please check out the reference. You can also have a look at a full example in the second part of the Beginner’s Guide.

5 - Migration

Set of guides to migrate from other tools to Hyperfoil

This section includes some guides to make the migration from other benchmarking tools easier.

5.1 - Migrating from wrk/wrk2

How and why should I migrate from wrk/wrk2 to Hyperfoil?

Both Will Glozer’s wrk and Gil Tene’s wrk2 are great tools but maybe you’ve realized that you need more functionalities, need to hit more endpoints in parallel or simply have to scale horizontally to more nodes. Hyperfoil offers an adapter that tries to mimic the behaviour of these load drivers. This guide will show how to use these and translate the test into a full-fledged Hyperfoil benchmark.

Warning

Please make sure that you are using Hyperfoil version >= 0.22

Let’s start with this command:

./wrk -c 10 -t 2 -d 10s -H 'accept: application/json' http://example.com

that will produce something like:

Running 10s test @ http://example.com
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    97.64ms  489.01us 101.11ms   73.66%
    Req/Sec    50.73      4.81    80.00     91.92%
  1010 requests in 10.01s, 1.53MB read
Requests/sec:    100.85
Transfer/sec:    156.91KB

You can do exactly the same thing with Hyperfoil, either using bin/wrk.sh/bin/wrk2.sh or using container:

podman run --rm quay.io/hyperfoil/hyperfoil wrk -c 10 -t 2 -d 10s  -H 'accept: application/json' http://example.com/

that will generate something like:

Running 10s test @ http://example.com/
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    98.61ms    9.58ms 402.65ms   98.92%
    Req/Sec   101.70      3.35   110.00     80.00
  1017 requests in 10.023s,   1.59MB read
Requests/sec: 101.47
Transfer/sec: 162.84kB

Not much of a difference as you can see. Note that if you want to test something in localhost you’d need to use host networking (--net host). You could also run it from CLI using the wrk/wrk2 command. In that case you’d either connect to a controller using the connect command or start a controller inside the CLI with start-local command. When the (remote) controller is clustered you can use -A/--agent option (e.g. -Amy-agent=host=my-server.my-company.com,port=22) to drive the load from a different node.

Unlike original tools we support HTTP2 - this is disabled by default and you need to pass --enable-http2 to allow it explicitly.

When you run the command from the CLI the benchmark stays in controller, so you can have a look on the statistics the same way as with any other Hyperfoil run (stats, report, compare, …). You can also execute another run of the benchmark and observe the status as it progresses:

podman run -it --rm quay.io/hyperfoil/hyperfoil cli

[hyperfoil]$ start-local --quiet
[hyperfoil@in-vm]$ wrk -t 2 -c 10 -H 'accept: application/json' -d 10s http://example.com
Running 10s test @ http://example.com
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    97.99ms  618.36?s 101.19ms   62.63%
    Req/Sec   102.50      2.94   110.00     90.00
  1025 requests in 10.006s,   1.61MB read
Requests/sec: 102.44
Transfer/sec: 164.41kB

[hyperfoil@in-vm]$ stats
Total stats from run 0000
Extensions (use -e to show): transfer
PHASE        METRIC   THROUGHPUT    REQUESTS  MEAN      p50       p90        p99        p99.9      p99.99     TIMEOUTS  ERRORS  BLOCKED  2xx   3xx  4xx  5xx  CACHE
calibration  request   96.58 req/s       608  99.39 ms  98.57 ms  100.14 ms  117.96 ms  406.85 ms  406.85 ms         0       0     0 ns   608    0    0    0      0
test         request  101.44 req/s      1025  97.99 ms  98.04 ms   99.09 ms  100.14 ms  101.19 ms  101.19 ms         0       0     0 ns  1015    0    0    0      0

[hyperfoil@in-vm]$ run wrk
Started run 0001
Run 0001, benchmark wrk
Agents: in-vm[READY]
Started: 2022/09/08 17:59:05.283
NAME  STATUS   STARTED       REMAINING      COMPLETED  TOTAL DURATION  DESCRIPTION
test  RUNNING  17:59:11.372  88 ms (88 ms)                             10 users always

Under the hood the command creates a benchmark that should reflect what the original wrk/wrk2 implementation does. These do not have a YAML representation so you cannot view it with info/edit but you can use inspect to dig deep into the programmatic representation. The source YAML for the wrk invocation above would look like this:

name: wrk
threads: 2 # option -t
http:
  host: http://example.com
  allowHttp2: false
  sharedConnections: 10 # option -c
ergonomics:
  repeatCookies: false
  userAgentFromSession: false
phases:
- calibration:
    always:
      users: 10 # option -c
      duration: 6s
      maxDuration: 70s # This is duration + default timeout 60s
      scenario: &scenario
      - request:
        - httpRequest:
            GET: /
            timeout: 60s # option --timeout
            headers:
            - accept: application/json # option -H
            handler:
              rawBytes: ... # Handler to record the data for 'Transfer/sec'
- test:
    always:
      users: 10 # option -c
      duration: 10s # option -d
      maxDuration: 10s # option -d
      startAfterStrict: calibration
      scenario: *scenario

As you can see there’s a ‘calibration’ phase - wrk implementations might use it for different purposes but in our case it’s convenient at least for a basic JVM warmup. The ’test’ phase starts only when all requests from calibration phase complete. The scenario and number of users is identical, though.

With wrk2 we use open model, you have to set request rate using the -R option - with -R 20; the phases would look like:

phases:
- calibration:
    constantRate:
      usersPerSec: 20
      variance: false
      maxSessions: 300 # request rate x 15
      duration: 10s # option -d
      # ...

Hopefully this gives you some headstart and you can get familiar with Hyperfoil before diving into the details of benchmark syntax.

6 - Reference

List of all possible steps and handlers that can be used for the benchmark development

Before exploring this reference you should be familiar with the basic structure of a benchmark. If you’re not sure what is the difference between the phase, scenario and sequence check out the concepts in user guide.

This reference lists all the steps and handlers used in a scenario. The lists below are not finite; you can also easily develop and use your own components, but Hyperfoil provides generic components for the most common tasks out of the box.

There are few generic types of components:

This documentation is auto-generated from Javadoc in source code, explaining format for each key-value pair in benchmark YAML. If there is an issue with these docs (e.g. property showing no description) please file an issue on GitHub.

This is the basic structure of the docs:

EXAMPLE

Example description.

Property	Type	Description
Key	Class	Explanation for the value

YAML syntax

EXAMPLE:
  Key: Value

For example, the POST definition in httpRequest step looks like this:

POST

Generic builder for generating a string.

Property	Type	Description
fromVar	String	Load the string from session variable.
pattern	String	Use pattern replacing session variables.
value	String	String value used verbatim.

YAML syntax

POST:
  pattern: /user/${userId}/info

You might be wondering why the documentation above does not mention anything about issuing a HTTP request. In fact the top-level POST property httpRequest says “Issue HTTP POST request to given path.” but the POST() method returns a generic string builder; this generic builder is used as the path for the HTTP request with POST method.

If the ’type’ is not a scalar value, the key in ‘property’ works as a link to further property mapping. It’s also possible that the property has multiple options, e.g. accepting both property mapping and list of values.

For brevity some components have inline definition like this:

SET

Set variable in session to certain value.

Inline definition
Use `var <- value`.

YAML syntax

set: myVar <- "This is my value"

Hyperfoil defines automatically generated JSON schema for the benchmark; you can use that in Visual Studio Code to automatically check the YAML syntax.

6.1 - Steps

Steps are the basic building blocks that form each sequence of a scenario, similar to statements in a programming language. Steps are potentially blocking (the sequence cannot continue with next step until previous one finishes).

Note that every action can be also used as a step that simply never blocks, as actions do not require any extra input.

6.1.1 - addItem

Appends value to an array stored in another variable.

Property	Type	Description
fromVar	String	Fetch value from session variable.
toVar	String	Destination variable with the array.
value	String	Verbatim value.

6.1.2 - addToInt

Add value to integer variable in the session.

Inline definition
Accepting one of: `var++`, `var–`, `var += value`, `var -= value`.

Property	Type	Description
orElseSetTo	int	If the variable is currently not set, set it to this value instead of addition.
value	int	Value added (can be negative).
var	String	Variable name.

6.1.3 - addToSharedCounter

Adds value to a counter shared by all sessions in the same executor.

Inline definition
Use on of: `counter++`, `counter–`, `counter += <value>`,
`counter -= <value>`

Property	Type	Description
fromVar	String	Input variable name.
key	String	Identifier for the counter.
operator	enum	Operation to perform on the counter. Default is `ADD`. Options: `ADD` `SUBTRACT`
value	int	Value (integer).

6.1.4 - awaitAllResponses

Block current sequence until all requests receive the response.

6.1.5 - awaitDelay

Block this sequence until referenced delay point.

Inline definition
Delay point created in `scheduleDelay.key`.

Property	Type	Description
key	String	Delay point created in `scheduleDelay.key`.

6.1.6 - awaitInt

Block current sequence until condition becomes true.

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.
var	String	Variable name storing the compared value.

equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.1.7 - awaitVar

Block current sequence until this variable gets set/unset.

Inline definition
Variable name or `!variable` if we are waiting for it to be unset.

Property	Type	Description
var	String	Variable name or `!variable` if we are waiting for it to be unset.

6.1.8 - breakIfFinished

Stop execution of current sequence if the phase is in finished state.

This is useful for a long-running (looping) sequence that should not extend the duration of its phase.

6.1.9 - breakSequence

Prematurely stops execution of this sequence if the condition is satisfied.

Property	Type	Description
allConditions	Builder	Condition combining multiple other conditions with ‘AND’ logic.
boolCondition	Builder	Condition comparing boolean variables.
dependency	String	This step is blocked if this variable does not have set value (none by default).
intCondition	Builder	Condition comparing integer variables.
onBreak	Action.Builder	Action performed when the condition is true and the sequence is to be ended.
stringCondition	Builder	Condition comparing string variables.

allConditions

Test more conditions and combine the results using AND logic.

Property	Type	Description
<list of mappings>	<list of builders>	List of conditions.

allConditions.<list of mappings>

Selector for condition type.

Property	Type	Description
allConditions	Builder	Condition combining multiple other conditions with ‘AND’ logic.
boolCondition	Builder	Condition comparing boolean variables.
intCondition	Builder	Condition comparing integer variables.
stringCondition	Builder	Condition comparing string variables.

allConditions.<list of mappings>.allConditions

Test more conditions and combine the results using AND logic.

Property	Type	Description
<list of mappings>	<list of builders>	List of conditions.

allConditions.<list of mappings>.boolCondition

Tests session variable containing boolean value.

Property	Type	Description
fromVar	String	Variable name.
value	boolean	Expected value.

allConditions.<list of mappings>.intCondition

Condition comparing integer in session variable.

Inline definition
Parses condition in the form <variable> <operator> <value>
where operator is one of: `==`, `!=`,
`<>` (the same as `!=`),
`>=`, `>`, `<=`, `<`.

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
fromVar	Object	Variable name.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

allConditions.<list of mappings>.stringCondition

Condition comparing string in session variable.

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
fromVar	Object	Variable name.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

boolCondition

Tests session variable containing boolean value.

Property	Type	Description
fromVar	String	Variable name.
value	boolean	Expected value.

intCondition

Condition comparing integer in session variable.

Inline definition
Parses condition in the form <variable> <operator> <value>
where operator is one of: `==`, `!=`,
`<>` (the same as `!=`),
`>=`, `>`, `<=`, `<`.

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
fromVar	Object	Variable name.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

intCondition.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition

Condition comparing string in session variable.

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
fromVar	Object	Variable name.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

stringCondition.length

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

stringCondition.length.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.1.10 - clearHttpCache

Drops all entries from HTTP cache in the session.

6.1.11 - conditional

Perform an action or sequence of actions conditionally.

Property	Type	Description
actions	Action.Builder	Actions that should be executed should the condition hold.
allConditions	Builder	Condition combining multiple other conditions with ‘AND’ logic.
boolCondition	Builder	Condition comparing boolean variables.
intCondition	Builder	Condition comparing integer variables.
stringCondition	Builder	Condition comparing string variables.

allConditions

Test more conditions and combine the results using AND logic.

Property	Type	Description
<list of mappings>	<list of builders>	List of conditions.

allConditions.<list of mappings>

Selector for condition type.

Property	Type	Description
allConditions	Builder	Condition combining multiple other conditions with ‘AND’ logic.
boolCondition	Builder	Condition comparing boolean variables.
intCondition	Builder	Condition comparing integer variables.
stringCondition	Builder	Condition comparing string variables.

allConditions.<list of mappings>.allConditions

Test more conditions and combine the results using AND logic.

Property	Type	Description
<list of mappings>	<list of builders>	List of conditions.

allConditions.<list of mappings>.boolCondition

Tests session variable containing boolean value.

Property	Type	Description
fromVar	String	Variable name.
value	boolean	Expected value.

allConditions.<list of mappings>.intCondition

Condition comparing integer in session variable.

Inline definition
Parses condition in the form <variable> <operator> <value>
where operator is one of: `==`, `!=`,
`<>` (the same as `!=`),
`>=`, `>`, `<=`, `<`.

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
fromVar	Object	Variable name.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

allConditions.<list of mappings>.stringCondition

Condition comparing string in session variable.

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
fromVar	Object	Variable name.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

boolCondition

Tests session variable containing boolean value.

Property	Type	Description
fromVar	String	Variable name.
value	boolean	Expected value.

intCondition

Condition comparing integer in session variable.

Inline definition
Parses condition in the form <variable> <operator> <value>
where operator is one of: `==`, `!=`,
`<>` (the same as `!=`),
`>=`, `>`, `<=`, `<`.

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
fromVar	Object	Variable name.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

intCondition.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition

Condition comparing string in session variable.

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
fromVar	Object	Variable name.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

stringCondition.length

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

stringCondition.length.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.1.12 - fail

Fail the phase with given message. Used mostly for testing/debugging.

Inline definition
Message to fail with (unconditionally).

Property	Type	Description
allConditions	Builder	Condition combining multiple other conditions with ‘AND’ logic.
boolCondition	Builder	Condition comparing boolean variables.
intCondition	Builder	Condition comparing integer variables.
message	String	Message attached to the failure exception.
stringCondition	Builder	Condition comparing string variables.

allConditions

Test more conditions and combine the results using AND logic.

Property	Type	Description
<list of mappings>	<list of builders>	List of conditions.

allConditions.<list of mappings>

Selector for condition type.

Property	Type	Description
allConditions	Builder	Condition combining multiple other conditions with ‘AND’ logic.
boolCondition	Builder	Condition comparing boolean variables.
intCondition	Builder	Condition comparing integer variables.
stringCondition	Builder	Condition comparing string variables.

allConditions.<list of mappings>.allConditions

Test more conditions and combine the results using AND logic.

Property	Type	Description
<list of mappings>	<list of builders>	List of conditions.

allConditions.<list of mappings>.boolCondition

Tests session variable containing boolean value.

Property	Type	Description
fromVar	String	Variable name.
value	boolean	Expected value.

allConditions.<list of mappings>.intCondition

Condition comparing integer in session variable.

Inline definition
Parses condition in the form <variable> <operator> <value>
where operator is one of: `==`, `!=`,
`<>` (the same as `!=`),
`>=`, `>`, `<=`, `<`.

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
fromVar	Object	Variable name.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

allConditions.<list of mappings>.stringCondition

Condition comparing string in session variable.

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
fromVar	Object	Variable name.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

boolCondition

Tests session variable containing boolean value.

Property	Type	Description
fromVar	String	Variable name.
value	boolean	Expected value.

intCondition

Condition comparing integer in session variable.

Inline definition
Parses condition in the form <variable> <operator> <value>
where operator is one of: `==`, `!=`,
`<>` (the same as `!=`),
`>=`, `>`, `<=`, `<`.

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
fromVar	Object	Variable name.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

intCondition.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition

Condition comparing string in session variable.

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
fromVar	Object	Variable name.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

stringCondition.length

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

stringCondition.length.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.1.13 - foreach

Instantiate new sequences based on array variable content.

Property	Type	Description
counterVar	String	Variable to be set to the number of created sequences (optional).
dependency	String	This step is blocked if this variable does not have set value (none by default).
fromVar	String	Variable holding the array.
sequence	String	Name of the instantiated sequence.

6.1.14 - getIndex

Lookup index of an item in an array/collection.

Property	Type	Description
collection	String	Variable to fetch the collection from.
item	Builder	Item that should be looked up (using equality).
toVar	String	Integer variable to store the index, or -1 if the item is not found.

item

Inline definition
Verbatim value.

Property	Type	Description
fromVar	String	Fetch value from session variable.
value	String	Verbatim value.

6.1.15 - getItem

Retrieves n-th item from an array or collection.

Property	Type	Description
fromVar	String	Source variable with an array or list.
index	Builder	Source for index into the array/list.
toVar	String	Destination variable for the n-th element.

index

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.1.16 - getSharedCounter

Retrieves value from a counter shared by all sessions in the same executor and stores that in a session variable.

Retrieves value from a counter shared by all sessions in the same executor and stores that in a session variable.
If the value exceeds allowed integer range (-2^31 .. 2^31 - 1) it is capped.

Inline definition
Both the key and variable name.

Property	Type	Description
key	String	Identifier for the counter.
toVar	String	Session variable for storing the value.

6.1.17 - getSize

Calculates size of an array/collection held in variable into another variable

Property	Type	Description
boolFilter	Builder	Count only items matching the condition.
fromVar	String	Variable holding the collection.
intFilter	Builder	Count only items matching the condition.
stringFilter	Builder	Count only items matching the condition.
toVar	String	Variable storing the size.
undefinedValue	int	Value to use when `fromVar` is unset or it does not contain any array/collection.

boolFilter

Property	Type	Description
value	boolean	Expected value.

intFilter

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

intFilter.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intFilter.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intFilter.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intFilter.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intFilter.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intFilter.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringFilter

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

stringFilter.length

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

stringFilter.length.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringFilter.length.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringFilter.length.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringFilter.length.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringFilter.length.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringFilter.length.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.1.18 - hotrodRequest

Issues a HotRod request and registers handlers for the response.

Property	Type	Description
cacheName	String	Name of the cache used for the operation. This can be a pattern.
cacheName (alternative)	Builder	<no description>
get	String	Get specified entry in the remote cache.
key	String	Key used for the operation. This can be a pattern.
key (alternative)	Builder	<no description>
metric	String	Requests statistics will use this metric name.
metric (alternative)	<list of strings>	Allows categorizing request statistics into metrics based on the request path.
operation	enum	Options: `PUT`Adds or overrides each specified entry in the remote cache. `GET`Get specified entry in the remote cache.
put	String	Adds or overrides each specified entry in the remote cache.
value	String	Value for the operation. This can be a pattern.
value (alternative)	Builder	<no description>

cacheName

Generic builder for generating a string.

Inline definition
A pattern for string interpolation.

Property	Type	Description
fromVar	Object	Load the string from session variable.
pattern	String	Use pattern replacing session variables.
value	String	String value used verbatim.

key

Generic builder for generating a string.

Inline definition
A pattern for string interpolation.

Property	Type	Description
fromVar	Object	Load the string from session variable.
pattern	String	Use pattern replacing session variables.
value	String	String value used verbatim.

metric

Allows categorizing request statistics into metrics based on the request path.

Property	Type	Description
<list of strings>	<list of strings>	Allows categorizing request statistics into metrics based on the request path. The expressions are evaluated in the order as provided in the list. Use one of: `regexp -> replacement`, e.g. `([^?])(?.)? -> $1` to drop the query part. `regexp` (don’t do any replaces and use the full path), e.g. `.*.jpg` `-> name` (metric applied if none of the previous expressions match).

value

Generic builder for generating a string.

Inline definition
A pattern for string interpolation.

Property	Type	Description
fromVar	Object	Load the string from session variable.
pattern	String	Use pattern replacing session variables.
value	String	String value used verbatim.

6.1.19 - httpRequest

Issues a HTTP request and registers handlers for the response.

Property	Type	Description
authority	String	HTTP authority (host:port) this request should target. Must match one of the entries in `http` section. The string can use string interpolation.
authority (alternative)	Builder	HTTP authority (host:port) this request should target. Must match one of the entries in `http` section.
body	Builder	HTTP request body.
body (alternative)	String	HTTP request body (possibly a pattern).
compensation	Builder	Configures additional metric compensated for coordinated omission.
compression	Builder	Configure response compression.
compression (alternative)	String	Request server to respond with compressed entity using specified content encoding.
CONNECT	String	Issue HTTP CONNECT request to given path. This can be a pattern.
CONNECT (alternative)	Builder	Issue HTTP CONNECT request to given path.
DELETE	String	Issue HTTP DELETE request to given path. This can be a pattern.
DELETE (alternative)	Builder	Issue HTTP DELETE request to given path.
endpoint	Builder	HTTP endpoint this request should target. Must match to the `name` of the entries in `http` section.
GET	String	Issue HTTP GET request to given path. This can be a pattern.
GET (alternative)	Builder	Issue HTTP GET request to given path.
handler	Builder	HTTP response handlers.
HEAD	String	Issue HTTP HEAD request to given path. This can be a pattern.
HEAD (alternative)	Builder	Issue HTTP HEAD request to given path.
headers	Builder	HTTP headers sent in the request.
method	enum	HTTP method used for the request. Options: `GET` `HEAD` `POST` `PUT` `DELETE` `OPTIONS` `PATCH` `TRACE` `CONNECT`
metric	String	Requests statistics will use this metric name.
metric (alternative)	<list of strings>	Allows categorizing request statistics into metrics based on the request path.
OPTIONS	String	Issue HTTP OPTIONS request to given path. This can be a pattern.
OPTIONS (alternative)	Builder	Issue HTTP OPTIONS request to given path.
PATCH	String	Issue HTTP PATCH request to given path. This can be a pattern.
PATCH (alternative)	Builder	Issue HTTP PATCH request to given path.
path	String	HTTP path (absolute or relative), including query and fragment. The string can use string interpolation.
path (alternative)	Builder	HTTP path (absolute or relative), including query and fragment.
POST	String	Issue HTTP POST request to given path. This can be a pattern.
POST (alternative)	Builder	Issue HTTP POST request to given path.
PUT	String	Issue HTTP PUT request to given path. This can be a pattern.
PUT (alternative)	Builder	Issue HTTP PUT request to given path.
sla	Builder	List of SLAs the requests are subject to.
sync	boolean	This request is synchronous; execution of the sequence does not continue until the full response is received. If this step is executed from multiple parallel instances of this sequence the progress of all sequences is blocked until there is a request in flight without response. Default is `true`.
timeout	String	Request timeout - after this time the request will be marked as failed and connection will be closed. Defaults to value set globally in `http` section.
TRACE	String	Issue HTTP TRACE request to given path. This can be a pattern.
TRACE (alternative)	Builder	Issue HTTP TRACE request to given path.

authority

Generic builder for generating a string.

Inline definition
A pattern for string interpolation.

Property	Type	Description
fromVar	Object	Load the string from session variable.
pattern	String	Use pattern replacing session variables.
value	String	String value used verbatim.

body

Allows building HTTP request body from session variables.

Property	Type	Description
form	Builder	Build form as if we were sending the request using HTML form. This option automatically adds `Content-Type: application/x-www-form-urlencoded` to the request headers.
fromFile	String	Send contents of the file. Note that this method does NOT set content-type automatically.
fromVar	String	Use variable content as request body.
pattern	String	Pattern replacing `${sessionvar}` with variable contents in a string.
text	String	String sent as-is.

body.form

Property	Type	Description
<list of mappings>	<list of builders>	Add input pair described in the mapping.

body.form.<list of mappings>

Form element (e.g. as if coming from an INPUT field).

Property	Type	Description
fromVar	String	Input field value from session variable.
name	String	Input field name.
pattern	String	Input field value replacing session variables in a pattern, e.g. `foo${myvariable}var`
value	String	Input field value (verbatim).

compensation

Property	Type	Description
metric	String	Metric name for the compensated results.
metric (alternative)	<list of strings>	Configure a custom metric for the compensated results.
targetRate	double	Desired rate of new virtual users per second. This is similar to `constantRate.usersPerSec` phase settings but works closer to legacy benchmark drivers by fixing the concurrency.
targetRate (alternative)	Builder	Desired rate of new virtual users per second. This is similar to `constantRate.usersPerSec` phase settings but works closer to legacy benchmark drivers by fixing the concurrency.

compensation.metric

Configure a custom metric for the compensated results.

Property	Type	Description
<list of strings>	<list of strings>	Allows categorizing request statistics into metrics based on the request path. The expressions are evaluated in the order as provided in the list. Use one of: `regexp -> replacement`, e.g. `([^?])(?.)? -> $1` to drop the query part. `regexp` (don’t do any replaces and use the full path), e.g. `.*.jpg` `-> name` (metric applied if none of the previous expressions match).

compensation.targetRate

Property	Type	Description
base	double	Base value used for first iteration.
increment	double	Value by which the base value is incremented for each (but the very first) iteration.

compression

Property	Type	Description
encoding	String	Encoding used for `Accept-Encoding`/`TE` header. The only currently supported is `gzip`.
type	enum	Type of compression (resource vs. transfer based). Options: `CONTENT_ENCODING`Use `Accept-Encoding` in request and expect `Content-Encoding` in response. `TRANSFER_ENCODING`Use `TE` in request and expect `Transfer-Encoding` in response.

CONNECT

Generic builder for generating a string.

Inline definition
A pattern for string interpolation.

Property	Type	Description
fromVar	Object	Load the string from session variable.
pattern	String	Use pattern replacing session variables.
value	String	String value used verbatim.

DELETE

Generic builder for generating a string.

Inline definition
A pattern for string interpolation.

Property	Type	Description
fromVar	Object	Load the string from session variable.
pattern	String	Use pattern replacing session variables.
value	String	String value used verbatim.

endpoint

Generic builder for generating a string.

Inline definition
A pattern for string interpolation.

Property	Type	Description
fromVar	Object	Load the string from session variable.
pattern	String	Use pattern replacing session variables.
value	String	String value used verbatim.

GET

Generic builder for generating a string.

Inline definition
A pattern for string interpolation.

Property	Type	Description
fromVar	Object	Load the string from session variable.
pattern	String	Use pattern replacing session variables.
value	String	String value used verbatim.

handler

Manages processing of HTTP responses.

Property	Type	Description
autoRangeCheck	boolean	Inject status handler that marks the request as invalid on status 4xx or 5xx. Default value depends on `ergonomics.autoRangeCheck` (see User Guide).
body	Processor.Builder	Handle HTTP response body.
followRedirect	enum	Automatically fire requests when the server responds with redirection. Default value depends on `ergonomics.followRedirect` (see User Guide). Options: `NEVER`Do not insert any automatic redirection handling. `LOCATION_ONLY`Redirect only upon status 3xx accompanied with a ’location’ header. Status, headers, body and completions handlers are suppressed in this case (only raw-bytes handlers are still running). This is the default option. `HTML_ONLY`Handle only HTML response with META refresh header. Status, headers and body handlers are invoked both on the original response and on the response from subsequent requests. Completion handlers are suppressed on this request and invoked after the last response arrives (in case of multiple redirections). `ALWAYS`Implement both status 3xx + location and HTML redirects.
header	HeaderHandler.Builder	Handle HTTP response headers.
onCompletion	Action.Builder	Action executed when the HTTP response is fully received.
rawBytes	RawBytesHandler.Builder	Handler processing HTTP response before parsing.
status	StatusHandler.Builder	Handle HTTP response status.
stopOnInvalid	boolean	Inject completion handler that will stop the session if the request has been marked as invalid. Default value depends on `ergonomics.stopOnInvalid` (see User Guide).

handler.header

Handle HTTP response headers.

Property	Type	Description
conditional	ConditionalHeaderHandler.Builder	Passes the headers to nested handler if the condition holds. Note that the condition may be evaluated multiple times and therefore any nested handlers should not change the results of the condition.
countHeaders	CountHeadersHandler.Builder	Stores number of occurences of each header in custom statistics (these can be displayed in CLI using the `stats -c` command).
filter	FilterHeaderHandler.Builder	Compares if the header name matches expression and invokes a processor with the value.
logInvalid	LogInvalidHandler.HeaderHandlerBuilder	Logs headers from requests marked as invalid.
recordHeaderTime	RecordHeaderTimeHandler.Builder	Records alternative metric based on values from a header (e.g. when a proxy reports processing time).

handler.header.conditional

Passes the headers to nested handler if the condition holds. Note that the condition may be evaluated multiple times and therefore any nested handlers should not change the results of the condition.

Property	Type	Description
allConditions	Builder	Condition combining multiple other conditions with ‘AND’ logic.
boolCondition	Builder	Condition comparing boolean variables.
handler	HeaderHandler.Builder	One or more header handlers that should be invoked.
intCondition	Builder	Condition comparing integer variables.
stringCondition	Builder	Condition comparing string variables.

handler.header.conditional.allConditions

Test more conditions and combine the results using AND logic.

Property	Type	Description
<list of mappings>	<list of builders>	List of conditions.

handler.header.conditional.allConditions.<list of mappings>

Selector for condition type.

Property	Type	Description
allConditions	Builder	Condition combining multiple other conditions with ‘AND’ logic.
boolCondition	Builder	Condition comparing boolean variables.
intCondition	Builder	Condition comparing integer variables.
stringCondition	Builder	Condition comparing string variables.

handler.header.conditional.allConditions.<list of mappings>.allConditions

Test more conditions and combine the results using AND logic.

Property	Type	Description
<list of mappings>	<list of builders>	List of conditions.

handler.header.conditional.allConditions.<list of mappings>.boolCondition

Tests session variable containing boolean value.

Property	Type	Description
fromVar	String	Variable name.
value	boolean	Expected value.

handler.header.conditional.allConditions.<list of mappings>.intCondition

Condition comparing integer in session variable.

Inline definition
Parses condition in the form <variable> <operator> <value>
where operator is one of: `==`, `!=`,
`<>` (the same as `!=`),
`>=`, `>`, `<=`, `<`.

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
fromVar	Object	Variable name.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

handler.header.conditional.allConditions.<list of mappings>.stringCondition

Condition comparing string in session variable.

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
fromVar	Object	Variable name.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

handler.header.conditional.boolCondition

Tests session variable containing boolean value.

Property	Type	Description
fromVar	String	Variable name.
value	boolean	Expected value.

handler.header.conditional.handler

One or more header handlers that should be invoked.

Property	Type	Description
conditional	ConditionalHeaderHandler.Builder	Passes the headers to nested handler if the condition holds. Note that the condition may be evaluated multiple times and therefore any nested handlers should not change the results of the condition.
countHeaders	CountHeadersHandler.Builder	Stores number of occurences of each header in custom statistics (these can be displayed in CLI using the `stats -c` command).
filter	FilterHeaderHandler.Builder	Compares if the header name matches expression and invokes a processor with the value.
logInvalid	LogInvalidHandler.HeaderHandlerBuilder	Logs headers from requests marked as invalid.
recordHeaderTime	RecordHeaderTimeHandler.Builder	Records alternative metric based on values from a header (e.g. when a proxy reports processing time).

handler.header.conditional.intCondition

Condition comparing integer in session variable.

Inline definition
Parses condition in the form <variable> <operator> <value>
where operator is one of: `==`, `!=`,
`<>` (the same as `!=`),
`>=`, `>`, `<=`, `<`.

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
fromVar	Object	Variable name.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

handler.header.conditional.intCondition.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

handler.header.conditional.intCondition.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

handler.header.conditional.intCondition.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

handler.header.conditional.intCondition.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

handler.header.conditional.intCondition.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

handler.header.conditional.intCondition.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

handler.header.conditional.stringCondition

Condition comparing string in session variable.

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
fromVar	Object	Variable name.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

handler.header.conditional.stringCondition.length

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

handler.header.conditional.stringCondition.length.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

handler.header.conditional.stringCondition.length.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

handler.header.conditional.stringCondition.length.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

handler.header.conditional.stringCondition.length.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

handler.header.conditional.stringCondition.length.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

handler.header.conditional.stringCondition.length.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

handler.header.filter

Compares if the header name matches expression and invokes a processor with the value.

Property	Type	Description
header	Builder	Condition on the header name.
processor	Processor.Builder	Add one or more processors.

handler.header.filter.header

Inline definition
Literal value the string should match.

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

handler.header.filter.header.length

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

handler.header.recordHeaderTime

Records alternative metric based on values from a header (e.g. when a proxy reports processing time).

Property	Type	Description
header	String	Header carrying the time.
metric	String	Name of the created metric.
unit	String	Time unit in the header; use either `ms` or `ns`.

handler.rawBytes

Handler processing HTTP response before parsing.

Property	Type	Description
transferSizeRecorder	TransferSizeRecorder.Builder	Accumulates request and response sizes into custom metrics.

handler.rawBytes.transferSizeRecorder

Accumulates request and response sizes into custom metrics.

Property	Type	Description
key	String	Name of the custom metric for collecting request/response bytes.

handler.status

Handle HTTP response status.

Property	Type	Description
action	ActionStatusHandler.Builder	Perform certain actions when the status falls into a range.
counter	StatusToCounterHandler.Builder	Counts how many times given status is received.
multiplex	MultiplexStatusHandler.Builder	Multiplexes the status based on range into different status handlers.
range	RangeStatusValidator.Builder	Marks requests that don’t fall into the desired range as invalid.
stats	StatusToStatsHandler.Builder	Records number of occurrences of each status counts into custom statistics (these can be displayed in CLI using `stats -c`).
store	StoreStatusHandler.Builder	Stores the status into session variable.

handler.status.action

Perform certain actions when the status falls into a range.

Property	Type	Description
<any>	Builder	Perform a sequence of actions if the range matches. Use range as the key and action in the mapping. Possible values of the status should be separated by commas (,). Ranges can be set using low-high (inclusive) (e.g. 200-299), or replacing lower digits with ‘x’ (e.g. 2xx).

handler.status.counter

Counts how many times given status is received.

Property	Type	Description
add	int	Number to be added to the session variable.
expectStatus	int	Expected status (others are ignored). All status codes match by default.
init	int	Initial value for the session variable.
set	int	Do not accumulate (add), just set the variable to this value.
var	String	Variable name.

handler.status.multiplex

Multiplexes the status based on range into different status handlers.

Property	Type	Description
<any>	Builder	Run another handler if the range matches. Use range as the key and another status handler in the mapping. Possible values of the status should be separated by commas (,). Ranges can be set using low-high (inclusive) (e.g. 200-299), or replacing lower digits with ‘x’ (e.g. 2xx).

handler.status.multiplex.<any>

Run another handler if the range matches. Use range as the key and another status handler in the mapping. Possible values of the status should be separated by commas (,). Ranges can be set using low-high (inclusive) (e.g. 200-299), or replacing lower digits with ‘x’ (e.g. 2xx).

Property	Type	Description
action	ActionStatusHandler.Builder	Perform certain actions when the status falls into a range.
counter	StatusToCounterHandler.Builder	Counts how many times given status is received.
multiplex	MultiplexStatusHandler.Builder	Multiplexes the status based on range into different status handlers.
range	RangeStatusValidator.Builder	Marks requests that don’t fall into the desired range as invalid.
stats	StatusToStatsHandler.Builder	Records number of occurrences of each status counts into custom statistics (these can be displayed in CLI using `stats -c`).
store	StoreStatusHandler.Builder	Stores the status into session variable.

handler.status.range

Marks requests that don’t fall into the desired range as invalid.

Inline definition
Single status code (`204`), masked code (`2xx`) or range (`200-399`).

Property	Type	Description
max	int	Highest accepted status code.
min	int	Lowest accepted status code.

handler.status.store

Stores the status into session variable.

Inline definition
Variable name.

Property	Type	Description
toVar	Object	Variable name.

HEAD

Generic builder for generating a string.

Inline definition
A pattern for string interpolation.

Property	Type	Description
fromVar	Object	Load the string from session variable.
pattern	String	Use pattern replacing session variables.
value	String	String value used verbatim.

headers

Property	Type	Description
<any>	String	Use header name (e.g. `Content-Type`) as key and value (possibly a pattern).
<any> (alternative)	Builder	Use header name (e.g. `Content-Type`) as key and specify value in the mapping.

headers.<any>

Specifies value that should be sent in headers.

Inline definition
The value. This can be a pattern.

Property	Type	Description
fromVar	String	Load header value from session variable.
pattern	String	Load header value using a pattern.

metric

Allows categorizing request statistics into metrics based on the request path.

Property	Type	Description
<list of strings>	<list of strings>	Allows categorizing request statistics into metrics based on the request path. The expressions are evaluated in the order as provided in the list. Use one of: `regexp -> replacement`, e.g. `([^?])(?.)? -> $1` to drop the query part. `regexp` (don’t do any replaces and use the full path), e.g. `.*.jpg` `-> name` (metric applied if none of the previous expressions match).

OPTIONS

Generic builder for generating a string.

Inline definition
A pattern for string interpolation.

Property	Type	Description
fromVar	Object	Load the string from session variable.
pattern	String	Use pattern replacing session variables.
value	String	String value used verbatim.

PATCH

Generic builder for generating a string.

Inline definition
A pattern for string interpolation.

Property	Type	Description
fromVar	Object	Load the string from session variable.
pattern	String	Use pattern replacing session variables.
value	String	String value used verbatim.

path

Generic builder for generating a string.

Inline definition
A pattern for string interpolation.

Property	Type	Description
fromVar	Object	Load the string from session variable.
pattern	String	Use pattern replacing session variables.
value	String	String value used verbatim.

POST

Generic builder for generating a string.

Inline definition
A pattern for string interpolation.

Property	Type	Description
fromVar	Object	Load the string from session variable.
pattern	String	Use pattern replacing session variables.
value	String	String value used verbatim.

PUT

Generic builder for generating a string.

Inline definition
A pattern for string interpolation.

Property	Type	Description
fromVar	Object	Load the string from session variable.
pattern	String	Use pattern replacing session variables.
value	String	String value used verbatim.

sla

Defines a list of Service Level Agreements (SLAs) - conditions that must hold for benchmark to be deemed successful.

Property	Type	Description
<list of mappings>	<list of builders>	One or more SLA configurations.

sla.<list of mappings>

Defines a Service Level Agreement (SLA) - conditions that must hold for benchmark to be deemed successful.

Property	Type	Description
blockedRatio	double	Maximum allowed ratio of time spent waiting for usable connection to sum of response latencies and blocked time. Default is 0 - client must not be blocked. Set to 1 if the client can block without limits.
errorRatio	double	Maximum allowed ratio of errors: connection failures or resets, timeouts and internal errors. Valid values are 0.0 - 1.0 (inclusive). Note: 4xx and 5xx statuses are NOT considered errors for this SLA parameter. Use `invalidRatio` for that.
invalidRatio	double	Maximum allowed ratio of requests with responses marked as invalid. Valid values are 0.0 - 1.0 (inclusive). Note: With default settings 4xx and 5xx statuses are considered invalid. Check out `ergonomics.autoRangeCheck` or `httpRequest.handler.autoRangeCheck` to change this.
limits	Builder	Percentile limits.
meanResponseTime	String	Maximum allowed mean (average) response time. Use suffix `ns`, `us`, `ms` or `s` to specify units.
window	String	Period over which the stats should be collected. By default the SLA applies to stats from whole phase.

sla.<list of mappings>.limits

Percentile limits.

Property	Type	Description
<any>	String	Use percentile (value between 0.0 and 1.0) as key and response time with unit (e.g. `ms`) in suffix as value.

TRACE

Generic builder for generating a string.

Inline definition
A pattern for string interpolation.

Property	Type	Description
fromVar	Object	Load the string from session variable.
pattern	String	Use pattern replacing session variables.
value	String	String value used verbatim.

6.1.20 - json

Parse JSON in variable into another variable.

Property	Type	Description
delete	boolean	If this is set to true, the selected key will be deleted from the JSON and the modified JSON will be passed to the `processor`.
format	enum	Conversion to apply on the matching parts with ’toVar’ or ’toArray’ shortcuts. Options: `BYTEBUF`Store the buffer directly. Beware that this may cause memory leaks! `BYTES`Store data as byte array. `STRING`Interprets the bytes as UTF-8 string.
fromVar	String	Variable to load JSON from.
processor	Processor.Builder	Add one or more processors.
query	String	Query selecting the part of JSON.
replace	Transformer.Builder	Custom transformation executed on the value of the selected item. Note that the output value must contain quotes (if applicable) and be correctly escaped.
replace (alternative)	String	Replace value of selected item with value generated through a pattern. Note that the result must contain quotes and be correctly escaped.
toArray	String	Shortcut to store selected parts in an array in the session. Must follow the pattern `variable[maxSize]`
toVar	String	Shortcut to store first match in given variable. Further matches are ignored.
unquote	boolean	Automatically unquote and unescape the input values. By default true.

replace

Custom transformation executed on the value of the selected item. Note that the output value must contain quotes (if applicable) and be correctly escaped.

Property	Type	Description
actions	ActionsTransformer.Builder	This transformer stores the (defragmented) input into a variable, using requested format. After that it executes all the actions and fetches transformed value using the pattern.
pattern	Pattern.TransformerBuilder	Use pattern replacing session variables.

replace.actions

This transformer stores the (defragmented) input into a variable, using requested format. After that it executes all the actions and fetches transformed value using the pattern.

Property	Type	Description
actions	Action.Builder	Actions to be executed.
format	enum	Format into which should this transformer convert the buffers before storing. Default is `STRING`. Options: `BYTEBUF`Store the buffer directly. Beware that this may cause memory leaks! `BYTES`Store data as byte array. `STRING`Interprets the bytes as UTF-8 string.
pattern	String	Pattern to use when fetching the transformed value.
var	String	Variable used as the intermediate storage for the data.

replace.pattern

Use pattern replacing session variables.

Inline definition
The pattern formatting string.

Property	Type	Description
pattern	String	Use pattern replacing session variables.

6.1.21 - log

Log a message and variable values.

Inline definition
A pattern for string interpolation.

Property	Type	Description
message	String	Message format pattern. Use string interpolation for variables.

6.1.22 - loop

Repeats a set of steps fixed number of times.

This step is supposed to be inserted as the first step of a sequence and the counterVar should not be set during the first invocation.

Before the loop the counterVar is initialized to zero, and in each after the last step in the steps sequence the counter is incremented. If the result is lesser than repeats this sequence is restarted from the beginning (this is also why the step must be the first step in the sequence).

It is legal to place steps after the looped steps.

Example:

scenario:
- mySequence:
  - loop:
      counterVar: myCounter
      repeats: 5
      steps:
      - httpRequest:
          GET: /foo/${myCounter}
          # ...
      - someOtherStep: ...
  - anotherStepExecutedAfterThoseFiveLoops

Property	Type	Description
counterVar	String	Variable holding number of iterations.
repeats	int	Number of iterations that should be executed.
steps	Builder	List of steps that should be looped.

6.1.23 - markRequestInvalid

Unconditionally mark currently processed request as invalid.

6.1.24 - newSequence

Instantiates a sequence for each invocation.

Inline definition
Sequence name.

Property	Type	Description
concurrencyPolicy	enum	Options: `FAIL` `WARN`
forceSameIndex	boolean	Forces that the sequence will have the same index as the currently executing sequence. This can be useful if the sequence is passing some data to the new sequence using sequence-scoped variables. Note that the new sequence must have same concurrency factor as the currently executing sequence.
sequence	String	Name of the instantiated sequence.

6.1.25 - noop

Does nothing. Only for demonstration purposes.

6.1.26 - publishAgentData

Makes the data available to all sessions in the same agent, including those using different executors.

Inline definition
Both name of source variable and the key used to read the data.

Property	Type	Description
fromVar	String	Source session variable name.
name	String	Arbitrary unique identifier for the data.

6.1.27 - publishGlobalCounters

Gathers values from session variables and publishes them globally (to all agents).

Gathers values from session variables and publishes them globally (to all agents).
You can name the counters individually (example 1) or use the variable names (example 2):

# Example 1:
- publishGlobalCounters:
    key: myKey
    vars: [ foo, bar ]
# Example 2:
- publishGlobalCounters:
    key: someOtherKey
    vars:
    - foo: myFoo
    - bar: bbb
</code>```


| Property | Type | Description |
| ------- | ------- | -------- |
| key | String | Identifier of the global record. |
| vars | [Builder](#vars) | List of names and session variables. |

### vars

| Property | Type | Description |
| ------- | ------- | ------- |
| &lt;any&gt; | &lt;unknown&gt; | <font color="#606060">&lt;no description&gt;</font> |
| &lt;list of strings&gt; | &lt;unknown&gt; | <font color="#606060">&lt;no description&gt;</font> |

6.1.28 - pullSharedMap

Move values from a map shared across all sessions using the same executor into session variables.

The executor can host multiple shared maps, each holding an entry with several variables. This step moves variables from either a random entry (if no match is set) or with an entry that has the same value for given variable as the current session. When data is moved to the current session the entry is dropped from the shared map. If the map contains records for which the {@link #vars()} don’t contain a destination variable the contents is lost.

Property	Type	Description
key	String	Key identifying the shared map.
match	String	Name of the session variable that stores value identifying the entry in the shared map.
vars	<list of strings>	List of variables the map should be pulled into.

vars

List of variables the map should be pulled into.

Property	Type	Description
<list of strings>	<unknown>	<no description>

6.1.29 - pushSharedMap

Store values from session variables into a map shared across all sessions using the same executor into session variables.

The executor can host multiple shared maps, each holding an entry with several variables. This step creates one entry in the map, copying values from session variables into the entry.

Property	Type	Description
key	String	Key identifying the shared map.
vars	<list of strings>	List of variable names that should be stored in the entry.

vars

List of variable names that should be stored in the entry.

Property	Type	Description
<list of strings>	<unknown>	<no description>

6.1.30 - randomCsvRow

Stores random row from a CSV-formatted file to variables.

Property	Type	Description
columns	Builder	Defines mapping from columns to session variables.
file	String	Path to the CSV file that should be loaded.
separator	char	Set character used for column separation. By default it is comma (`,`).
skipComments	boolean	Skip lines starting with character ‘#’. By default set to false.

columns

Property	Type	Description
<any>	String	Use 0-based column as the key and variable name as the value.

6.1.31 - randomFile

Reads bytes from a randomly chosen file into a variable.

Reads bytes from a randomly chosen file into a variable.
Two formats are supported:
Example 1 - without weights:

toVar: myVar
files:
- /path/to/file1.txt
- file2_relative_to_benchmark.txt

Example 2 - with weights (the second file will be returned twice as often):

toVar: myVar
files:
  /path/to/file1.txt: 1
  file2_relative_to_benchmark.txt: 2

Property	Type	Description
filenameVar	String	Optional variable to store the filename of the random file.
files	Builder	Potentially weighted list of files to choose from.
toVar	String	Variable where the chosen byte array should be stored.

files

Property	Type	Description
<any>	<list of strings>	Item as the key and weight (arbitrary floating-point number, defaults to 1.0) as the value.
<list of strings>	<list of strings>	Item as the key and weight (arbitrary floating-point number, defaults to 1.0) as the value.

6.1.32 - randomInt

Stores random (linearly distributed) integer into session variable.

Property	Type	Description
max	Builder	Highest possible value (inclusive). Default is Integer.MAX_VALUE.
min	Builder	Lowest possible value (inclusive). Default is 0.
toVar	String	Variable name to store the result.

max

Inline definition
Constant value.

Property	Type	Description
fromVar	String	Initialize with a value from session variable.
value	int	Initialize with a constant value.

min

Inline definition
Constant value.

Property	Type	Description
fromVar	String	Initialize with a value from session variable.
value	int	Initialize with a constant value.

6.1.33 - randomItem

Stores random item from a list or array into session variable.

Property	Type	Description
file	String	This file will be loaded into memory and the step will choose on line as the item.
fromVar	String	Variable containing an array or list.
list	Builder	Potentially weighted list of items to choose from.
toVar	String	Variable where the chosen item should be stored.

list

Property	Type	Description
<any>	<list of strings>	Item as the key and weight (arbitrary floating-point number, defaults to 1.0) as the value.
<list of strings>	<list of strings>	Item as the key and weight (arbitrary floating-point number, defaults to 1.0) as the value.

6.1.34 - randomUUID

Stores random string into session variable based on the UUID generator.

Property	Type	Description
toVar	String	Variable name to store the result.

6.1.35 - readAgentData

Reads data from agent-wide scope into session variable.

Reads data from agent-wide scope into session variable.
The data must be published in a phase that has terminated before this phase starts: usually this is achieved using the startAfterStrict property on the phase.

Inline definition
Both the identifier and destination session variable.

Property	Type	Description
name	String	Unique identifier for the data.
toVar	String	Destination session variable name.

6.1.36 - removeItem

Removes element from an array of variables.

Property	Type	Description
fromVar	String	Variable containing an existing array of object variables.
index	Builder	Set index at which the item should be removed. Elements to the right of this are moved to the left.

index

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.1.37 - restartSequence

Restarts current sequence from beginning.

6.1.38 - scheduleDelay

Define a point in future until which we should wait. Does not cause waiting.

Property	Type	Description
duration	String	Duration of the delay with appropriate suffix (e.g. `ms` or `s`).
fromLast	<none>	Set previous delay point reference as the reference for next delay point; it will be computed as `(previous delay point or now) + duration`. Note: property does not have any value
fromNow	<none>	Set this step invocation as the delay point reference; it will be computed as `now + duration`. Note: property does not have any value
key	String	Key that is referenced later in `awaitDelay` step. If you’re introducing the delay through `thinkTime` do not use this property.
max	String	Upper cap on the duration (if randomized).
min	String	Lower cap on the duration (if randomized).
random	enum	Randomize the duration. Options: `CONSTANT`Do not randomize; use constant duration. `LINEAR`Use linearly random duration between `min` and `max` (inclusively). `NEGATIVE_EXPONENTIAL`Use negative-exponential duration with expected value of `duration`, capped at `min` and `max` (inclusively).
type	enum	Alternative way to set delay reference point. See `fromNow` and `fromLast` property setters. Options: `FROM_LAST` `FROM_NOW`

6.1.39 - set

Set variable in session to certain value.

Inline definition
Use `var <- value`.

Property	Type	Description
intArray	Builder	Set variable to an (unset) integer array.
objectArray	Builder	Set variable to an (unset) object array.
value	String	String value.
var	String	Variable name.

intArray

Creates integer arrays to be stored in the session.

Property	Type	Description
fromVar	String	Contents of the new array. If the variable contains an array or a list, items will be copied to the elements with the same index up to the size of this array. If the variable contains a different value all elements will be initialized to this value.
size	int	Size of the array.

objectArray

Creates object arrays to be stored in the session.

Property	Type	Description
fromVar	String	Contents of the new array. If the variable contains an array or a list, items will be copied to the elements with the same index up to the size of this array. If the variable contains a different value all elements will be initialized to this value.
size	int	Size of the array.

6.1.40 - setInt

Set session variable to an integral value.

Inline definition
Use `var <- value`.

Property	Type	Description
fromVar	String	Input variable name.
intCondition	Builder	Set variable only if the current value satisfies certain condition.
max	Builder	Set to value that is the maximum of this list of values.
min	Builder	Set to value that is the minimum of this list of values.
onlyIfNotSet	boolean	Set variable to the value only if it is not already set.
value	int	Value (integer).
var	String	Variable name.

intCondition

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

intCondition.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

max

Property	Type	Description
<list of mappings>	<list of builders>	<no description>

max.<list of mappings>

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

min

Property	Type	Description
<list of mappings>	<list of builders>	<no description>

6.1.41 - setItem

Set element in a collection on given position.

Property	Type	Description
fromVar	String	Fetch value from session variable.
index	Builder	Integer session variable with an index into the collection.
toVar	String	Session variable with the collection.
value	String	Verbatim value.

index

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.1.42 - setSharedCounter

Sets value in a counter shared by all sessions in the same executor.

Inline definition
Both the name of the counter and variable name.

Property	Type	Description
fromVar	String	Input variable name.
key	String	Identifier for the counter.
value	int	Value (integer).

6.1.43 - stop

Immediately stop the user session (break all running sequences).

6.1.44 - stopwatch

Run nested sequence of steps, recording execution time.

6.1.45 - stringToInt

Parse string into integer and store it in a variable.

If the parsing fails the target variable is not modified.

Inline definition
Use `fromVar -> toVar`

Property	Type	Description
fromVar	String	Source variable name.
toVar	String	Target variable name.

6.1.46 - template

Format pattern into session variable.

Property	Type	Description
pattern	String	Pattern to be encoded, e.g. `foo${variable}bar${another-variable}`
toVar	String	Variable name to store the result.

6.1.47 - thinkTime

Block current sequence for specified duration.

Inline definition
Duration of the delay with appropriate suffix (e.g. `ms` or `s`).

Property	Type	Description
duration	String	Duration of the delay with appropriate suffix (e.g. `ms` or `s`).
fromLast	<none>	Set previous delay point reference as the reference for next delay point; it will be computed as `(previous delay point or now) + duration`. Note: property does not have any value
fromNow	<none>	Set this step invocation as the delay point reference; it will be computed as `now + duration`. Note: property does not have any value
key	String	Key that is referenced later in `awaitDelay` step. If you’re introducing the delay through `thinkTime` do not use this property.
max	String	Upper cap on the duration (if randomized).
min	String	Lower cap on the duration (if randomized).
random	enum	Randomize the duration. Options: `CONSTANT`Do not randomize; use constant duration. `LINEAR`Use linearly random duration between `min` and `max` (inclusively). `NEGATIVE_EXPONENTIAL`Use negative-exponential duration with expected value of `duration`, capped at `min` and `max` (inclusively).
type	enum	Alternative way to set delay reference point. See `fromNow` and `fromLast` property setters. Options: `FROM_LAST` `FROM_NOW`

6.1.48 - timestamp

Stores the current time in milliseconds as string to a session variable.

Inline definition
Variable name.

Property	Type	Description
localeCountry	String	2-letter ISO country code used in the formatter locale. Defaults to ‘US’.
pattern	String	Format the timestamp using `SimpleDateFormat` pattern.
toVar	String	Target variable name.

6.1.49 - unset

Undefine variable name.

Inline definition
Variable name.

Property	Type	Description
var	Object	Variable name.

6.2 - Processors

Processors can work either as consumers of input bytes (e.g. storing part of the input into session variables), as filters (passing modified version of the input to delegated processors) or combination of the above.

Some processors can expect extra context in the session, such as an ongoing (HTTP) request. It is possible to use processors that don’t expect specific type of request in places where a more specific type is provided; opposite is not allowed.

Also it is possible to use an action instead of a processor; Hyperfoil automatically inserts an adapter. That’s why the list below includes actions as well.

6.2.1 - addItem

Appends value to an array stored in another variable.

Property	Type	Description
fromVar	String	Fetch value from session variable.
toVar	String	Destination variable with the array.
value	String	Verbatim value.

6.2.2 - addToInt

Add value to integer variable in the session.

Inline definition
Accepting one of: `var++`, `var–`, `var += value`, `var -= value`.

Property	Type	Description
orElseSetTo	int	If the variable is currently not set, set it to this value instead of addition.
value	int	Value added (can be negative).
var	String	Variable name.

6.2.3 - addToSharedCounter

Adds value to a counter shared by all sessions in the same executor.

Inline definition
Use on of: `counter++`, `counter–`, `counter += <value>`,
`counter -= <value>`

Property	Type	Description
fromVar	String	Input variable name.
key	String	Identifier for the counter.
operator	enum	Operation to perform on the counter. Default is `ADD`. Options: `ADD` `SUBTRACT`
value	int	Value (integer).

6.2.4 - array

Stores data in an array stored as session variable.

Inline definition
Use format `toVar[maxSize]`.

Property	Type	Description
format	enum	Format into which should this processor convert the buffers before storing. Default is `STRING`. Options: `BYTEBUF`Store the buffer directly. Beware that this may cause memory leaks! `BYTES`Store data as byte array. `STRING`Interprets the bytes as UTF-8 string.
maxSize	int	Maximum size of the array.
silent	boolean	Do not log warnings when the maximum size is exceeded.
toVar	String	Variable name.

6.2.5 - clearHttpCache

Drops all entries from HTTP cache in the session.

6.2.6 - closeConnection

Prevents reuse connection after the response has been handled.

6.2.7 - collection

Collects results of processor invocation into a unbounded list.

Collects results of processor invocation into a unbounded list.
WARNING: This processor should be used rarely as it allocates memory during the benchmark.

Inline definition
Variable name to store the list.

Property	Type	Description
format	enum	Format into which should this processor convert the buffers before storing. Default is `STRING`. Options: `BYTEBUF`Store the buffer directly. Beware that this may cause memory leaks! `BYTES`Store data as byte array. `STRING`Interprets the bytes as UTF-8 string.
toVar	String	Variable name.

6.2.8 - conditional

Passes the data to nested processor if the condition holds.

Passes the data to nested processor if the condition holds.
Note that the condition may be evaluated multiple times and therefore any nested processors should not change the results of the condition.

Property	Type	Description
allConditions	Builder	Condition combining multiple other conditions with ‘AND’ logic.
boolCondition	Builder	Condition comparing boolean variables.
intCondition	Builder	Condition comparing integer variables.
processor	Processor.Builder	One or more processors that should be invoked if the condition holds.
stringCondition	Builder	Condition comparing string variables.

allConditions

Test more conditions and combine the results using AND logic.

Property	Type	Description
<list of mappings>	<list of builders>	List of conditions.

allConditions.<list of mappings>

Selector for condition type.

Property	Type	Description
allConditions	Builder	Condition combining multiple other conditions with ‘AND’ logic.
boolCondition	Builder	Condition comparing boolean variables.
intCondition	Builder	Condition comparing integer variables.
stringCondition	Builder	Condition comparing string variables.

allConditions.<list of mappings>.allConditions

Test more conditions and combine the results using AND logic.

Property	Type	Description
<list of mappings>	<list of builders>	List of conditions.

allConditions.<list of mappings>.boolCondition

Tests session variable containing boolean value.

Property	Type	Description
fromVar	String	Variable name.
value	boolean	Expected value.

allConditions.<list of mappings>.intCondition

Condition comparing integer in session variable.

Inline definition
Parses condition in the form <variable> <operator> <value>
where operator is one of: `==`, `!=`,
`<>` (the same as `!=`),
`>=`, `>`, `<=`, `<`.

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
fromVar	Object	Variable name.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

allConditions.<list of mappings>.stringCondition

Condition comparing string in session variable.

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
fromVar	Object	Variable name.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

boolCondition

Tests session variable containing boolean value.

Property	Type	Description
fromVar	String	Variable name.
value	boolean	Expected value.

intCondition

Condition comparing integer in session variable.

Inline definition
Parses condition in the form <variable> <operator> <value>
where operator is one of: `==`, `!=`,
`<>` (the same as `!=`),
`>=`, `>`, `<=`, `<`.

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
fromVar	Object	Variable name.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

intCondition.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition

Condition comparing string in session variable.

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
fromVar	Object	Variable name.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

stringCondition.length

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

stringCondition.length.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.2.9 - count

Records number of parts this processor is invoked on.

Inline definition
Name of variable for the number of occurrences.

Property	Type	Description
toVar	String	Variable where to store number of occurrences.

6.2.10 - fail

Fail the phase with given message. Used mostly for testing/debugging.

Inline definition
Message to fail with (unconditionally).

Property	Type	Description
allConditions	Builder	Condition combining multiple other conditions with ‘AND’ logic.
boolCondition	Builder	Condition comparing boolean variables.
intCondition	Builder	Condition comparing integer variables.
message	String	Message attached to the failure exception.
stringCondition	Builder	Condition comparing string variables.

allConditions

Test more conditions and combine the results using AND logic.

Property	Type	Description
<list of mappings>	<list of builders>	List of conditions.

allConditions.<list of mappings>

Selector for condition type.

Property	Type	Description
allConditions	Builder	Condition combining multiple other conditions with ‘AND’ logic.
boolCondition	Builder	Condition comparing boolean variables.
intCondition	Builder	Condition comparing integer variables.
stringCondition	Builder	Condition comparing string variables.

allConditions.<list of mappings>.allConditions

Test more conditions and combine the results using AND logic.

Property	Type	Description
<list of mappings>	<list of builders>	List of conditions.

allConditions.<list of mappings>.boolCondition

Tests session variable containing boolean value.

Property	Type	Description
fromVar	String	Variable name.
value	boolean	Expected value.

allConditions.<list of mappings>.intCondition

Condition comparing integer in session variable.

Inline definition
Parses condition in the form <variable> <operator> <value>
where operator is one of: `==`, `!=`,
`<>` (the same as `!=`),
`>=`, `>`, `<=`, `<`.

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
fromVar	Object	Variable name.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

allConditions.<list of mappings>.stringCondition

Condition comparing string in session variable.

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
fromVar	Object	Variable name.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

boolCondition

Tests session variable containing boolean value.

Property	Type	Description
fromVar	String	Variable name.
value	boolean	Expected value.

intCondition

Condition comparing integer in session variable.

Inline definition
Parses condition in the form <variable> <operator> <value>
where operator is one of: `==`, `!=`,
`<>` (the same as `!=`),
`>=`, `>`, `<=`, `<`.

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
fromVar	Object	Variable name.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

intCondition.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition

Condition comparing string in session variable.

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
fromVar	Object	Variable name.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

stringCondition.length

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

stringCondition.length.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.2.11 - getIndex

Lookup index of an item in an array/collection.

Property	Type	Description
collection	String	Variable to fetch the collection from.
item	Builder	Item that should be looked up (using equality).
toVar	String	Integer variable to store the index, or -1 if the item is not found.

item

Inline definition
Verbatim value.

Property	Type	Description
fromVar	String	Fetch value from session variable.
value	String	Verbatim value.

6.2.12 - getItem

Retrieves n-th item from an array or collection.

Property	Type	Description
fromVar	String	Source variable with an array or list.
index	Builder	Source for index into the array/list.
toVar	String	Destination variable for the n-th element.

index

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.2.13 - getSharedCounter

Retrieves value from a counter shared by all sessions in the same executor and stores that in a session variable.

Retrieves value from a counter shared by all sessions in the same executor and stores that in a session variable.
If the value exceeds allowed integer range (-2^31 .. 2^31 - 1) it is capped.

Inline definition
Both the key and variable name.

Property	Type	Description
key	String	Identifier for the counter.
toVar	String	Session variable for storing the value.

6.2.14 - getSize

Calculates size of an array/collection held in variable into another variable

Property	Type	Description
boolFilter	Builder	Count only items matching the condition.
fromVar	String	Variable holding the collection.
intFilter	Builder	Count only items matching the condition.
stringFilter	Builder	Count only items matching the condition.
toVar	String	Variable storing the size.
undefinedValue	int	Value to use when `fromVar` is unset or it does not contain any array/collection.

boolFilter

Property	Type	Description
value	boolean	Expected value.

intFilter

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

intFilter.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intFilter.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intFilter.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intFilter.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intFilter.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intFilter.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringFilter

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

stringFilter.length

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

stringFilter.length.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringFilter.length.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringFilter.length.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringFilter.length.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringFilter.length.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringFilter.length.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.2.15 - gzipInflator

Decompresses a GZIP data and pipes the output to delegated processors.

Decompresses a GZIP data and pipes the output to delegated processors.
If the data contains multiple concatenated GZIP streams it will pipe multiple decompressed objects with isLastPart set to true at the end of each stream.

Property	Type	Description
encodingVar	Object	Variable used to pass header value from header handlers.
processor	Processor.Builder	Add one or more processors.

6.2.16 - json

Parses JSON responses using simple queries.

Property	Type	Description
delete	boolean	If this is set to true, the selected key will be deleted from the JSON and the modified JSON will be passed to the `processor`.
format	enum	Conversion to apply on the matching parts with ’toVar’ or ’toArray’ shortcuts. Options: `BYTEBUF`Store the buffer directly. Beware that this may cause memory leaks! `BYTES`Store data as byte array. `STRING`Interprets the bytes as UTF-8 string.
processor	Processor.Builder	Add one or more processors.
processor (alternative)	Processor.Builder	If neither `delete` or `replace` was set this processor will be called with the selected parts. In the other case the processor will be called with chunks of full (modified) JSON. Note that the `processor.before()` and `processor.after()` methods are called only once for each request, not for the individual filtered items.
query	String	Query selecting the part of JSON.
replace	Transformer.Builder	Custom transformation executed on the value of the selected item. Note that the output value must contain quotes (if applicable) and be correctly escaped.
replace (alternative)	String	Replace value of selected item with value generated through a pattern. Note that the result must contain quotes and be correctly escaped.
toArray	String	Shortcut to store selected parts in an array in the session. Must follow the pattern `variable[maxSize]`
toVar	String	Shortcut to store first match in given variable. Further matches are ignored.
unquote	boolean	Automatically unquote and unescape the input values. By default true.

replace

Custom transformation executed on the value of the selected item. Note that the output value must contain quotes (if applicable) and be correctly escaped.

Property	Type	Description
actions	ActionsTransformer.Builder	This transformer stores the (defragmented) input into a variable, using requested format. After that it executes all the actions and fetches transformed value using the pattern.
pattern	Pattern.TransformerBuilder	Use pattern replacing session variables.

replace.actions

This transformer stores the (defragmented) input into a variable, using requested format. After that it executes all the actions and fetches transformed value using the pattern.

Property	Type	Description
actions	Action.Builder	Actions to be executed.
format	enum	Format into which should this transformer convert the buffers before storing. Default is `STRING`. Options: `BYTEBUF`Store the buffer directly. Beware that this may cause memory leaks! `BYTES`Store data as byte array. `STRING`Interprets the bytes as UTF-8 string.
pattern	String	Pattern to use when fetching the transformed value.
var	String	Variable used as the intermediate storage for the data.

replace.pattern

Use pattern replacing session variables.

Inline definition
The pattern formatting string.

Property	Type	Description
pattern	String	Use pattern replacing session variables.

6.2.17 - log

Log a message and variable values.

Inline definition
A pattern for string interpolation.

Property	Type	Description
message	String	Message format pattern. Use string interpolation for variables.

6.2.18 - logInvalid

Logs body chunks from requests marked as invalid.

6.2.19 - markRequestInvalid

Unconditionally mark currently processed request as invalid.

6.2.20 - newSequence

Instantiates a sequence for each invocation.

Inline definition
Sequence name.

Property	Type	Description
concurrencyPolicy	enum	Options: `FAIL` `WARN`
forceSameIndex	boolean	Forces that the sequence will have the same index as the currently executing sequence. This can be useful if the sequence is passing some data to the new sequence using sequence-scoped variables. Note that the new sequence must have same concurrency factor as the currently executing sequence.
sequence	String	Name of the instantiated sequence.

6.2.21 - parseHtml

Parses HTML tags and invokes handlers based on criteria.

Property	Type	Description
onEmbeddedResource	Builder	Handler firing upon reference to other resource, e.g. image, stylesheet…
onTagAttribute	Builder	<no description>

onEmbeddedResource

Handles <img src="…">, <link href="…">, <embed src="…">, <frame src="…">, <iframe src="…">, <object data="…"> and <script src="…">.

Does not handle <source src="…"> or <track src="…"> because browser would choose only one of the options.

Property	Type	Description
fetchResource	Builder	Automatically download referenced resource.
ignoreExternal	boolean	Ignore resources hosted on servers that are not covered in the `http` section.
processor	Processor.Builder	Add one or more processors.
processor (alternative)	Processor.Builder	Custom processor invoked pointing to attribute data - e.g. in case of `<img>` tag the processor gets contents of the `src` attribute.

onEmbeddedResource.fetchResource

Automates download of embedded resources.

Property	Type	Description
concurrency	int	Maximum number of resources fetched concurrently. Default is 8.
maxResources	int	Maximum number of resources that can be fetched.
metric	<list of strings>	Metrics selector for downloaded resources.
onCompletion	Action.Builder	Action performed when the download of all resources completes.

onEmbeddedResource.fetchResource.metric

Metrics selector for downloaded resources.

Property	Type	Description
<list of strings>	<list of strings>	Allows categorizing request statistics into metrics based on the request path. The expressions are evaluated in the order as provided in the list. Use one of: `regexp -> replacement`, e.g. `([^?])(?.)? -> $1` to drop the query part. `regexp` (don’t do any replaces and use the full path), e.g. `.*.jpg` `-> name` (metric applied if none of the previous expressions match).

onTagAttribute

Property	Type	Description
attribute	String	Name of the attribute in this element you want to process, e.g. `action`
format	enum	Conversion to apply on the matching parts with ’toVar’ or ’toArray’ shortcuts. Options: `BYTEBUF`Store the buffer directly. Beware that this may cause memory leaks! `BYTES`Store data as byte array. `STRING`Interprets the bytes as UTF-8 string.
processor	Processor.Builder	Add one or more processors.
tag	String	Name of the tag this handler should look for, e.g. `form`
toArray	String	Shortcut to store selected parts in an array in the session. Must follow the pattern `variable[maxSize]`
toVar	String	Shortcut to store first match in given variable. Further matches are ignored.

6.2.22 - publishAgentData

Makes the data available to all sessions in the same agent, including those using different executors.

Inline definition
Both name of source variable and the key used to read the data.

Property	Type	Description
fromVar	String	Source session variable name.
name	String	Arbitrary unique identifier for the data.

6.2.23 - publishGlobalCounters

Gathers values from session variables and publishes them globally (to all agents).

Gathers values from session variables and publishes them globally (to all agents).
You can name the counters individually (example 1) or use the variable names (example 2):

# Example 1:
- publishGlobalCounters:
    key: myKey
    vars: [ foo, bar ]
# Example 2:
- publishGlobalCounters:
    key: someOtherKey
    vars:
    - foo: myFoo
    - bar: bbb
</code>```


| Property | Type | Description |
| ------- | ------- | -------- |
| key | String | Identifier of the global record. |
| vars | [Builder](#vars) | List of names and session variables. |

### vars

| Property | Type | Description |
| ------- | ------- | ------- |
| &lt;any&gt; | &lt;unknown&gt; | <font color="#606060">&lt;no description&gt;</font> |
| &lt;list of strings&gt; | &lt;unknown&gt; | <font color="#606060">&lt;no description&gt;</font> |

6.2.24 - queue

Stores defragmented data in a queue.

Stores defragmented data in a queue.
For each item in the queue a new sequence instance will be started (subject the concurrency constraints) with sequence index that allows it to read an object from an array using sequence-scoped access.

Property	Type	Description
concurrency	int	Maximum number of started sequences that can be running at one moment.
format	enum	Conversion format from byte buffers. Default format is STRING. Options: `BYTEBUF`Store the buffer directly. Beware that this may cause memory leaks! `BYTES`Store data as byte array. `STRING`Interprets the bytes as UTF-8 string.
maxSize	int	Maximum number of elements that can be stored in the queue.
onCompletion	Action.Builder	Custom action that should be performed when the last consuming sequence reports that it has processed the last element from the queue. Note that the sequence is NOT automatically augmented to report completion.
sequence	String	Name of the started sequence.
var	String	Variable storing the array that it used as a output object from the queue.

6.2.25 - readAgentData

Reads data from agent-wide scope into session variable.

Inline definition
Both the identifier and destination session variable.

Property	Type	Description
name	String	Unique identifier for the data.
toVar	String	Destination session variable name.

6.2.26 - removeItem

Removes element from an array of variables.

Property	Type	Description
fromVar	String	Variable containing an existing array of object variables.
index	Builder	Set index at which the item should be removed. Elements to the right of this are moved to the left.

index

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.2.27 - restartSequence

Schedules a restart of this sequence.

6.2.28 - set

Set variable in session to certain value.

Inline definition
Use `var <- value`.

Property	Type	Description
intArray	Builder	Set variable to an (unset) integer array.
objectArray	Builder	Set variable to an (unset) object array.
value	String	String value.
var	String	Variable name.

intArray

Creates integer arrays to be stored in the session.

Property	Type	Description
fromVar	String	Contents of the new array. If the variable contains an array or a list, items will be copied to the elements with the same index up to the size of this array. If the variable contains a different value all elements will be initialized to this value.
size	int	Size of the array.

objectArray

Creates object arrays to be stored in the session.

Property	Type	Description
fromVar	String	Contents of the new array. If the variable contains an array or a list, items will be copied to the elements with the same index up to the size of this array. If the variable contains a different value all elements will be initialized to this value.
size	int	Size of the array.

6.2.29 - setInt

Set session variable to an integral value.

Inline definition
Use `var <- value`.

Property	Type	Description
fromVar	String	Input variable name.
intCondition	Builder	Set variable only if the current value satisfies certain condition.
max	Builder	Set to value that is the maximum of this list of values.
min	Builder	Set to value that is the minimum of this list of values.
onlyIfNotSet	boolean	Set variable to the value only if it is not already set.
value	int	Value (integer).
var	String	Variable name.

intCondition

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

intCondition.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

max

Property	Type	Description
<list of mappings>	<list of builders>	<no description>

max.<list of mappings>

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

min

Property	Type	Description
<list of mappings>	<list of builders>	<no description>

6.2.30 - setItem

Set element in a collection on given position.

Property	Type	Description
fromVar	String	Fetch value from session variable.
index	Builder	Integer session variable with an index into the collection.
toVar	String	Session variable with the collection.
value	String	Verbatim value.

index

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.2.31 - setSharedCounter

Sets value in a counter shared by all sessions in the same executor.

Inline definition
Both the name of the counter and variable name.

Property	Type	Description
fromVar	String	Input variable name.
key	String	Identifier for the counter.
value	int	Value (integer).

6.2.32 - store

Stores data in a session variable (overwriting on multiple occurences).

Inline definition
Variable name.

Property	Type	Description
format	enum	Format into which should this processor convert the buffers before storing. Default is `STRING`. Options: `BYTEBUF`Store the buffer directly. Beware that this may cause memory leaks! `BYTES`Store data as byte array. `STRING`Interprets the bytes as UTF-8 string.
toVar	Object	Variable name.

6.2.33 - storeInt

Converts buffers into integral value and stores it in a variable.

Inline definition
Name of integer variable where to store the value.

Property	Type	Description
override	boolean	Allow the value to be set multiple times (last write wins). Defaults to false.
toVar	String	Name of variable where to store the integral value.

6.2.34 - stringToInt

Parse string into integer and store it in a variable.

If the parsing fails the target variable is not modified.

Inline definition
Use `fromVar -> toVar`

Property	Type	Description
fromVar	String	Source variable name.
toVar	String	Target variable name.

6.2.35 - unset

Undefine variable name.

Inline definition
Variable name.

Property	Type	Description
var	Object	Variable name.

6.3 - Actions

Actions are non-blocking handlers for various events.

6.3.1 - addItem

Appends value to an array stored in another variable.

Property	Type	Description
fromVar	String	Fetch value from session variable.
toVar	String	Destination variable with the array.
value	String	Verbatim value.

6.3.2 - addToInt

Add value to integer variable in the session.

Inline definition
Accepting one of: `var++`, `var–`, `var += value`, `var -= value`.

Property	Type	Description
orElseSetTo	int	If the variable is currently not set, set it to this value instead of addition.
value	int	Value added (can be negative).
var	String	Variable name.

6.3.3 - addToSharedCounter

Adds value to a counter shared by all sessions in the same executor.

Inline definition
Use on of: `counter++`, `counter–`, `counter += <value>`,
`counter -= <value>`

Property	Type	Description
fromVar	String	Input variable name.
key	String	Identifier for the counter.
operator	enum	Operation to perform on the counter. Default is `ADD`. Options: `ADD` `SUBTRACT`
value	int	Value (integer).

6.3.4 - clearHttpCache

Drops all entries from HTTP cache in the session.

6.3.5 - conditional

Perform an action or sequence of actions conditionally.

Property	Type	Description
actions	Action.Builder	Actions that should be executed should the condition hold.
allConditions	Builder	Condition combining multiple other conditions with ‘AND’ logic.
boolCondition	Builder	Condition comparing boolean variables.
intCondition	Builder	Condition comparing integer variables.
stringCondition	Builder	Condition comparing string variables.

allConditions

Test more conditions and combine the results using AND logic.

Property	Type	Description
<list of mappings>	<list of builders>	List of conditions.

allConditions.<list of mappings>

Selector for condition type.

Property	Type	Description
allConditions	Builder	Condition combining multiple other conditions with ‘AND’ logic.
boolCondition	Builder	Condition comparing boolean variables.
intCondition	Builder	Condition comparing integer variables.
stringCondition	Builder	Condition comparing string variables.

allConditions.<list of mappings>.allConditions

Test more conditions and combine the results using AND logic.

Property	Type	Description
<list of mappings>	<list of builders>	List of conditions.

allConditions.<list of mappings>.boolCondition

Tests session variable containing boolean value.

Property	Type	Description
fromVar	String	Variable name.
value	boolean	Expected value.

allConditions.<list of mappings>.intCondition

Condition comparing integer in session variable.

Inline definition
Parses condition in the form <variable> <operator> <value>
where operator is one of: `==`, `!=`,
`<>` (the same as `!=`),
`>=`, `>`, `<=`, `<`.

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
fromVar	Object	Variable name.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

allConditions.<list of mappings>.stringCondition

Condition comparing string in session variable.

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
fromVar	Object	Variable name.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

boolCondition

Tests session variable containing boolean value.

Property	Type	Description
fromVar	String	Variable name.
value	boolean	Expected value.

intCondition

Condition comparing integer in session variable.

Inline definition
Parses condition in the form <variable> <operator> <value>
where operator is one of: `==`, `!=`,
`<>` (the same as `!=`),
`>=`, `>`, `<=`, `<`.

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
fromVar	Object	Variable name.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

intCondition.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition

Condition comparing string in session variable.

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
fromVar	Object	Variable name.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

stringCondition.length

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

stringCondition.length.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.3.6 - fail

Fail the phase with given message. Used mostly for testing/debugging.

Inline definition
Message to fail with (unconditionally).

Property	Type	Description
allConditions	Builder	Condition combining multiple other conditions with ‘AND’ logic.
boolCondition	Builder	Condition comparing boolean variables.
intCondition	Builder	Condition comparing integer variables.
message	String	Message attached to the failure exception.
stringCondition	Builder	Condition comparing string variables.

allConditions

Test more conditions and combine the results using AND logic.

Property	Type	Description
<list of mappings>	<list of builders>	List of conditions.

allConditions.<list of mappings>

Selector for condition type.

Property	Type	Description
allConditions	Builder	Condition combining multiple other conditions with ‘AND’ logic.
boolCondition	Builder	Condition comparing boolean variables.
intCondition	Builder	Condition comparing integer variables.
stringCondition	Builder	Condition comparing string variables.

allConditions.<list of mappings>.allConditions

Test more conditions and combine the results using AND logic.

Property	Type	Description
<list of mappings>	<list of builders>	List of conditions.

allConditions.<list of mappings>.boolCondition

Tests session variable containing boolean value.

Property	Type	Description
fromVar	String	Variable name.
value	boolean	Expected value.

allConditions.<list of mappings>.intCondition

Condition comparing integer in session variable.

Inline definition
Parses condition in the form <variable> <operator> <value>
where operator is one of: `==`, `!=`,
`<>` (the same as `!=`),
`>=`, `>`, `<=`, `<`.

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
fromVar	Object	Variable name.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

allConditions.<list of mappings>.stringCondition

Condition comparing string in session variable.

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
fromVar	Object	Variable name.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

boolCondition

Tests session variable containing boolean value.

Property	Type	Description
fromVar	String	Variable name.
value	boolean	Expected value.

intCondition

Condition comparing integer in session variable.

Inline definition
Parses condition in the form <variable> <operator> <value>
where operator is one of: `==`, `!=`,
`<>` (the same as `!=`),
`>=`, `>`, `<=`, `<`.

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
fromVar	Object	Variable name.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

intCondition.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition

Condition comparing string in session variable.

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
fromVar	Object	Variable name.
isSet	boolean	Check if the value is set or unset. By default the variable must be set.
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

stringCondition.length

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

stringCondition.length.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringCondition.length.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.3.7 - getIndex

Lookup index of an item in an array/collection.

Property	Type	Description
collection	String	Variable to fetch the collection from.
item	Builder	Item that should be looked up (using equality).
toVar	String	Integer variable to store the index, or -1 if the item is not found.

item

Inline definition
Verbatim value.

Property	Type	Description
fromVar	String	Fetch value from session variable.
value	String	Verbatim value.

6.3.8 - getItem

Retrieves n-th item from an array or collection.

Property	Type	Description
fromVar	String	Source variable with an array or list.
index	Builder	Source for index into the array/list.
toVar	String	Destination variable for the n-th element.

index

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.3.9 - getSharedCounter

Retrieves value from a counter shared by all sessions in the same executor and stores that in a session variable.

Retrieves value from a counter shared by all sessions in the same executor and stores that in a session variable.
If the value exceeds allowed integer range (-2^31 .. 2^31 - 1) it is capped.

Inline definition
Both the key and variable name.

Property	Type	Description
key	String	Identifier for the counter.
toVar	String	Session variable for storing the value.

6.3.10 - getSize

Calculates size of an array/collection held in variable into another variable

Property	Type	Description
boolFilter	Builder	Count only items matching the condition.
fromVar	String	Variable holding the collection.
intFilter	Builder	Count only items matching the condition.
stringFilter	Builder	Count only items matching the condition.
toVar	String	Variable storing the size.
undefinedValue	int	Value to use when `fromVar` is unset or it does not contain any array/collection.

boolFilter

Property	Type	Description
value	boolean	Expected value.

intFilter

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

intFilter.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intFilter.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intFilter.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intFilter.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intFilter.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intFilter.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringFilter

Property	Type	Description
caseSensitive	boolean	True if the case must match, false if the check is case-insensitive.
endsWith	CharSequence	Suffix for the string.
equalTo	CharSequence	Literal value the string should match (the same as {@link #value}).
length	int	Check the length of the string.
length (alternative)	Builder	Check the length of the string.
matchVar	String	Fetch the value from a variable.
negate	boolean	Invert the logic of this condition. Defaults to false.
notEqualTo	CharSequence	Value that the string must not match.
startsWith	CharSequence	Prefix for the string.
value	CharSequence	Literal value the string should match.

stringFilter.length

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

stringFilter.length.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringFilter.length.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringFilter.length.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringFilter.length.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringFilter.length.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

stringFilter.length.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.3.11 - log

Log a message and variable values.

Inline definition
A pattern for string interpolation.

Property	Type	Description
message	String	Message format pattern. Use string interpolation for variables.

6.3.12 - markRequestInvalid

Unconditionally mark currently processed request as invalid.

6.3.13 - newSequence

Instantiates a sequence for each invocation.

Inline definition
Sequence name.

Property	Type	Description
concurrencyPolicy	enum	Options: `FAIL` `WARN`
forceSameIndex	boolean	Forces that the sequence will have the same index as the currently executing sequence. This can be useful if the sequence is passing some data to the new sequence using sequence-scoped variables. Note that the new sequence must have same concurrency factor as the currently executing sequence.
sequence	String	Name of the instantiated sequence.

6.3.14 - publishAgentData

Makes the data available to all sessions in the same agent, including those using different executors.

Inline definition
Both name of source variable and the key used to read the data.

Property	Type	Description
fromVar	String	Source session variable name.
name	String	Arbitrary unique identifier for the data.

6.3.15 - publishGlobalCounters

Gathers values from session variables and publishes them globally (to all agents).

Gathers values from session variables and publishes them globally (to all agents).
You can name the counters individually (example 1) or use the variable names (example 2):

# Example 1:
- publishGlobalCounters:
    key: myKey
    vars: [ foo, bar ]
# Example 2:
- publishGlobalCounters:
    key: someOtherKey
    vars:
    - foo: myFoo
    - bar: bbb
</code>```


| Property | Type | Description |
| ------- | ------- | -------- |
| key | String | Identifier of the global record. |
| vars | [Builder](#vars) | List of names and session variables. |

### vars

| Property | Type | Description |
| ------- | ------- | ------- |
| &lt;any&gt; | &lt;unknown&gt; | <font color="#606060">&lt;no description&gt;</font> |
| &lt;list of strings&gt; | &lt;unknown&gt; | <font color="#606060">&lt;no description&gt;</font> |

6.3.16 - readAgentData

Reads data from agent-wide scope into session variable.

Inline definition
Both the identifier and destination session variable.

Property	Type	Description
name	String	Unique identifier for the data.
toVar	String	Destination session variable name.

6.3.17 - removeItem

Removes element from an array of variables.

Property	Type	Description
fromVar	String	Variable containing an existing array of object variables.
index	Builder	Set index at which the item should be removed. Elements to the right of this are moved to the left.

index

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.3.18 - restartSequence

Schedules a restart of this sequence.

6.3.19 - set

Set variable in session to certain value.

Inline definition
Use `var <- value`.

Property	Type	Description
intArray	Builder	Set variable to an (unset) integer array.
objectArray	Builder	Set variable to an (unset) object array.
value	String	String value.
var	String	Variable name.

intArray

Creates integer arrays to be stored in the session.

Property	Type	Description
fromVar	String	Contents of the new array. If the variable contains an array or a list, items will be copied to the elements with the same index up to the size of this array. If the variable contains a different value all elements will be initialized to this value.
size	int	Size of the array.

objectArray

Creates object arrays to be stored in the session.

Property	Type	Description
fromVar	String	Contents of the new array. If the variable contains an array or a list, items will be copied to the elements with the same index up to the size of this array. If the variable contains a different value all elements will be initialized to this value.
size	int	Size of the array.

6.3.20 - setInt

Set session variable to an integral value.

Inline definition
Use `var <- value`.

Property	Type	Description
fromVar	String	Input variable name.
intCondition	Builder	Set variable only if the current value satisfies certain condition.
max	Builder	Set to value that is the maximum of this list of values.
min	Builder	Set to value that is the minimum of this list of values.
onlyIfNotSet	boolean	Set variable to the value only if it is not already set.
value	int	Value (integer).
var	String	Variable name.

intCondition

Property	Type	Description
equalTo	Builder	Compared variable must be equal to this value.
greaterOrEqualTo	Builder	Compared variable must be greater or equal to this value.
greaterThan	Builder	Compared variable must be greater than this value.
lessOrEqualTo	Builder	Compared variable must be lower or equal to this value.
lessThan	Builder	Compared variable must be lower than this value.
notEqualTo	Builder	Compared variable must not be equal to this value.

intCondition.equalTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.greaterThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessOrEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.lessThan

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

intCondition.notEqualTo

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

max

Property	Type	Description
<list of mappings>	<list of builders>	<no description>

max.<list of mappings>

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

min

Property	Type	Description
<list of mappings>	<list of builders>	<no description>

6.3.21 - setItem

Set element in a collection on given position.

Property	Type	Description
fromVar	String	Fetch value from session variable.
index	Builder	Integer session variable with an index into the collection.
toVar	String	Session variable with the collection.
value	String	Verbatim value.

index

Inline definition
Uses the argument as a constant value.

Property	Type	Description
fromVar	String	Input variable name.
value	int	Value (integer).

6.3.22 - setSharedCounter

Sets value in a counter shared by all sessions in the same executor.

Inline definition
Both the name of the counter and variable name.

Property	Type	Description
fromVar	String	Input variable name.
key	String	Identifier for the counter.
value	int	Value (integer).

6.3.23 - stringToInt

Parse string into integer and store it in a variable.

If the parsing fails the target variable is not modified.

Inline definition
Use `fromVar -> toVar`

Property	Type	Description
fromVar	String	Source variable name.
toVar	String	Target variable name.

6.3.24 - unset

Undefine variable name.

Inline definition
Variable name.

Property	Type	Description
var	Object	Variable name.

7 - Extensions

How to develop your own extensions

You have probably already read the Custom steps and handlers quickstart which shows how to create a simple component. It can get more tricky when the component embeds other components, though.

The build of scenario happens in two phases. In first phase the sequences, steps and components call method prepareBuild(). Most often that method uses the default (empty) implementation, but if your component (e.g. custom step) embeds another one (e.g. instance of a Processor) it should call its prepareBuild() method, too. The purpose of this method is mutatation of builders, for example adding extra steps to the scenario or registering handlers elsewhere. We’ll see how to do that later on.

In the second phase build(...) is called. At this point the builders tree must not be mutated further as some components might be already built and the change could not be reflected; this method should validate the builder and return the component.

Mutations of the scenario can be position dependent (e.g. adding one step before or after current step). Each builder that needs to know its position therefore must override two methods, setLocator(Locator) and copy(Locator). The former usually just sets a field storing the Locator and delegates the call to embedded components, the latter performs a deep copy of this builder, storing the provided Locator in the copy.

If you want to add some extra steps elsewhere in the scenario, you can implement the prepareBuild() method this way:

Java


private Locator locator;

/* ... */

@Override
public void prepareBuild() {
   // insert custom step before this step in this sequence
   locator.sequence().insertBefore(locator)
      .step(new CustomStep(42));

   // insert custom step after this step in this sequence, using a builder
   locator.sequence().insertAfter(locator)
      .stepBuilder(new CustomStep.Builder().foo(42));

   // insert a new sequence 'foo' with single custom step to the scenario
   locator.scenario().sequence("foo")
      .step(new CustomStep(42));
}

Note that when you insert any builders in the prepareBuild() methods it is possible that its prepare phase won’t be executed (if inserting to already prepared sequence), though it might be (if inserting to sequence that is yet to be prepared). It’s up to the calling code to make sure that the inserted component will be prepared.

As mentioned above, components often embed other components. To service-load a component, e.g. an Action you define these methods in the builder:

Java


// This method is not different from regular fluent setter
// and it's useful for programmatic configuration.
public Builder onEvent(Action.Builder onEvent) {
    // you can ensure here that this is called only once
    this.onEvent = onEvent;
    return this;
}

// This is the service-loading method.
public ServiceLoadedBuilderProvider<Action.Builder> onEvent() {
    return new ServiceLoadedBuilderProvider<>(Action.Builder.class, locator, this::onEvent);
}

The parser instantiates concrete implementation of the Action.Builder, calls its setters and then passes the builder to the consumer method referenced as this::onEvent. Note that the call to ServiceLoadedBuilderProvider constructor requires a Locator parameter, as the embedded Action can mutate the scenario later on.

8 - Controller API

OpenAPI 3 specification of the Hyperfoil controller

As a user you’ll probably interact with the Controller through CLI. However when you set up e.g. regression runs in CI you’ll need to control the test programmatically. Some limited capabilities are already exposed through Ansible Galaxy scripts but to get full control you can use the REST API - the same as the CLI or Ansible scripts connect to.

Hyperfoil Controller API is defined through OpenAPI 3 Specification. Our OpenAPI reference defines some types as free-form objects (e.g. the benchmark definition); refer to source code in these cases.

8.1 - Swagger UI

Hyperfoil controller OpenAPI swagger UI

9 - Architecture

Deep dive into the Hyperfoil architecture

While we have already explained basic concepts in the benchmark and last quickstart shows how to create a custom steps or handlers here we will show how Hyperfoil internally works and give you better idea how to create non-trivial extensions.

Building the scenario

The road from a YAML file to executing the benchmark starts with creating the builder tree. Either the CLI or controller presents this file to the parser (mostly classes from the io.hyperfoil.core.parser package) which reads it token-by-token and invoke methods on the io.hyperfoil.api.config.BenchmarkBuilder instance. Some parts of the parser are hard-coded, but most of them use reflection - that’s why you don’t need to write the parser yourselves.

Generally speaking each mapping (foo: bar) results in invoking method foo() on the builder; if this method accepts an argument (bar) the return value could be ignored - the method mutates the builder and that is all. Other methods do not accept any arguments and return another builder instance - the YAML subtree is then applies to this builder. The builder tree then roughly maps to the YAML tree in the original file.

When the YAML is fully read we execute the first phase of building the benchmark itself. We recursively (depth-first) call prepareBuild() methods on the tree; these methods are allowed to invoke further mutations on the builder tree, such as adding other steps and sequences. An empty (default) implementation of this method is perfectly fine if you don’t need anything complex, but if your extension delegates to children extensions it should recursively call the method on children builders. Make sure to iterate through a shallow copy of any collection of children as the children can mutate its parent, failing the iteration.

Hyperfoil does not track new components created in prepareBuild() and therefore it won’t call prepareBuild() on these if the part of builder tree was already processed - the code creating new components must call the method.

When the first phase completes we invoke the build() method on the builder tree. As the build method is invoked recursively again we end up with the benchmark tree that again mirrors the builder tree. The mapping between builders and benchmark components doesn’t need to be 1:1, e.g. StepBuilder can return several steps, builder can return wrapped components etc. However no mutations are allowed in this phase.

So we end up with the io.hyperfoil.api.config.Benchmark instance that holds this tree. It is important to make sure that this is immutable and serializable as it will be sent over the wire from controller to agents. This means that the components must not reference the builders. A common oversight is using a lambda that uses one of builder’s fields - while you only need the actual value of the field the lambda would capture a reference to the builder; this is quite easy to fix by assigning the field value to a local var, though.

Creating the sessions

Hyperfoil tries to minimize allocations during the benchmark as while Java garbage collector is a good friend of every developer it has a negative effect on the real-time properties of the program. You should use collectors that minimize pause times (such as Shenandoah or ZGC) rather than those that maximize throughput. This is why we allocate all what we could need ahead.

Before the benchmark starts each agent creates all the sessions (based on the maxSessions property in case of open-model phases). When Session.reserve() is called it calls reserve() method on all steps that implement ResourceUtilizer interface. In this method the step must call Session.declareResource() on all the resources it uses.

Note: in previous versions of Hyperfoil it was necessary to also explicitly declare that the step can write into a session variable, and recursively call the ResourceUtilizer.reserve() method on all children components. Since version 0.16 Hyperfoil discovers all ResourceUtilizer implementors in the scenario tree; the recursive invocation is no longer necessary.

Scenario execution

Session execution starts with:

one instance of each sequence declared in the initialSequences list
one instance of the first sequence in orderedSequences list
one instance of the first sequence in the implicit list of sequences (right under the scenario: declaration - this is mutually exclusive to the options above)

The session keeps a list of active sequence instances, each with an index of the step that should execute next. There can be several instances of the same sequence, up to its concurrency (the number in brackets next to sequence name). Whenever the session is notified (using Session.proceed() which schedules Session.call() invocation in its executor) it goes through all active instances and calls Step.invoke() on the current step.

There are two possible results from the invoke() method that returns a boolean:

true: This means that step has been successfuly executed, it’s complete and the sequence can continue with the next step (which it will, immediately) or complete if this was the last step.
false: We say that the step was blocked - it cannot execute immediately. It could be either because the step is short of resources (e.g. httpRequest cannot get any available connection from the pool) but most often this is because the purpose of this step is to wait for certain condition: variable being set/reaching certain value, request being completed etc. The sequence will not progress towards next step and this step’s invoke() method will be called again (when the Session.proceed() is called). An important property is that if the step returns false the step must not have any side-effects - it must not fire a request, set a session variable, simply it was a no-op. If the step acquired a resource from a pool it should return it prior to returning false.

Session variables

The session contains a map of variables the scenario uses. The keys are usually strings but this is not mandated; some steps may e.g. choose to use an unique object as the key. The values in the map are wrapper objects that hold a boolean flag whether this variable is set and the value itself. To avoid boxing and unboxing there’s a different wrapper for integers and other objects - it’s up to step to check the wrapper type and convert the value if necessary.

The map is not manipulated directly - a builder for a step that should work with variable foo should call SessionFactory.readAccess("foo"), SessionFactory.intAccess("foo") or SessionFactory.objectAccess("foo") in its build() method and pass the received Access to the step it creates. The step then operates exclusively using Access methods. And example of this can be found in the getting started: custom steps guide.

Sequence-scoped access

In quickstarts there are examples of sequence-scoped access - variables with [.] suffix, e.g. unset: myVar[.]. This is used when the variable holds an array of variables (wrappers) created using ObjectVar.newArray() or IntVar.newArray() in the ResourcesUtilizer.reserve() method - a common pattern would be

Java


@Override
public void reserve(Session session) {
    if (!var.isSet(session)) {
        var.setObject(session, ObjectVar.newArray(session, concurrency));
    }
}

The array is often created in a non-conurrent sequence that starts several concurrent instances of another sequence - the var would be used with the simple access (without the [.] suffix) in the original sequence, and with [.] in the concurrent sequences. Each of the concurrent sequences would get a different SequenceInstance.index() and with sequence-scoped access these would work on the variable on this position in the array. The step does not need to be tailored specifically to work on sequence-scoped variables; when creating the Access instance using SessionFactory.objectAccess() the presence of the suffix is automatically checked and the returned Access will relay the operations to the slot in the array.

Session resources

If a step (or a set of cooperating steps) needs to keep some internal state that is not available to users through arbitrary identifiers as session variables these can use concept called session resources. It is again an immutable map of objects in the session (while the map itself is immutable the values are meant to be mutated).

To make the code type-safe you start with the Resource implementation and matching ResourceKey:

Java


public class FooResource implements Session.Resource {
    /* ... */
}

public class FooResourceKey implements Session.ResourceKey<FooResource> {}

The resource key does not need any methods - it is just an unique marker object that will serve as the key in a map. If the resource will be used exclusively in this very step (action, processor…) you can implement the ResourceKey in there and use this when calling session.getResource():

Java


public class FooStep implements Step, ResourceUtilizer,
        SessionResourceKey<FooStep.FooResource> {
    /* ... */
}

Both session variables and session resources are declared in the reserve() method and retrieved (and mutated) in the business method (invoke() in the case of a step):

Java


public class FooStep implements Step, ResourceUtilizer {
    private final FooResourceKey resourceKey; // set in constructor

    @Override
    public void reserve(Session session) {
        // we are using supplier rather than creating instance directly because
        // if this sequence is concurrent we will create N resources, the state
        // of concurrent sequences will be isolated by default.
        session.declareResource(resourceKey, FooResource::new);
    }

    @Override
    public void invoke(Session session) {
        FooResource resource = session.getResource(resourceKey);
        /* work on the resource */
    }
}

Component adapters

There are several types of extension components: steps, actions and processors get extra attention but it is possible to use other interfaces as well. Actions are the simplest of these: these do not require any input (but the session) and do always execute without blocking. Therefore it is possible to use an action on any place where a step or processor would fit. When loading the component by name Hyperfoil automatically wraps the action into an adapter to the target component type.

Thread-local, agent-local and global data

Besides session variables Hyperfoil offers 3 more levels of memory. Neither of those is limited to the currently executing phase: this data is not reset until the run completes.

First level is the thread-local memory: since each session runs using single executor it is possible to share some data between sessions using the same executor without any need for synchronization. Currently this model supports shared counters (see addToSharedCounter and getSharedCounter) and shared map-like objects (see pushSharedMap and pullSharedMap). The latter keeps a pool of maps for each executor; when the map is pulled to a session it is removed from the pool, and it’s up to the user to return it back. This is useful e.g. for simulating stateful virtual users when we don’t want to modify one user concurrently in multiple sessions.

Second level is the agent-local memory. This is intended for caching data that needs to be initialized once and then used throughout the test; the initializing phase should invoke publishAgentData. It’s up to you to make sure that when this is read using readAgentData the data is already available - usually the reading phase should be ordered after the publishing phase using startAfterStrict property. Agent data are identified using keys (names); the data for each key can be published only once and cannot be updated afterwards. This limitation is imposed to minimize synchronization of executors.

Third level is the global mem ory. Again this could be used for distributing initialization data but also for gathering data from other agents and threads. The idea is that each thread or agent produces a reduce-able object; Hyperfoil then combines these objects on the agent level (in arbitrary order), sends it to the controller where data from all agents are combined again and the result is distributed back to all agents. As with the agent data you should strictly order the producing and consuming phases, otherwise the data might not be available yet and the run would fail. Currently Hyperfoil does not provide any general-use steps/actions to work with global data; you should implement GlobalData.Element in an extension and provide steps to create & publish instances of these.