Phases - advanced

Delve into more advanced phase configuration

Previous quickstart presented a benchmark with three phases that all started at the same moment (when the benchmark was started) and had the same duration - different phases represented different workflows (types of user). In this example we will adjust the benchmark to scale the load gradually up.

At this point it would be useful to mention the lifecycle of phases; phase is in one of these states:

  • not started: As the name clearly says, the phase is not yet started.
  • running: The agent started running the phase, i.e., performing the configured load.
  • finished: When the duration elapses, no more new users are started. However, some might be still executing their scenarios.
  • terminated: When all users complete their scenarios the phase becomes terminated. Users may be forcibly interrupted by setting maxDuration on the phase.
  • cancelled If the benchmark cannot continue further, all remaining stages are cancelled.

Let’s take a look into the example, where we’ll slowly (over 5 seconds) increase load to 10+5 users/sec, run with this load for 10 seconds, again increase it by another 10+5 users/sec and so forth until we reach 100+50 users per second. As we define maxIterations for these phases the benchmark will actually contain phases browsingUserRampUp/0, browsingUserRampUp/1, browsingUserRampUp/2 and so forth.

name: eshop-scale
http:
  host: http://localhost:8080
  sharedConnections: 80
phases:
- browsingUserRampUp:
    # This type of phase is similar to constantRate in the way how new users
    # are started but gradually increases the rate from `initialUsersPerSec`
    # to `targetUsersPerSec`.
    increasingRate:
      duration: 5s
      # In Hyperfoil, everything is pre-allocated = limited in size. Here we'll
      # set that we won't run more than 10 iterations of this phase.
      maxIterations: 10
      # Number of started users per sec increases with the iteration; in first
      # iteration we'll go from 0 to 10 users/second, in second from 10 to 20
      # and in last (10th) we'll reach 100 users/second.
      initialUsersPerSec:
        base: 0
        increment: 10
      targetUsersPerSec:
        base: 10
        increment: 10
      # Nth iteration of this phase will start when (N-1)th iteration of other
      # steady-state phases are finished. First iteration can start
      # immediatelly, of course.
      startAfter:
      - phase: browsingUserSteady
        iteration: previous
      - phase: buyingUserSteady
        iteration: previous
      # The &browsingUser syntax below creates YAML alias: we can later
      # reference this scenario and it will be used verbatim in another phase.
      # It is possible to use aliases for both scenarios and sequences.
      scenario: &browsingUser
      # We'll use the same scenario as in eshop.hf.yaml
      - browse:
        - httpRequest:
            GET: /quickstarts/eshop/items
- browsingUserSteady:
    constantRate:
      duration: 10s
      maxIterations: 10
      usersPerSec:
        base: 10
        increment: 10
      # Nth iteration of this phase will start when Nth iteration of ramp-up
      # phases is finished.
      # Note that there's implicit rule that Nth iteration of given phase will
      # start only after (N-1)th iteration terminates.
      startAfter:
      - phase: browsingUserRampUp
        iteration: same
      - phase: buyingUserRampUp
        iteration: same
      # This refers to the alias created above; in steady state we'll use the
      # same scenario.
      scenario: *browsingUser
# These two phases will be very similar to browsingUserSteady and RampUp
- buyingUserRampUp:
    increasingRate:
      duration: 5s
      maxIterations: 10
      initialUsersPerSec:
        base: 0
        increment: 5
      targetUsersPerSec:
        base: 5
        increment: 5
      startAfter:
      - phase: browsingUserSteady
        iteration: previous
      - phase: buyingUserSteady
        iteration: previous
      # Again we'll use the same scenario as in eshop.hf.yaml
      scenario: &buyingUser
      - browse:
        - httpRequest:
            GET: /quickstarts/eshop/items
            handler:
              body:
                json:
                  query: .[].id
                  toArray: itemIds[10]
      - buy:
        - randomItem: itemId <- itemIds
        - httpRequest:
            POST: /quickstarts/eshop/items/${itemId}/buy
- buyingUserSteady:
    constantRate:
      duration: 10s
      maxIterations: 10
      usersPerSec:
        base: 5
        increment: 5
      startAfter:
      - phase: browsingUserRampUp
        iteration: same
      - phase: buyingUserRampUp
        iteration: same
      scenario: *buyingUser
# Operator phase is omitted for brevity as we wouldn't scale that up

Don’t forget to start the mock server as we’ve used in the previous quickstart.

podman run --rm -p 8080:8083 quay.io/hyperfoil/hyperfoil-examples

Synchronizing multiple workloads across iteration can become a bit cumbersome. That’s why we can keep similar types of workflow together, and split the phase into forks. In fact forks will become different phases, but these will be linked together so that you can refer to all of them as to a single phase. Take a look at the benchmark rewritten to use forks:

name: eshop-forks
http:
  host: http://localhost:8080
  sharedConnections: 80
phases:
- rampUp:
    increasingRate:
      duration: 5s
      maxIterations: 10
      # Note that we have increased both the base and increment from 10 and 5
      # to 15. This value is split between the forks based on their weight.
      initialUsersPerSec:
        base: 0
        increment: 15
      targetUsersPerSec:
        base: 15
        increment: 15
      startAfter:
        phase: steadyState
        iteration: previous
      forks:
        browsingUser:
          weight: 2
          scenario: &browsingUser
          - browse:
            - httpRequest:
                GET: /quickstarts/eshop/items
        buyingUser:
          weight: 1
          scenario: &buyingUser
          - browse:
            - httpRequest:
                GET: /quickstarts/eshop/items
                handler:
                  body:
                    json:
                      query: .[].id
                      toArray: itemIds[10]
          - buy:
            - randomItem: itemId <- itemIds
            - httpRequest:
                POST: /quickstarts/eshop/items/${itemId}/buy
- steadyState:
    constantRate:
      duration: 10s
      maxIterations: 10
      usersPerSec:
        base: 15
        increment: 15
      startAfter:
        phase: rampUp
        iteration: same
      forks:
        browsingUser:
          weight: 2
          scenario: *browsingUser
        buyingUser:
          weight: 1
          scenario: *buyingUser
# Operator phase is omitted for brevity as we wouldn't scale that up

This definition will create phases rampUp/0/browsingUser, rampUp/0/buyingUser, rampUp/1/browsingUser etc. - you’ll see them in statistics.

You could orchestrate the phases as it suits you, using startAfter, startAfterStrict (this requires the referenced phase to me terminated instead of finished as with startAfter) or startTime with relative time since benchmark start.

This sums up basic principles, in next quickstart you’ll see how to start and use Hyperfoil in distributed mode.


Last modified November 20, 2024: Update website refs for 0.27.1 release (7c0ac65)