Defining Automatic Block-Retry Policies

Defining Automatic Block-Retry Policies

In order for a submission’s processing to be completed, each block in its flow must finish its task successfully. If a block fails, the submission becomes halted, which may cause the submission to breach its SLA. When you define automatic retry policies for blocks, you minimize the chances of submissions being halted due to temporary outages, which prevents processing delays from occurring. For each retry policy, you can specify the number of times blocks should be retried, along with the amount of time that should pass between retry attempts.

Defining block-retry policies

You can define block-retry policies at the following levels:

When defining these policies, note that each retry may not occur after the exact amount of time specified for the retry interval. Retries are scheduled to occur after their retry intervals pass, but depending on the priority of the blocks’ tasks, they may not be executed at their scheduled times.

This article describes how to define block-retry policies in the application. If you’re creating custom flows, see the Flows SDK documentation to learn how to define these policies with the Flows SDK.

Defining the default block-retry policy

To define the system-level block-retry policy:

  1. Go to /admin/hyperflow/wfeconfig/, and click the number in the ID column.
  2. In the Default error handling policy field, enter the details of your retry policy in JSON format:
   {
     "block_error_retry_policy": {
       "method": "",
       "retry_count": ,
       "retry_interval_seconds":
     }
   }

The table below describes the valid method values and their effects on retry-attempt intervals:

method value Effect on retry-attempt intervals
FIXED The number of seconds specified in retry_interval_seconds is used as the retry-attempt interval for all retry attempts.
LINEAR_BACKOFF The number of seconds between retry attempts is calculated as follows:
retry_interval_seconds * the number of the attempt
Example
If retry_interval_seconds = 5, the system waits 5 seconds to initiate the first retry attempt (5 * 1). If that attempt fails, the system waits 10 seconds to initiate the second attempt (5 * 2), and so on.
EXPONENTIAL_BACKOFF The number of seconds between retry attempts is calculated as follows:
retry_interval_seconds * 2^(the number of the attempt - 1)
Example
If retry_interval_seconds = 5, the system waits 5 seconds to initiate the first retry attempt (5 * 1 = 5). If that attempt fails, the system waits 10 seconds to initiate the second attempt (5 * 2 = 10), and so on.
Note that the difference between LINEAR_BACKOFF and EXPONENTIAL_BACKOFF only takes effect from the third attempt onward.

Example policies

Defining a block-retry policy at the flow level

To define a block-retry policy at the flow level:

  1. Go to the Flows page, and click on the name of the flow you want to define a block-retry policy for.
  2. In the flow settings, find the Retry Failed Blocks section, and click Edit.
  3. Select one of the following options:
    1. System Default
    2. Do not retry failed blocks in this flow
    3. Custom rule for this flow
  4. If you selected Custom rule for this flow, click More Options to reveal the options available to you, and:
    1. Enter the number of times blocks should be retried after they fail.
    2. Enter the number of minutes and seconds, that should pass before the initial retry attempt.
    3. To determine subsequent retry intervals, choose one of the following:
      • Linear Function — initial retry-attempt interval * the number of the attempt
      • Exponential Function — initial retry-attempt interval * 2^(the number of the attempt - 1)
      • Keep wait time constant — Same retry-attempt interval is used between all attempts.
  5. Click Confirm and Save to save your changes.

Defining a block-retry policy for an individual block

To define a block-retry policy for a specific block:

  1. Go to the Flows page, and click on the name of the flow containing the block you want.
  2. Click on the block in Flow Studio, find the Retry Failed Blocks section in its settings, and click Edit.
  3. Select one of the following options:
    1. Inherit from flow retry policy
    2. Do not retry when failed
    3. Custom rule for this block
  4. If you selected Custom rule for this block, click More Options to reveal the options available to you and:
    1. Provide the number of times the block should be retried after it fails.
    2. Provide the number of minutes and seconds for the initial retry attempt.
    3. Choose the subsequent retry intervals as described above.
  5. Click Confirm and Save to save your changes.

Applying retry policies to sub-flows

If you’ve connected a flow block to another flow, and that flow is not a notification flow, the block’s retry policy applies to the connected flow and its sub-flows.

If you’ve connected a flow block to a notification flow, you need to define a separate retry policy for the connected flow.

Best practices for retry policies

When determining the length of your retry-attempt intervals, consider the following: