4.8.8. Retry Failed Steps

In this chapter, you’ll learn how to configure steps to allow retry on failure.

What is a Step Retry?#

A step retry is a mechanism that allows a step to be retried automatically when it fails. This is useful for handling transient errors, such as network issues or temporary unavailability of a service.

By default, when a step fails, the workflow execution stops, and the workflow is marked as failed. However, you can configure a step to retry on failure.

When a step fails, you can configure the workflow engine to automatically retry the step a specified number of times before marking the workflow as failed. This can help improve the reliability and resilience of your workflows.

You can also configure the interval between retries, allowing you to wait for a certain period before attempting the step again. This is useful when the failure is due to a temporary issue that may resolve itself after some time.

For example, if a step captures a payment, you may want to retry it daily until the payment is successful or the maximum number of retries is reached.

Configure a Step’s Retry#

By default, when an error occurs in a step, the step and the workflow fail, and the execution stops.

You can configure the step to retry on failure. The createStep function can accept a configuration object instead of the step’s name as a first parameter.

For example:

import { MedusaError } from "@medusajs/framework/utils"

const step1 = createStep(
  {
    name: "step-1",
    maxRetries: 2,
  },
  async () => {
    console.log("Executing step 1")

    throw new MedusaError(
      MedusaError.Types.UNEXPECTED_STATE,
      "Oops! Something happened."
    )
  }
)

const myWorkflow = createWorkflow(
  "hello-world", 
  function () {
  const str1 = step1()

  return new WorkflowResponse({
    message: str1,
  })
})

export default myWorkflow

The step’s configuration object accepts a maxRetries property, which is a number indicating the number of times a step can be retried when it fails.

When you execute the above workflow, you’ll see the following result in the terminal:

Executing step 1
Executing step 1
Executing step 1
error:   Oops! Something happened.
Error: Oops! Something happened.

The first line indicates the first time the step was executed, and the next two lines indicate the times the step was retried. After that, the step and workflow fail.

Disable Automatic Retries#

By default, a step configured with the maxRetries property will be retried automatically when it fails.

You can disable automatic retries by setting the autoRetry property to false in the step's configuration object. Then, when the step fails, its status will be set to temporary failure, and you'll need to manually retry it using the Workflow Engine Module Service.

For example, to disable automatic retries:

const step1 = createStep(
  {
    name: "step-1",
    maxRetries: 2,
    autoRetry: false, // Disable automatic retries
  },
  async () => {
    console.log("Executing step 1")

    throw new MedusaError(
      MedusaError.Types.UNEXPECTED_STATE,
      "Oops! Something happened."
    )
  }
)

This step will not be retried automatically when it fails. Instead, you'll need to manually retry it, as explained in the Manually Retry a Step section.

Step Retry Intervals#

By default, a step is retried immediately after it fails. To specify a wait time before a step is retried, pass a retryInterval property to the step's configuration object. Its value is a number of seconds to wait before retrying the step.

For example:

const step1 = createStep(
  {
    name: "step-1",
    maxRetries: 2,
    retryInterval: 2, // 2 seconds
  },
  async () => {
    console.log("Executing step 1")

    throw new MedusaError(
      MedusaError.Types.UNEXPECTED_STATE,
      "Oops! Something happened."
    )
  }
)

In this example, if the step fails, it will be retried after two seconds.

Maximum Retry Interval#

The retryInterval property's maximum value is Number.MAX_SAFE_INTEGER. So, you can set a very long wait time before the step is retried, allowing you to retry steps after a long period.

For example, to retry a step after a day:

const step1 = createStep(
  {
    name: "step-1",
    maxRetries: 2,
    retryInterval: 86400, // 1 day
  },
  async () => {
    console.log("Executing step 1")

    throw new MedusaError(
      MedusaError.Types.UNEXPECTED_STATE,
      "Oops! Something happened."
    )
  }
)

In this example, if the step fails, it will be retried after 86400 seconds (one day).

Interval Changes Workflow to Long-Running#

By setting retryInterval on a step, a workflow that uses that step becomes a long-running workflow that runs asynchronously in the background. This is useful when creating workflows that may fail and should run for a long time until they succeed, such as waiting for a payment to be captured or a shipment to be delivered.

However, since the long-running workflow runs in the background, you won't receive its result or errors immediately when you execute the workflow.

Instead, you must subscribe to the workflow's execution using the Workflow Engine Module's service. Learn more about this in the Long-Running Workflows chapter.

Retry Waiting Async Steps#

An async step whose function doesn't return a StepResponse stays in a waiting state until you set its status with the Workflow Engine Module's setStepSuccess or setStepFailure methods. The maxRetries and retryInterval properties don't apply to a waiting step, because the step hasn't failed.

So, an async step can stay waiting indefinitely. For example:

• The process running the step dies or restarts before the step finishes, so nothing sets the step's status.
• The step starts a job in a third-party service, and you want to check the job's status periodically until it finishes.

To execute a step again while it's still waiting, pass the following properties to the step's configuration object:

• retryIntervalAwaiting: The number of seconds to wait after the step's last execution before executing it again, if the step is still waiting.
• maxAwaitingRetries: The maximum number of times the step can execute, including its first execution. If you don't set it, the workflow engine keeps executing the step until its status changes.

For example:

import {
  createStep,
  StepResponse,
} from "@medusajs/framework/workflows-sdk"

const waitForImportStep = createStep(
  {
    name: "wait-for-import",
    async: true,
    retryIntervalAwaiting: 60, // 1 minute
    maxAwaitingRetries: 10,
  },
  async ({ job_id }: { job_id: string }, { container }) => {
    const erpModuleService = container.resolve("erp")

    const job = await erpModuleService.getImportJob(job_id)

    if (job.status !== "completed") {
      return
    }

    return new StepResponse(job.result)
  }
)

In this example, the step retrieves the status of an import job from a custom ERP Module. If the job isn't completed, the step returns nothing, so it stays waiting. The workflow engine executes the step again after one minute.

Once the job is completed, the step returns a StepResponse, and the workflow continues to the next step.

How the Workflow Engine Counts Awaiting Retries#

The maxAwaitingRetries and maxRetries properties count executions differently:

• The workflow engine compares maxAwaitingRetries to the step's attempt counter. The counter includes every execution of the step, including the first one. So, a step with maxAwaitingRetries: 10 executes at most ten times in total.
• The workflow engine compares maxRetries to the number of times the step failed. So, a step with maxRetries: 2 executes at most three times in total: the first execution and two retries.

You can access the current attempt number in the step function using the attempt property of the step's second parameter.

Step Stays Waiting After the Last Awaiting Retry#

When a step reaches maxAwaitingRetries and is still waiting, the workflow engine stops executing it. However, the step doesn't fail. It stays waiting, and the workflow doesn't continue.

To fail the step after a period of time instead, set the step's timeout property, as explained in the Workflow Timeout chapter. You can also set the step's status with the Workflow Engine Module's setStepSuccess or setStepFailure methods, or retry it as explained in the Manually Retry a Step section.

Make the Step Safe to Execute Multiple Times#

The workflow engine executes a waiting step again even if a previous execution is still running, for example, when an external job takes longer than retryIntervalAwaiting. So, make sure that executing the step multiple times doesn't cause side effects, such as creating duplicate records or charging a customer twice.

Manually Retry a Step#

In some cases, you may need to manually retry a step. For example:

• If a step's autoRetry property is set to false.
• If the machine running the Medusa application in development or worker mode dies or shuts down.
• If the step takes longer than expected to complete.

For async steps, you can also configure the workflow engine to execute a waiting step again automatically, as explained in the Retry Waiting Async Steps section.

To retry a step manually, resolve the Workflow Engine Module's service from the Medusa container and call its retryStep method.

For example, to do it in an API route:

import { MedusaRequest, MedusaResponse } from "@medusajs/framework/http"
import {
  TransactionHandlerType,
  Modules,
} from "@medusajs/framework/utils"

export async function GET(
  req: MedusaRequest,
  res: MedusaResponse
) {
  const workflowEngine = req.scope.resolve(
    Modules.WORKFLOW_ENGINE
  )

  workflowEngine.retryStep({
    idempotencyKey: {
      action: TransactionHandlerType.INVOKE,
      transactionId: req.validatedQuery.transaction_id as string,
      stepId: "step-1",
      workflowId: "hello-world",
    },
  })

  res.json({ message: "Step retry initiated" })
}

When you send a request to the API route, the workflow execution will resume, retrying the specified step.

Learn more about the retryStep method in the Workflow Engine Module Service reference.