Parallel Agents

Fan out work to multiple agents, then combine the results. Failed agents are retried independently while successful results are preserved.

Fan out work to multiple agents, then combine the results. Restate runs the agents in parallel with automatic retries and recovery. If one agent fails, only that agent is retried, the successful results are preserved.

Example: parallel claim analysis

Three specialist agents analyze a claim concurrently. A decision agent combines their results.

Vercel AI

workflow-parallel.ts

const run = async (ctx: restate.Context, claim: ClaimInput) => {
  const [eligibility, rateComparison, fraudCheck] = await RestatePromise.all([
    ctx.serviceClient(eligibilityAgent).run(claim),
    ctx.serviceClient(rateComparisonAgent).run(claim),
    ctx.serviceClient(fraudCheckAgent).run(claim),
  ]);

  const model = wrapLanguageModel({
    model: openai("gpt-5.4"),
    middleware: durableCalls(ctx, { maxRetryAttempts: 3 }),
  });

  const { text } = await generateText({
    model,
    system: "You are a claim decision engine.",
    prompt: `Decide about claim ${JSON.stringify(claim)}.
        Base your decision on the following analyses:
        Eligibility: ${eligibility}, Cost: ${rateComparison} Fraud: ${fraudCheck}`,
  });
  return text;
};

Try out parallel agents

Install Restate and launch it:

npm install --global @restatedev/restate-server@latest @restatedev/restate@latest
restate-server

Get the example:

restate example typescript-vercel-ai-tour-of-agents && cd typescript-vercel-ai-tour-of-agents
npm install

Export your OpenAI API key and run the agent:

export OPENAI_API_KEY=sk-...

npx tsx ./src/workflow-parallel.ts

Register the agents with Restate:

restate deployments register http://localhost:9080 --force --yes # dev only: overrides previous registrations

Start a request for a claim that needs to be analyzed by multiple agents in parallel:

curl localhost:8080/restate/call/ParallelAgentClaimApproval/run --json '{
    "date":"2024-10-01",
    "category":"orthopedic",
    "reason":"hospital bill for a broken leg",
    "amount":3000,
    "placeOfService":"General Hospital"
}'

In the UI, you can see that the handler called the sub-agents in parallel. Once all sub-agents return, the main agent makes a decision.

OpenAI Agents

workflow_parallel.py

@agent_service.handler()
async def run(restate_context: restate.Context, claim: InsuranceClaim) -> str:
    # Start multiple agents in parallel with auto retries and recovery
    eligibility = restate_context.service_call(run_eligibility_agent, claim)
    cost = restate_context.service_call(run_rate_comparison_agent, claim)
    fraud = restate_context.service_call(run_fraud_agent, claim)

    # Wait for all responses
    await restate.gather(eligibility, cost, fraud)

    # Run decision agent on outputs
    result = await DurableRunner.run(
        Agent(
            name="ClaimApprovalAgent", instructions="You are a claim decision engine."
        ),
        input=f"Decide about claim: {claim.model_dump_json()}. "
        "Base your decision on the following analyses:"
        f"Eligibility: {await eligibility} Cost {await cost} Fraud: {await fraud}",
    )
    return result.final_output

Try out parallel agents

Install Restate and launch it:

restate-server

Get the example:

restate example python-openai-agents-tour-of-agents && cd python-openai-agents-tour-of-agents

Export your OpenAI API key and run the agent:

export OPENAI_API_KEY=sk-...

uv run app/workflow_parallel.py

Register the agents with Restate:

restate deployments register http://localhost:9080 --force --yes # dev only: overrides previous registrations

Start a request for a claim that needs to be analyzed by multiple agents in parallel:

curl localhost:8080/restate/call/ParallelAgentClaimApproval/run --json '{
    "date":"2024-10-01",
    "category":"orthopedic",
    "reason":"hospital bill for a broken leg",
    "amount":3000,
    "placeOfService":"General Hospital"
}'

In the UI, you can see that the handler called the sub-agents in parallel. Once all sub-agents return, the main agent makes a decision.

Google ADK

workflow_parallel.py

@agent_service.handler()
async def run(ctx: restate.ObjectContext, claim: InsuranceClaim) -> str | None:

    # Start multiple agents in parallel with auto retries and recovery
    eligibility = ctx.service_call(run_eligibility_agent, claim)
    cost = ctx.service_call(run_rate_comparison_agent, claim)
    fraud = ctx.service_call(run_fraud_agent, claim)

    # Wait for all responses
    await restate.gather(eligibility, cost, fraud)

    # Get the results
    eligibility_result = await eligibility
    cost_result = await cost
    fraud_result = await fraud

    # Run decision agent on outputs
    prompt = f"""Decide about claim: {claim.model_dump_json()}. Assessments:
    Eligibility: {eligibility_result} Cost: {cost_result} Fraud: {fraud_result}"""

    events = runner.run_async(
        user_id=ctx.key(),
        session_id=claim.session_id,
        new_message=Content(role="user", parts=[Part.from_text(text=prompt)]),
    )
    return await parse_agent_response(events)

Try out parallel agents

Install Restate and launch it:

restate-server

Get the example:

restate example python-google-adk-tour-of-agents && cd python-google-adk-tour-of-agents

Export your Google API key and run the agent:

export GOOGLE_API_KEY=your-api-key

uv run app/workflow_parallel.py

Register the agents with Restate:

restate deployments register http://localhost:9080 --force --yes # dev only: overrides previous registrations

Start a request for a claim that needs to be analyzed by multiple agents in parallel:

curl localhost:8080/restate/call/ParallelAgentClaimApproval/user123/run --json '{
    "amount": 3000,
    "category": "orthopedic",
    "date": "2024-10-01",
    "placeOfService": "General Hospital",
    "reason": "hospital bill for a broken leg",
    "sessionId": "session-123"
}'

In the UI, you can see that the handler called the sub-agents in parallel. Once all sub-agents return, the main agent makes a decision.

Pydantic AI

workflow_parallel.py

@agent_service.handler()
async def run(ctx: restate.Context, claim: InsuranceClaim) -> str:
    # Start multiple agents in parallel with auto retries and recovery
    eligibility = ctx.service_call(run_eligibility_agent, claim)
    cost = ctx.service_call(run_rate_comparison_agent, claim)
    fraud = ctx.service_call(run_fraud_agent, claim)

    # Wait for all responses
    await restate.gather(eligibility, cost, fraud)

    # Run decision agent on outputs
    result = await restate_decision_agent.run(
        f"Decide about claim: {claim.model_dump_json()}. "
        "Base your decision on the following analyses:"
        f"Eligibility: {await eligibility} Cost {await cost} Fraud: {await fraud}",
    )
    return result.output

Try out parallel agents

Install Restate and launch it:

restate-server

Get the example:

restate example python-pydantic-ai-tour-of-agents && cd python-pydantic-ai-tour-of-agents

Export your OpenAI API key and run the agent:

export OPENAI_API_KEY=sk-...

uv run app/workflow_parallel.py

Register the agents with Restate:

restate deployments register http://localhost:9080 --force --yes # dev only: overrides previous registrations

Start a request for a claim that needs to be analyzed by multiple agents in parallel:

curl localhost:8080/restate/call/ParallelAgentClaimApproval/run --json '{
    "date":"2024-10-01",
    "category":"orthopedic",
    "reason":"hospital bill for a broken leg",
    "amount":3000,
    "placeOfService":"General Hospital"
}'

In the UI, you can see that the handler called the sub-agents in parallel. Once all sub-agents return, the main agent makes a decision.

LangChain

workflow_parallel.py

decision = create_agent(
    model=init_chat_model("openai:gpt-5.4"),
    system_prompt="You are a claim decision engine.",
    middleware=[RestateMiddleware()],
)


agent_service = restate.Service("ParallelAgentClaimApproval")


@agent_service.handler()
async def run(ctx: restate.Context, claim: InsuranceClaim) -> str:
    # Start multiple sub-agents in parallel with auto-retries and recovery.
    eligibility = ctx.service_call(run_eligibility_agent, claim)
    cost = ctx.service_call(run_rate_comparison_agent, claim)
    fraud = ctx.service_call(run_fraud_agent, claim)

    await restate.gather(eligibility, cost, fraud)

    result = await decision.ainvoke(
        {"messages": f"""Decide about claim: {claim.model_dump_json()}.
        Base your decision on the following analyses:
        Eligibility: {await eligibility} Cost {await cost} Fraud: {await fraud}"""}
    )
    return result["messages"][-1].content

Try out parallel agents

Install Restate and launch it:

restate-server

Get the example:

restate example python-langchain-tour-of-agents && cd python-langchain-tour-of-agents

Export your OpenAI API key and run the agent:

export OPENAI_API_KEY=sk-...

uv run app/workflow_parallel.py

Register the agents with Restate:

restate deployments register http://localhost:9080 --force --yes # dev only: overrides previous registrations

Start a request for a claim that needs to be analyzed by multiple agents in parallel:

curl localhost:8080/restate/call/ParallelAgentClaimApproval/run --json '{
    "date":"2024-10-01",
    "category":"orthopedic",
    "reason":"hospital bill for a broken leg",
    "amount":3000,
    "placeOfService":"General Hospital"
}'

In the UI, you can see that the handler called the sub-agents in parallel. Once all sub-agents return, the main agent makes a decision.

Restate TS

workflow-parallel.ts

async function run(ctx: Context, claim: ClaimInput) {
  // Create parallel tasks - each runs independently
  const claimJson = JSON.stringify(claim);
  const eligibility = ctx.run(
    "Eligibility agent",
    async () => llmCall(
        "Decide whether the following claim is eligible for reimbursement." +
        "Respond with eligible if it's a medical claim, and not eligible otherwise." +
        "\n\nClaim: " + claimJson,
    ),
    { maxRetryAttempts: 3 },
  )
  const fraud = ctx.run(
    "Fraud agent",
      async () => llmCall(
          "Decide whether the claim is fraudulent." +
          "Always respond with low risk, medium risk, or high risk." +
          "\n\nClaim: " + claimJson,
    ),
    { maxRetryAttempts: 3 },
  )
  const cost = ctx.run(
    "Rate comparison agent",
      async () => llmCall(
          "Decide whether the cost of the claim is reasonable given the treatment." +
          "Respond with reasonable or not reasonable." +
          "\n\nClaim: " + claimJson,
    ),
    { maxRetryAttempts: 3 },
  )

  // Wait for all tasks to complete and return the results
  await RestatePromise.all([eligibility, cost, fraud]);

  // Make final decision
  const { text } = await ctx.run(
      "Decision agent",
      async () => llmCall( `Decide about claim ${JSON.stringify(claim)}.
        Base your decision on the following analyses:
        - Eligibility: ${(await eligibility).text}, 
        - Cost: ${(await cost).text},
        - Fraud: ${(await fraud).text}`)
  );
  return text
}

Run this example

Install Restate and launch it:

restate-server

Get the example:

restate example typescript-restate-tour-of-agents && cd typescript-restate-tour-of-agents
npm install

Export your API key:

export OPENAI_API_KEY=sk-...

npx tsx ./src/workflow-parallel.ts

Register the services with Restate:

restate deployments register http://localhost:9080 --force --yes # dev only: overrides previous registrations

Send a request:

curl localhost:8080/restate/call/ParallelAgentClaimApproval/run --json '{
    "date":"2024-10-01",
    "category":"orthopedic",
    "reason":"hospital bill for a broken leg",
    "amount":3000,
    "placeOfService":"General Hospital"
}'

Restate Py

workflow_parallel.py

parallelization_svc = restate.Service("ParallelAgentClaimApproval")


@parallelization_svc.handler()
async def run(ctx: restate.Context, claim: ClaimData) -> str | None:
    """Analyzes a claim in parallel with specialized agents."""

    # Create parallel tasks - each runs independently
    claim_json = json.dumps(claim.model_dump())
    eligibility = ctx.run_typed(
        "Eligibility agent",
        llm_call,
        RunOptions(max_attempts=3),
        messages="Decide whether the following claim is eligible for reimbursement."
        " Respond with eligible if it's a medical claim, and not eligible otherwise."
        f"\n\nClaim: {claim_json}",
    )
    fraud = ctx.run_typed(
        "Fraud agent",
        llm_call,
        RunOptions(max_attempts=3),
        messages="Decide whether the claim is fraudulent."
        " Always respond with low risk, medium risk, or high risk."
        f"\n\nClaim: {claim_json}",
    )
    rate = ctx.run_typed(
        "Rate comparison agent",
        llm_call,
        RunOptions(max_attempts=3),
        messages="Decide whether the cost of the claim is reasonable given the treatment."
        " Respond with reasonable or not reasonable."
        f"\n\nClaim: {claim_json}",
    )

    # Wait for all tasks to complete
    await restate.gather(eligibility, fraud, rate)

    # Make final decision
    decision = await ctx.run_typed(
        "Decision agent",
        llm_call,
        RunOptions(max_attempts=3),
        messages=f"Decide about claim: {claim.model_dump_json()}. "
        "Base your decision on the following analyses:"
        f"Eligibility: {(await eligibility).content} "
        f"Cost: {(await rate).content} "
        f"Fraud: {(await fraud).content}",
    )
    return decision.content

Run this example

Install Restate and launch it:

restate-server

Get the example:

restate example python-restate-tour-of-agents && cd python-restate-tour-of-agents

Export your API key:

export OPENAI_API_KEY=sk-...

uv run app/workflow_parallel.py

Register the services with Restate:

restate deployments register http://localhost:9080 --force --yes # dev only: overrides previous registrations

Send a request:

curl localhost:8080/restate/call/ParallelAgentClaimApproval/run --json '{
    "date":"2024-10-01",
    "category":"orthopedic",
    "reason":"hospital bill for a broken leg",
    "amount":3000,
    "placeOfService":"General Hospital"
}'