Cost and Safety Controls

Cost and safety controls bound LLM agent execution through iteration caps, timeouts, rate limits, and concurrency limits – preventing runaway API spend and unbounded computation.

The Risk

LLM API calls cost money. An agent loop with no bounds can iterate indefinitely, accumulating token costs. Parallel agents multiply the problem. A single misconfigured workflow can generate thousands of API calls in minutes.

DagNats does not track token counts or dollar amounts directly. Instead, it provides structural bounds that limit the number of LLM calls an agent can make and the time it can spend making them. These are blunt but effective proxies for cost.

Max Iterations as Token Budget Proxy

MaxIterations on an agent loop caps the number of reasoning cycles. Each iteration typically involves one LLM call, so MaxIterations is a direct proxy for maximum API calls per agent step.

agent := wf.AgentLoop("agent", "llm-task").
    WithMaxIterations(20).     // at most 20 LLM calls
    WithMaxDuration(10 * time.Minute)

If an agent hits MaxIterations, the step fails. Design the agent prompt to complete well before the cap. The cap is a safety net, not a target.

Timeouts as Spend Caps

MaxDuration bounds total wall-clock time. Timeout bounds per-iteration time. Together they prevent both slow accumulation and individual hung calls.

agent := wf.AgentLoop("agent", "llm-task").
    WithMaxIterations(30).
    WithMaxDuration(15 * time.Minute). // total time cap
    WithTimeout(60 * time.Second).     // per-call cap
    WithLoopDelay(500 * time.Millisecond)

MaxDuration is the hard ceiling. If the agent has done 5 iterations in 14 minutes, it will not start iteration 6 if MaxDuration is 15 minutes. Timeout catches individual LLM calls that hang (model provider issues, network problems).

For normal (non-loop) steps that call LLMs, the step-level timeout serves the same purpose.

Rate Limits per Model/Provider

Rate limiting controls how many tasks of a given type execute per time window. Use this to stay within API provider rate limits:

wf := dag.NewWorkflow("rate-limited-pipeline")

// Each step using the LLM gets rate-limited
agent := wf.AgentLoop("agent", "llm-gpt4").
    WithMaxIterations(20).
    WithRateLimit(10, time.Minute) // 10 calls per minute

Rate limits are enforced at the engine level. If the limit is reached, tasks queue until the window resets. This prevents bursting through provider rate limits and incurring 429 errors.

For different models with different rate limits, register separate task types:

w.Handle("llm-gpt4", gpt4Handler)      // rate limited to 10/min
w.Handle("llm-claude", claudeHandler)   // rate limited to 20/min

Concurrency Limits for Parallel API Calls

Concurrency limits bound how many LLM tasks execute simultaneously. This is important for map steps that fan out tool calls:

tools := wf.Map("tools", "llm-tool-call").
    After(plan).
    WithMaxItems(20).
    WithConcurrency(5) // at most 5 simultaneous API calls

Without concurrency limits, a map step with 20 items dispatches all 20 tasks simultaneously. If each calls an LLM API, that is 20 concurrent requests. Concurrency limits cap the parallelism.

Planner Step Bounds

Planner steps that let LLMs generate DAG fragments have their own bounds:

plan := wf.Planner("plan", "generate-plan", dag.PlannerConfig{
    MaxSteps:     10,               // cap generated steps
    MaxDepth:     3,                // cap dependency chain depth
    AllowedTasks: []string{         // restrict available task types
        "code-edit", "test-run",
    },
})

Configuration Summary

wf := dag.NewWorkflow("bounded-agent")

agent := wf.AgentLoop("agent", "llm-task").
    WithMaxIterations(20).             // iteration cap
    WithMaxDuration(10 * time.Minute). // time cap
    WithTimeout(60 * time.Second).     // per-iteration timeout
    WithLoopDelay(1 * time.Second).    // spacing between iterations
    WithRateLimit(10, time.Minute).    // provider rate limit
    WithRetries(2)                     // retry transient failures

plan := wf.Planner("plan", "generate-plan", dag.PlannerConfig{
    MaxSteps:     10,
    MaxDepth:     3,
    AllowedTasks: []string{"code-edit", "test-run"},
}).After(agent)

def, _ := wf.Build()

Every bound is explicit in the workflow definition. There are no implicit defaults that silently allow unbounded execution.

Related

• Agent Loops – MaxIterations and MaxDuration configuration
• Rate Limiting – per-task-type rate controls
• Concurrency Limits – parallel execution bounds
• Timeouts – per-step and per-iteration timeouts
• Planner Pattern – safety constraints on generated plans