Skip to content

Add a hard dollar cap to the query loop - #179

Open
Mattbusel wants to merge 1 commit into
lingcoder:mainfrom
Mattbusel:deps/llm-budget-loop-cap
Open

Mattbusel wants to merge 1 commit into
lingcoder:mainfrom
Mattbusel:deps/llm-budget-loop-cap

Conversation

@Mattbusel

Copy link
Copy Markdown

query_loop has no spending limit. CostAccumulator adds up the cost after each call and nothing reads the total back, so a runaway tool loop keeps billing until someone cancels it.

This change gives CostAccumulator an optional shared AtomicBudget (with_budget). Every recorded call is charged to it, and query_loop returns an error before the next request once the cap is spent. Give sub-agent accumulators the same Arc and the whole team draws from one budget. Without a budget, nothing changes.

Adds llm-budget 0.10.0 (default features off) to crab-session. Matthew Busel maintains llm-budget.

Disclosure: I maintain llm-budget.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant