A hard dollar limit on one GraphRAG indexing run (settings.yaml only)

[domondi1]

Indexing cost is one of the most common GraphRAG surprises: entity extraction, gleanings, summarization and community reports add up to thousands of calls, many in flight at once, and the bill for one graphrag index depends on the corpus in ways that are hard to predict up front. Here's one way to put a hard dollar ceiling on a single indexing run using only settings.yaml.

call_args are sent with every model request through LiteLLM, so the completion model can carry two extra headers (the run's id and its budget) to a small OpenAI-compatible gateway running next to GraphRAG:

pip install inferrail
export OPENAI_API_KEY=sk-...
inferrail serve --quickstart --app-mode   # http://127.0.0.1:8000/v1

# settings.yml
completion_models:
  default_completion_model:
    model_provider: openai
    model: gpt-4.1-mini
    auth_method: api_key
    api_key: unused                       # the gateway holds the real key
    api_base: http://127.0.0.1:8000/v1
    call_args:
      max_completion_tokens: 2000
      extra_headers:
        X-Inferrail-Attribute-Work-Id: ${INDEX_RUN_ID}
        X-Inferrail-Budget-Usd: "5.00"
embedding_models:
  default_embedding_model:              # unchanged: embeddings go straight to OpenAI
    model_provider: openai
    model: text-embedding-3-large
    auth_method: api_key
    api_key: ${GRAPHRAG_API_KEY}

INDEX_RUN_ID=index-$(date +%s) graphrag index --root .
inferrail work $INDEX_RUN_ID     # what that run cost

Each call reserves its worst-case cost before it's sent, so the concurrent extraction requests can't all spend the same remaining budget, and a call that doesn't fit is refused with a 402 before it reaches OpenAI. Once the budget is spent the run stops with that error instead of continuing to bill.

Caveats: use a new INDEX_RUN_ID for each run (a run's budget is fixed the first time its id is seen), set max_completion_tokens (the reservation is based on it; without it each call reserves a 4,096-token default), and only the completion model goes through the gateway, so embeddings aren't counted. The model needs a price in Inferrail; gpt-4.1-mini, gpt-4.1 and gpt-4o-mini are built in. The LiteLLM extra_headers path is what we tested; I haven't run a full index of a large corpus through it, so reports are welcome.

Setup details: guide. I maintain Inferrail (open source, Apache-2.0), so take the suggestion with that in mind.