Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

Set Model PricesBeta

Beta
This is coming — this page describes a feature that's still being rolled out and may not be available to you yet.

See which price Haystack Enterprise Platform uses for each model, and set your own price to change how LLM costs are calculated, for example when you negotiated a rate or use a model without a public price.

About This Task​

We calculate costs from the tokens a model used and the model's price. Every model has a default price: the price its provider publishes, which we sync every night. A custom price replaces the model's default price, either in all workspaces or in one workspace. A workspace price takes precedence over a price for all workspaces. One price applies at a time: a new price ends the one that's running when the new one starts.

The price a call is charged is the custom price if there is one, otherwise the default price. If a model has neither, its calls are not counted in your spend.

Permissions​

  • Everyone who can open the Cost Management page can see prices.
  • Users with the Costs permission in a workspace see the custom prices that apply in that workspace: its own and the ones for all workspaces.
  • Only Organization Admins can add, edit, or delete prices.

View Model Prices​

To check what your costs are based on, go to the Cost Management>Prices.

To include calls left out of your total spend because their model has no matching price, use the warning on the Cost Management page, which shows how many calls are left out. Click Show unmatched models in the warning, then click a model name to set its price. For details, see Monitor LLM Costs and Token Usage.

View Model Overview​

Use the Overview tab to check the price behind each model's cost. It lists up to 100 models, based on the scope, date range, and filters you set on the page.

For each model, the tab shows the price a call is charged now:

  • A custom price, if one applies.
  • Otherwise, the default price.
  • Otherwise, No price · not counted in your spend.

The columns show the input price, the output price, and the cached input price per 1 million tokens. If a model has cache write prices, the cache write columns appear too.

A recorded model name, such as gpt-4o-2024-08-06 or azure/gpt-4o, matches a price set for gpt-4o, the same way we match names when we calculate costs. In a workspace, the workspace's own custom price takes precedence over the price for all workspaces.

Below the models, the custom prices table lists the custom prices that apply.

View All Prices​

Use the All prices tab to see the default price of every model we know. Models that appear in your usage are tagged.

Add a Price​

You must be an Organization Admin.

  1. Go to the Cost Management>Prices.
  2. Start the price in one of these ways:
    • In the Overview or All prices tab, find the model and click Set custom price next to it. The form opens with the model and its default price filled in.
    • In the unpriced calls warning on the Cost Management page, click a model name.
  3. In Model, check the model name. It must match the name as it appears on your calls.
  4. In Provider, enter the lower-case provider ID, for example anthropic or openai. The provider decides how cached tokens are billed.
  5. Choose if it applies to all workspaces or just one workspace.
  6. Enter the Input price and the Output price in dollars per million tokens. Enter 0 for a model that's free.
  7. Under Starts, choose when the price starts:
    • From the beginning recalculates all past usage of the model.
    • Today keeps the cost of calls made before today.
    • On a date starts the price on that date. If the date is in the past, usage from that date is recalculated.
  8. Optional: In Ends, pick the last day the price applies. After that day, the default price applies.
  9. Optional: Set cache prices. For details, see Set Cache Prices.
  10. Click Save price. If the new price ends a price that's running, we ask you to confirm first.

Result: The price appears in the custom prices table. Costs are updated, which usually takes about a minute.

If costs don't change, check that Model matches the model name as it appears on your calls, and that the price's start date covers the calls.

Set Cache Prices​

Some models support prompt caching and have separate prices for cached tokens. Set cache prices so cached tokens are billed at your rate for them instead of falling back to the input price. To set them, expand Advanced in the price form:

  • Cache read price: The price of tokens served from the prompt cache.
  • Cache write price: The price of tokens newly written to the prompt cache.
  • 1-hour cache write price: The price of cache writes kept for 1 hour instead of 5 minutes (Anthropic).

If you leave a field empty, the price falls back to another one:

FieldWhen empty, tokens are billed at
Cache read priceThe input price
Cache write priceThe input price
1-hour cache write priceThe cache write price, or the input price if that's empty too

An empty field isn't the same as 0. Enter 0 to make those tokens free.

You can also set these prices through the API. The 1-hour cache write price is the cache_write_1h_price_per_1k_tokens field, priced per 1,000 tokens.

For providers that count cached tokens as part of the prompt, such as Anthropic and OpenAI, cache reads and writes are taken out of the prompt tokens before the input price applies. This way, cached tokens aren't counted twice.

Edit or Delete a Price​

You must be an Organization Admin. Other users see the drawer as read-only.

  1. Go to the Cost Management page and click Prices.
  2. To see prices that have ended, click Show ended prices.
  3. In the custom price's row, choose Edit or Delete.

You can't change Applies to on an existing price. To change it, delete the price and add it again. When you delete a price, usage in that period is recalculated with the default price.