Monitor LLM Costs and Token UsageBeta
To keep track of your LLM spend and find out which workspaces, services, pipelines, providers, and models drive it, use the Cost Management page. It shows how much your LLM calls cost and how many tokens they use.
About This Task
The Cost Management page shows costs for your whole organization or, depending on your role, for a workspace. Costs are estimates: we calculate them from the token usage recorded in your traces and the model's price. To check credits and document storage units instead, see Understand Your Pipeline Usage. To change a model's price, see Set Model Prices.
The page refreshes automatically every minute while it's open. Click Auto-refresh to refresh right away.
Permissions
Organization Admins can see the cost data of the whole organization and of any workspace. Other users can open the page if their role has the Costs permission. They see the data of the workspace they're in and can't switch to another one. For details, see User Roles and Permissions.
How Days Are Counted
The Cost Management page counts every day in the date range as a UTC day, the same way providers bill. When you pick a day, the page shows the spend from 00:00 to 24:00 UTC of that day, even if your browser is in another timezone. For example, if you pick October 16 in Berlin, you see the spend of October 16 UTC, not of October 16 in Berlin time.
This means the total for a day matches the same day in your provider's console or invoice, and matches the cost exports. To reconcile spend with a provider, compare the same UTC day on both sides. The daily spend chart places each bar on its UTC day, and the Projected this month figure is also calculated from UTC days.
Open the Cost Management Page
In Haystack Enterprise Platform, click Cost Management in the sidebar, or open the command palette and search for Cost Management. You can see the spend for the whole organization over the date range you choose. You can also choose the workspaces you want to see.
The page URL keeps the scope, date range, and filters, so to share your current view, copy the URL and send it.
Understand the Summary
To check how much you've spent and where the month is heading, read the summary at the top of the page. It shows these figures for the scope and date range you selected:
- Total spend: The cost of all priced calls in the range, with a comparison against the previous period of the same length. If there is no earlier data, the page says No prior period to compare yet.
- Requests: The number of LLM calls.
- Tokens: The number of tokens used by those calls.
- Avg / request: The average cost of one call.
- Avg / day: The average cost per day in the range.
- Projected this month: The month-to-date spend plus the average daily spend of the last 7 days applied to the remaining days of the month. Use it to see where the month is heading.
Calls That Aren't in the Total
If some calls use a model without a matching price, they can't be priced and stay out of the total. Calls that are still being priced are added automatically, usually within seconds.
To find out which models need a price, click Show unmatched models in the warning. The Unmatched models list shows each model with its number of calls and tokens, starting with the model that affects the most calls, so you can price that one first. If you're an Organization Admin, click a model name to set a price for it, which applies to all of its calls. For details, see Set Model Prices. Click Hide details to collapse the list.
Explore the Breakdown
To compare spend across groups for the whole date range, use the Spend breakdown table below the Cost flow chart. It lists spend by the dimension you choose in its Group by control. The options depend on your scope:
- Whole organization: Workspace, Service, Pipeline, Model, and Provider.
- One workspace: Service, Pipeline, Model, and Provider.
- One service or pipeline: Model and Provider. The table groups by provider by default.
Each row shows:
- Share of spend: The row's share of the total.
- Requests: The number of calls.
- Tokens: The number of tokens.
- Avg / req: The average cost of one call.
- Cost: The row's total cost.
Click a workspace, service, or pipeline row to scope the whole page to it. Click a provider row to filter the page by that provider.
- To see which providers take the largest share of spend, check the By provider card.
- To find your most expensive models, check the Top models card, which lists the five models with the highest cost.
- To follow spend from your organization through workspaces and services to models, use the Cost flow chart. It shows which workspaces and services send the most spend to which models. Hover over a flow to see its cost.
Switch the Cost Flow Between Services and Pipelines
A service and the pipeline it serves can look like two separate services in the same column. To avoid this, the Cost flow chart lists only one kind of query target at a time and sums up the other kind in a single node.
- Choose Services to compare what each service costs. The chart shows one node for each service. All queries that ran directly agains pipelines go into one node named Deployed Pipelines.
- Choose Pipelines to see the cost of queries that ran directly against pipelines. The chart shows one node for each pipeline, based on the queries that ran directly against it. All queries that ran through a service go into one node named Services.
- Hover over a node to see its cost.
The chart lists nodes from most to least expensive. The node that sums up the other kind always comes last in the column, even when it's the most expensive one. A query made through a service counts as a service query in both views, even though it also records the pipeline it served.
See the Prices Behind Your Costs
To find out which price each model's cost is based on, click Prices. The drawer shows the price in effect for the models in your current period and filters, and the default price of every model. It also lists the custom prices that apply in your workspace. Everyone who can open the page can view prices; only Organization Admins can change them. For details, see Set Model Prices.
Get Daily Cost Over the API
To use daily cost data outside the Cost Management page, for example in your own reports, see Get Daily Cost. To find out which price each model's cost is based on, click Prices at the top of the page. This view shows the price in effect for the models in your current period and filters, and the default price of every model. It also lists the custom prices that apply in your workspace. Everyone who can open the page can view prices; only Organization Admins can change them. For details, see Set Model Prices.
Check the LLM Cost of a Service
To check the recent LLM cost of a single deployed pipeline, go to Services, open the service, and click Overview. The LLM Cost (30d) card shows the total estimated cost and number of requests for that service over the last 30 days.
Related Information
Was this page helpful?