Create a ServiceBeta
A service runs your pipeline and serves queries. Services decouple the pipeline from the endpoint that serves it. This way, you can continue to iterate on the pipeline in the Builder while keeping a stable and reliable endpoint for external callers. You create and manage services in Gateway.
About This Task
A service is a stable endpoint that exposes a published pipeline version to your callers. You can set the hosting settings for the service to control how it scales and how long it stays active.
A single service can have one pipeline version published to it. You can publish a new version to a service at any time and the endpoint URL stays the same.
A single pipeline version can be published to multiple services.
When you create a service through the API without an instance size, the service uses Medium, regardless of your organization's default size. Pass an instance size to use another size.
If you pass an instance size through the API, the SDK, or MCP, it must be one that your organization has enabled. Otherwise, the request fails with a 422 error that names the enabled sizes. Ask an organization admin to enable the size, or pick one of the listed sizes. The same check applies when you change a service's instance size later. This check doesn't apply when the service uses a fleet environment, because the environment sets the instance size.
To learn more, see Gateway.
Prerequisites
You must have read and write permissions to services to perform this task. For details, see User Roles and Permissions.
Create a Service
- In the navigation, go to Gateway > Create service.
- Give your service a name and, optionally, a description to easily identify it.
- Optional: Add up to three tags to label the service. Tags must be unique, and each tag name must follow the same naming rules as tags you add to an existing service. If any tag is invalid or the limit is exceeded, the service isn't created and you see an error message.
- Configure the hosting settings for the service. By default, this section is collapsed and shows a one-line summary of the selected instance size and scaling preset, for example "Medium instance · Low demand". These are the recommended defaults, so you can keep them and continue. To change them, click Advanced hosting settings:Select the Instance size, which sets how much compute each replica gets: Small, Medium, Large, or Extra large. The field is preselected with your organization's default size, or Medium if no default is set, and lists the sizes your organization has enabled. Keep the preselected size unless your pipeline needs more or less compute to handle each query. Organization admins set the default and the enabled sizes. For details, see Managing Compute Settings. It's separate from the scaling preset below, which sets how many replicas run.
Hosting determines
- High demand: Choose this option for production pipelines that need high availability and scalability.
- Low demand: Choose this option for testing and pipelines that are not used frequently.
- Custom: Configure your own hosting settings.
Create a Service with the CLI
If your pipeline is in code, you can create the service and publish your first version in one step with the Haystack Enterprise SDK. Only Haystack pipelines and agents are supported.
- In Gateway, on the Create service page, click Have a pipeline in code? Create the service with the CLI instead. If your workspace has no services yet, click Create with the CLI in the empty state.
- In the Create a service with the CLI drawer, select With a coding agent or With the CLI.
- Copy the prompt or the commands and run them. Replace
my-servicewith your service name and<your_pipeline.py>with the Python file that defines your pipeline or agent.
The first run creates a new service with your pipeline as its first version. The service starts with default hosting settings, which you can change later in the service settings. The command fails if the service name is already taken. To publish an update later, run it again with --no-create instead of --create.
What To Do Next
- Click the service name to view active deployments and update service settings.
- Publish a pipeline version to the service.
- Call the service from your app, an OpenAI-compatible client, or an MCP client.
Related Information
Was this page helpful?