Choose Files to Index
Choose which files your index processes. You can filter files by name prefix
About This Task
By default, an index processes all files in your workspace. You can restrict indexing to a subset of those files:
- By file name prefix: Index only files whose names start with one of the prefixes you set. For example,
legal*matcheslegal-contract.pdfandlegal_memo.docx.
- By metadata: Index only files whose metadata matches a condition you set. For example, index files where
category == reports.
Only files that match your filters are indexed, counted, and listed on the Index Files tab. All other files are skipped.
You can use both filters at the same time. When both are set, a file must match the prefix filter and the metadata filter to be indexed.
You can update these filters only while the index is disabled. Disable the index first, then change the filters. Changing the filter does not re-index files that were already indexed.
Changing a file's metadata after it was already indexed does not automatically re-index that file.
Prerequisites
- A disabled index. If the index is enabled, disable it before you change filters.
- File metadata, if you want to filter by metadata. For details, see Add Metadata to Your Files.
Set a File Name Prefix Filter
- In Haystack Enterprise Platform, go to Indexes and click the index you want to configure. This opens the Index in Builder.
- Open the Settings tab.
- In the File Name Prefix Filter section, type one or more prefixes in the Prefixes field.
- Separate multiple prefixes with commas. A file is indexed if its name matches any of the prefixes.
- You can end a prefix with
*as a wildcard, for exampledoc*orcontracts/. - Each prefix can include letters, digits,
.,_,-, and/.
Clear the File Name Prefix Filter
- Open the Settings tab for the disabled index.
- Clear the Prefixes field in the File Name Prefix Filter section.
Set a File Name Prefix Filter with the API
To set or clear the prefix filter with the REST API, see Filter Files to Index with the REST API.
Set a Metadata Filter in the UI
The File Meta Filter in the UI uses one equality condition: you specify a metadata field name and a value, and only files where that field equals that value are indexed. Files without that field, or with a different value, are skipped.
- In Haystack Enterprise Platform, go to Indexes and click the index you want to configure. This opens the Index in Builder.
- Open the Settings tab.
- In the File Meta Filter section, type the metadata field name in the first field. For example,
category. - The operator is fixed to
==(equals). - Type the value to match in the second field. For example,
reports. - Click Save.
The index now processes only files whose metadata field matches the value you entered.
Clear the File Meta Filter
To remove the filter and index all files again:
- Open the Settings tab for the disabled index.
- In the File Meta Filter section, clear both the field name and the value fields.
The filter applies to file metadata, not to file content. Add the relevant metadata to your files before you enable the index. For details, see Add Metadata to Your Files.
In the UI you can set one metadata rule at a time, and the operator is always ==. To use != or combine several conditions, set the filter with the REST API.
Set a Metadata Filter with the API
To set or clear the metadata filter with the REST API, see Filter Files to Index with the REST API.
What To Do Next
After you update the filters, enable the index so it applies the new conditions to incoming files.
Was this page helpful?