Ingest pipelines

Ingest pipelines transform documents as they are indexed. Querybara edits a pipeline as the JSON its PUT _ingest/pipeline request takes, and lets you try an edit on sample documents before you save it.

Edit and simulate a pipeline

1. Double-click Templates and pipelines in the sidebar and pick the Ingest pipelines tab.

2. Pick a pipeline from the list, or choose New. A new pipeline starts with a set processor as an example.

3. Edit the definition.

4. Under Sample documents (a JSON array of sources), enter the documents to try, for example:

   [{ "order_id": 1042, "product": "Oak dining table", "total": "899.00" }]

5. Choose Simulate.

6. When the result is right, enter the Name (for a new pipeline) and choose Save….

The simulation runs the pipeline as it is in the editor, not as saved. For each document it says whether it passed, was dropped or failed, and lists each processor with its outcome. Simulating is a read, so it also runs on read-only profiles.

The list shows each pipeline’s description. Delete… removes the selected pipeline; deleting is destructive, so it always shows the request and asks.

Related

• Aliases and templates
• ILM policies
• Console