Creating a standardizer

A standardizer takes validated data from one or more pipelines and converts it into a clean output schema. This guide explains how to create one.

Choose the input

A standardizer reads from a pipeline output. Select the pipeline that produces the validated file you want to reshape.

The pipeline must be configured to stage its output so the standardizer can use it as an input. Make sure internal staging is enabled in the pipeline's delivery settings.

Define output columns

The heart of a standardizer is its list of output columns. For each column you define:

The output column name.
How the value is produced: a simple source field, a chain of transform steps, an expression, or an LLM mapping.
The target data type, such as string, integer, decimal, boolean, date, datetime, or time.

Test as you build

Run the standardizer against a recent successful pipeline run to preview the output. Check that:

Column names match your target schema.
Data types are correct.
Lookups return expected values.
Expressions compute the right results.

Add delivery

By default, the standardizer stages its output for downstream use. You can also deliver the output directly to SFTP, FTP, S3, HTTP API, Database, or a lookup dataset.

When you are satisfied with the preview, activate the standardizer. You can run it manually, trigger it from a pipeline's validated output event, or schedule it. Standardizers can also declare dependencies on other pipeline or transformation runs with an optional freshness window.

Common first standardizer

A typical first standardizer:

Renames vendor-specific columns to your internal names.
Trims whitespace and normalizes case.
Converts date strings to a standard format.
Looks up supplier codes to produce canonical supplier names.