Joining inputs

A standardizer can combine data from two pipeline inputs using a join. This is useful when a single output row needs fields from more than one source.

When to use a join

• Combine order lines with customer master data.
• Enrich product records with supplier information.
• Merge two sheets from a workbook pipeline.

How joins work

Add two inputs to the standardizer, each with a unique alias. Then define a join config that matches rows by key columns. left_key and right_key can be a single column name or a list of column names for composite keys.

input_config:
  - source_type: pipeline
    source_id: "<orders-pipeline-id>"
    alias: orders
  - source_type: pipeline
    source_id: "<customers-pipeline-id>"
    alias: customers

join_config:
  join_type: inner
  left_alias: orders
  right_alias: customers
  left_key: customer_id
  right_key: customer_id
  conflict_resolution: error

Join types

• inner: keep only rows that match in both inputs.
• left: keep all rows from the left input and matching rows from the right.
• right: keep all rows from the right input and matching rows from the left.
• outer: keep all rows from both inputs.
• cross: produce every combination of rows from both inputs.

Conflict resolution

When both inputs define a column with the same name, choose how to handle it:

• prefer_left: keep the value from the left input.
• prefer_right: keep the value from the right input.
• error: fail the run.
• both: emit both columns with suffixes derived from the input aliases, such as _orders and _customers.

Mapping after a join

After the join, reference either alias in your mappings:

mapping_config:
  - output_column: order_id
    mode: simple
    source:
      alias: orders
      field: order_id

  - output_column: customer_name
    mode: simple
    source:
      alias: customers
      field: name