Output and delivery

A standardizer always produces a delimited output file. You can deliver that file to an external system or leave it staged for downstream use.

Output format

The output file can be CSV or TSV. Specify the filename and format in the standardizer configuration. You can also customize the delimiter and quote_char.

output_config:
  filename: standardized_orders.csv
  format: csv
  delimiter: ","
  quote_char: '"'

Optional delivery

If you want to send the output somewhere else, add a delivery destination.

S3 / SFTP / FTP

output_config:
  filename: output.csv
  format: csv
  delivery:
    type: s3              # or sftp / ftp
    bucket: customer-bucket
    path: standardized/
    credential_id: "<credential-id>"
    overwrite: true

HTTP API

delivery:
  type: http_api
  url: https://api.example.com/upload
  method: POST
  credential_id: "<credential-id>"
  field_mapping:
    sku: product_sku
    quantity: qty
  constants:
    source: vendorgate

Database

delivery:
  type: database
  credential_id: "<credential-id>"
  table: standardized_orders
  mode: insert            # insert | upsert | replace | append
  column_mapping:
    sku: product_sku
    quantity: qty
  upsert_keys:
    - sku

Lookup dataset

delivery:
  type: lookup_dataset
  dataset_id: "<dataset-id>"
  mode: upsert            # insert | upsert | replace
  field_mapping:
    sku: product_sku
    quantity: qty
  upsert_keys:
    - sku

Internal staging

Even when you configure delivery, a copy of the output is staged internally. This lets other standardizers or pipelines consume the output as an input.