Core Concepts

Understanding a few core concepts makes it easier to build reliable data workflows in VendorGate.

Account and workspace

Your VendorGate account is the top-level container for everything you build. Inside an account, you can invite team members and configure billing. All resources belong to the account.

Credentials

Credentials are secure records that store the details VendorGate needs to connect to external systems. Examples include SFTP usernames and passwords, API keys, database connection strings, and webhook signing secrets. Credentials are referenced by name in pipelines and monitors, never embedded in configuration files.

Contracts

A contract defines what a valid file looks like. It specifies the source format, the columns a file must contain, expected data types (string, integer, decimal, boolean, date, datetime, time, email, url, uuid, json), and business rules that rows must satisfy. For each column, required controls whether the column must appear in the source, and nullable controls whether cells may be empty. When a pipeline runs, the contract is applied to the incoming file and invalid rows are flagged.

Pipelines

A pipeline moves data from one place to another. Each pipeline has a source, a contract for validation, a delivery destination, and optional execution settings. Single pipelines use one contract; workbook pipelines use one contract per sheet. Sources include SFTP, FTP, S3, HTTP APIs, and databases. Destinations include SFTP, FTP, S3, HTTP APIs, databases, and lookup datasets. Pipelines can be triggered on a schedule or run manually on demand. You can also enable fail-fast execution to stop processing after a configured number of errors.

Transformations (Standardizers)

Transformations, also called standardizers, take validated input data and convert it into a clean, predictable output schema. You define output columns and choose a mapping mode for each one: simple field mapping, multi-step transforms, expression evaluation, or llm prompts that call a provider-owned LLM. Transformations can also reference lookup datasets and join multiple inputs.

Lookup datasets

A lookup dataset is a reference file or table that a transformation can use to enrich or validate data. For example, you can map supplier IDs to standardized names or validate country codes against a reference list.

API monitors

An API monitor checks a remote endpoint on a schedule. It verifies status codes, response times, response content, headers, and TLS certificate expiry. Monitors support OAuth2 client credentials, mTLS, bearer tokens, basic auth, query-param, and header-based API credential authentication. Monitors are independent of pipelines and transformations.

Runs and alerts

Every pipeline and monitor execution creates a run record. You can inspect run history, view errors, and configure alerts to notify your team by email or webhook when something fails or succeeds.