Skip to main content

Pipelines

A data pipeline is a collection of data components to complete a task. These data components include:

Together they allow you to move your data from one place to another.

Pipelines

Every source connector will emit data records into a stream. Destination pull and process data from streams in real-time.

Specific sources suport Change Data Capture (e.g., Postgres using Logical Replication and emit events (inserts, updates, and deletes) as data records; out of the box, these sources also support mirroring data from source to destination.

Creating Pipelines#

You may create pipelines in the Dashboard or using the CLI. For example:

meroxa pipelines create my-new-pipeline

Check out the Guides for specific pipeline use cases and examples.