Orchestration Consolidation Matrix

Count the places a job can be scheduled in your data stack. Include the cron on that one VM.

When I do this with teams around ten or twenty pipelines, the answer is rarely one. The ingestion tool has its own schedules. dbt Cloud has jobs. Airflow or Dagster runs some things. A Lambda fires on a timer, and a VM has a crontab someone set up during a migration.

Each choice made sense on the day. Together they mean a failure at 6am could live in four consoles, and usually one person knows which.

For each scheduler I write down what runs there, who sees a failure, and who owns it by name.

Rows with no alert and no owner get folded first. That’s usually the crontab.

I often leave the syncs in the ingestion tool and trigger everything downstream from one orchestrator. That way there’s one place to look when something didn’t run.

How many did you count? Ownership and decision rights are one section of the assessment I use: https://thomasnys.com/data-platform-assessment-checklist/