Stop getting paged for schema changes you didn't make.
8 micro-lessons · ~78 min · Real Docker images
Data contracts for data platforms
Stop getting paged for schema changes you didn't make.
Real skills, real career delta.
Skills you'll gain
10- ODCS v3.1.0 contract authoringWorking
Write and lint ODCS v3.1.0 YAML contracts covering the fundamentals, schema, quality, slaProperties, and server blocks using datacontract-cli 0.10.x. Diff contracts across versions to identify breaking field changes before they reach downstream consumers.
- dbt model contracts with enforced: trueProduction
Configure dbt 1.9 model contracts in schema.yml with enforced: true so that column removal or type changes cause dbt build to exit non-zero. Implement multi-version models with deprecation_date and cross-project ref() in a dbt Mesh producer-consumer setup on DuckDB.
- dbt-checkpoint pre-commit hook configurationWorking
Wire dbt-checkpoint 2.x hooks (check-model-has-contracts, check-model-has-description, check-source-has-freshness-defined) into .pre-commit-config.yaml to block commits that introduce contract-free models before dbt build ever runs.
- Confluent Schema Registry compatibility mode enforcementProduction
Register Avro schemas against Confluent Schema Registry 7.8 via its REST API and configure BACKWARD, FORWARD, and FULL compatibility modes per subject. Trigger HTTP 409 conflict responses in CI when a schema evolution violates the configured compatibility rule.
- buf breaking for Protobuf CI gatesWorking
Write buf.yaml v2 configs with FILE and WIRE_JSON breaking rule sets and run buf breaking --against .git#branch=main in GitHub Actions to block PRs that remove fields or reuse field numbers in .proto files.
- GX Core 1.x Checkpoint authoring and Airflow integrationWorking
Define Expectation Suites and Checkpoints using the GX Core 1.x FileDataContext API and attach them as Airflow 2.9 task gates that fail the DAG and produce Data Docs HTML when freshness, completeness, or value-range expectations are violated.
- SodaCL checks for freshness, completeness, and accuracy SLOsWorking
Write SodaCL checks.yml files for Soda Core 3.x that enforce freshness thresholds, completeness percentages, and null-column accuracy rules against a Postgres data source. Map each SodaCL check field to the corresponding ODCS slaProperties key.
- datacontract-cli code generation pipelineWorking
Use datacontract-cli 0.10.x export commands to generate dbt schema.yml, Great Expectations Expectation Suite JSON, and Avro .avsc files from a single ODCS v3.1.0 YAML. Enforce artifact freshness in CI by regenerating all outputs and failing if any committed file differs from the generated version.
- Kafka schema evolution testing with Avro and Docker ComposeWorking
Stand up a Kafka plus Confluent Schema Registry 7.8 stack with Docker Compose and write Python producer scripts using confluent-kafka 2.x to register schemas and demonstrate the contrast between a BACKWARD-compatible additive field change and a breaking field deletion that returns HTTP 409.
- Backstage catalog-info.yaml for data contract discoverabilityWorking
Author catalog-info.yaml entity descriptors that link ODCS contract YAML URLs as annotations and configure mkdocs-based TechDocs to render contract schema, SLO, and ownership metadata inside a Backstage instance so consumers can locate contract terms without contacting the producing team.