Data Architecture and Platforms

About this course

Data architecture for operational and analytical systems. Learners start with how storage engines, indexes, replication and partitioning shape what a database can promise, then design lakehouse platforms on open table formats, analytical models, batch, streaming and change-data-capture pipelines, and the ownership, contract, quality, lineage and privacy controls (data mesh where it fits) that keep a data estate trustworthy, affordable and ready for AI.

For: architect, data professional, software engineer, tech leader.

Learning outcomes

On successful completion you will be able to:

1. 1Explain how storage engines, indexes, transactions, replication and partitioning determine the behaviour and guarantees of operational and analytical data stores.
2. 2Evaluate data models and stores (relational, document, key-value, wide-column, graph, search and time-series) for a workload and justify a polyglot persistence strategy.
3. 3Design analytical data platforms using lakehouse patterns, open table formats and dimensional or data-vault models with a semantic layer.
4. 4Design batch, streaming and change-data-capture pipelines with explicit guarantees for ordering, late data and delivery semantics.
5. 5Establish data ownership, contracts, quality, lineage and privacy controls - adopting data mesh only where it fits - and take accountability for data governance and GDPR obligations.
6. 6Judge the cost, performance and AI-readiness trade-offs of a data platform and defend a data architecture strategy to business stakeholders.

Modules and lessons

1. Week 1Data stores and data models⌄

   • Storage engines, indexing and query execution
   • Data models and polyglot persistence
2. Week 2Scaling data⌄

   • Replication and partitioningcase study
   • Caching, CQRS and event sourcing
3. Week 3Analytical platforms⌄

   • Warehouses, lakes and the lakehousecase study
   • Analytical modelling: dimensional, data vault and semantic layerslab
4. Week 4Moving data⌄

   • Batch, streaming and change data capture
   • Stream processing semanticslab
5. Week 5Ownership, quality and governance⌄

   • Data mesh, data products and data contracts
   • Data quality, lineage and privacy controls
6. Week 6Data economics and strategy⌄

   • Cost and performance of data platforms
   • Data architecture for AI and the data strategy

Assessment

• Quiz (15%)

  Six weekly quizzes drawn from the lesson quiz banks; best five count.

• Design review (40%)

  Data platform design for a provided case - store choices, lakehouse layout, analytical model, pipeline designs with stated delivery guarantees, and ADRs.

• Case analysis (20%)

  Written analysis of a published data-platform case, assessing ownership, governance, cost and the alternatives the team could have taken.

• Viva (25%)

  Twenty-five-minute defence of the platform design and its governance model before mentors acting as CDO and DPO.

Pass mark 60%. Grades: Distinction, Merit, Pass.