Building an ML Platform on Azure

A small, low-ops ML platform built twice: first as a Docker Compose proof of concept, then ported to Azure Container Apps behind an explicit environment contract.

A step-by-step build of a deliberately small, low-ops MLOps platform for a team of ML engineers who are not full-time platform/DevOps engineers. We use the fewest moving parts that still deliver reproducible training, honest evaluation, scheduled and on-demand batch/inference workflows, and observable operations — and we add machinery only when a concrete need forces it.

This course is the authoritative build guide. Each chapter develops one slice of the platform, and the chapters accumulate toward the completed implementation in projects/ml-platform/. The project directory contains the final cumulative result; following the course in order reconstructs how it is built.

Local-first phased build

The platform is built twice on purpose. Part I builds every feature on a Docker Compose sandbox: Postgres and MinIO underneath, self-hosted MLflow, a small runner service as the execution plane, serving, dashboard, and local LLM commands. The feedback loop stays short while the design settles.

Part Chapters Environment Outcome
I: Local platform 02–08 Docker Compose Every feature working, interfaces fixed
II: Production 09–13 Azure Container Apps + Terraform Same workload images deployed and operated
Off the critical path 14 Azure ML Exception track, adopted only if forced
Capstone 15 Compose + Azure adapters Separate suites preserve one behavioral definition of done

Part II ports the settled result to Azure: a Terraform foundation, Container Apps Jobs as the execution plane, managed Postgres and Blob storage, then CI/CD and operations. Workload behavior stays fixed; deployment adapters change.

Source audit
Ch Chapter Builds in projects/ml-platform/
01 Overview & the golden path — (conceptual)
02 Local platform foundation demo/
03 Reproducible training & registry src/train_job/, src/ml_platform/common/
04 Results DB & batch workflows src/ml_platform/results/, src/batch_job/
05 Online serving & promotion src/serving_app/, demo/promote.py
06 Observability & dashboard src/dashboard/, infra/modules/observability/
07 LLM release artifacts src/ml_platform/llm/
08 The environment contract tools/check_env_contract.py
09 Just enough Terraform infra/
10 Azure foundation infra/, src/mlflow_app/
11 Porting the workflows to ACA deploy/, infra/modules/
12 CI/CD .github/workflows/
13 Azure operations infra/modules/dashboard/, infra/modules/observability/
14 Multi-GPU training src/train_aml/, infra/modules/aml/
15 End-to-end integration demo/golden_path.py, deploy/smoke-tests.*

Chapter 14 is off the critical path: an exception track included so the boundary is explicit but never required to ship the baseline.

Four planes architecture

Everything in the platform is a consequence of four planes plus a thin dashboard.

Plane Responsibility Azure building block
Execution Run every workflow as an ephemeral, image-pinned task Azure Container Apps Jobs
Model lifecycle Track experiments, register versions, store artifacts Self-hosted MLflow (ACA App + Postgres + Blob)
Operational state Record status/output/error for every run, with batch granularity Generic results DB (Postgres)
Serving Optional online HTTP inference at an exact model version Azure Container Apps Apps

There is no bespoke control plane: no Durable Functions, no orchestration engine, and no application broker in the baseline. Linear multi-step workflows are an ordinary Python script inside one Job; batch fan-out is expressed as parent/child rows in the results DB plus a small stateless rule.

Cloud deployment contract

What makes the Part II move a port and not a rewrite is a short list of seams, fixed once in chapter 08. Here, a contract is the set of interfaces and invariants that both deployment environments must preserve. Azure implements the same contract through different infrastructure adapters; it is not a second application with different behavior.

  • Environmental variables: services read the same documented environment variables, and a contract check1 fails when a variable has no provider or documented production injector.
  • Workload entrypoints: Compose and ACA reference the same train, batch, serving, dashboard, and shared LLM image sources.
  • Results DB: the same schema records every run in both worlds.
  • Promotion semantics: flip the MLflow production alias, then repin only the long-running consumer to the exact version.
  • Behavioral checks: the local golden path and cloud smoke adapters use separate trigger mechanisms but assert terminal success, results state, readiness, model identity, and prediction behavior.

Part I stands each plane up on Compose equivalents; Part II moves them onto Azure blocks and managed identities without adding local-only feature implementations.

How to read the rest of the course

The source is not authored inside the notebooks. Each chapter:

  1. Defines the outcome and behavior added to the cumulative platform.
  2. Builds the relevant modules/scripts under projects/ml-platform/src/ (and demo/ in Part I, infra/ / deploy/ in Part II).
  3. References that source when demonstrating a stage — e.g. triggering a training run through the runner API, or loading a model version in the serving app.
  4. Ends with an Extensions section covering optional capabilities or production hardening beyond the chapter’s baseline.

The course owns the build sequence and explanation. The project directory holds the completed, reviewable implementation.

What you need before you start

  • Part I: Docker and Python 3.11+, basic ML familiarity. No cloud account.
  • Part II: Azure CLI with an active subscription; container builds via ACR Tasks (no local Docker required there).
  • Budget is small with tear-down discipline; every Part II chapter that provisions resources ends by tearing them down.
Back to top

Footnotes

  1. tools/check_env_contract.py↩︎