01. Overview & the golden path

The four planes, the fixed invariants, the golden path a model travels, and the phased plan the course follows.

Outcome

By the end of this chapter you can describe the whole platform on one page: the four planes it is built from, the invariants that never change, the golden path a model travels from data to production, and the phased order in which we build it. Every later chapter fills in one phase of this map.

The guiding rule, adopted from docs/00, is the fewest moving parts that deliver reproducibility, honest evaluation, scheduled/on-demand workflows, and observable operations — and no more. We deliberately reject an enterprise control-plane build (durable orchestration engine, hash-chained release ledgers, a bespoke broker) because it costs more to build and operate than a small ML team can sustain and buys guarantees we do not yet need.

## The four planes

The platform is four planes plus a thin dashboard. Everything else follows.

Plane Responsibility Azure building block
Execution Run every workflow as an ephemeral, image-pinned task Azure Container Apps Jobs
Model lifecycle Track experiments, register versions, store artifacts Self-hosted MLflow (ACA App + Postgres + Blob)
Operational state Record status/output/error for every run, with batch granularity Generic results DB (Postgres)
Serving Optional online HTTP inference at an exact model version Azure Container Apps Apps 
flowchart TD
    DASH["Dashboard (ACA App, Entra)<br/>catalog + launcher + links"]
    JOBS["ACA Jobs (execution)<br/>train / eval / batch / task"]
    MLF["Self-hosted MLflow<br/>registry + tracking"]
    RDB["Results DB<br/>run state"]
    BLOB["Blob<br/>artifacts"]
    OBS["Grafana / Log Analytics<br/>dashboards"]

    DASH -->|reads status| JOBS
    DASH -->|deep-links| MLF
    DASH -->|deep-links| OBS
    JOBS -->|read model version| MLF
    JOBS -->|write runs| RDB
    JOBS -->|large payloads| BLOB
    MLF --> BLOB

The dashboard is a read-and-launch surface over these planes; it holds no authoritative state of its own.

## Fixed invariants

These hold across every chapter. Changing any of them is an architecture decision, not an implementation detail (docs/00).

  1. ACA Jobs are the execution plane — every workflow (train, eval, batch, ad-hoc) runs as an ephemeral Job from a pinned image digest, scaled to zero when idle. A code deploy is just a Job-definition image bump, so it can never strand a stale worker on old code.
  2. No workflow control-plane service — linear multi-step workflows are one script in one Job. No Durable Functions, no orchestration engine.
  3. No application broker by default — no Service Bus, Redis broker, or Celery fleet in the baseline. Fan-out is parent/child rows in the results DB.
  4. Self-hosted MLflow is the tracking + model registry, pinned to an exact version — the registered version number is the canonical model identity.
  5. A generic results DB is the run store — one table records status/output/ error for every job, with parent/child rows for batch granularity.
  6. MLflow is scoped to model lifecycle — batch inference only reads a pinned version; its state lives in the results DB, not as MLflow runs.
  7. GitHub Actions is CI/CD only — build/test/scan images and update Job/App definitions. Never a scheduler or orchestrator.
  8. Least-privilege managed identities; OIDC for CI — each workload has its own identity with minimal roles; no shared broad identity, no secrets in images.
  9. Distributed/multi-GPU training is an admission-gated exception — Azure ML command jobs against min-zero clusters, logging to the same MLflow.

## The golden path

A single path a model travels from data to production. Every step is an ACA workload, every model is a registry version, every run is a results-DB record, and every human action is audited.

flowchart TD
    DATA["tracked dataset"]
    TRAIN["train Job (ACA)"]
    VER["MLflow run + registered version"]
    EVAL["eval Job (ACA)<br/>metrics + results-DB record"]
    PROMOTE["promote version"]
    BATCH["batch Job (ACA, scheduled/manual)<br/>reads models:/name/version<br/>parent/child result rows"]
    SERVE["serving App (ACA, optional)<br/>loads models:/name/version<br/>/health reports version"]
    OBS["dashboard + Grafana + alerts"]

    DATA --> TRAIN --> VER --> EVAL --> PROMOTE
    PROMOTE --> BATCH
    PROMOTE --> SERVE
    BATCH --> OBS
    SERVE --> OBS

## The phased build — and how the course maps to it

We build in the phase order of docs/07, where each phase is independently useful. The course chapter that implements each phase is shown alongside.

Phase What ships Useful because Chapter
0 — Foundation Registry, ACA env + Log Analytics, Postgres (mlflow + results), storage, Key Vault, Grafana, identities; self-hosted MLflow Everything has a home and an identity 02
1 — Reproducible training + registry train/eval ACA Jobs logging to MLflow + results-DB records Reproducible models with recoverable lineage 03
2 — Results DB + first workflow results-DB module + one scheduled/batch workflow (parent/child + continuation) The operational backbone exists and is exercised 04
3 — Serving + promotion/rollback serving App + version-based promotion/rollback Models reach consumers with safe rollback 05
4 — Observability + dashboard alerts, Grafana, catalog/launcher dashboard The team can see and launch everything 06
5 — LLM artifacts pyfunc packaging + evaluator reusing the same paths An LLM ships through existing machinery 07
Exc. — Multi-GPU admission-gated Azure ML path Distributed training without derailing the baseline 08
Upg. — Broker Celery-as-a-library on KEDA-triggered ACA Jobs + Redis Only if fan-out routinely exceeds a few hundred units 09

Chapter 10 integrates the whole golden path end to end.

## How to read the rest of the course

Each chapter is structured the same way, so the build stays predictable:

  • Outcome — what you can do when the chapter is done.
  • Design — the decisions from the relevant docs/ document, in brief.
  • Build in projects/ml-platform/ — the modules, images, IaC, or scripts the chapter adds to the project (the source is authored in the project, not the notebook).
  • Golden-path position — where this chapter’s slice sits in the diagram above.
  • Acceptance evidence — what demonstrates the slice actually works (never “a local demo ran”).
  • Extensions — what the production contract asks for that this MVP defers, and the docs/ section that specifies it.

Next: 02 — Platform foundation stands up the footprint every later phase depends on.

Back to top