Typed ML APIs with FastAPI + pydantic + OpenAPI → TS

End-to-end type safety from tensor to React prop, with the schema generated from the server rather than maintained twice. No hand-written client, no drift.

The usual failure is not dramatic. A field is renamed on the server, the frontend keeps reading the old name, undefined flows into a chart, and the chart renders an empty state that looks plausible. Nobody notices for a week.

The fix is structural: stop maintaining the contract in two places. Define it once in Python, let FastAPI publish it as OpenAPI, and generate the TypeScript from that. The generator is the only thing that has to be correct.

The chain

Figure 1 — One direction of authority. Nothing downstream of the pydantic model is hand-written, so nothing downstream can drift. The dashed return path is the CI check that makes this property enforceable rather than aspirational.

The pieces

pydantic v2 — request and response models, with validation
FastAPI — serves them, and publishes /openapi.json from the same objects
openapi-typescript — turns that document into a .d.ts
A CI check that regenerates and fails on any difference

The last one is what makes the rest true. Without it, the generated file is a snapshot that someone will eventually edit by hand.

The model server

from typing import Annotated, Literal

from fastapi import FastAPI
from pydantic import BaseModel, Field

app = FastAPI(title="inference", version="class="tok-num">1.0.class="tok-num">0")

class EmbedRequest(BaseModel):
class=class="tok-str">"tok-com">    # Constraints belong in the schema, not in a docstring: they are
class=class="tok-str">"tok-com">    # validated at runtime AND exported to OpenAPI, so the frontend sees
class=class="tok-str">"tok-com">    # the same bounds the server enforces.
    texts: Annotated[list[str], Field(min_length=class="tok-num">1, max_length=class="tok-num">64)]
    normalize: bool = True
    model: Literal["e5-large", "bge-m3"] = "e5-large"

class EmbedResponse(BaseModel):
    vectors: list[list[float]]
    dim: int
    model: str
class=class="tok-str">"tok-com">    # Serialised as a string in JSON; declaring it keeps the TS type honest.
    elapsed_ms: float

@app.post("/embed", response_model=EmbedResponse)
def embed(req: EmbedRequest) -> EmbedResponse:
    ...

Two habits that pay for themselves:

Use Literal for closed sets. It becomes a TypeScript union, so an invalid model name is a compile error in the frontend rather than a 422 at runtime.

Declare constraints in Field. max_length=64 is enforced by the server, documented in OpenAPI, and visible to whoever writes the client. The same fact in three places, written once.

Version the API in the path, not only in the title. /v1/embed lets you generate two clients during a migration. A version that only exists in metadata cannot be depended on.

Generating the client

"tok-com"># Regenerating is cheap; keeping the output in git makes drift reviewable.
.PHONY: types check-types

types:
    python -c "import json, app.main as m; print(json.dumps(m.app.openapi(), indent=2, sort_keys=True))" > openapi.json
    npx openapi-typescript openapi.json -o frontend/src/api.d.ts
"tok-com">
# CI: regenerate into a temp dir and diff. Fails if someone changed the
"tok-com"># server without regenerating, or edited the generated file by hand.
check-types: types
    git diff --exit-code openapi.json frontend/src/api.d.ts

sort_keys=True matters more than it looks: without it, dictionary ordering changes produce diffs that are not semantic changes, the check cries wolf, and the team learns to bypass it.

Generating from app.openapi() rather than from a running server keeps the step hermetic — no port, no startup, no flake.

The frontend side

import type { components } from "./api";

type EmbedRequest = components["schemas"]["EmbedRequest"];
type EmbedResponse = components["schemas"]["EmbedResponse"];

class=class="tok-str">"tok-com">// One typed wrapper, used everywhere. The point is not elegance — it is
class=class="tok-str">"tok-com">// that the type arguments cannot be wrong, because they are imported.
async function embed(req: EmbedRequest): Promise<EmbedResponse> {
  const r = await fetch("/api/embed", {
    method: "POST",
    headers: { "content-type": "application/json" },
    body: JSON.stringify(req),
  });
  if (!r.ok) throw new Error(\`embed failed: \${r.status}\`);
  return r.json();
}

Note what this does not do: validate the response. The types are a compile-time claim that the server keeps its contract, not a runtime guarantee. For an internal API behind your own CI check, that is usually the right trade. For anything crossing a trust boundary, parse the response with a runtime validator too — generated types will happily describe a payload that never arrives.

Where it breaks

Any propagates. A single dict[str, Any] in a pydantic model becomes Record<string, unknown>, and the guarantee stops there. If a field is genuinely polymorphic, model it as a discriminated union rather than surrendering to Any.

NumPy types are not JSON types. np.float32 is not float, and pydantic will refuse it. Convert at the boundary — .tolist() or .item() — and do it in the response construction, not in a serialiser hook where it becomes invisible.

Large arrays are the wrong shape for JSON. An embedding response with 64 × 1024 floats is several megabytes of decimal text. Once the payload matters, the honest answer is a binary encoding (Arrow, npy, or base64 float32) with the schema still describing the envelope.

Optional versus nullable. str | None and "field may be absent" are different statements, and they generate different TypeScript. Decide deliberately; the default is rarely what you meant.

Testing the contract, not just generating it

The CI check proves the generated types match the server. It says nothing about whether the server matches reality — different claims, and the second one fails in its own way.

Test the schema, not only the handler. A response model declaring dim: int while the handler returns a NumPy integer passes every unit test that calls the function directly, then fails at serialisation. Exercise the route through the test client so pydantic actually runs:

from fastapi.testclient import TestClient

client = TestClient(app)

def test_embed_response_matches_schema():
    r = client.post("/embed", json={"texts": ["hello"], "model": "e5-large"})
    assert r.status_code == class="tok-num">200
class=class="tok-str">"tok-com">    # Re-validating with the declared model IS the assertion: if the handler
class=class="tok-str">"tok-com">    # returns a shape the schema does not describe, this raises.
    EmbedResponse.model_validate(r.json())

def test_constraints_are_enforced():
class=class="tok-str">"tok-com">    # class="tok-num">65 items against max_length=class="tok-num">64. The bound is written in exactly one
class=class="tok-str">"tok-com">    # place, so this test protects the contract rather than the handler.
    r = client.post("/embed", json={"texts": ["x"] * class="tok-num">65})
    assert r.status_code == class="tok-num">422

Treat a schema change as an API change. Adding an optional field is safe. Making an optional field required, narrowing a Literal, or renaming anything is not. The generated diff in CI is where that call gets made — the second reason to commit generated files rather than build them silently.

Snapshot the OpenAPI document across releases. Keeping the previous openapi.json lets you diff two versions and answer "did we break a client" mechanically instead of from memory.

Why this holds up

The property that makes it work is not the type system — it is that there is exactly one place where the contract is written, and a check that fails when anything downstream disagrees with it.

You could get the same guarantee with protobuf, or with a hand-written schema and a code generator. What you cannot get is the guarantee without the CI check: a generated file with no enforcement is a hand-written file with extra steps.