Typing an ML API end to end: FastAPI, pydantic and a generated TypeScript client

The classic failure is not spectacular. A field gets renamed on the server, the frontend still reads the old name, undefined lands in a chart, and the chart shows an empty state that looks normal. Nobody notices for a week.

The fix is structural: stop maintaining the contract in two places. I define it once in Python, FastAPI publishes it as OpenAPI, and the TypeScript is generated from there. The only component that has to be right is the generator.

The chain

The pieces

• pydantic v2: request and response models, with validation
• FastAPI: serves them, and publishes /openapi.json from the same objects
• openapi-typescript: turns that document into a .d.ts
• A CI check that regenerates and fails on any difference

The last point is what makes the rest reliable. Without it, the generated file becomes a snapshot that someone will eventually edit by hand.

The model server

from typing import Annotated, Literal

from fastapi import FastAPI
from pydantic import BaseModel, Field

app = FastAPI(title="inference", version="class="tok-num">1.0.class="tok-num">0")

class EmbedRequest(BaseModel):
class=class="tok-str">"tok-com">    # Constraints go in the schema: they are validated at runtime AND exported
class=class="tok-str">"tok-com">    # to OpenAPI, so the frontend sees the same bounds the server enforces.
    texts: Annotated[list[str], Field(min_length=class="tok-num">1, max_length=class="tok-num">64)]
    normalize: bool = True
    model: Literal["e5-large", "bge-m3"] = "e5-large"

class EmbedResponse(BaseModel):
    vectors: list[list[float]]
    dim: int
    model: str
class=class="tok-str">"tok-com">    # Serialised as a JSON number; declaring it keeps the TS type accurate.
    elapsed_ms: float

@app.post("/embed", response_model=EmbedResponse)
def embed(req: EmbedRequest) -> EmbedResponse:
    ...

Three habits that pay off quickly.

Literal for closed sets. It becomes a TypeScript union: an invalid model name is a compile error on the frontend, long before a 422 at runtime.

Constraints in Field. max_length=64 is enforced by the server, documented in OpenAPI and visible to whoever writes the client. The same rule in three places, written once.

The API version in the path. /v1/embed lets you generate two clients during a migration. A version that only lives in the metadata is useless the day two versions have to coexist.

Generating the client

"tok-com"># Regenerating is free; keeping the output in git makes drift reviewable.
.PHONY: types check-types

types:
    python -c "import json, app.main as m; print(json.dumps(m.app.openapi(), indent=2, sort_keys=True))" > openapi.json
    npx openapi-typescript openapi.json -o frontend/src/api.d.ts
"tok-com">
# CI: regenerate, then diff. Fails if someone changed the server without
"tok-com"># regenerating, or hand-edited the generated file.
check-types: types
    git diff --exit-code openapi.json frontend/src/api.d.ts

sort_keys=True matters more than it looks. Without it, a mere change in key order produces diffs that change nothing in meaning, the check fires for nothing, and the team learns to route around it.

I generate straight from app.openapi(), without starting a server: no port, no startup, nothing random.

On the frontend

import type { components } from "./api";

type EmbedRequest = components["schemas"]["EmbedRequest"];
type EmbedResponse = components["schemas"]["EmbedResponse"];

class=class="tok-str">"tok-com">// One typed function, used everywhere. The types are imported:
class=class="tok-str">"tok-com">// the parameters cannot be wrong.
async function embed(req: EmbedRequest): Promise<EmbedResponse> {
  const r = await fetch("/api/embed", {
    method: "POST",
    headers: { "content-type": "application/json" },
    body: JSON.stringify(req),
  });
  if (!r.ok) throw new Error(class="tok-str">`embed failed: ${r.status}`);
  return r.json();
}

This code does not validate the response. The types assert, at compile time, that the server keeps its contract, and nothing checks it at runtime. For an internal API guarded by its own CI check, that is usually a good trade-off. For anything crossing a trust boundary, I also validate the response at runtime: generated types will calmly describe a payload that never arrives in that shape.

Where it breaks

Any spreads. A single dict[str, Any] in a pydantic model becomes Record<string, unknown>, and the guarantee stops there. If a field really is polymorphic, I model it as a discriminated union.

NumPy types are not JSON types. np.float32 is not a float, and pydantic will reject it. I convert at the boundary, with .tolist() or .item(), while building the response. Hidden in a serialisation hook, the problem becomes invisible.

Large arrays fit JSON badly. An embeddings response of 64 × 1024 floats weighs several megabytes of decimal text. Once size matters, move to a binary encoding (Arrow, npy or base64 float32), with the schema still describing the envelope.

Optional and nullable are two different things. str | None and "the field may be missing" generate different TypeScript. The default behaviour is rarely the one you wanted, so I choose explicitly.

Testing the contract, on top of generating it

The CI check proves the generated types match the server. It says nothing about whether the server matches what it actually returns, and that second point fails in its own way.

Test the schema through the route. A response model declaring dim: int while the handler returns a NumPy integer passes every unit test that calls the function directly, then fails at serialisation. I push the route through the test client so pydantic really runs:

from fastapi.testclient import TestClient

client = TestClient(app)

def test_embed_response_matches_schema():
    r = client.post("/embed", json={"texts": ["hello"], "model": "e5-large"})
    assert r.status_code == class="tok-num">200
class=class="tok-str">"tok-com">    # Re-validating with the declared model IS the assertion: if the handler
class=class="tok-str">"tok-com">    # returns a shape the schema does not describe, this raises.
    EmbedResponse.model_validate(r.json())

def test_constraints_are_enforced():
class=class="tok-str">"tok-com">    # class="tok-num">65 items against max_length=class="tok-num">64. The bound is written in one place
class=class="tok-str">"tok-com">    # only: this test protects the contract itself.
    r = client.post("/embed", json={"texts": ["x"] * class="tok-num">65})
    assert r.status_code == class="tok-num">422

A schema change is an API change. Adding an optional field breaks nothing. Making an optional field required, narrowing a Literal or renaming anything does. The decision is made by reading the generated diff in CI, which is a second reason to commit generated files.

Keep the OpenAPI document for each version. With the previous openapi.json at hand, you compare two versions and answer "did we break a client?" mechanically, without relying on anyone's memory.

The maths, intuitively: one source, zero copies

Why does a contract written in one place hold better than two carefully maintained ones? Because the number of ways to drift grows with the number of copies.

With k copies of the same contract (the pydantic model, the TypeScript client, the docs…), there are k(k − 1)/2 pairs that have to stay in agreement: 1 pair for 2 copies, 3 for 3, 6 for 4. And every change has to be carried over by hand into the k − 1 other copies. If each carry-over is right with probability p, the probability that no drift appears after m changes is:

P(no drift) = p^(m(k − 1))

With carry-overs right 98 times out of 100, 50 changes and 3 copies: 0.98^100 ≈ 13%. In 87% of cases, at least one drift has crept in somewhere. Even careful work loses this game, because small error probabilities add up. Generating the client from a single source brings k − 1 down to zero, and p^0 = 1, provided a CI check verifies that the generated file matches the source. Without that check, the generated file becomes just another copy.

Why it holds

The type system helps, but what really makes the whole thing work is that there is one place where the contract is written, and a check that fails as soon as anything downstream disagrees with it.

You would get the same guarantee with protobuf, or with a hand-written schema and a code generator. In every case, the guarantee comes from the CI check. A generated file that nobody verifies ends up living its own life, like a hand-written one.