MLOps field notes

The menu: full POCs with their maths, new models tried while they are fresh, opinion pieces and quick tips to go.

• EmbeddingGemma 2: a test bench for replacing BGE-M3 in my RAG — Plat du jour, Oct 30, 2026
  Google released EmbeddingGemma 2 on 6 October: 271 million parameters for the text part, 768-dimension vectors you can cut down to 256, Apache 2.0 licence. On my server, BGE-M3 shares an eighth of a GPU with a reranker. The POC to compare both on my own documents, and the fp16 trap.
• A Hugging Face model inside a Docker image: four settings to make sure it is really there — To go, Oct 30, 2026
  The model is downloaded at build time, and at startup the container looks for it somewhere else or downloads it again. Four settings I now apply to every image, learned the hard way on my ml-runner.
• Mistral Large 4 goes open weight: good news, but I won't be running it — Apéro, Oct 30, 2026
  At the end of October, Mistral published the weights of Large 4, a model of roughly a trillion parameters. For Europe it is real progress. For someone serving models on an L4, the numbers add up fast, and the licence has stepped back compared with Large 3.
• Orchestrating specialised agents: choosing, synthesising, showing what is going on — À la carte, Sep 12, 2026
  An agent with forty tools ends up picking the wrong one. Several narrow agents behind an orchestrator hold up better, under four conditions: choose before calling, receive structured findings, be able to say "I don't know", and display a real state. What I took away from building Chimera.
• Evaluating a model before shipping it: my four-layer harness — À la carte, Apr 14, 2026
  A good test-set score guarantees nothing about production. How I build an evaluation pipeline that catches regressions before users do: behavioural tests, a regression set, a CI gate, and the calculation that tells you whether a score difference means anything.
• vLLM, TGI, SGLang at batch=1: how I measure latency — À la carte, Apr 2, 2026
  Throughput benchmarks answer a different question than an assistant, where a single user waits for an answer. The single-stream protocol I use, the metrics to track, the physical floor set by memory, and the pitfalls that make most published numbers impossible to compare.
• Feature store: when the served value stops matching training — À la carte, Mar 24, 2026
  Train/serve skew hides in the handoff between the offline and online paths. No error shows up and offline metrics stay good. The three forms I run into, a simple calculation for the age of a served value, and the check that makes it visible.
• Typing an ML API end to end: FastAPI, pydantic and a generated TypeScript client — À la carte, Mar 11, 2026
  The API contract written once in Python, published as OpenAPI by FastAPI, and the frontend TypeScript generated from it. A CI check makes sure nothing has drifted. No hand-written client.
• I read MLOps maturity models from the top down — Apéro, Feb 28, 2026
  The level 0 → level 3 staircase describes the destination well, and the order of the steps is made up. Following it to the letter is the best way to automate a model nobody uses. My own order, and the small calculation behind it.
• Monitoring an LLM from day one: latency, drift, quality, cost — À la carte, Feb 14, 2026
  Four signals to track on an LLM service from the day it goes live, why classic application monitoring misses them, and the order in which I instrument them.