A Hugging Face model inside a Docker image: four settings to make sure it is really there

The model is downloaded at build time, and at startup the container looks for it somewhere else or downloads it again. Four settings I now apply to every image, learned the hard way on my ml-runner.

My ml-runner ships BGE-M3 inside its Docker image, so it can start without downloading anything. On paper, that is simple. In practice, I got a model downloaded at build time and nowhere to be found at startup. Since then, I apply four settings.

1. Set HF_HOME before any import

HF_* variables are read when huggingface_hub is imported. An os.environ["HF_HOME"] = ... written after import sentence_transformers or import transformers has no effect anymore: the library has already computed its paths.

The rule: these variables go in the Dockerfile (ENV) or in the container environment, never in the middle of the code.

2. Know where the model lands

With HF_HOME=/app/hf_cache, the cache is stored in /app/hf_cache/hub/models--<org>--<name>/. The hub subfolder comes from HF_HUB_CACHE, which defaults to $HF_HOME/hub.

The trap: the cache_folder parameter of sentence-transformers expects the folder that directly contains the models--… entries, so /app/hf_cache/hub. Passing it /app/hf_cache makes it look one level too high. The simplest option is to not pass cache_folder at all and let HF_HOME decide, at build time and at runtime alike.

3. Keep the cache out of volumes

If the cache sits in a folder mounted as a volume at startup (/models in my case), the volume hides the image's content. The model downloaded at build time vanishes, and the volume comes with the host's permissions, often root. The container can then neither read the old cache nor write a new one.

So I put the cache in a folder of the image (/app/hf_cache), owned by the runtime user.

4. Check offline during the build

The last setting is the most useful: reload the model with HF_HUB_OFFLINE=1 in the Dockerfile. In that mode the library makes no network call and only uses the cache. If the model is not exactly where the code will look for it, the build fails, and the broken image never ships.

# Cache inside the image, outside any volume.
ENV HF_HOME=/app/hf_cache

# 1) Download at build time.
RUN python -c "from huggingface_hub import snapshot_download; snapshot_download('BAAI/bge-m3')"

# 2) Reload without network, exactly as at startup: failure = failed build.
RUN HF_HUB_OFFLINE=1 python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('BAAI/bge-m3')"

# 3) The cache belongs to the user that runs the service.
RUN chown -R runtime:runtime /app/hf_cache   # "runtime": adapt to your image

Bonus: even with the model cached, huggingface_hub normally sends a request to check whether a newer version exists. With HF_HUB_OFFLINE=1 at runtime, that request goes away and loading is faster. Turn it on if the container is not meant to update the model by itself.

Source