Self-Hosting LLMs Securely: Hardening Ollama and vLLM

About this course

“This course contains the use of artificial intelligence.”

Running an open-weight model on your own hardware keeps prompts and data in house, but it also makes you the operator of a new kind of server. This course treats a model server as a server: you measure what the default install exposes, harden it layer by layer, and prove every change with a check that runs.

You start by looking, not fixing. Ollama and a CPU build of vLLM are installed the documented way, their API surfaces are mapped endpoint by endpoint, and a verification script records every check that fails. That baseline scoreboard is what the rest of the course turns green.

Ollama is hardened on the host first: bind address, CORS, a dedicated service user, systemd sandboxing and resource limits, with the documentation's own statement that it does not require authentication taken at face value. Then the same service goes into a container that runs as non-root, read-only, without capabilities and published on loopback only, with the image pinned by digest and scanned.vLLM gets its own section. You read its security guidance line by line, set the bind address explicitly, and measure exactly which endpoints --api-key protects and which still answer without it. What the flags cannot close, the gateway does: nginx with TLS from a private CA, one key per client, an endpoint allowlist, rate and size limits, and an LLM proxy layer for per-client budgets.

The host is then isolated with a default-deny firewall and no route out, and models reach it only through a staging step. Model files are treated as code-adjacent: you compare pickle, safetensors and GGUF, scan files, pin Hugging Face downloads to a full commit, verify digests, sign and verify a model, and serve only what a model manifest lists.

Read the full description on Udemy on Udemy (opens in a new tab)

Prices, discounts and availability are set by Udemy. We may earn a commission when you purchase through links on this site.