Basically one has two real choices for local LLMs: llama.cpp (if single user) or vLLM (if multi-user/enterprise).
Basically one has two real choices for local LLMs: llama.cpp (if single user) or vLLM (if multi-user/enterprise).