Private model hosting and GPU experimentation using local infrastructure rather than relying entirely on remote AI services.
The self-hosted AI project uses the homelab’s NVIDIA GPU capacity to experiment with local inference, private model serving, API-compatible endpoints and application integrations.
Learn GPU passthrough, model/runtime compatibility, container orchestration, authentication, performance limits and where local inference is genuinely useful.
Local inference can keep selected workflows and datasets inside the lab.
The project is as much about GPU/runtime integration as it is about models.
Dedicated 16GB workstation GPU for CUDA workloads and local model inference.
Self-hosted model/API layer used to expose local inference services.
AI services run inside the existing Proxmox and Docker environment.
A natural next step is retrieval over locally hosted documentation and project knowledge.
APPLICATIONS
│
▼
LOCAL AI API
│
▼
MODEL RUNTIME
│
▼
RTX A4000
│
▼
LOCAL MODELS