Self-Hosted Large Language Model (LLM) – Open WebUI And Ollama

I set up a self-hosted large language model (LLM) workflow using Open WebUI and Ollama on my local machines a week ago. By self-hosting, I don’t need API keys or cloud dependency… just pure local inference! With Open WebUI serving as a sleek front-end and Ollama managing the back-end LLM runtime, it’s been pretty fun to use LLMs like Llama 3.2, Mistral, and Gemma in a private, secure environment.

I figured the RTX 4090 on the latter would offer more raw computing to run LLMs, so I started this process by installing Ollama on my gaming PC. To expose Ollama on my network to other devices, I needed to change the default bind address by editing an environment variable. On Windows 11, navigate to Start → Settings → Control Panel → Search for “Edit Environment Variables For Your Account.” In the section for User variables for (username), select New, create a variable called OLLAMA_HOST, and make the value 0.0.0.0:11434. When you’re finished, you should see an entry similar to this. Look at the taskbar (bottom right of the screen), find the Ollama icon, exit the service, and restart it.

Now to install the user interface. My Synology DS1821+ serves as my network-attached storage and a Docker home-lab server running over 50 containers (including Open WebUI). Here is the relevant docker-compose.yml I used to spin up an Open WebUI container.

services:
  webui:
    container_name: OpenWebUI
    image: ghcr.io/open-webui/open-webui:0.5
    restart: unless-stopped
    volumes:
      - /volume1/docker/openwebui:/app/backend/data:rw
    environment:
      OLLAMA_BASE_URL: http://192.168.7.253:11434
    ports:
      - 8271:8080

Note the bolded IP address for the OLLAMA_BASE-URL environment variable. This should match the IP address of your Ollama device (in my workflow, my PC). Now when I navigate to my container’s IP address, I’ll set up an Open WebUI account and see the large language models I downloaded through Ollama.

Drop me a comment below with questions!