Turn your computer into an AI machine - Install Open WebUI

2024-11-26 2024-11-26 1649 words 8 minutes

Discuss and setup hard- and software requirements to install Open WebUI - the UI for (nearly) all AI needs

Contents

Introduction

I recently played around with hosting various AI stuff locally - so no Internet connection needed and also no OpenAI/ChatGPT (but still can be used if needed). I wanted to see how good self-hosted AI is already working. Keeping things private is for sure a reason to do so if you want to process some documents that should not leave your home or your company e.g. It turned out that it’s already working pretty good if you meet some prerequisites.

This blog series will install all the tools, AI models, and other stuff needed for various use cases. The focus is to have everything running locally but also being able to use whatever APIs that makes sense like the OpenAI API.

Note: I’m by no means an AI expert. I’m just very interested in this technology and this blog series just summarizes my findings and how it works for me - esp. running everything at home 😉

The goals

Have Open WebUI running as a frontend with ollama as backend hosting various AI models
Using Open WebUI as a chat frontend for self-hosted AI models and for OpenAI/ChatGPT
Using Open WebUI to generate AI images with self-hosted ComfyUI or DALL-E from OpenAI
Have ComfyUI running to generate AI images with Stable Diffusion using AI models like Juggernaut XL or FLUX.1-dev
Connect Open WebUI and ComfyUI to use text generated in Open WebUI as an input for ComfyUI to generate images

The hardware

To make it short: More is better 😄 I’ve no idea how well that all works with a newer Mac and maybe also with newer CPUs but the difference between running a Chatbot with my older AMD Ryzen 3900X CPU (12 cores) CPU and the same on a Nvidia RTX 4080 Super GPU is quite dramatic. And if we are talking about GPU already: A Nvidia RTX 4080 with 16 GByte of VRAM (video memory) is basically a must if it comes to generating pictures with AI. A Nivida RTX 4090 with 24 GByte of VRAM would be even better… But even with that high end consumer GPU you still can’t do everything AI related as it’s just not enough VRAM… While AI support for AMD Radeon GPUs (e.g. via ROCm) improved quite a lot within the last 12 month, Nvidia is still the way to go - at least until end of 2025 (and Nvidia might already launch RTX 5090/5080 in January 2025).

So the minimal hardware setup for this tutorials is like this (to have at least a bit of fun…):

CPU: AMD Ryzen 3900X (some AMD/Intel CPU with 8 cores or more at least)
GPU: Nvidia RTX 4080 with at least 16 GByte video memory (VRAM) esp. for image generation
RAM: 32 or even 64 GByte RAM is definitely recommended
HDD: A fast disk like a SSD or NVMe with at least 200 GByte (AI models need lot of space)

You can also try with a GPU with only 8 or 12 GByte VRAM. But then you can only load models with less parameters. Sometimes that’s already good enough and models improve all the time. But you really need to keep an eye on GPU VRAM usage.

For some AI tasks a fast CPU is already good enough like transcription of audio to text e.g. But you need to experiment a bit. For some tasks a CPU is quite slow. But if that’s not a problem for you and still have a lot of RAM > 64 GByte e.g. you might be able to load quite big models into RAM e.g. using it as a local coding assistant.

The software

The only requirement is basically Ubuntu Linux version 24.04 or newer. Ubuntu 22.04 might be also good enough and other other Linux OS but I’m concentrating on Ubuntu 24.04.

My hardware and software setup

My setup looks a bit different than the minimal setup described above. I run Archlinux as my daily operating system and use KDE 6 as desktop. My workstation has two graphics card installed and in general looks like this:

CPU: AMD Ryzen 3900X
RAM: 128 GByte
GPU: AMD Radeon RX 6700XT with 12 GByte VRAM
GPU: Nvidia RTX 4080 Super with 16 GByte VRAM

The Radeon GPU is used for Archlinux and KDE (so my default GPU so to say). I’ve various virtual machines running using QEMU/KVM. One of those VMs is using Ubuntu 24.04. To this VM I passed through the Nvidia GPU. I wrote a blog post how to do so. The advantage of this approach is that the Nvidia GPU is completely dedicated for the AI stuff. So I don’t need to spend any VRAM for my desktop and the 16 GByte VRAM available are completely free. I also passed 12 CPU cores to this VM. The other nice thing about that setup is that I can just shutdown that “AI” VM and start Windows 11 to play some games. The Nvidia GPU is also passed through to a Windows VM. You just can’t run two or more VMs sharing the same GPU. But that’s a different story 😉

Setting up Nvidia GPU driver, CUDA toolkit and Nvidia utils

If you have Ubuntu 24.04 with your Nvidia GPU running already then you most probably also have the Nvidia driver installed. How to setup Nvidia driver, CUDA and utils (nvidia-smi e.g.) I’ve already described in my blog post Setting up environment for Stable Diffusion for image creation. Please install the components if not already done so and get back if done:

Install git utility

Since a few source code repositories need to be downloaded (or cloned) from github.com later, git utility is needed:

bash

sudo apt install git

Install uv utility

uv is an extremely fast Python package and project manager - and it’s really fast. While this THE one stop Python utility for me meanwhile there are two specific use-cases:

Setup a Python virtual environment
Use specified Python version in the Python virtual environment

Please read the Install uv utility paragraph I already wrote in a previous blog post and then come back.

Install Open WebUI

Before installing Open WebUI we want to make sure that it gets installed in a Python virtual environment (venv). During the installation of Open WebUI lots of dependencies will get downloaded. To not mess around with the Python installation of your operating system the venv makes sure that everything Open WebUI related gets installed in its very own directory. By deleting this directory later everything is gone. So no leftovers later.

The Github website of Open WebUI mentions different installation options e.g. using Docker. So if you want to use Docker please look there.

As of writing this blog post Open WebUI runs best with Python 3.11. That avoids compatibility issues. To install the venv and have Python 3.11 within the venv, uv comes to rescue. uv venv specifies that a virtual environment should be created in the directory open-webui (last parameter). --python 3.11 specifies that the Python interpreter should be 3.11 within the Python venv. --seed specifies that also tools like pip should be installed and I’ll use that pip to install Open WebUI later. So in my $HOME directory I run this command:

bash

uv venv --python 3.11 --seed open-webui

Next activate the venv. That means that the PATH variable will be adjusted to have the venv version of python and pip binary in the PATH e.g.:

bash

source open-webui/bin/activate

Next switch to the new (venv) directory:

bash

cd open-webui

Make sure that python and pip binary are the ones installed in the venv. This can be verified by using which command. E.g.

bash

which pip

The output will look similar to /home/your-user/open-webui/bin/pip. If it’s something like /usr/bin/pip something went wrong and you still use the binary of your OS.

Finally Open WebUI can be installed (that’s version 0.4.7 currently):

bash

pip install open-webui

This could take quite a while depending on your Internet speed. After the command is done you have around 7 GByte less space on your HDD 😉

Using direnv

direnv is a little utility that can run various commands when you enter a directory. So instead of running source bin/activate all the time to activate the Open WebUI venv I’ve added a file called .envrc into the open-webui directory with the following content:

bash

source ./bin/activate

Next install direnv:

bash

sudo apt install direnv

Additionally there needs to be a hook added to $HOME/.bashrc to make direnv work:

bash

eval "$(direnv hook bash)"

To get it enabled immediately:

bash

source $HOME/.bashrc

The next time you enter /home/your-user/open-webui directory you get an error as you need to allow direnv to load the .envrc file in this directory. E.g.:

bash

direnv allow .

If you now leave the directory and re-enter again the commands in .envrc will be executed. In this case it means that the Python venv gets activated and python and pip executables of that venv will be used.

Start Open WebUI

Starting Open WebUI is as easy as:

bash

open-webui serve

This will download another bunch of files fist and finally starts a webserver. If the server logs INFO: Uvicorn running on http://0.0.0.0:8080 you can start using it. If you installed Open WebUI on your local computer just use http://127.0.0.1:8080. If you installed on a remote host use the IP/hostname of that host and make sure that your firewall settings allow connections to port 8080.

Setup authentication

When you open Open WebUI UI the very first time you need to specify a user. That user will become kinda “super-user”. You need to specify a username, email and a password. Store that information in a secure place and share it only with administrators if really needed.

That’s it for today and the installation of Open WebUI. Next time I’ll show how to configure the most important options for Open WebUI to make it somewhat useful.