# How to add AI to your home server: self-hosted services guide

> AI Release · @ai_release1 · https://ai-release.org/guides/kak-podklyuchit-ii-k-domashnemu-serveru-poshagovoe-ruko_en.html

Self-hosting an AI service means running the model and its API on hardware you own instead of sending every request to someone else's cloud. This guide walks you through the whole path on a home Linux server (Ubuntu/Debian LTS or similar), a spare PC, or a NAS that can run Docker; the same steps work on Windows or macOS if you use Docker Desktop instead of the Linux install commands. The only thing you need to decide before you start is which self-hosted AI image you want to run — everything else below is generic plumbing that fits any of them.

## Requirements and preparation
![Illustration: Requirements and preparation](https://ai-release.org/guides/img/kak-podklyuchit-ii-k-domashnemu-serveru-poshagovoe-ruko-en-1.jpg)


- A machine that stays on: home server, mini-PC, old laptop or NAS. Linux with sudo rights is assumed for the commands; Windows/macOS work with Docker Desktop and PowerShell.
- Docker Engine and the Docker Compose plugin. Docker is the tool that packs a service and all its dependencies into a "container" — an isolated box that runs the same way on any machine.
- Enough free RAM and disk space for the model you plan to run, plus a few GB extra for logs and updates. Check before you start, not after.
- One free TCP port. We use `8080` for the API and `3000` for a web interface in the examples. A port is just a numbered door on the machine; two services cannot use the same number.
- An API key. Self-hosted services generate their own, but if you also want a cloud model as a fallback, you need a key from that provider — the process is described in [How to Use GPT-6.1 Sol via API](https://ai-release.org/guides/kak-ispolzovat-gpt-6-1-sol-cherez-api-polnoe-rukovodstv.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide).
- A client to talk to the server: a chat app, a browser, or a desktop client such as [DeepSeek Harness Desktop](https://ai-release.org/guides/deepseek-harness-desktop-kak-ustanovit-i-podklyuchit-pl.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide).

Pre-flight checklist:

- [ ] `sudo` access confirmed
- [ ] Docker and Compose installed and tested
- [ ] Project folder created outside of system directories
- [ ] `.env` file added to `.gitignore`
- [ ] Port 8080 free
- [ ] A backup plan for the model files (they are large)

## Step-by-step instructions
![Illustration: Step-by-step instructions](https://ai-release.org/guides/img/kak-podklyuchit-ii-k-domashnemu-serveru-poshagovoe-ruko-en-2.jpg)


1. **Check the hardware and OS.** Run `uname -a`, `lsb_release -a`, `free -h` and `df -h`. Expected result: you see the kernel, the distribution version, free RAM and free disk in human-readable form.

**Пример:**
```bash
free -h
#               total        used        free
# Mem:           15Gi       1.2Gi        13Gi
df -h /home
# Filesystem      Size  Used Avail Use% Mounted on
# /dev/sda2       468G   62G  382G  14% /
```
*Why it works:* container memory limits are inherited from the host, so if the host has no free RAM, the container will be killed later no matter how you configure it.

2. **Install Docker and Compose.** On Ubuntu/Debian: `curl -fsSL https://get.docker.com | sh`, then `sudo usermod -aG docker $USER` and `newgrp docker`. Expected result: `docker --version` prints a version, `docker compose version` prints a version, and `docker ps` works without `sudo`.
*Why it works:* the convenience script adds Docker's own repository and the Compose plugin, and the `usermod` line puts your user into the `docker` group so you stop typing `sudo` for every command.

3. **Create the project folder and the key file.** Run `mkdir -p ~/ai-server && cd ~/ai-server`, then create `.env` with `AI_API_KEY=change-me-please` and `API_PORT=8080`. Add `.env` to `.gitignore`. Expected result: `ls -a` shows both files, and `cat .env` prints your two lines.
*Why it works:* Compose reads `.env` automatically, so secrets stay out of the YAML file and out of version control.

4. **Write `docker-compose.yml`.** Paste this next to `.env`, replacing the image line with the service you chose:

```yaml
services:
  ai:
    image: your-vendor/your-ai-service:latest   # replace with your image
    restart: unless-stopped
    ports:
      - "${API_PORT}:8080"
    env_file:
      - .env
    volumes:
      - ./models:/models
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 5s
      retries: 5
```

Expected result: `docker compose config` prints the resolved file with `8080` substituted for `${API_PORT}` and no error.
*Why it works:* `volumes` keeps model files on the host so a container update does not delete them, and `restart: unless-stopped` brings the service back after a reboot.

5. **Start the stack.** Run `docker compose up -d`, then `docker compose ps`. Expected result: the container shows `STATUS: Up` (or `Up (healthy)` after the first health check).
*Why it works:* `-d` runs it in the background; `ps` is your fastest way to see whether it is actually alive.

6. **Verify the API from the server itself.** Run `curl http://localhost:8080/v1/models`. Expected result: a JSON list of available models, for example:

```json
{"object":"list","data":[{"id":"local-model","object":"model"}]}
```

*Why it works:* talking to `localhost` skips the firewall and network entirely, so if this fails, the problem is inside the container, not in your router.

7. **Open the port only to your home network.** With UFW: `sudo ufw allow from 192.168.1.0/24 to any port 8080 proto tcp`. Expected result: `sudo ufw status` shows the rule. Do not run `ufw allow 8080` — that opens the port to the entire internet.
*Why it works:* the `/24` restricts access to addresses in your own subnet.

8. **Point a client at the server.** In your app's settings, set Base URL to `http://192.168.1.50:8080/v1` (replace with your server's LAN address from `hostname -I`) and paste the API key from `.env`. Expected result: the client lists your model and answers a test prompt.
*Why it works:* nearly all AI clients speak the same OpenAI-style HTTP format, so any self-hosted service that mimics it drops in without code changes.

9. **Optional: expose it safely outside the home.** Put a reverse proxy (Nginx or Caddy) in front and issue a TLS certificate, then forward `https://ai.example.com` to `127.0.0.1:8080`. Expected result: a padlock in the browser and no open port besides 443.
*Why it works:* the proxy terminates encryption and the AI container never touches the public network directly.

10. **Update later without losing anything.** Run `docker compose pull` and then `docker compose up -d`. Expected result: `docker compose ps` shows a new container ID, and your chats, models and settings are still there.
*Why it works:* only the image layer is replaced; the `./models` folder and `.env` stay on disk.

## Common problems and solutions
![Illustration: Common problems and solutions](https://ai-release.org/guides/img/kak-podklyuchit-ii-k-domashnemu-serveru-poshagovoe-ruko-en-3.jpg)


**Symptom: `Error response from daemon: driver failed programming external connectivity ... port is already allocated`.** Something else owns port 8080.

1. Run `sudo ss -tlnp | grep 8080` to see which process holds it.
2. Stop that process, or change `API_PORT=8081` in `.env`.
3. Re-run `docker compose up -d` and check `docker compose ps`.

**Symptom: `permission denied while trying to connect to the Docker daemon socket`.** Your user is not in the `docker` group yet.

1. Run `sudo usermod -aG docker $USER`.
2. Log out and back in (or run `newgrp docker` in the current shell).
3. Confirm with `docker ps` — it should print a table instead of an error.

**Symptom: the container restarts every few seconds; `docker compose logs -f` ends with `Killed` or `OOMKilled`.** The model is bigger than the free RAM.

1. Run `docker inspect <container> --format '{{.State.OOMKilled}}'` — `true` confirms it.
2. Switch to a smaller version of the model, or add swap space.
3. Restart with `docker compose up -d` and watch `docker stats` until memory stabilizes.

## FAQ

**Can I run this on a Raspberry Pi or an old laptop?**
Yes, if it has enough RAM and disk for the model you pick. Both run Docker fine. CPU-only inference is slower than a GPU, so expect a delay of several seconds before the first word appears.

**Do I need a graphics card?**
No. Everything above works on CPU. A GPU only shortens response time; it changes nothing about the setup, ports or configuration.

**How do I reach the server from outside my home?**
Use a reverse proxy with a TLS certificate, or better, a VPN into your home network. Never forward port 8080 straight from your router — the service has no built-in protection against internet-wide scans.

**What about MCP and RAG?**
MCP (Model Context Protocol — a standard way for an AI app to call outside tools and data sources) is what you add when you want your self-hosted assistant to read files, query databases or run commands. Start here: [MCP explained: what it is and how to connect your first MCP server](https://ai-release.org/guides/mcp-chto-eto-i-kak-podklyuchit-server.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide). RAG (retrieval-augmented generation — the model first finds relevant documents, then answers from them) is the next step after that, and it runs on the same server.