(refactor) Add native systemd production deployment

Make native systemd services the recommended production path for pobsync while keeping Docker Compose available for development and optional test installs. Add web, worker, and scheduler systemd unit templates, a native environment example, an optional nginx reverse proxy template, and an installer that creates the venv, service user, env file, units, and runs migrations/static collection. Allow native deployments to configure POBSYNC_BACKUP_ROOT directly and document the new production layout and update flow.
2026-05-19 15:59:07 +02:00
parent 1297a839d4
commit b93e19a7c8
8 changed files with 290 additions and 46 deletions
--- a/README.md
+++ b/README.md
@@ -7,7 +7,7 @@ The refactor direction is SQL-first:
 - Django is the management layer and source of truth.
 - SQLite is the default database; MariaDB is optional.
 - Backups still use the existing rsync snapshot engine internally.
- Scheduling is handled by a Django/Docker scheduler process, not host cron.
+- Scheduling is handled by a Django scheduler service, not host cron.
 - Legacy YAML import/export exists only for migration and inspection.

 ## Requirements
@@ -18,6 +18,7 @@ On the backup server or in the container:
 - rsync
 - ssh
 - SSH key-based access from the backup server to remotes
+- systemd for the recommended production deployment

 ## Local Development

@@ -109,8 +110,83 @@ python3 manage.py export_pobsync_configs --prefix /opt/pobsync

 These commands are migration helpers, not the normal operating model.

+## Production With Systemd
+
+The recommended production deployment is native systemd services on the backup server. This avoids Docker friction around
+SSH, filesystems, large backup mounts, and host-level service logs.
+
+Recommended layout:
+
+```
+/opt/pobsync/app          # git checkout
+/opt/pobsync/venv         # Python virtualenv
+/etc/pobsync/pobsync.env  # settings and secrets
+/var/lib/pobsync          # SQLite database, state, runtime SSH key files, static files
+/backups                  # backup storage, or set POBSYNC_BACKUP_ROOT to another absolute path
+```
+
+Install OS packages first:
+
+```
+apt install python3 python3-venv rsync openssh-client
+```
+
+Clone or update the app at `/opt/pobsync/app`, then run:
+
+```
+cd /opt/pobsync/app
+sudo scripts/install-systemd
+```
+
+The installer creates:
+
+- `pobsync-web.service` for Gunicorn on `127.0.0.1:8010`
+- `pobsync-worker.service` for queued backup runs
+- `pobsync-scheduler.service` for SQL-backed schedules
+- `/etc/pobsync/pobsync.env` if it does not exist
+
+Edit `/etc/pobsync/pobsync.env` before exposing the service:
+
+```
+POBSYNC_DJANGO_ALLOWED_HOSTS=backup.example.com,localhost,127.0.0.1
+POBSYNC_DJANGO_CSRF_TRUSTED_ORIGINS=https://backup.example.com
+POBSYNC_BACKUP_ROOT=/backups
+POBSYNC_WEB_BIND=127.0.0.1:8010
+```
+
+Restart after changes:
+
+```
+sudo systemctl restart pobsync-web pobsync-worker pobsync-scheduler
+```
+
+Check service state and logs:
+
+```
+systemctl status pobsync-web pobsync-worker pobsync-scheduler
+journalctl -u pobsync-worker -f
+```
+
+Update an existing native install:
+
+```
+cd /opt/pobsync/app
+git pull
+sudo scripts/install-systemd
+```
+
+Use an existing reverse proxy by forwarding to `http://127.0.0.1:8010`. To install a simple nginx site file as a
+starting point:
+
+```
+sudo scripts/install-systemd --with-nginx --server-name backup.example.com
+```
+
 ## Docker With SQLite

+Docker Compose is still useful for local development and disposable test installs. Native systemd is preferred for
+production backup servers.
+
 ```
 docker compose up --build web
 ```
@@ -140,55 +216,14 @@ POBSYNC_BACKUP_ROOT=/mnt/backups/pobsync docker compose up --build web scheduler
 The Django setup UI keeps the backup root fixed at `/backups`; only the Docker mount decides which host directory
 that points to.

-## Remote Server Deployment
-
-For a single backup server, use Docker Compose with the SQLite services and put a reverse proxy such as Caddy, nginx,
-or Traefik in front of `web`.
-
-Create a `.env` from the example:
-
-```
-cp .env.example .env
-```
-
-Set at least:
-
-```
-POBSYNC_BACKUP_ROOT=/mnt/backups/pobsync
-POBSYNC_DJANGO_ALLOWED_HOSTS=backup.example.com,localhost,127.0.0.1
-POBSYNC_DJANGO_SECRET_KEY=<long-random-secret>
-POBSYNC_DJANGO_DEBUG=0
-POBSYNC_WEB_BIND=127.0.0.1
-```
-
-Deploy or update:
-
-```
-git pull
-docker compose build web scheduler worker
-docker compose up -d --force-recreate web scheduler worker
-docker compose exec web python manage.py migrate
-```
-
-Check service state:
-
-```
-docker compose ps
-docker compose logs --tail=100 worker
-docker compose logs --tail=100 scheduler
-```
-
-`web`, `scheduler`, and `worker` use `restart: unless-stopped` and Docker healthchecks. If `POBSYNC_WEB_BIND` is
-`127.0.0.1`, expose the app through your reverse proxy instead of directly publishing it to the internet.
-
 ## Django-Managed SSH Keys

 SSH keys can be managed from the Django UI at `/ssh-credentials/`. Add a private key there, optionally paste
 `known_hosts` entries, and select the credential either as the global default or as a per-host override.

-When a backup starts, the worker writes the selected key to `/opt/pobsync/state/ssh-credentials/<id>/identity`
-inside the container with `0600` permissions and injects `IdentityFile` into the rsync SSH command. If `known_hosts`
-is configured, the worker also writes a matching `known_hosts` file and injects `UserKnownHostsFile`.
+When a backup starts, the worker writes the selected key to `$POBSYNC_HOME/state/ssh-credentials/<id>/identity`
+with `0600` permissions and injects `IdentityFile` into the rsync SSH command. If `known_hosts` is configured, the
+worker also writes a matching `known_hosts` file and injects `UserKnownHostsFile`.

 ## Docker With MariaDB

--- a/deploy/nginx/pobsync.conf
+++ b/deploy/nginx/pobsync.conf
@@ -0,0 +1,15 @@
+server {
+    listen 80;
+    server_name @POBSYNC_SERVER_NAME@;
+
+    client_max_body_size 16m;
+
+    location / {
+        proxy_pass http://127.0.0.1:8010;
+        proxy_http_version 1.1;
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+    }
+}
--- a/deploy/pobsync.env.example
+++ b/deploy/pobsync.env.example
@@ -0,0 +1,15 @@
+POBSYNC_DJANGO_DEBUG=0
+POBSYNC_DJANGO_SECRET_KEY=change-me-to-a-long-random-secret
+POBSYNC_DJANGO_ALLOWED_HOSTS=backup.example.com,localhost,127.0.0.1
+POBSYNC_DJANGO_CSRF_TRUSTED_ORIGINS=https://backup.example.com
+
+POBSYNC_HOME=/var/lib/pobsync
+POBSYNC_BACKUP_ROOT=/backups
+POBSYNC_SQLITE_PATH=/var/lib/pobsync/pobsync.sqlite3
+POBSYNC_STATIC_ROOT=/var/lib/pobsync/static
+
+POBSYNC_WEB_BIND=127.0.0.1:8010
+POBSYNC_GUNICORN_WORKERS=2
+POBSYNC_GUNICORN_TIMEOUT=120
+POBSYNC_WORKER_INTERVAL=15
+POBSYNC_SCHEDULER_INTERVAL=60
--- a/deploy/systemd/pobsync-scheduler.service
+++ b/deploy/systemd/pobsync-scheduler.service
@@ -0,0 +1,17 @@
+[Unit]
+Description=pobsync schedule dispatcher
+After=network-online.target pobsync-web.service
+Wants=network-online.target
+
+[Service]
+Type=simple
+User=@POBSYNC_USER@
+Group=@POBSYNC_GROUP@
+WorkingDirectory=@POBSYNC_APP_DIR@
+EnvironmentFile=@POBSYNC_ENV_FILE@
+ExecStart=/bin/sh -c 'exec @POBSYNC_VENV_DIR@/bin/python manage.py run_pobsync_scheduler --loop --interval "${POBSYNC_SCHEDULER_INTERVAL:-60}"'
+Restart=on-failure
+RestartSec=5
+
+[Install]
+WantedBy=multi-user.target
--- a/deploy/systemd/pobsync-web.service
+++ b/deploy/systemd/pobsync-web.service
@@ -0,0 +1,19 @@
+[Unit]
+Description=pobsync Django control panel
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+User=@POBSYNC_USER@
+Group=@POBSYNC_GROUP@
+WorkingDirectory=@POBSYNC_APP_DIR@
+EnvironmentFile=@POBSYNC_ENV_FILE@
+ExecStartPre=@POBSYNC_VENV_DIR@/bin/python manage.py migrate --noinput
+ExecStartPre=@POBSYNC_VENV_DIR@/bin/python manage.py collectstatic --noinput --clear
+ExecStart=/bin/sh -c 'exec @POBSYNC_VENV_DIR@/bin/gunicorn pobsync_server.wsgi:application --bind "${POBSYNC_WEB_BIND:-127.0.0.1:8010}" --workers "${POBSYNC_GUNICORN_WORKERS:-2}" --timeout "${POBSYNC_GUNICORN_TIMEOUT:-120}"'
+Restart=on-failure
+RestartSec=5
+
+[Install]
+WantedBy=multi-user.target
--- a/deploy/systemd/pobsync-worker.service
+++ b/deploy/systemd/pobsync-worker.service
@@ -0,0 +1,17 @@
+[Unit]
+Description=pobsync queued backup worker
+After=network-online.target pobsync-web.service
+Wants=network-online.target
+
+[Service]
+Type=simple
+User=@POBSYNC_USER@
+Group=@POBSYNC_GROUP@
+WorkingDirectory=@POBSYNC_APP_DIR@
+EnvironmentFile=@POBSYNC_ENV_FILE@
+ExecStart=/bin/sh -c 'exec @POBSYNC_VENV_DIR@/bin/python manage.py run_pobsync_worker --loop --interval "${POBSYNC_WORKER_INTERVAL:-15}"'
+Restart=on-failure
+RestartSec=5
+
+[Install]
+WantedBy=multi-user.target
--- a/scripts/install-systemd
+++ b/scripts/install-systemd
@@ -0,0 +1,123 @@
+#!/bin/sh
+set -eu
+
+APP_DIR=${POBSYNC_APP_DIR:-$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)}
+VENV_DIR=${POBSYNC_VENV_DIR:-/opt/pobsync/venv}
+ENV_FILE=${POBSYNC_ENV_FILE:-/etc/pobsync/pobsync.env}
+SERVICE_USER=${POBSYNC_SERVICE_USER:-pobsync}
+SERVICE_GROUP=${POBSYNC_SERVICE_GROUP:-pobsync}
+INSTALL_EXTRAS=${POBSYNC_INSTALL_EXTRAS:-}
+SERVER_NAME=${POBSYNC_SERVER_NAME:-_}
+WITH_NGINX=0
+
+while [ "$#" -gt 0 ]; do
+    case "$1" in
+        --with-nginx)
+            WITH_NGINX=1
+            shift
+            ;;
+        --server-name)
+            SERVER_NAME=$2
+            shift 2
+            ;;
+        *)
+            echo "Unknown argument: $1" >&2
+            exit 2
+            ;;
+    esac
+done
+
+if [ "$(id -u)" -ne 0 ]; then
+    echo "Run this installer as root." >&2
+    exit 1
+fi
+
+if ! command -v python3 >/dev/null 2>&1; then
+    echo "python3 is required." >&2
+    exit 1
+fi
+
+if ! command -v rsync >/dev/null 2>&1; then
+    echo "rsync is required." >&2
+    exit 1
+fi
+
+if ! command -v ssh >/dev/null 2>&1; then
+    echo "openssh-client is required." >&2
+    exit 1
+fi
+
+if ! getent group "$SERVICE_GROUP" >/dev/null 2>&1; then
+    groupadd --system "$SERVICE_GROUP"
+fi
+
+if ! id "$SERVICE_USER" >/dev/null 2>&1; then
+    useradd --system --home /var/lib/pobsync --shell /usr/sbin/nologin --gid "$SERVICE_GROUP" "$SERVICE_USER"
+fi
+
+mkdir -p /etc/pobsync /var/lib/pobsync /var/log/pobsync "$(dirname "$VENV_DIR")"
+chown "$SERVICE_USER:$SERVICE_GROUP" /var/lib/pobsync /var/log/pobsync
+chmod 0750 /var/lib/pobsync /var/log/pobsync
+
+python3 -m venv "$VENV_DIR"
+"$VENV_DIR/bin/python" -m pip install --upgrade pip
+"$VENV_DIR/bin/python" -m pip install -e "$APP_DIR$INSTALL_EXTRAS"
+
+if [ ! -f "$ENV_FILE" ]; then
+    secret=$("$VENV_DIR/bin/python" -c "import secrets; print(secrets.token_urlsafe(48))")
+    cat > "$ENV_FILE" <<EOF
+POBSYNC_DJANGO_DEBUG=0
+POBSYNC_DJANGO_SECRET_KEY=$secret
+POBSYNC_DJANGO_ALLOWED_HOSTS=localhost,127.0.0.1
+POBSYNC_DJANGO_CSRF_TRUSTED_ORIGINS=
+
+POBSYNC_HOME=/var/lib/pobsync
+POBSYNC_BACKUP_ROOT=/backups
+POBSYNC_SQLITE_PATH=/var/lib/pobsync/pobsync.sqlite3
+POBSYNC_STATIC_ROOT=/var/lib/pobsync/static
+
+POBSYNC_WEB_BIND=127.0.0.1:8010
+POBSYNC_GUNICORN_WORKERS=2
+POBSYNC_GUNICORN_TIMEOUT=120
+POBSYNC_WORKER_INTERVAL=15
+POBSYNC_SCHEDULER_INTERVAL=60
+EOF
+    chmod 0640 "$ENV_FILE"
+    chown "root:$SERVICE_GROUP" "$ENV_FILE"
+    echo "Created $ENV_FILE. Edit allowed hosts and backup root before exposing the service."
+fi
+
+install_unit() {
+    src=$1
+    dest=$2
+    sed \
+        -e "s|@POBSYNC_APP_DIR@|$APP_DIR|g" \
+        -e "s|@POBSYNC_VENV_DIR@|$VENV_DIR|g" \
+        -e "s|@POBSYNC_ENV_FILE@|$ENV_FILE|g" \
+        -e "s|@POBSYNC_USER@|$SERVICE_USER|g" \
+        -e "s|@POBSYNC_GROUP@|$SERVICE_GROUP|g" \
+        "$src" > "$dest"
+    chmod 0644 "$dest"
+}
+
+install_unit "$APP_DIR/deploy/systemd/pobsync-web.service" /etc/systemd/system/pobsync-web.service
+install_unit "$APP_DIR/deploy/systemd/pobsync-worker.service" /etc/systemd/system/pobsync-worker.service
+install_unit "$APP_DIR/deploy/systemd/pobsync-scheduler.service" /etc/systemd/system/pobsync-scheduler.service
+
+systemctl daemon-reload
+"$VENV_DIR/bin/python" "$APP_DIR/manage.py" migrate --noinput
+"$VENV_DIR/bin/python" "$APP_DIR/manage.py" collectstatic --noinput --clear
+systemctl enable --now pobsync-web.service pobsync-worker.service pobsync-scheduler.service
+
+if [ "$WITH_NGINX" -eq 1 ]; then
+    if ! command -v nginx >/dev/null 2>&1; then
+        echo "nginx is not installed; skipping nginx config." >&2
+    else
+        sed "s|@POBSYNC_SERVER_NAME@|$SERVER_NAME|g" "$APP_DIR/deploy/nginx/pobsync.conf" > /etc/nginx/sites-available/pobsync.conf
+        ln -sf /etc/nginx/sites-available/pobsync.conf /etc/nginx/sites-enabled/pobsync.conf
+        nginx -t
+        systemctl reload nginx
+    fi
+fi
+
+systemctl --no-pager --full status pobsync-web.service pobsync-worker.service pobsync-scheduler.service || true
--- a/src/pobsync_server/settings.py
+++ b/src/pobsync_server/settings.py
@@ -12,6 +12,9 @@ DEBUG = os.getenv("POBSYNC_DJANGO_DEBUG", "0").lower() in {"1", "true", "yes", "
 _allowed_hosts = os.getenv("POBSYNC_DJANGO_ALLOWED_HOSTS", "localhost,127.0.0.1")
 ALLOWED_HOSTS = [host.strip() for host in _allowed_hosts.split(",") if host.strip()]

+_csrf_trusted_origins = os.getenv("POBSYNC_DJANGO_CSRF_TRUSTED_ORIGINS", "")
+CSRF_TRUSTED_ORIGINS = [origin.strip() for origin in _csrf_trusted_origins.split(",") if origin.strip()]
+
 INSTALLED_APPS = [
    "django.contrib.admin",
    "django.contrib.auth",
@@ -95,4 +98,4 @@ STORAGES = {
 DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"

 POBSYNC_HOME = os.getenv("POBSYNC_HOME", "/opt/pobsync")
-POBSYNC_BACKUP_ROOT = "/backups"
+POBSYNC_BACKUP_ROOT = os.getenv("POBSYNC_BACKUP_ROOT", "/backups")