Skip to content

Proxmox Backup & Retention Policy

River Oaks Church - Created April 2026

Overview

Three-tier backup strategy: local fast-access, NAS mid-term, and cloud offsite. Critical services (webserver, Bitwarden) get more frequent local backups.

On October 1, 2026, a read-only pvesh get /cluster/backup query on rocc01 confirmed that all three Proxmox jobs below were enabled with the documented schedules, storage targets, and guest selection. This verifies configuration, not that recent backups succeeded, that gofer2 is healthy, or that any backup can be restored. The LXC 109 offsite cron and Drive contents were not part of this check.

Tier 1 — Local (Proxmox Host)

Nightly Backups

  • Storage: local (/var/lib/vz)
  • Schedule: Monday–Friday at 21:00
  • Retention: Keep last 3 backups
  • Selection: All VMs/CTs except 109 (storage-mgr), 110 (Bitwarden), 100 (webserver)
  • Purpose: Fast restores for recent issues

Critical Services (High Frequency)

  • Storage: local (/var/lib/vz)
  • Schedule: Every 6 hours (*/6:00)
  • Retention: Keep last 5 backups
  • Selection: 100 (webserver), 110 (Bitwarden)
  • Purpose: Minimize data loss for critical services

Tier 2 — NAS (SMB/CIFS)

  • Storage: gofer2 (/mnt/pve/gofer2)
  • Schedule: Daily at 22:30
  • Retention: Keep last 5 backups
  • Selection: All VMs/CTs
  • Purpose: Mid-term local backup with more history

Tier 3 — Offsite (Google Team Drive)

  • Storage: td-backups:ProxmoxBackups via rclone
  • Schedule: Daily at midnight via cron on storage-mgr (LXC 109)
  • Retention: Weekly snapshots, kept for 13 weeks (~3 months)
  • Mechanism: Script /root/proxmox-backup-sync.sh on LXC 109
  • Copies from /mnt/proxmox-backups (bind mount of host's /var/lib/vz)
  • Organizes by ISO week folder (e.g., 2026-W16/)
  • Runs an idempotent copy every night, even when the current week already contains files
  • Verifies existing objects and uploads only missing or changed files, so quota-interrupted weeks resume on the next run
  • Stops promptly when Google Drive reports its upload cap instead of retrying for hours
  • Uses a nonblocking lock to prevent overlapping syncs
  • Deletes remote folders older than 13 weeks only after the weekly copy succeeds
  • --min-age 60m prevents uploading in-progress backups
  • Cron: 0 0 * * * /root/proxmox-backup-sync.sh >> /var/log/proxmox-backup-sync.log 2>&1

Architecture

Proxmox vzdump (*/6:00)     ──► local (keep 5) ── 100 (webserver), 110 (Bitwarden)
Proxmox vzdump (mon-fri 21) ──► local (keep 3) ── everything else (except 109)
Proxmox vzdump (22:30)      ──► gofer2/NAS (keep 5) ── all VMs/CTs
LXC 109 cron   (00:00)      ──► rclone ──► Google Team Drive (weekly, keep 13 weeks)

Notes

  • LXC 109 (storage-mgr) is excluded from all local backup jobs to avoid recursion
  • The Team Drive remote is configured as td-backups: in rclone on LXC 109
  • Local backups are bind-mounted read-only into LXC 109 at /mnt/proxmox-backups
  • Config: mp1: /var/lib/vz/dump,mp=/mnt/proxmox-backups,ro=1 in /etc/pve/lxc/109.conf
  • First weekly upload is ~230-350 GB depending on local retention at time of sync
  • Monitor sync logs at /var/log/proxmox-backup-sync.log on LXC 109

Verification and recovery

Check the scheduled backup jobs and storage status on the Proxmox host:

pvesh get /cluster/backup
pvesm status

Check the offsite schedule and recent sync output in LXC 109:

pct exec 109 -- crontab -l
pct exec 109 -- tail -n 200 /var/log/proxmox-backup-sync.log

Preview a sync without uploading or deleting anything:

pct exec 109 -- /root/proxmox-backup-sync.sh --dry-run

An exit code of 7 with userRateLimitExceeded means Google Drive's upload cap was reached. The upload remains incomplete but recoverable; the next nightly run compares the same ISO-week folder and resumes missing objects. Do not create an empty marker or skip the existing week. Confirm completion by comparing the local eligible-file inventory with the remote week and by checking any specifically reported missing object.

The deployed script is maintained in the riveroaks/infra repository at storage/hosts/storage-mgr/files/usr/local/sbin/riveroaks-proxmox-backup-sync. Update the repository first, pass CI, then deploy the reviewed copy to LXC 109. Do not place rclone credentials or backup contents in Git, logs, tickets, or documentation.