Proxmox Backup & Retention Policy
River Oaks Church - Created April 2026
Overview
Three-tier backup strategy: local fast-access, NAS mid-term, and cloud offsite. Critical services (webserver, Bitwarden) get more frequent local backups.
On October 1, 2026, a read-only pvesh get /cluster/backup query on rocc01
confirmed that all three Proxmox jobs below were enabled with the documented
schedules, storage targets, and guest selection. This verifies configuration,
not that recent backups succeeded, that gofer2 is healthy, or that any
backup can be restored. The LXC 109 offsite cron and Drive contents were not
part of this check.
Tier 1 — Local (Proxmox Host)
Nightly Backups
- Storage:
local(/var/lib/vz) - Schedule: Monday–Friday at 21:00
- Retention: Keep last 3 backups
- Selection: All VMs/CTs except 109 (storage-mgr), 110 (Bitwarden), 100 (webserver)
- Purpose: Fast restores for recent issues
Critical Services (High Frequency)
- Storage:
local(/var/lib/vz) - Schedule: Every 6 hours (*/6:00)
- Retention: Keep last 5 backups
- Selection: 100 (webserver), 110 (Bitwarden)
- Purpose: Minimize data loss for critical services
Tier 2 — NAS (SMB/CIFS)
- Storage:
gofer2(/mnt/pve/gofer2) - Schedule: Daily at 22:30
- Retention: Keep last 5 backups
- Selection: All VMs/CTs
- Purpose: Mid-term local backup with more history
Tier 3 — Offsite (Google Team Drive)
- Storage:
td-backups:ProxmoxBackupsvia rclone - Schedule: Daily at midnight via cron on
storage-mgr(LXC 109) - Retention: Weekly snapshots, kept for 13 weeks (~3 months)
- Mechanism: Script
/root/proxmox-backup-sync.shon LXC 109 - Copies from
/mnt/proxmox-backups(bind mount of host's/var/lib/vz) - Organizes by ISO week folder (e.g.,
2026-W16/) - Runs an idempotent copy every night, even when the current week already contains files
- Verifies existing objects and uploads only missing or changed files, so quota-interrupted weeks resume on the next run
- Stops promptly when Google Drive reports its upload cap instead of retrying for hours
- Uses a nonblocking lock to prevent overlapping syncs
- Deletes remote folders older than 13 weeks only after the weekly copy succeeds
--min-age 60mprevents uploading in-progress backups- Cron:
0 0 * * * /root/proxmox-backup-sync.sh >> /var/log/proxmox-backup-sync.log 2>&1
Architecture
Proxmox vzdump (*/6:00) ──► local (keep 5) ── 100 (webserver), 110 (Bitwarden)
Proxmox vzdump (mon-fri 21) ──► local (keep 3) ── everything else (except 109)
Proxmox vzdump (22:30) ──► gofer2/NAS (keep 5) ── all VMs/CTs
LXC 109 cron (00:00) ──► rclone ──► Google Team Drive (weekly, keep 13 weeks)
Notes
- LXC 109 (storage-mgr) is excluded from all local backup jobs to avoid recursion
- The Team Drive remote is configured as
td-backups:in rclone on LXC 109 - Local backups are bind-mounted read-only into LXC 109 at
/mnt/proxmox-backups - Config:
mp1: /var/lib/vz/dump,mp=/mnt/proxmox-backups,ro=1in/etc/pve/lxc/109.conf - First weekly upload is ~230-350 GB depending on local retention at time of sync
- Monitor sync logs at
/var/log/proxmox-backup-sync.logon LXC 109
Verification and recovery
Check the scheduled backup jobs and storage status on the Proxmox host:
pvesh get /cluster/backup
pvesm status
Check the offsite schedule and recent sync output in LXC 109:
pct exec 109 -- crontab -l
pct exec 109 -- tail -n 200 /var/log/proxmox-backup-sync.log
Preview a sync without uploading or deleting anything:
pct exec 109 -- /root/proxmox-backup-sync.sh --dry-run
An exit code of 7 with userRateLimitExceeded means Google Drive's upload cap was reached. The upload remains incomplete but recoverable; the next nightly run compares the same ISO-week folder and resumes missing objects. Do not create an empty marker or skip the existing week. Confirm completion by comparing the local eligible-file inventory with the remote week and by checking any specifically reported missing object.
The deployed script is maintained in the riveroaks/infra repository at storage/hosts/storage-mgr/files/usr/local/sbin/riveroaks-proxmox-backup-sync. Update the repository first, pass CI, then deploy the reviewed copy to LXC 109. Do not place rclone credentials or backup contents in Git, logs, tickets, or documentation.