Skip to content

User Guide

Archival & Retention

The server runs an automatic cleanup every hour. When archival is enabled, expired logs are compressed to gzipped CSV files organized by month before being removed from the database. Audit events are never archived.

How retention works

A cleanup job runs every hour. It checks the Keep logs for setting and removes any log entries older than that threshold.

  • Range: 7 to 90 days
  • Default: 30 days
  • Changes save instantly via AJAX — no restart required
  • Changes take effect at the next hourly cleanup

The retention range is enforced server-side; values outside 7–90 are clamped automatically.

Retention preview

When you change the retention value in Settings → Database, the UI calls a preview endpoint that tells you how many logs would be deleted before you save. This lets you avoid surprises like "I dropped retention to 7 days and suddenly lost 80% of my history."

Archival

When Archive before deleting is enabled (default), the hourly job compresses expired logs to monthly gzipped CSV files before removing them from the database.

Field Content
id Original database row ID
timestamp When the message was received
host Source IP or hostname
severity Parsed severity
facility Parsed facility
message The raw syslog message

The id field is preserved so re-importing an archive is straightforward and idempotent.

Filenames

Archives are named by month:

2026-01.csv.gz
2026-02.csv.gz
2026-03.csv.gz

Multiple cleanup runs in the same month append to the same file — they don't overwrite. Already-archived rows are detected by ID and skipped.

Audit events are never archived

Audit entries (logins, settings changes, exports, user actions) are stored in the same table as syslog data, but they're excluded from archival. They remain in the live database indefinitely so the audit trail is always complete.

This means a server that's been running for years will have a small amount of audit data in syslog.db even when syslog data has been aggressively archived.

Where archives live

C:\Program Files\LuceDev Syslog\archives\

/var/lib/lucedev-syslog/archives/

These directories are preserved across upgrades and reinstalls.

Managing archives

The Database tab in Settings lists every archive file with:

  • Filename
  • Month it covers
  • Number of log entries
  • Size on disk
  • Modified timestamp

From here you can:

  • Download — fetch the gzipped CSV
  • Delete — remove it permanently from disk
  • Run archive now — trigger the archival job manually (also runs hourly)

Every download and delete is recorded in the audit trail.

Disk space planning

Logs per day 30-day DB size Monthly archive (compressed)
100,000 ~50 MB ~3 MB
500,000 ~250 MB ~15 MB
1,000,000 ~500 MB ~30 MB
2,000,000 ~1 GB ~60 MB

Estimates assume an average message length of ~120 bytes. Compressed archives typically run 5–7% of the original size.

SSD strongly recommended above 500K/day

The dashboard queries SQLite directly. On spinning disks, the database becomes the bottleneck around 500K logs/day. SSDs eliminate this completely.

After archival

When the cleanup job finishes, three things happen automatically:

  1. Stats cache rebuild — the cached counters used by the dashboard (total log count, per-host counts, per-severity counts) are rebuilt to match the new state.
  2. FTS index rebuild — the full-text search index is rebuilt to reclaim space and stay in sync.
  3. PRAGMA optimize — SQLite is told to update its query planner statistics.

For long-running deployments, run a manual VACUUM from Settings → Database → Maintenance once a month or so to reclaim disk space that SQLite holds onto after large deletes.