Stop the retention-preview timer from polling every page forever and index audit_events

This commit is contained in:
marvin
2026-09-19 22:42:35 -04:00
parent 578d39a3ad
commit 926296d0d7
6 changed files with 21 additions and 6 deletions
+2
View File
@@ -210,3 +210,5 @@ Roughly in priority order:
`v0.16.40` moves the Administration → System tab's Version panel Uptime figure onto the same 7-second `/api/system/health` poll driving the hero panel above it, instead of only updating when the page's slower, general dashboard snapshot refetches (on load, or every ~30 seconds while the Dashboard view specifically is active). Found while reviewing the v0.16.39 change: the hero's own Uptime slot was now current to the second, but the separate Version-panel line right below it — the same number, shown twice on the same page — could still be stale by up to half a minute or more. `renderSystemStatus()`'s Uptime is now wrapped in its own `#system-version-uptime` span and updated by a small `updateSystemVersionUptime(health)` helper, called both from the initial render and from the same 7-second timer that already refreshes the hero, so the two Uptime figures on that page can no longer drift apart.
`v0.16.41` cuts redundant work out of the app's shared `refresh()` cycle -- the single function that populates nearly every page (Hosted Sites, Proxy Hosts, Redirects, Streams, Access Lists, Dashboard, and Certificates all pull from it) -- after a user reported the whole site feeling slow to refresh, most concretely on a plain reload of Hosted Sites or Proxy Hosts. Two real causes, found by reading the actual request path rather than guessing: first, `certificateInventory()` (which walks the certificate directories and parses every `.crt`/`.pem` file on disk) was being fully recomputed from scratch on every single call, and `refresh()` calls it twice per cycle -- once via `/api/dashboard`, once via `/api/certificates` -- so a normal page load did that walk-and-parse work twice for identical results. It now carries a short (3 second) in-memory cache, well under the 7-second hero-poll interval, so back-to-back calls within a cycle share one real disk walk instead of two, and nothing on screen goes more than one cycle stale. Second, and the bigger one: `refreshPendingProxies()` -- triggered whenever a page loads with any enabled proxy that doesn't have cached upstream-health data yet, which is the common case right after a page load or a new proxy -- was calling the *entire* `refresh()` again at +1s, +2s, and +3s until every proxy's health came back. That meant a single pending proxy could quietly trigger three additional full 8-endpoint refetches (each with its own pair of certificate walks) in the six seconds after a page appeared to have finished loading. It now re-fetches only `/api/proxies` on those retries, since upstream health is all it was ever waiting on. Together these remove the two largest sources of duplicated, unnecessary work from the most-used code path in the app; whether they fully account for the reported slowness or whether a client-side rendering cost remains to find is still open and being evaluated against a real before/after comparison.
`v0.16.42` finds and fixes the real, dominant cause of the site-wide slowness reported after v0.16.41: a live Network-tab capture from the user's own browser showed a flood of requests to `/api/logs/prune/preview`, some queued for over 15 seconds, with unrelated requests (`/api/dashboard`, `/api/system/health`, `/api/system/security`, `/api/system/storage`) stuck at nearly identical multi-second times in the same batch -- the signature of one blocking operation stalling everything behind it, not several independently slow endpoints. Root cause: `renderRetentionPreview()`'s `setInterval(..., 2000)` polls that endpoint every 2 seconds forever, on every page of the app, not just Administration -> Logs & retention, because its "does the panel exist" guard checks a `<section>` that's written into `index.html` from page load and only ever CSS-hidden -- so the guard was always true, everywhere. There was also no protection against a new poll firing while a previous one was still in flight, so once the server answered slower than 2 seconds even once, requests piled up and never caught back up. Compounding it: `previewPruneEvents()` runs five synchronous SQLite COUNT queries, and one of them (`audit_events`) had no index at all -- a full table scan, every call -- and because this app's SQLite queries run synchronously, that scan doesn't just slow its own request, it blocks the entire Node process for every other request being served at that moment. Fixed on both sides: `renderRetentionPreview()` and the sibling `renderRetentionRunStatus()` (previously also running unconditionally every 500ms) now check that the retention panel is actually visible, not just present in the DOM, before doing any work, and an in-flight guard stops a new preview poll from starting until the last one has landed; `audit_events` now has the same `(instance_id, created_at)` index every sibling events table already had. Together these should remove the vast majority of the "8-10 seconds to load a simple page" behavior reported after v0.16.41 -- that fix (deduplicating certificate-inventory work) was real but minor by comparison to this one.