Cache certificate inventory and stop refreshPendingProxies from re-triggering a full refresh

README.md

@@ -12,7 +12,7 @@
     <img alt="Docker" src="https://img.shields.io/badge/Docker-ready-2496ED?logo=docker&logoColor=white">
     <img alt="Architectures" src="https://img.shields.io/badge/platform-amd64%20%7C%20arm64-5965F2">
     <img alt="Caddy" src="https://img.shields.io/badge/powered%20by-Caddy-1F88C0">
-    <img alt="Version" src="https://img.shields.io/badge/version-0.16.39-62E6A7">
+    <img alt="Version" src="https://img.shields.io/badge/version-0.16.41-62E6A7">
   </p>
   <p>
     <a href="#why-site-gateway">Why Site Gateway</a> ·

ROADMAP.md

@@ -205,3 +205,8 @@ Roughly in priority order:
 
 
 `v0.16.39` retires the Dashboard Uptime tile's separate 1-second client-side ticker, the same mechanism behind the v0.16.38 "resets to 0m on page load" bug, in favor of treating Uptime as just another field on the shared 7-second `/api/system/health` poll that already drives CPU, memory, swap, disk, and network on both the Dashboard and the Administration > System tab's hero panel. Revisiting the ticker after fixing its race condition, it turned out to be solving a precision problem the display doesn't actually have: `formatDuration()` only ever renders minute-level granularity ("2h 59m"), never seconds, so a per-second tick never changed what was on screen between one 7-second poll and the next. `systemHealthSnapshot()` now includes `uptimeSeconds` (from `process.uptime()`, the same source `dashboardSnapshot()` already used), and `renderHeroPanel()`'s former `includeThroughput` boolean became a `sixthSlot` option ("throughput" for the System tab, which has no other requests/min display, or "uptime" for the Dashboard, which already shows Throughput in its own chip) so the sixth hero slot can be either stat without a special case. The client ticker, its anchor state, and the whole race-condition class it enabled are gone: one poll, one code path, six stats, no anchor to get out of sync.
+
+
+`v0.16.40` moves the Administration → System tab's Version panel Uptime figure onto the same 7-second `/api/system/health` poll driving the hero panel above it, instead of only updating when the page's slower, general dashboard snapshot refetches (on load, or every ~30 seconds while the Dashboard view specifically is active). Found while reviewing the v0.16.39 change: the hero's own Uptime slot was now current to the second, but the separate Version-panel line right below it — the same number, shown twice on the same page — could still be stale by up to half a minute or more. `renderSystemStatus()`'s Uptime is now wrapped in its own `#system-version-uptime` span and updated by a small `updateSystemVersionUptime(health)` helper, called both from the initial render and from the same 7-second timer that already refreshes the hero, so the two Uptime figures on that page can no longer drift apart.
+
+`v0.16.41` cuts redundant work out of the app's shared `refresh()` cycle -- the single function that populates nearly every page (Hosted Sites, Proxy Hosts, Redirects, Streams, Access Lists, Dashboard, and Certificates all pull from it) -- after a user reported the whole site feeling slow to refresh, most concretely on a plain reload of Hosted Sites or Proxy Hosts. Two real causes, found by reading the actual request path rather than guessing: first, `certificateInventory()` (which walks the certificate directories and parses every `.crt`/`.pem` file on disk) was being fully recomputed from scratch on every single call, and `refresh()` calls it twice per cycle -- once via `/api/dashboard`, once via `/api/certificates` -- so a normal page load did that walk-and-parse work twice for identical results. It now carries a short (3 second) in-memory cache, well under the 7-second hero-poll interval, so back-to-back calls within a cycle share one real disk walk instead of two, and nothing on screen goes more than one cycle stale. Second, and the bigger one: `refreshPendingProxies()` -- triggered whenever a page loads with any enabled proxy that doesn't have cached upstream-health data yet, which is the common case right after a page load or a new proxy -- was calling the *entire* `refresh()` again at +1s, +2s, and +3s until every proxy's health came back. That meant a single pending proxy could quietly trigger three additional full 8-endpoint refetches (each with its own pair of certificate walks) in the six seconds after a page appeared to have finished loading. It now re-fetches only `/api/proxies` on those retries, since upstream health is all it was ever waiting on. Together these remove the two largest sources of duplicated, unnecessary work from the most-used code path in the app; whether they fully account for the reported slowness or whether a client-side rendering cost remains to find is still open and being evaluated against a real before/after comparison.

package.json

@@ -1,6 +1,6 @@
 {
   "name": "site-gateway",
-  "version": "0.16.39",
+  "version": "0.16.41",
   "private": true,
   "description": "Site Gateway: simple self-hosted website publishing, reverse proxying, and automatic HTTPS.",
   "type": "module",

src/public/app.js

@@ -533,12 +533,22 @@ function render() {
 
 // --- Data refresh helpers ------------------------------------------------------------------
 async function refresh() { const requests = [api("/api/sites"), api("/api/proxies"), api("/api/redirects"), api("/api/streams"), api("/api/access-lists"), canAdmin() ? api("/api/groups") : Promise.resolve([]), api("/api/dashboard"), api("/api/certificates")]; const results = await Promise.allSettled(requests); results.forEach((result, index) => { if (result.status !== "fulfilled") return; const keys = ["sites", "proxies", "redirects", "streams", "accessLists", "groups", "dashboard", "certificates"]; state[keys[index]] = result.value; }); state.loaded = true; render(); window.renderExtendedViews?.(); const pending = state.proxies.filter(proxy => proxy.enabled !== false && !proxy.upstream).map(proxy => proxy.id); if (pending.length && !state.pendingProxyRefresh) { state.pendingProxyRefresh = true; refreshPendingProxies(pending).finally(() => { state.pendingProxyRefresh = false; }); } }
+// Polls just /api/proxies for upstream health that wasn't ready yet on the last refresh() --
+// e.g. right after a page load or a new proxy, before its first health check has completed.
+// This used to call the full refresh() (all 8 endpoints, including two redundant certificate
+// walks via /api/dashboard + /api/certificates), up to 3 times in a row -- meaning a single
+// pending proxy could quietly trigger 3 extra full-app refetches over 6 seconds. Since all it
+// actually needs is fresh upstream status, it now re-fetches only /api/proxies.
 async function refreshPendingProxies(ids = []) {
   const pending = new Set(ids.map(String));
   for (const delay of [1000, 2000, 3000]) {
     if (!pending.size) return;
     await new Promise(resolve => setTimeout(resolve, delay));
-    await refresh();
+    try {
+      state.proxies = await api("/api/proxies");
+      render();
+      window.renderExtendedViews?.();
+    } catch { /* Keep the last-known proxy list if this poll fails; the next delay tries again. */ }
     for (const proxy of state.proxies) if (pending.has(String(proxy.id)) && proxy.upstream) pending.delete(String(proxy.id));
   }
 }

src/public/features.js

@@ -691,11 +691,19 @@ function renderSystemPanel() {
     if (!state.systemHealthTimer) state.systemHealthTimer = setInterval(() => {
       const systemPanel = document.querySelector('[data-admin-panel="system"]');
       if (state.view !== "administration" || !systemPanel || systemPanel.classList.contains("hidden")) return;
-      api("/api/system/health").then(health => renderHeroPanel("system-hero", health)).catch(() => {});
+      api("/api/system/health").then(health => { renderHeroPanel("system-hero", health); updateSystemVersionUptime(health); }).catch(() => {});
     }, 7000);
   }
   renderSystemStatus(panel);
 }
+// Updates just the Version panel's Uptime figure from a fresh /api/system/health payload --
+// kept separate from the rest of renderSystemStatus() so it can be called on the fast 7s hero
+// poll without re-rendering (or re-fetching) everything else in that panel.
+function updateSystemVersionUptime(health) {
+  const el = document.querySelector("#system-version-uptime");
+  if (!el) return;
+  el.textContent = Number.isFinite(health?.uptimeSeconds) ? formatDuration(health.uptimeSeconds) : "Unavailable";
+}
 async function renderSystemStatus(panel) {
   panel = panel || document.querySelector('[data-admin-panel="system"]');
   if (!panel) return;
@@ -715,16 +723,19 @@ async function renderSystemStatus(panel) {
   }
   if (syncStatus) { const drift = (state.dashboard?.attention || []).some(item => item.kind === "drift"); syncStatus.textContent = drift ? "Configuration drift detected \u2014 the running gateway no longer matches the last known-good configuration." : `Gateway configuration is in sync. Last reload: ${state.dashboard?.gateway?.lastReload ? formatTime(state.dashboard.gateway.lastReload) : "unknown"}.`; syncStatus.className = drift ? "muted status-warning" : "muted"; }
   if (version) {
-    // Uptime, Caddy version, Database status, and Public IP used to live on the Dashboard's
-    // Runtime/System panel -- that panel is now the shared hero component (CPU/memory/swap/disk/
-    // network), so this operational metadata moved here instead, reusing the same system.* fields
-    // from the global dashboard snapshot rather than a separate fetch.
+    // Caddy version, Database status, and Public IP used to live on the Dashboard's Runtime/System
+    // panel -- that panel is now the shared hero component (CPU/memory/swap/disk/network), so this
+    // operational metadata moved here instead, reusing the same system.* fields from the global
+    // dashboard snapshot rather than a separate fetch. Uptime is the one exception: it's wrapped in
+    // its own #system-version-uptime span and kept current by updateSystemVersionUptime(), called
+    // from the same 7-second /api/system/health poll that drives the hero panel above, instead of
+    // only refreshing on the slower ~30s dashboard snapshot like the rest of this block.
     const sys = state.dashboard?.system || {};
     const uptime = Number.isFinite(sys.uptimeSeconds) ? formatDuration(sys.uptimeSeconds) : "Unavailable";
     const database = sys.databaseEngine ? `${extendedEscape(sys.databaseEngine)} \u00b7 ${extendedEscape(sys.databaseStatus || "unknown")} \u00b7 ${formatBytes(sys.databaseBytes)}` : "Unavailable";
     const publicIp = sys.publicIp || (sys.publicIpError ? "Unavailable" : "Checking\u2026");
     const publicIpDetail = sys.publicIpError ? `check failed \u00b7 ${extendedEscape(sys.publicIpError)}` : sys.publicIpCheckedAt ? `checked ${extendedEscape(formatTime(sys.publicIpCheckedAt))}` : "not yet checked";
-    version.innerHTML = `Site Gateway v${extendedEscape(state.config?.version || "unknown")} \u00b7 Caddy ${extendedEscape(sys.caddyVersion || "unknown")}<br>Uptime: ${uptime} \u00b7 Database: ${database} \u00b7 Public IP: ${extendedEscape(publicIp)} (${publicIpDetail})<br>Access this dashboard at: <code>${extendedEscape(location.origin)}</code><br>Data directory: <code>${extendedEscape(state.config?.storage?.databasePath ? state.config.storage.databasePath.replace(/\/database\/.*/, "") : "/data")}</code> &middot; Site ports: <code>${extendedEscape(String(state.config?.minPort ?? ""))}\u2013${extendedEscape(String(state.config?.maxPort ?? ""))}</code>`;
+    version.innerHTML = `Site Gateway v${extendedEscape(state.config?.version || "unknown")} \u00b7 Caddy ${extendedEscape(sys.caddyVersion || "unknown")}<br>Uptime: <span id="system-version-uptime">${uptime}</span> \u00b7 Database: ${database} \u00b7 Public IP: ${extendedEscape(publicIp)} (${publicIpDetail})<br>Access this dashboard at: <code>${extendedEscape(location.origin)}</code><br>Data directory: <code>${extendedEscape(state.config?.storage?.databasePath ? state.config.storage.databasePath.replace(/\/database\/.*/, "") : "/data")}</code> &middot; Site ports: <code>${extendedEscape(String(state.config?.minPort ?? ""))}\u2013${extendedEscape(String(state.config?.maxPort ?? ""))}</code>`;
   }
   try {
     const [sec, store, policy, health] = await Promise.all([
@@ -734,6 +745,7 @@ async function renderSystemStatus(panel) {
       api("/api/system/health").catch(() => null),
     ]);
     renderHeroPanel("system-hero", health);
+    if (health) updateSystemVersionUptime(health);
     if (security) security.innerHTML = [
       { ok: !sec.adminPasswordIsDefault, label: "ADMIN_PASSWORD", detail: sec.adminPasswordIsDefault ? "Still using the built-in default \u2014 set this before exposing the dashboard." : "Configured." },
       { ok: !sec.sessionSecretIsDefault, label: "SESSION_SECRET", detail: sec.sessionSecretIsDefault ? "Not set \u2014 sessions are keyed off the admin credentials instead of an independent secret." : "Configured." },

src/server.js

@@ -64,6 +64,14 @@ let caddyVersion = "Unknown";
 const recentActivity = [];
 const upstreamHealth = new Map();
 const certificateStatusCache = new Map();
+// Certificate inventory is expensive (it walks and parses every certificate file on disk) and is
+// recomputed on every call with no memoization. It's called twice per client refresh() cycle --
+// once from /api/dashboard, once from /api/certificates -- and that whole cycle can itself repeat
+// several times in a row (see refreshPendingProxies in app.js), so a short time-based cache here
+// collapses that duplicate work into a single real disk walk every few seconds. The window is kept
+// well under the 7s hero-poll interval so nothing ever appears more than one cycle stale.
+const CERTIFICATE_INVENTORY_CACHE_MS = 3000;
+let certificateInventoryCache = null; // { at: number, value: object }
 const loginAttempts = new Map();
 const rateLimitBuckets = new Map();
 let dockerSocketMounted = false;
@@ -882,6 +890,7 @@ function certificateNames(certificate) {
 }
 
 async function certificateInventory() {
+  if (certificateInventoryCache && Date.now() - certificateInventoryCache.at < CERTIFICATE_INVENTORY_CACHE_MS) return certificateInventoryCache.value;
   const configured = [...sites.map(item => ({ ...item, kind: "Hosted site" })), ...proxies.map(item => ({ ...item, kind: "Proxy host" })), ...redirects.map(item => ({ ...item, kind: "Redirect host" }))]
     .filter(item => item.enabled && item.domain && item.tls !== "http");
   const configuredDomains = configured.flatMap(item => normalizeDomains(item.domain, item.domains).map(domain => ({ ...item, domain })));
@@ -908,7 +917,9 @@ async function certificateInventory() {
   });
   for (const certificate of certificates) { const previous = certificateStatusCache.get(certificate.domain); if (previous && previous !== certificate.status) recordActivity(`Certificate status changed for ${certificate.domain}: ${previous} → ${certificate.status}.`, certificate.status === "healthy" ? "ok" : "error"); certificateStatusCache.set(certificate.domain, certificate.status); }
   const latestError = recentActivity.find(item => item.status === "error" && /cert|tls|acme|caddy|gateway/i.test(item.message)) || null;
-  return { checkedAt: new Date().toISOString(), thresholds: settings.certificateHealth, latestError, summary: { total: certificates.length, healthy: certificates.filter(item => item.status === "healthy").length, within30Days: certificates.filter(item => item.daysRemaining != null && item.daysRemaining <= 30 && item.daysRemaining > 0).length, within7Days: certificates.filter(item => item.daysRemaining != null && item.daysRemaining <= 7 && item.daysRemaining > 0).length, warning: certificates.filter(item => item.status === "warning").length, critical: certificates.filter(item => item.status === "critical").length, expired: certificates.filter(item => item.status === "expired").length, pending: certificates.filter(item => item.status === "pending").length, mismatch: certificates.filter(item => item.status === "mismatch").length }, certificates };
+  const result = { checkedAt: new Date().toISOString(), thresholds: settings.certificateHealth, latestError, summary: { total: certificates.length, healthy: certificates.filter(item => item.status === "healthy").length, within30Days: certificates.filter(item => item.daysRemaining != null && item.daysRemaining <= 30 && item.daysRemaining > 0).length, within7Days: certificates.filter(item => item.daysRemaining != null && item.daysRemaining <= 7 && item.daysRemaining > 0).length, warning: certificates.filter(item => item.status === "warning").length, critical: certificates.filter(item => item.status === "critical").length, expired: certificates.filter(item => item.status === "expired").length, pending: certificates.filter(item => item.status === "pending").length, mismatch: certificates.filter(item => item.status === "mismatch").length }, certificates };
+  certificateInventoryCache = { at: Date.now(), value: result };
+  return result;
 }
 
 async function pruneOrphanedCertificates(candidateDomains) {