Cache certificate inventory and stop refreshPendingProxies from re-triggering a full refresh

README.md

@@ -12,7 +12,7 @@
     <img alt="Docker" src="https://img.shields.io/badge/Docker-ready-2496ED?logo=docker&logoColor=white">
     <img alt="Architectures" src="https://img.shields.io/badge/platform-amd64%20%7C%20arm64-5965F2">
     <img alt="Caddy" src="https://img.shields.io/badge/powered%20by-Caddy-1F88C0">
-    <img alt="Version" src="https://img.shields.io/badge/version-0.16.40-62E6A7">
+    <img alt="Version" src="https://img.shields.io/badge/version-0.16.41-62E6A7">
   </p>
   <p>
     <a href="#why-site-gateway">Why Site Gateway</a> ·

ROADMAP.md

@@ -208,3 +208,5 @@ Roughly in priority order:
 
 
 `v0.16.40` moves the Administration → System tab's Version panel Uptime figure onto the same 7-second `/api/system/health` poll driving the hero panel above it, instead of only updating when the page's slower, general dashboard snapshot refetches (on load, or every ~30 seconds while the Dashboard view specifically is active). Found while reviewing the v0.16.39 change: the hero's own Uptime slot was now current to the second, but the separate Version-panel line right below it — the same number, shown twice on the same page — could still be stale by up to half a minute or more. `renderSystemStatus()`'s Uptime is now wrapped in its own `#system-version-uptime` span and updated by a small `updateSystemVersionUptime(health)` helper, called both from the initial render and from the same 7-second timer that already refreshes the hero, so the two Uptime figures on that page can no longer drift apart.
+
+`v0.16.41` cuts redundant work out of the app's shared `refresh()` cycle -- the single function that populates nearly every page (Hosted Sites, Proxy Hosts, Redirects, Streams, Access Lists, Dashboard, and Certificates all pull from it) -- after a user reported the whole site feeling slow to refresh, most concretely on a plain reload of Hosted Sites or Proxy Hosts. Two real causes, found by reading the actual request path rather than guessing: first, `certificateInventory()` (which walks the certificate directories and parses every `.crt`/`.pem` file on disk) was being fully recomputed from scratch on every single call, and `refresh()` calls it twice per cycle -- once via `/api/dashboard`, once via `/api/certificates` -- so a normal page load did that walk-and-parse work twice for identical results. It now carries a short (3 second) in-memory cache, well under the 7-second hero-poll interval, so back-to-back calls within a cycle share one real disk walk instead of two, and nothing on screen goes more than one cycle stale. Second, and the bigger one: `refreshPendingProxies()` -- triggered whenever a page loads with any enabled proxy that doesn't have cached upstream-health data yet, which is the common case right after a page load or a new proxy -- was calling the *entire* `refresh()` again at +1s, +2s, and +3s until every proxy's health came back. That meant a single pending proxy could quietly trigger three additional full 8-endpoint refetches (each with its own pair of certificate walks) in the six seconds after a page appeared to have finished loading. It now re-fetches only `/api/proxies` on those retries, since upstream health is all it was ever waiting on. Together these remove the two largest sources of duplicated, unnecessary work from the most-used code path in the app; whether they fully account for the reported slowness or whether a client-side rendering cost remains to find is still open and being evaluated against a real before/after comparison.

package.json

@@ -1,6 +1,6 @@
 {
   "name": "site-gateway",
-  "version": "0.16.40",
+  "version": "0.16.41",
   "private": true,
   "description": "Site Gateway: simple self-hosted website publishing, reverse proxying, and automatic HTTPS.",
   "type": "module",

src/public/app.js

@@ -533,12 +533,22 @@ function render() {
 
 // --- Data refresh helpers ------------------------------------------------------------------
 async function refresh() { const requests = [api("/api/sites"), api("/api/proxies"), api("/api/redirects"), api("/api/streams"), api("/api/access-lists"), canAdmin() ? api("/api/groups") : Promise.resolve([]), api("/api/dashboard"), api("/api/certificates")]; const results = await Promise.allSettled(requests); results.forEach((result, index) => { if (result.status !== "fulfilled") return; const keys = ["sites", "proxies", "redirects", "streams", "accessLists", "groups", "dashboard", "certificates"]; state[keys[index]] = result.value; }); state.loaded = true; render(); window.renderExtendedViews?.(); const pending = state.proxies.filter(proxy => proxy.enabled !== false && !proxy.upstream).map(proxy => proxy.id); if (pending.length && !state.pendingProxyRefresh) { state.pendingProxyRefresh = true; refreshPendingProxies(pending).finally(() => { state.pendingProxyRefresh = false; }); } }
+// Polls just /api/proxies for upstream health that wasn't ready yet on the last refresh() --
+// e.g. right after a page load or a new proxy, before its first health check has completed.
+// This used to call the full refresh() (all 8 endpoints, including two redundant certificate
+// walks via /api/dashboard + /api/certificates), up to 3 times in a row -- meaning a single
+// pending proxy could quietly trigger 3 extra full-app refetches over 6 seconds. Since all it
+// actually needs is fresh upstream status, it now re-fetches only /api/proxies.
 async function refreshPendingProxies(ids = []) {
   const pending = new Set(ids.map(String));
   for (const delay of [1000, 2000, 3000]) {
     if (!pending.size) return;
     await new Promise(resolve => setTimeout(resolve, delay));
-    await refresh();
+    try {
+      state.proxies = await api("/api/proxies");
+      render();
+      window.renderExtendedViews?.();
+    } catch { /* Keep the last-known proxy list if this poll fails; the next delay tries again. */ }
     for (const proxy of state.proxies) if (pending.has(String(proxy.id)) && proxy.upstream) pending.delete(String(proxy.id));
   }
 }

src/public/index.html

@@ -8,7 +8,7 @@
   <title>Site Gateway</title>
   <meta name="description" content="Host sites, proxy services, and manage HTTPS from one simple dashboard.">
   <link rel="icon" type="image/png" href="/site-gateway-icon-approved.png">
-  <link rel="stylesheet" href="/styles.css?v=0.16.40">
+  <link rel="stylesheet" href="/styles.css?v=0.16.41">
 </head>
 
 <!-- ================================================================
@@ -432,6 +432,6 @@
   <div id="toast" class="toast" role="status"></div>
   <div id="update-banner" class="update-banner hidden" role="status"><span>A new version of Site Gateway is available.</span><div class="update-banner-actions"><button id="update-banner-refresh" class="button primary">Refresh</button><button id="update-banner-dismiss" class="text-button">Dismiss</button></div></div>
   <!-- App scripts: core (app.js) then extended views/admin (features.js) -->
-  <script src="/app.js?v=0.16.40" defer></script><script src="/features.js?v=0.16.40" defer></script><script src="/select-enhance.js?v=0.16.40" defer></script>
+  <script src="/app.js?v=0.16.41" defer></script><script src="/features.js?v=0.16.41" defer></script><script src="/select-enhance.js?v=0.16.41" defer></script>
 </body>
 </html>

src/server.js

@@ -64,6 +64,14 @@ let caddyVersion = "Unknown";
 const recentActivity = [];
 const upstreamHealth = new Map();
 const certificateStatusCache = new Map();
+// Certificate inventory is expensive (it walks and parses every certificate file on disk) and is
+// recomputed on every call with no memoization. It's called twice per client refresh() cycle --
+// once from /api/dashboard, once from /api/certificates -- and that whole cycle can itself repeat
+// several times in a row (see refreshPendingProxies in app.js), so a short time-based cache here
+// collapses that duplicate work into a single real disk walk every few seconds. The window is kept
+// well under the 7s hero-poll interval so nothing ever appears more than one cycle stale.
+const CERTIFICATE_INVENTORY_CACHE_MS = 3000;
+let certificateInventoryCache = null; // { at: number, value: object }
 const loginAttempts = new Map();
 const rateLimitBuckets = new Map();
 let dockerSocketMounted = false;
@@ -882,6 +890,7 @@ function certificateNames(certificate) {
 }
 
 async function certificateInventory() {
+  if (certificateInventoryCache && Date.now() - certificateInventoryCache.at < CERTIFICATE_INVENTORY_CACHE_MS) return certificateInventoryCache.value;
   const configured = [...sites.map(item => ({ ...item, kind: "Hosted site" })), ...proxies.map(item => ({ ...item, kind: "Proxy host" })), ...redirects.map(item => ({ ...item, kind: "Redirect host" }))]
     .filter(item => item.enabled && item.domain && item.tls !== "http");
   const configuredDomains = configured.flatMap(item => normalizeDomains(item.domain, item.domains).map(domain => ({ ...item, domain })));
@@ -908,7 +917,9 @@ async function certificateInventory() {
   });
   for (const certificate of certificates) { const previous = certificateStatusCache.get(certificate.domain); if (previous && previous !== certificate.status) recordActivity(`Certificate status changed for ${certificate.domain}: ${previous} → ${certificate.status}.`, certificate.status === "healthy" ? "ok" : "error"); certificateStatusCache.set(certificate.domain, certificate.status); }
   const latestError = recentActivity.find(item => item.status === "error" && /cert|tls|acme|caddy|gateway/i.test(item.message)) || null;
-  return { checkedAt: new Date().toISOString(), thresholds: settings.certificateHealth, latestError, summary: { total: certificates.length, healthy: certificates.filter(item => item.status === "healthy").length, within30Days: certificates.filter(item => item.daysRemaining != null && item.daysRemaining <= 30 && item.daysRemaining > 0).length, within7Days: certificates.filter(item => item.daysRemaining != null && item.daysRemaining <= 7 && item.daysRemaining > 0).length, warning: certificates.filter(item => item.status === "warning").length, critical: certificates.filter(item => item.status === "critical").length, expired: certificates.filter(item => item.status === "expired").length, pending: certificates.filter(item => item.status === "pending").length, mismatch: certificates.filter(item => item.status === "mismatch").length }, certificates };
+  const result = { checkedAt: new Date().toISOString(), thresholds: settings.certificateHealth, latestError, summary: { total: certificates.length, healthy: certificates.filter(item => item.status === "healthy").length, within30Days: certificates.filter(item => item.daysRemaining != null && item.daysRemaining <= 30 && item.daysRemaining > 0).length, within7Days: certificates.filter(item => item.daysRemaining != null && item.daysRemaining <= 7 && item.daysRemaining > 0).length, warning: certificates.filter(item => item.status === "warning").length, critical: certificates.filter(item => item.status === "critical").length, expired: certificates.filter(item => item.status === "expired").length, pending: certificates.filter(item => item.status === "pending").length, mismatch: certificates.filter(item => item.status === "mismatch").length }, certificates };
+  certificateInventoryCache = { at: Date.now(), value: result };
+  return result;
 }
 
 async function pruneOrphanedCertificates(candidateDomains) {