Add database integrity and disk usage jobs to Scheduled jobs

ROADMAP.md

@@ -232,3 +232,7 @@ Roughly in priority order:
 `v0.16.53` fixes the "Configure thresholds" popup on the Certificates page (added in v0.16.51) rendering with its first input floating oddly beside the title instead of below it. The cause: `.settings-form` is a shared two-column CSS grid, and the dialog's heading block was placed as a plain grid child instead of spanning both columns like every other full-width element in a settings form already does (`.dialog-actions`, error text, a `<label>` wrapping a textarea). That left the "Renewing-soon warning" field sitting in the grid's second column of row one, directly beside the heading text. `.settings-form>.dialog-heading` is now added to that same existing full-span rule, so the heading spans the full width and the three threshold fields lay out normally beneath it.
 
 `v0.16.54` gives the Certificates table's Upstream column an accurate three-state color treatment instead of collapsing every non-numeric outcome into a single generic "no response." The underlying `upstreamHealth` data already distinguished four real states -- healthy (has a status code), unmonitored (monitoring intentionally turned off for that host), pending (no check has run yet), and a genuine failure (the check ran and errored or timed out) -- and the Hosted Site / Proxy host cards already labeled these correctly ("Monitoring paused", "Upstream check pending", etc.), but the new table only checked for a numeric status code and printed "no response" for everything else, including deliberately paused monitoring. The column now reads: green "running" dot with the status code for a healthy check, amber "idle" dot with "Monitoring paused" or "Check pending" for the two non-issue states, and a new red `.status-dot.bad` (added to styles.css, reusing the existing `--danger` token) with the actual error message for a real failure -- so a glance at the column now tells you whether something needs attention or is simply not being checked by design.
+
+`v0.16.55` is a batch covering three items reported against the Certificates and Logs pages. First, the "Configure thresholds" link on the Certificates page moves from floating level with the page title down onto the same line as the descriptive text beneath it, using a dedicated `.section-heading-row` flex row instead of the whole heading block being one flex row. Second, the three-state Upstream coloring added to the Certificates table in v0.16.54 (healthy / monitoring paused or check pending / genuine failure) is now applied consistently everywhere else that shows the same `upstreamHealth` data -- the Hosted Site, Proxy Host, and Streaming Host cards' upstream text now also gets a matching amber `.upstream-copy.idle` state for "Monitoring paused" and "Upstream/Target check pending", instead of rendering identically green to a real healthy check the way it did before. Third, the Logs page's "Gateway events" panel is converted from a stacked list of `.event-row` cards into a sticky-header table (`Time | Severity | Category | Message`), matching the Access requests table directly above it on the same page; the Message column is left free to wrap rather than forced onto one line, since event messages are free-form and of varying length, unlike the fixed-format columns elsewhere.
+
+`v0.16.56` adds the two background jobs from v0.16.45 and v0.16.48 (the SQLite integrity re-scan and the /data disk-usage walk, both moved off the request path and onto their own timers to fix the multi-second dashboard freezes reported at the time) to the Scheduled jobs list shown on the Dashboard and Administration -> System page. Both caches already tracked their own `checkedAt` timestamp internally, they just were never surfaced in the `jobs` array both pages already render from -- "Database integrity check" (every 30 minutes) and "Disk usage refresh" (every 60 seconds) now appear alongside Upstream checks, Scheduled backups, Log pruning, Access-log import, Public IP check, and Configuration drift check, with live last-run timestamps the same as every other entry in that list.

package.json

@@ -1,6 +1,6 @@
 {
   "name": "site-gateway",
-  "version": "0.16.54",
+  "version": "0.16.56",
   "private": true,
   "description": "Site Gateway: simple self-hosted website publishing, reverse proxying, and automatic HTTPS.",
   "type": "module",

src/public/app.js

@@ -241,12 +241,16 @@ function canAdmin() { return state.user?.role === "administrator"; }
 
 
 // --- Hosted Sites & Proxy Hosts: card templates ------------------------------------
+function upstreamStateClass(enabled, upstream) {
+  if (!enabled || upstream?.status === "unmonitored" || !upstream || upstream.status === "pending") return "idle";
+  return upstream.status === "healthy" ? "" : "bad";
+}
 function hostedCard(site) {
   const status = site.status === "running" ? "running" : site.status === "error" ? "error" : "disabled";
   const upstream = !site.enabled || site.upstream?.status === "unmonitored" ? "Monitoring paused" : !site.upstream || site.upstream.status === "pending" ? "Upstream check pending" : site.upstream.status === "healthy" ? `Upstream ${site.upstream.httpStatus} · ${site.upstream.responseMs} ms` : `Upstream unavailable · ${escapeHtml(site.upstream.error || "check failed")}`;
   const menu = canManage() ? `<div class="menu-wrap"><button class="icon-button menu-button" aria-label="Site options" aria-expanded="false">•••</button><div class="menu"><button data-action="settings">Domain & TLS</button><button data-action="icon">Change icon</button><button data-action="caddy-config">View Caddy config</button><button data-action="replace">Replace files</button><button data-action="delete" class="danger-text">Delete site</button></div></div>` : "";
   const toggle = canManage() ? `<button class="toggle ${site.enabled ? "on" : ""}" data-action="toggle" aria-label="${site.enabled ? "Disable" : "Enable"} ${escapeHtml(site.name)}"><span></span></button>` : "";
-  return `<article class="site-card" data-id="${site.id}" data-kind="hosted"><div class="card-top"><div class="site-icon">${iconMarkup(site)}</div>${menu}</div><h2>${escapeHtml(site.name)}</h2><p class="address">${escapeHtml(site.domain || `Port ${site.port}`)}</p>${site.domain ? `<p class="gateway-address ${site.tls !== "http" ? "secure" : ""}">${escapeHtml(publicUrl(site))}</p>` : ""}<p class="upstream-copy ${site.upstream?.status === "unhealthy" ? "bad" : ""}">${upstream}</p><div class="card-footer"><span class="status-pill"><span class="status-dot ${status}"></span>${status === "error" ? "Needs attention" : status[0].toUpperCase() + status.slice(1)}</span><div class="card-actions">${toggle}<a class="launch" href="${publicUrl(site)}" target="_blank" rel="noopener" aria-label="Open ${escapeHtml(site.name)}">↗</a></div></div></article>`;
+  return `<article class="site-card" data-id="${site.id}" data-kind="hosted"><div class="card-top"><div class="site-icon">${iconMarkup(site)}</div>${menu}</div><h2>${escapeHtml(site.name)}</h2><p class="address">${escapeHtml(site.domain || `Port ${site.port}`)}</p>${site.domain ? `<p class="gateway-address ${site.tls !== "http" ? "secure" : ""}">${escapeHtml(publicUrl(site))}</p>` : ""}<p class="upstream-copy ${upstreamStateClass(site.enabled, site.upstream)}">${upstream}</p><div class="card-footer"><span class="status-pill"><span class="status-dot ${status}"></span>${status === "error" ? "Needs attention" : status[0].toUpperCase() + status.slice(1)}</span><div class="card-actions">${toggle}<a class="launch" href="${publicUrl(site)}" target="_blank" rel="noopener" aria-label="Open ${escapeHtml(site.name)}">↗</a></div></div></article>`;
 }
 function proxyCard(proxy) {
   const status = proxy.status === "running" ? "running" : proxy.status === "error" ? "error" : "disabled";
@@ -254,7 +258,7 @@ function proxyCard(proxy) {
   const menu = canManage() ? `<div class="menu-wrap"><button class="icon-button menu-button" aria-label="Proxy options" aria-expanded="false">•••</button><div class="menu"><button data-action="settings">Edit proxy</button><button data-action="icon">Change icon</button><button data-action="caddy-config">View Caddy config</button><button data-action="delete" class="danger-text">Delete proxy</button></div></div>` : "";
   const toggle = canManage() ? `<button class="toggle ${proxy.enabled ? "on" : ""}" data-action="toggle" aria-label="${proxy.enabled ? "Disable" : "Enable"} ${escapeHtml(proxy.name)}"><span></span></button>` : "";
   const access = proxy.accessListId ? (state.accessLists.find(item => item.id === proxy.accessListId)?.name || "Access List") : "Public · no Access List";
-  return `<article class="site-card proxy" data-id="${proxy.id}" data-kind="proxy"><div class="card-top"><div class="site-icon">${iconMarkup(proxy)}</div>${menu}</div><h2>${escapeHtml(proxy.name)}</h2><p class="address">${escapeHtml(proxy.target)}</p><p class="gateway-address ${proxy.tls !== "http" ? "secure" : ""}">${escapeHtml(publicUrl(proxy))}</p><p class="upstream-copy ${proxy.upstream?.status === "unhealthy" ? "bad" : ""}">${upstream}</p><p class="access-summary">${escapeHtml(access)}</p><div class="card-footer"><span class="status-pill"><span class="status-dot ${status}"></span>${status === "error" ? "Needs attention" : status[0].toUpperCase() + status.slice(1)}</span><div class="card-actions">${toggle}<a class="launch" href="${publicUrl(proxy)}" target="_blank" rel="noopener" aria-label="Open ${escapeHtml(proxy.name)}">↗</a></div></div></article>`;
+  return `<article class="site-card proxy" data-id="${proxy.id}" data-kind="proxy"><div class="card-top"><div class="site-icon">${iconMarkup(proxy)}</div>${menu}</div><h2>${escapeHtml(proxy.name)}</h2><p class="address">${escapeHtml(proxy.target)}</p><p class="gateway-address ${proxy.tls !== "http" ? "secure" : ""}">${escapeHtml(publicUrl(proxy))}</p><p class="upstream-copy ${upstreamStateClass(proxy.enabled, proxy.upstream)}">${upstream}</p><p class="access-summary">${escapeHtml(access)}</p><div class="card-footer"><span class="status-pill"><span class="status-dot ${status}"></span>${status === "error" ? "Needs attention" : status[0].toUpperCase() + status.slice(1)}</span><div class="card-actions">${toggle}<a class="launch" href="${publicUrl(proxy)}" target="_blank" rel="noopener" aria-label="Open ${escapeHtml(proxy.name)}">↗</a></div></div></article>`;
 }
 
 
@@ -306,7 +310,7 @@ function renderLogs() {
   const categoryOf = message => /cert|tls|https/i.test(message) ? "certificate" : /health|upstream|response|fetch/i.test(message) ? "health" : /login|user|password|access/i.test(message) ? "authentication" : /backup|restore/i.test(message) ? "backup" : /config|route|host|gateway|reload/i.test(message) ? "configuration" : "system";
   const severity = $("#event-severity").value, category = $("#event-category").value;
   const activity = data.activity.filter(item => (!severity || item.status === severity) && (!category || categoryOf(item.message) === category));
-  $("#gateway-log-list").innerHTML = activity.length ? activity.map(item => { const eventCategory = categoryOf(item.message); const indicatorClass = item.status === "error" ? "disabled" : item.status === "warning" ? "error" : "running"; return `<div class="event-row"><span class="status-dot ${indicatorClass}" aria-label="${escapeHtml(item.status || "ok")}"></span><span><strong>${escapeHtml(item.message)}</strong><small>${escapeHtml(eventCategory)} · ${escapeHtml(formatTime(item.at))}</small></span></div>`; }).join("") : '<div class="gateway-empty-state"><span class="status-dot"></span><strong>No matching gateway events</strong><small>Try a different severity or category filter.</small></div>';
+  $("#gateway-log-list").innerHTML = activity.length ? activity.map(item => { const eventCategory = categoryOf(item.message); const indicatorClass = item.status === "error" ? "disabled" : item.status === "warning" ? "error" : "running"; const severityLabel = item.status === "error" ? "Error" : item.status === "warning" ? "Warning" : "Normal"; return `<tr><td>${escapeHtml(formatTime(item.at))}</td><td><span class="status-dot ${indicatorClass}"></span>${severityLabel}</td><td>${escapeHtml(eventCategory)}</td><td>${escapeHtml(item.message)}</td></tr>`; }).join("") : '<tr><td colspan="4" class="quiet-state">No matching gateway events. Try a different severity or category filter.</td></tr>';
 }
 
 

src/public/features.js

@@ -25,7 +25,7 @@ function renderStreams() {
     const upstream = item.enabled === false || item.upstream?.status === "unmonitored" ? "Monitoring paused" : !item.upstream || item.upstream.status === "pending" ? "Target check pending" : item.upstream.status === "healthy" ? `Target reachable · ${item.upstream.responseMs} ms` : `Target unreachable · ${extendedEscape(item.upstream.error || "check failed")}`;
     const protocols = [item.tcp !== false ? "TCP" : null, item.udp ? "UDP" : null].filter(Boolean).map(value => `<span class="chip">${value}</span>`).join("");
     const toggle = `<button class="toggle ${item.enabled === false ? "" : "on"}" data-stream-action="toggle" aria-label="${item.enabled === false ? "Enable" : "Disable"} ${extendedEscape(item.name)}"><span></span></button>`;
-    return `<article class="site-card stream-card" data-stream-id="${item.id}" data-kind="stream"><div class="card-top"><div class="site-icon">${featureIcon(item,"SH")}</div><div class="menu-wrap"><button class="icon-button menu-button" aria-label="Streaming host options" aria-expanded="false">•••</button><div class="menu"><button data-stream-action="edit">Edit streaming host</button><button data-stream-action="icon">Change icon</button><button data-stream-action="delete" class="danger-text">Delete streaming host</button></div></div></div><h2>${extendedEscape(item.name)}</h2><p class="address">Port ${item.port}</p><p class="gateway-address">→ ${extendedEscape(item.target)}</p><p class="upstream-copy ${item.upstream?.status === "unhealthy" ? "bad" : ""}">${upstream}</p><div class="card-footer"><span class="status-pill"><span class="status-dot ${status}"></span>${status === "error" ? "Needs attention" : status[0].toUpperCase() + status.slice(1)}</span><div class="card-actions">${toggle}${protocols}</div></div></article>`;
+    return `<article class="site-card stream-card" data-stream-id="${item.id}" data-kind="stream"><div class="card-top"><div class="site-icon">${featureIcon(item,"SH")}</div><div class="menu-wrap"><button class="icon-button menu-button" aria-label="Streaming host options" aria-expanded="false">•••</button><div class="menu"><button data-stream-action="edit">Edit streaming host</button><button data-stream-action="icon">Change icon</button><button data-stream-action="delete" class="danger-text">Delete streaming host</button></div></div></div><h2>${extendedEscape(item.name)}</h2><p class="address">Port ${item.port}</p><p class="gateway-address">→ ${extendedEscape(item.target)}</p><p class="upstream-copy ${item.enabled === false || item.upstream?.status === "unmonitored" || !item.upstream || item.upstream.status === "pending" ? "idle" : item.upstream.status === "healthy" ? "" : "bad"}">${upstream}</p><div class="card-footer"><span class="status-pill"><span class="status-dot ${status}"></span>${status === "error" ? "Needs attention" : status[0].toUpperCase() + status.slice(1)}</span><div class="card-actions">${toggle}${protocols}</div></div></article>`;
   }).join("");
 }
 

src/public/index.html

@@ -8,7 +8,7 @@
   <title>Site Gateway</title>
   <meta name="description" content="Host sites, proxy services, and manage HTTPS from one simple dashboard.">
   <link rel="icon" type="image/png" href="/site-gateway-icon-approved.png">
-  <link rel="stylesheet" href="/styles.css?v=0.16.54">
+  <link rel="stylesheet" href="/styles.css?v=0.16.56">
 </head>
 
 <!-- ================================================================
@@ -161,7 +161,7 @@
         <div class="feature-summary">
           <div><strong id="cert-healthy">0</strong><span>Healthy</span></div><div><strong id="cert-30">0</strong><span>Within 30 days</span></div><div><strong id="cert-7">0</strong><span>Within 7 days</span></div><div><strong id="cert-warning">0</strong><span>Needs attention</span></div><div><strong id="cert-pending">0</strong><span>Not detected</span></div>
         </div>
-        <div class="diagnostic-section-heading"><div><p class="eyebrow">Certificate inventory</p><h2>Certificates</h2><p class="muted">Managed certificates, TLS readiness, and renewal status for every configured domain. Select a row for full detail.</p></div><button type="button" id="cert-threshold-trigger" class="text-button">Configure thresholds →</button></div>
+        <div class="diagnostic-section-heading"><p class="eyebrow">Certificate inventory</p><h2>Certificates</h2><div class="section-heading-row"><p class="muted">Managed certificates, TLS readiness, and renewal status for every configured domain. Select a row for full detail.</p><button type="button" id="cert-threshold-trigger" class="text-button">Configure thresholds →</button></div></div>
         <div class="table-wrap performance-table-wrap cert-table-wrap"><table class="performance-table cert-table"><thead><tr><th>Domain</th><th>Status</th><th>Days remaining</th><th>Issuer</th><th>DNS</th><th>TLS</th><th>Upstream</th></tr></thead><tbody id="certificate-list"><tr><td colspan="7" class="quiet-state">Loading certificates…</td></tr></tbody></table></div>
       </section>
 
@@ -192,7 +192,7 @@
           <p id="log-summary" class="muted feature-note">No requests in the current view. <span id="log-last-checked">Not checked yet.</span></p>
           <div class="table-wrap log-table-wrap"><table class="log-table"><thead><tr><th>Time</th><th>Domain</th><th>Request</th><th>Status</th><th>Duration</th></tr></thead><tbody id="log-rows"></tbody></table></div>
         </section>
-        <section class="dashboard-panel log-activity"><div class="panel-heading"><div><p class="eyebrow">Gateway events</p><h2>Activity and errors</h2><p class="muted">Configuration, certificate, and health events recorded by Site Gateway.</p></div></div><div class="event-filters"><label>Severity<select id="event-severity"><option value="">All severities</option><option value="ok">Normal</option><option value="warning">Warnings</option><option value="error">Errors</option></select></label><label>Category<select id="event-category"><option value="">All categories</option><option value="configuration">Configuration</option><option value="certificate">Certificates / TLS</option><option value="health">Upstream health</option><option value="authentication">Authentication</option><option value="backup">Backups</option><option value="system">System</option></select></label></div><div id="gateway-log-list" class="dashboard-list event-list diagnostic-list"></div></section>
+        <section class="dashboard-panel log-activity"><div class="panel-heading"><div><p class="eyebrow">Gateway events</p><h2>Activity and errors</h2><p class="muted">Configuration, certificate, and health events recorded by Site Gateway.</p></div></div><div class="event-filters"><label>Severity<select id="event-severity"><option value="">All severities</option><option value="ok">Normal</option><option value="warning">Warnings</option><option value="error">Errors</option></select></label><label>Category<select id="event-category"><option value="">All categories</option><option value="configuration">Configuration</option><option value="certificate">Certificates / TLS</option><option value="health">Upstream health</option><option value="authentication">Authentication</option><option value="backup">Backups</option><option value="system">System</option></select></label></div><div class="table-wrap performance-table-wrap event-table-wrap"><table class="performance-table event-table"><thead><tr><th>Time</th><th>Severity</th><th>Category</th><th>Message</th></tr></thead><tbody id="gateway-log-list"></tbody></table></div></section>
       </section>
 
       <!-- Performance view: request trend chart + per-domain throughput table -->
@@ -451,6 +451,6 @@
   <div id="toast" class="toast" role="status"></div>
   <div id="update-banner" class="update-banner hidden" role="status"><span>A new version of Site Gateway is available.</span><div class="update-banner-actions"><button id="update-banner-refresh" class="button primary">Refresh</button><button id="update-banner-dismiss" class="text-button">Dismiss</button></div></div>
   <!-- App scripts: core (app.js) then extended views/admin (features.js) -->
-  <script src="/app.js?v=0.16.54" defer></script><script src="/features.js?v=0.16.54" defer></script><script src="/select-enhance.js?v=0.16.54" defer></script>
+  <script src="/app.js?v=0.16.56" defer></script><script src="/features.js?v=0.16.56" defer></script><script src="/select-enhance.js?v=0.16.56" defer></script>
 </body>
 </html>

src/public/styles.css

@@ -253,6 +253,14 @@ header{align-items:flex-end}
 @media(max-width:760px){.mobile-nav{display:grid;grid-template-columns:repeat(5,1fr);gap:5px;margin:0 0 30px;padding:var(--space-1);border:1px solid var(--line);border-radius:var(--radius-md);background:var(--panel)}.mobile-nav button{justify-content:center;text-align:center;padding:9px 5px;font-size:.72rem}.dashboard-view{margin-top:30px}.metric-grid{grid-template-columns:repeat(2,minmax(0,1fr));gap:10px}.metric-card{padding:var(--space-4)}.metric-card strong{font-size:1.65rem}.dashboard-columns{grid-template-columns:1fr}.system-grid{grid-template-columns:repeat(2,minmax(0,1fr))}.metric-strip{grid-template-columns:1fr}.metric-chip{padding:11px 14px}}
 .upstream-copy{margin:10px 0 0;color:var(--green);font-size:.72rem}
 .upstream-copy.bad{color:var(--danger)}
+.upstream-copy.idle{color:var(--warning)}
+.event-table-wrap{max-height:min(52vh,620px);overflow:auto;margin-top:var(--space-4)}
+.event-table-wrap .event-table thead th{position:sticky;top:0;background:var(--panel);z-index:1}
+.event-table th:nth-child(1),.event-table td:nth-child(1){width:15%;white-space:nowrap}
+.event-table th:nth-child(2),.event-table td:nth-child(2){width:11%;white-space:nowrap}
+.event-table th:nth-child(3),.event-table td:nth-child(3){width:14%;white-space:nowrap}
+.event-table th:nth-child(4),.event-table td:nth-child(4){white-space:normal;overflow-wrap:anywhere}
+.event-table td .status-dot{margin-right:6px;vertical-align:-1px}
 .activity-mark.bad{background:rgba(var(--danger-rgb),.12);color:var(--danger)}
 .feature-summary{display:grid;grid-template-columns:repeat(3,minmax(0,1fr));gap:14px;margin-bottom:18px}
 .feature-summary>div{padding:18px;border:1px solid var(--line);border-radius:var(--radius-2xl);background:var(--panel)}
@@ -825,7 +833,8 @@ select{appearance:none!important;-webkit-appearance:none!important;background-re
 .diagnostic-section-heading p:last-child{margin:var(--space-1) 0 0}
 .diagnostic-section-heading{margin-bottom:0}
 .diagnostic-section-heading + .log-table-wrap{border-top:0;border-radius:0 0 15px 15px}
-.diagnostic-section-heading{display:flex;justify-content:space-between;align-items:flex-start;gap:var(--space-4)}
+.section-heading-row{display:flex;justify-content:space-between;align-items:baseline;gap:var(--space-4)}
+.section-heading-row .muted{margin:0}
 .diagnostic-section-heading + .cert-table-wrap{border-top:0;border-radius:0 0 15px 15px}
 .cert-table-row{cursor:pointer}
 .cert-table-row:hover{background:rgba(var(--green-rgb),.06)}

src/server.js

@@ -1185,6 +1185,8 @@ async function dashboardSnapshot(precomputedCertificates) {
         { name: "Access-log import", enabled: true, schedule: "30s", lastRunAt: lastAccessLogImportAt },
         { name: "Public IP check", enabled: true, schedule: "60m", lastRunAt: publicIpState.checkedAt || null },
         { name: "Configuration drift check", enabled: true, schedule: "10m", lastRunAt: configDrift.checkedAt || null },
+        { name: "Database integrity check", enabled: true, schedule: "30m", lastRunAt: databaseIntegrityCache.checkedAt || null },
+        { name: "Disk usage refresh", enabled: true, schedule: "60s", lastRunAt: dataDirSizeCache.checkedAt || null },
       ]
     },
     activity: recentActivity