Compare commits

...

5 Commits

7 changed files with 68 additions and 37 deletions
+1 -1
View File
@@ -12,7 +12,7 @@
<img alt="Docker" src="https://img.shields.io/badge/Docker-ready-2496ED?logo=docker&logoColor=white">
<img alt="Architectures" src="https://img.shields.io/badge/platform-amd64%20%7C%20arm64-5965F2">
<img alt="Caddy" src="https://img.shields.io/badge/powered%20by-Caddy-1F88C0">
<img alt="Version" src="https://img.shields.io/badge/version-0.16.43-62E6A7">
<img alt="Version" src="https://img.shields.io/badge/version-0.16.49-62E6A7">
</p>
<p>
<a href="#why-site-gateway">Why Site Gateway</a> ·
+10 -2
View File
@@ -118,8 +118,6 @@ Roughly in priority order:
- **Richer certificate diagnostics** — on-demand checks that distinguish DNS, inbound port, TLS, and upstream failures per domain.
- **Wildcard/DNS-challenge certificates** — selected DNS-provider integrations for domains that can't use HTTP-01 validation. Needs encrypted secret storage for provider API credentials before it ships.
- **Browsable backup/restore history** — today a restore validates and rolls back safely, but there's no UI history of past backups beyond what's on disk.
- **Container picker for Proxy/Streaming targets** — letting a target be selected from a list of running Docker containers instead of typed as an IP/hostname, gated behind an opt-in Docker-socket mount since it needs real access to the Engine API. Also needs a shared Docker network between Site Gateway and the target container to actually be reachable, not just discoverable.
- **Tailscale integration** — documented patterns exist today (host-level Tailscale for private dashboard access, a sidecar container for proxying to tailnet-only targets, `tailscale serve`/`funnel` for exposing a route without opening router ports), but nothing is built into Site Gateway itself yet.
- **Dynamic DNS** and **deeper Caddy controls** for advanced users who outgrow the guided options.
- **Rate limiting** and other specialist gateway controls.
@@ -214,3 +212,13 @@ Roughly in priority order:
`v0.16.42` finds and fixes the real, dominant cause of the site-wide slowness reported after v0.16.41: a live Network-tab capture from the user's own browser showed a flood of requests to `/api/logs/prune/preview`, some queued for over 15 seconds, with unrelated requests (`/api/dashboard`, `/api/system/health`, `/api/system/security`, `/api/system/storage`) stuck at nearly identical multi-second times in the same batch -- the signature of one blocking operation stalling everything behind it, not several independently slow endpoints. Root cause: `renderRetentionPreview()`'s `setInterval(..., 2000)` polls that endpoint every 2 seconds forever, on every page of the app, not just Administration -> Logs & retention, because its "does the panel exist" guard checks a `<section>` that's written into `index.html` from page load and only ever CSS-hidden -- so the guard was always true, everywhere. There was also no protection against a new poll firing while a previous one was still in flight, so once the server answered slower than 2 seconds even once, requests piled up and never caught back up. Compounding it: `previewPruneEvents()` runs five synchronous SQLite COUNT queries, and one of them (`audit_events`) had no index at all -- a full table scan, every call -- and because this app's SQLite queries run synchronously, that scan doesn't just slow its own request, it blocks the entire Node process for every other request being served at that moment. Fixed on both sides: `renderRetentionPreview()` and the sibling `renderRetentionRunStatus()` (previously also running unconditionally every 500ms) now check that the retention panel is actually visible, not just present in the DOM, before doing any work, and an in-flight guard stops a new preview poll from starting until the last one has landed; `audit_events` now has the same `(instance_id, created_at)` index every sibling events table already had. Together these should remove the vast majority of the "8-10 seconds to load a simple page" behavior reported after v0.16.41 -- that fix (deduplicating certificate-inventory work) was real but minor by comparison to this one.
`v0.16.43` scopes page refreshes to the page actually being viewed, instead of every refresh across the entire app unconditionally re-fetching everything -- Hosted Sites, Proxy Hosts, Redirects, Streams, Access Lists, Groups, the full Dashboard snapshot, and Certificates -- regardless of which single page triggered it. This was confirmed directly from the user's own account of the behavior ("if I'm on Hosted Sites and click refresh, it appears the whole entire site refreshes") and traced to a single shared `refresh()` function that every action in the app called: creating or editing a hosted site or proxy, toggling one on or off, deleting an entry, saving gateway settings, and re-syncing the gateway all ran the identical 8-endpoint fetch no matter which page initiated it. `refresh()` and its endpoints are now built from one shared map (`REFRESH_ENDPOINTS`), and a new `refreshCurrentView()` fetches only the state keys a `VIEW_REFRESH_KEYS` table says the active view actually renders -- Hosted Sites now refetches just `sites`, Proxy Hosts just `proxies`, Streaming just `streams`, Redirects just `redirects`, Access Lists just `accessLists` and `groups`. Every action listed above that's only ever reachable from one specific view (creating/editing/toggling/deleting a hosted site or proxy) now calls `refreshCurrentView()` instead of the full `refresh()`. Overview keeps the full, unscoped fetch deliberately: its attention list and the sidebar's per-section counts summarize the whole gateway, not one section of it, so scoping it would defeat the page's purpose; the initial page load (`boot()`) and the gateway re-sync button (only reachable from Overview) are unchanged for the same reason. A new generic refresh button (the same "↻" icon `refresh-health` already used) now appears on every page except Logs (which keeps its own dedicated "Refresh logs" button) so every view has an explicit, page-scoped way to pull fresh data without a full browser reload -- previously several views (Hosted, Proxy Hosts, Streaming, Redirects, Access Lists) had no refresh control of their own at all and only ever picked up new data from the page's initial load or the next full-page reload. One deliberate trade-off: sidebar badge counts for sections other than the one currently being viewed are not part of a scoped refresh and can go briefly stale until the next full refresh (a fresh page load, or a visit to Overview) -- intentional, since fetching data a page doesn't display was the entire problem being fixed here.
`v0.16.44` is a temporary, diagnostic-only release -- no behavior changes, just logging -- added after v0.16.43 (which fixed the app from over-fetching per page) didn't resolve the user's reported 6-14 second page loads. A Network-tab Timing capture the user sent for a single `GET /api/sites` request showed DNS and TCP connection at 0-7ms but "Waiting" (time to first byte) at 7485ms -- almost the entire delay happened server-side, before the app sent back a single byte of what should be a near-instant, in-memory list. Since this codebase's database and JS execution is single-threaded, that pattern (a trivially cheap request taking seconds) points to something else blocking the whole process at that moment, not a cost specific to any one endpoint. The leading suspect: `importAccessLogsToSqlite()`, a job that runs every 30 seconds, reads Caddy's access-log files, JSON-parses and hashes up to 5000 lines, and batch-inserts them -- all synchronous work with nothing to yield the event loop partway through. Rather than ship a fourth guess-based fix, this release adds two pieces of logging visible in the container's own logs: a warning whenever that import job takes over 500ms (broken down into read/hash/insert time), and a warning whenever any request takes over 1 second to answer. The next slow page load should show, in the logs, either the import job's duration lining up with the slow request's timestamp (confirming the suspect) or a different pattern entirely (pointing somewhere else). Both log lines are marked as temporary instrumentation, intended to be removed once the real cause is confirmed and fixed.
`v0.16.45` fixes the confirmed root cause behind the multi-second page loads reported after v0.16.41-v0.16.43: the user's own container logs, captured with v0.16.44's temporary diagnostics, showed completely unrelated endpoints -- `/api/dashboard`, `/api/system/security`, `/api/logs/prune/preview` -- all finishing within moments of each other at nearly identical ~8.5-9 second durations, right after the container started. That pattern only happens when several requests are queued behind one shared blocking operation, not when each is independently slow. The culprit: `dashboardSnapshot()` (which every `/api/dashboard` fetch runs) called `storage.integrity()` -- a full `PRAGMA integrity_check`, a complete scan of the entire SQLite database file for corruption, one of the most expensive operations SQLite can run -- on every single call, purely to compute one cosmetic "Healthy"/"Needs attention" label. Because this app's SQLite queries run synchronously, that scan didn't just make its own request slow, it froze the entire single-threaded server for its whole duration, on every dashboard fetch, for every user. The fix moves that check off the request path entirely: a new `refreshDatabaseIntegrityCache()` runs the real scan once shortly after startup and then every 30 minutes in the background, caching just the resulting status string, and `dashboardSnapshot()` now reads that cached value instantly instead of re-scanning the whole database on every poll. The (rarely-used, explicitly manual) downloadable support report still runs a live, real-time integrity check, since that's an appropriate place for a slow, thorough scan. v0.16.44's temporary `[perf]` logging stays in place for this release so the fix's effect is directly visible in the container's own logs -- expect no more `[perf] GET ... took` warnings tied to `/api/dashboard` going forward.
`v0.16.47` makes the page-scoped refresh button (added in v0.16.43) consistent across every view instead of appearing on most pages but not Logs, and removes a now-redundant control. The button is repositioned to always sit top-right, immediately to the right of that page's green primary action button (“+ New hosted site”, “Run certificate check”, “Refresh logs”) when one is present, or in that same top-right spot when a page has no primary action button of its own; it now also appears on the Logs page rather than being hidden there. A dedicated CSS rule (`.page-refresh{width:44px;height:44px}`) makes the button exactly the same height as the app's existing 44px primary-button standard (the same convention already used for the Backups and Retention action rows), so it visually lines up with the button beside it instead of looking undersized next to it. The Live Health panel's own separate “↻” refresh icon has been removed from the Dashboard, since the page-level refresh button sitting a few pixels away now does the identical job (`refreshDashboard()`, which repopulates that same panel); `refreshDashboard()` itself is unchanged and still runs on its normal 30-second Overview timer, it just no longer drives a second, separate icon's spinner.
`v0.16.48` is a batch covering five separately-reported items. First, it fixes a real layout regression v0.16.47 introduced: reordering the header's action buttons so the page-refresh icon appeared after the green primary button caused `header`'s `justify-content:space-between` to treat every button as its own flex item and redistribute space between all of them, visibly shifting the green button ("Refresh logs", "Run certificate check", etc.) away from its usual position instead of leaving it in place with the icon simply appended beside it. The buttons are now wrapped in a single `.header-actions` container so `header` only ever splits space between the page title and that one group, and the group's own `gap` keeps its buttons hugging together at the right edge exactly as before v0.16.47. Second, it removes the temporary `[perf]` diagnostic logging added in v0.16.44 (the slow-request middleware and the `importAccessLogsToSqlite` timing breakdown), now fully superseded by v0.16.45's fix and no longer needed. Third, it removes the "Block common exploits" per-Proxy-Host toggle entirely -- its regex-based matcher only ever inspected the request path, never the query string, so it never provided the SQL-injection/XSS protection its label implied; the checkbox, its documentation entry, and every server-side and client-side reference to `blockCommonExploits` are gone. Fourth, it applies the same "cache expensive checks instead of recomputing them on every request" fix used for the database-integrity check in v0.16.45 to the System tab hero panel's disk-usage figure: when `DATA_DIR_LIMIT_GB` is set, the hero panel needs a real recursive walk of `/data` to compute its used-space percentage, and that walk was being redone on every single 7-second hero-panel poll, for every concurrent viewer. It's now computed once shortly after boot and refreshed every 60 seconds in the background (`refreshDataDirSizeCache()`), with the hot request path just reading the cached value -- deployments that don't set `DATA_DIR_LIMIT_GB` are unaffected, since they never triggered this walk in the first place. Fifth, the ROADMAP's own "What's next" section is reconciled against the "Shipped" section above it: two items it listed as upcoming (browsable backup/restore history, a Docker container picker for Proxy/Streaming targets) had already shipped and were removed from the list.
`v0.16.49` fixes the My Account and Documentation pages' cramped spacing between the header subtitle and the first box below it, reported against several earlier releases. The cause was pinned down precisely by measuring pixel gaps across side-by-side screenshots of a correctly-spaced page (Logs) against the two broken ones: Certificates, Performance, and Logs all get their deliberate spacing from one shared rule, `#certificates-view,#performance-view,#logs-view{margin-top:var(--space-7)}`, and My Account and Documentation were simply never added to that selector, so both fell back to a 0px top margin. The fix adds `#account-view` and `#documentation-view` to that same existing rule -- reusing the app's own established spacing value rather than introducing a new one.
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "site-gateway",
"version": "0.16.43",
"version": "0.16.49",
"private": true,
"description": "Site Gateway: simple self-hosted website publishing, reverse proxying, and automatic HTTPS.",
"type": "module",
+5 -6
View File
@@ -94,7 +94,7 @@ function advancedFormBody(form, body, scoped) {
const read = (name, fallback = "") => scoped ? scopedValue(scoped.formEl, scoped.scope, name, fallback) : (form.get(name) || fallback);
const checked = (name) => scoped ? Boolean(scoped.formEl.querySelector(`${scoped.scope} [name="${name}"]`)?.checked) : form.has(name);
body.domains = String(form.get("domainsText") || "").split(/[\n,]+/).map(value => value.trim()).filter(Boolean);
body.hsts = form.has("hsts"); body.hstsSubdomains = checked("hstsSubdomains"); body.healthEnabled = checked("healthEnabled"); body.upstreamTlsInsecure = checked("upstreamTlsInsecure"); body.blockCommonExploits = checked("blockCommonExploits");
body.hsts = form.has("hsts"); body.hstsSubdomains = checked("hstsSubdomains"); body.healthEnabled = checked("healthEnabled"); body.upstreamTlsInsecure = checked("upstreamTlsInsecure");
body.accessListId = read("accessListId", body.accessListId || "");
body.requestHeaders = parseHeaderLines(read("requestHeadersText")); body.responseHeaders = parseHeaderLines(read("responseHeadersText")); body.compression = read("compression", "automatic"); body.customConfig = read("customConfig");
body.locations = String(form.get("customLocationsText") || "").split("\n").map(line => { const [path, target, behavior] = line.split("|").map(value => value.trim()); return path && target ? { path, target, stripPrefix:behavior.toLowerCase() === "strip" } : null; }).filter(Boolean);
@@ -493,7 +493,7 @@ function render() {
$("#streaming-view").classList.toggle("hidden", state.view !== "streaming"); $("#redirects-view").classList.toggle("hidden", state.view !== "redirects"); $("#access-view").classList.toggle("hidden", state.view !== "access"); $("#documentation-view").classList.toggle("hidden", state.view !== "documentation");
const activeAdminTab = state.view === "administration" ? document.querySelector("[data-admin-tab].tab-active")?.dataset.adminTab : null;
const adminUsersActive = activeAdminTab === "users", adminGroupsActive = activeAdminTab === "groups", adminApiActive = activeAdminTab === "api";
$("#open-create").classList.toggle("hidden", !(management || adminUsersActive || adminGroupsActive || adminApiActive || ["streaming","redirects","access"].includes(state.view)) || !canManage()); $("#check-health").classList.toggle("hidden", state.view !== "certificates" || !canAdmin()); $("#refresh-logs").classList.toggle("hidden", state.view !== "logs"); $("#refresh-view").classList.toggle("hidden", state.view === "logs");
$("#open-create").classList.toggle("hidden", !(management || adminUsersActive || adminGroupsActive || adminApiActive || ["streaming","redirects","access"].includes(state.view)) || !canManage()); $("#check-health").classList.toggle("hidden", state.view !== "certificates" || !canAdmin()); $("#refresh-logs").classList.toggle("hidden", state.view !== "logs");
if (overview) {
$("#page-title").textContent = "Dashboard";
$("#page-subtitle").textContent = "Health, activity, and system status at a glance.";
@@ -606,9 +606,9 @@ async function refreshPendingProxies(ids = []) {
}
}
async function refreshDashboard() {
const button = $("#refresh-health"); button.disabled = true; button.classList.add("spinning"); $("#health-checked").innerHTML = '<span class="live-dot checking"></span>Checking services…';
$("#health-checked").innerHTML = '<span class="live-dot checking"></span>Checking services…';
try { state.dashboard = await api("/api/dashboard"); renderDashboard(); }
finally { button.disabled = false; button.classList.remove("spinning"); }
finally { /* no-op: the Live Health panel's own refresh icon was removed in favor of the page-level refresh button */ }
}
// Populates the Dashboard's hero panel (CPU/memory/swap/disk/network/uptime) directly from
// /api/system/health, the same call and the same renderHeroPanel() the Administration > System
@@ -769,7 +769,6 @@ document.addEventListener("keydown", event => { if (event.key === "Escape") clos
document.querySelectorAll("dialog").forEach(dialog => dialog.addEventListener("close", () => { closeMenus(); dialog.querySelectorAll('input[type="password"]').forEach(input => input.value = ""); }));
// --- Hosted Sites & Proxy Hosts: create form submit handlers --------------------------------
$("#refresh-health").addEventListener("click", () => refreshDashboard().catch(error => toast(error.message, "error")));
$("#create-form").addEventListener("submit", async event => { event.preventDefault(); const button = resolveSubmitter(event); button.disabled = true; button.textContent = "Publishing…"; $("#create-error").textContent = ""; try { await api("/api/sites", { method: "POST", body: new FormData(event.target) }); $("#create-dialog").close(); await refreshCurrentView(); toast("Hosted site created and gateway applied."); } catch (error) { $("#create-error").textContent = error.message; } finally { button.disabled = false; button.textContent = "Create & publish"; } });
$("#proxy-form").addEventListener("submit", async event => { event.preventDefault(); const button = resolveSubmitter(event); button.disabled = true; button.textContent = "Publishing…"; $("#proxy-error").textContent = ""; const form = new FormData(event.target), certificate = form.get("certificateFile"), privateKey = form.get("privateKeyFile"), wantsCustom = form.get("tls") === "custom"; if (wantsCustom && (!certificate?.size || !privateKey?.size)) { $("#proxy-error").textContent = "Choose both the certificate and private key for Custom HTTPS."; button.disabled = false; button.textContent = "Create & publish"; return; } const body = advancedFormBody(form, Object.fromEntries(form)); delete body.certificateFile; delete body.privateKeyFile; if (wantsCustom) body.tls = "http"; try { const created = await api("/api/proxies", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify(body) }); if (wantsCustom) { const files = new FormData(); files.append("certificate", certificate); files.append("privateKey", privateKey); await api(`/api/proxies/${created.id}/certificate`, { method:"POST", body:files }); } $("#proxy-dialog").close(); await refreshCurrentView(); toast(wantsCustom ? "Proxy host created with its custom certificate." : "Proxy host created. Certificate provisioning runs automatically."); } catch (error) { $("#proxy-error").textContent = error.message; } finally { button.disabled = false; button.textContent = "Create & publish"; } });
@@ -786,7 +785,7 @@ function openSettings(kind, id) {
form.elements.name.value = item.name || ""; form.elements.domain.value = item.domain || ""; form.elements.target.value = item.target || ""; form.elements.tls.value = item.tls || "automatic"; form.elements.hsts.checked = Boolean(item.hsts); if (form.elements.settingsAccessListId) form.elements.settingsAccessListId.value = item.accessListId || "";
if (kind === "proxy") {
const scope = "#settings-advanced";
setScoped(form, scope, "accessListId", item.accessListId || ""); setScoped(form, scope, "healthPath", item.healthPath || "/"); setScoped(form, scope, "healthMethod", item.healthMethod || "GET"); setScoped(form, scope, "healthExpected", item.healthExpected || "200-499"); setScoped(form, scope, "healthTimeoutSeconds", item.healthTimeoutSeconds || 4); setScoped(form, scope, "healthEnabled", item.healthEnabled !== false); setScoped(form, scope, "compression", item.compression || "automatic"); setScoped(form, scope, "blockCommonExploits", Boolean(item.blockCommonExploits));
setScoped(form, scope, "accessListId", item.accessListId || ""); setScoped(form, scope, "healthPath", item.healthPath || "/"); setScoped(form, scope, "healthMethod", item.healthMethod || "GET"); setScoped(form, scope, "healthExpected", item.healthExpected || "200-499"); setScoped(form, scope, "healthTimeoutSeconds", item.healthTimeoutSeconds || 4); setScoped(form, scope, "healthEnabled", item.healthEnabled !== false); setScoped(form, scope, "compression", item.compression || "automatic");
form.elements.customLocationsText.value = (item.locations || []).map(location => `${location.path} | ${location.target} | ${location.stripPrefix ? "strip" : "preserve"}`).join("\n");
setScoped(form, scope, "requestHeadersText", (item.requestHeaders || []).map(header => `${header.name}: ${header.value}`).join("\n")); setScoped(form, scope, "responseHeadersText", (item.responseHeaders || []).map(header => `${header.name}: ${header.value}`).join("\n"));
form.elements.upstreamTlsServerName.value = item.upstreamTlsServerName || ""; setScoped(form, scope, "upstreamTlsInsecure", Boolean(item.upstreamTlsInsecure)); setScoped(form, scope, "hstsSubdomains", Boolean(item.hstsSubdomains)); setScoped(form, scope, "customConfig", item.customConfig || ""); form.elements.upstreamsText.value = (item.upstreams || []).join("\n"); setScoped(form, scope, "lbPolicy", item.lbPolicy || "random");
File diff suppressed because one or more lines are too long
+3 -1
View File
@@ -64,6 +64,8 @@ h2{letter-spacing:-.025em}
.card-footer{position:absolute;left:20px;right:20px;bottom:20px}
.status-pill{display:flex;align-items:center;gap:var(--space-2);text-transform:capitalize;font-size:var(--font-size-sm);color:var(--muted)}
.icon-button,.launch{width:34px;height:34px;border-radius:var(--radius-sm);border:1px solid var(--line);display:grid;place-items:center;background:var(--icon-button-bg);color:var(--muted);cursor:pointer;text-decoration:none}
.page-refresh{width:44px;height:44px;flex:0 0 auto}
.header-actions{display:flex;align-items:center;gap:var(--space-5)}
.menu-wrap{position:relative}
.menu{display:none;position:absolute;right:0;top:var(--space-7);width:145px;background:var(--surface-raised);border:1px solid var(--line);border-radius:var(--radius-2xs);padding:6px;box-shadow:var(--shadow);z-index:3}
.menu-open .menu{display:block}
@@ -149,7 +151,7 @@ header{align-items:flex-end}
/* Dashboard */
.mobile-nav{display:none}
.dashboard-view{margin-top:38px}
#certificates-view,#performance-view,#logs-view{margin-top:var(--space-7)}
#certificates-view,#performance-view,#logs-view,#account-view,#documentation-view{margin-top:var(--space-7)}
.metric-grid{display:grid;grid-template-columns:repeat(3,minmax(0,1fr));gap:var(--space-4)}
.metric-card{min-width:0;padding:20px;border:1px solid var(--line);border-radius:var(--radius-2xl);background:linear-gradient(145deg,rgba(var(--panel2-rgb),.95),rgba(var(--card-shade-rgb),.95));color:var(--text);text-align:left;position:relative;overflow:hidden}
.metric-card::before{content:"";position:absolute;inset:0 0 auto 0;height:3px;background:var(--card-accent,var(--green));opacity:.85}
+41 -19
View File
@@ -193,22 +193,31 @@ async function sampleNetworkInterfaces() {
}
setInterval(sampleNetworkInterfaces, 5000).unref();
sampleNetworkInterfaces();
// A recursive walk of /data (directorySize()) is only needed when DATA_DIR_LIMIT_GB is set, and
// only to compute one denominator-relative percentage -- disk usage doesn't change fast enough to
// justify redoing that walk on every single hero-panel poll (every 7 seconds, times every
// concurrent viewer). Cached in the background instead, same pattern as refreshDatabaseIntegrityCache()
// above: compute once shortly after boot, then on a steady interval, and have the hot request path
// just read the cached number.
let dataDirSizeCache = { checkedAt: null, bytes: null };
async function refreshDataDirSizeCache() {
try { dataDirSizeCache = { checkedAt: new Date().toISOString(), bytes: await directorySize(dataDir) }; }
catch (error) { console.warn("Could not compute data directory size:", error.message); }
}
// One combined snapshot for the System tab's hero panel -- CPU/memory/swap/network are all
// container-scoped (cgroup v2 + this container's network namespace); disk reuses the same
// statfs-on-the-data-volume approach as /api/system/storage.
async function systemHealthSnapshot() {
const assignedLimitGb = numberEnv("DATA_DIR_LIMIT_GB", null);
const assignedLimitBytes = assignedLimitGb && assignedLimitGb > 0 ? assignedLimitGb * 1024 ** 3 : null;
const [cpu, memory, swap, disk, appUsedBytes] = await Promise.all([
const [cpu, memory, swap, disk] = await Promise.all([
cgroupCpuPercent(),
cgroupMemory(),
cgroupSwap(),
fsp.statfs(dataDir).catch(() => null),
// Only walk /data (the same directorySize() the storage breakdown below already uses) when
// an assigned limit is actually configured -- it's the one case that needs it, and the walk
// isn't free, so skip it when the panel is just going to show whole-volume stats anyway.
assignedLimitBytes !== null ? directorySize(dataDir) : Promise.resolve(null),
]);
// See refreshDataDirSizeCache() above -- this used to be a live directorySize() walk on every fetch.
const appUsedBytes = assignedLimitBytes !== null ? dataDirSizeCache.bytes : null;
return {
cpu,
memory,
@@ -544,7 +553,6 @@ function applyAdvancedSettings(item, body) {
if (body.accessListId !== undefined) item.accessListId = String(body.accessListId || "");
if (body.compression !== undefined) item.compression = ["off", "gzip", "automatic"].includes(body.compression) ? body.compression : "automatic";
if (body.hstsSubdomains !== undefined) item.hstsSubdomains = Boolean(body.hstsSubdomains);
if (body.blockCommonExploits !== undefined) item.blockCommonExploits = Boolean(body.blockCommonExploits);
if (body.requestHeaders !== undefined) item.requestHeaders = cleanHeaders(body.requestHeaders);
if (body.responseHeaders !== undefined) item.responseHeaders = cleanHeaders(body.responseHeaders);
if (body.upstreamTlsServerName !== undefined) item.upstreamTlsServerName = String(body.upstreamTlsServerName || "").trim().slice(0, 253);
@@ -603,19 +611,8 @@ function accessDirectives(accessListId) {
return output;
}
// Static, general-purpose ruleset for the "Block common exploits" toggle — not a full WAF. Rejects
// requests whose path matches common exploit-probe patterns before they reach the upstream: directory
// traversal, WordPress/PHP admin and scanner paths, dotfile exposure attempts, and SQL-injection-style
// query strings. One named matcher + one respond directive per host, so it's cheap to add or remove.
const COMMON_EXPLOIT_PATTERN = String.raw`(?i)(\.\./|\.\.\\|/etc/passwd|/wp-login\.php|/wp-admin(?:/|$)|/xmlrpc\.php|/\.env(?:$|\?)|/\.git/|/\.aws/|/vendor/phpunit|/phpunit(?:/|$)|eval\(|base64_decode\(|union(?:\s|%20|\+)+select|<script)`;
function exploitBlockDirectives(id) {
return [` @blocked-exploit-${id} {`, ` path_regexp ${caddyQuote(COMMON_EXPLOIT_PATTERN)}`, " }", ` respond @blocked-exploit-${id} 403`];
}
function commonHostDirectives(item) {
const output = [...accessDirectives(item.accessListId)];
if (item.blockCommonExploits) output.push(...exploitBlockDirectives(item.id));
if (item.compression !== "off") output.push(item.compression === "gzip" ? " encode gzip" : " encode zstd gzip");
for (const header of item.responseHeaders || []) output.push(` header ${header.name} ${caddyQuote(header.value)}`);
if (item.hsts && item.tls !== "http") output.push(` header Strict-Transport-Security ${caddyQuote(`max-age=31536000${item.hstsSubdomains ? "; includeSubDomains" : ""}`)}`);
@@ -807,6 +804,25 @@ async function syncCaddy() {
let configDrift = { checkedAt: null, drift: false, detail: null };
// storage.integrity() runs a full PRAGMA integrity_check -- a complete scan of the entire SQLite
// database file for corruption. It's one of the most expensive operations SQLite can run, its
// cost scales with total database size, and because this app's SQLite queries run synchronously,
// it blocks the whole single-threaded server for its full duration while it runs -- not just the
// request that triggered it. dashboardSnapshot() used to call it on EVERY /api/dashboard fetch
// just to compute one cosmetic "Healthy"/"Needs attention" label, which is why unrelated requests
// (confirmed via container logs: /api/system/security, /api/logs/prune/preview) were getting
// stuck behind it in lockstep, all finishing at nearly the same multi-second mark regardless of
// what they actually needed to do. A dashboard status badge doesn't need a fresh, exhaustive
// integrity scan on every single poll -- checking it periodically in the background and caching
// the result is more than sufficient, since real corruption doesn't appear and disappear between
// one 7-second poll and the next.
let databaseIntegrityCache = { checkedAt: null, status: "Healthy" };
function refreshDatabaseIntegrityCache() {
try {
const result = storage.integrity();
databaseIntegrityCache = { checkedAt: new Date().toISOString(), status: result.length === 1 && result[0] === "ok" ? "Healthy" : "Needs attention" };
} catch (error) { console.warn("Database integrity check failed:", error.message); }
}
let lastUpstreamCheckAt = null;
let lastAccessLogImportAt = null;
let lastKnownGoodCaddyConfig = null;
@@ -1130,7 +1146,7 @@ async function dashboardSnapshot(precomputedCertificates) {
for (const certificate of certificates.certificates.filter(item => ["warning", "critical", "expired", "mismatch"].includes(item.status))) attention.push({ kind: "certificate", target: "certificates", name: certificate.domain, message: certificate.status === "expired" ? "Certificate has expired." : certificate.status === "mismatch" ? "The uploaded certificate does not cover this domain." : `Certificate expires in ${certificate.daysRemaining} day${certificate.daysRemaining === 1 ? "" : "s"}.` });
if (configDrift.drift) attention.push({ kind: "drift", name: "Configuration drift", message: "Caddy\u2019s live configuration no longer matches the saved configuration.", target: "administration/defaults" });
const disk = await fsp.statfs(dataDir).catch(() => null);
const databaseIntegrity = storage.integrity();
// See refreshDatabaseIntegrityCache() above -- this used to be a live storage.integrity() call on every fetch.
return {
checkedAt: new Date().toISOString(),
gateway: { ...gatewayProbe, lastReload: lastGatewayReload },
@@ -1157,7 +1173,7 @@ async function dashboardSnapshot(precomputedCertificates) {
caddyVersion,
nodeVersion: process.version,
databaseEngine: "SQLite",
databaseStatus: databaseIntegrity.length === 1 && databaseIntegrity[0] === "ok" ? "Healthy" : "Needs attention",
databaseStatus: databaseIntegrityCache.status,
databaseBytes: (await fsp.stat(storage.databasePath).catch(() => null))?.size || 0,
publicIp: publicIpState.address,
publicIpCheckedAt: publicIpState.checkedAt,
@@ -2579,6 +2595,12 @@ setTimeout(() => cleanupOldPruneSnapshots().catch(error => console.warn("Startup
setInterval(() => checkAllProxies().then(() => { lastUpstreamCheckAt = new Date().toISOString(); }).catch(error => console.warn("Upstream checks failed:", error.message)), 60000).unref();
setTimeout(() => checkConfigDrift().catch(error => console.warn("Config drift check failed:", error.message)), 10000).unref();
setInterval(() => checkConfigDrift().catch(error => console.warn("Config drift check failed:", error.message)), 10 * 60000).unref();
// Runs the (expensive, synchronous, whole-server-blocking) database integrity scan once shortly
// after boot and then every 30 minutes in the background, rather than on every dashboard fetch.
setTimeout(refreshDatabaseIntegrityCache, 5000).unref();
setInterval(refreshDatabaseIntegrityCache, 30 * 60000).unref();
setTimeout(refreshDataDirSizeCache, 5000).unref();
setInterval(refreshDataDirSizeCache, 60000).unref();
// --- Scheduled jobs: automatic backups, log pruning, public IP checks, graceful shutdown ---------------------------------