Public: anyone can read this; the crew can change it.
Hardware health: know each machine, its spec sheet, and real wear against the ratings #10363
I added a hardware page at /admin/hardware (site admins only) that shows each disk with the vendor's ratings beside the drive's own SMART numbers.
stopped parked by the crew deploy dc94ebf
3 earlier comments
I added a hardware page at /admin/hardware (site admins only) that shows each disk with the vendor's ratings beside the drive's own SMART numbers. bun x tsc --noEmit, bun scripts/gate.ts, bun scripts/design-lint.ts and the tests I ran (the new hardware tests plus the admin-gate, route-timing, migration and settings tests) all pass. I did not run the full suite, because scripts/fast.ts named 1,589 test files for this diff. I haven't seen the page on a preview or against a real drive.
What the page shows
- CPU and memory, then one line per disk. A healthy NVMe drive is one quiet line with a "healthy" badge and "550 GB of 600 TB (0.092%)". Opening it shows every figure from the issue with its rating beside it: percentage used, spare against its threshold, media errors, critical warning bits, temperature against 82/85 °C, minutes above warning and critical, power-on hours, unsafe shutdowns.
- A drive that reports a problem is badged and opens by default.
- A PERC virtual disk is shown as "behind a controller; SMART not visible" and is never counted as an SSD.
/admin/hardware.jsonreturns the same data. The page is reachable from Account settings → This forge → Hardware.
Alerts Only what the drive reports as bad raises one:
- any critical-warning bit set
- spare below its threshold
- percentage used of 80% or more
- media errors rising
- time above the critical temperature rising
"Rising" compares with the previous stored reading, so a single reading never alerts on a count. A hot, busy or high-unsafe-shutdown drive stays a quiet number.
Ratings and projection
- The spec catalog is checked in and keyed by model. It holds only the 980 PRO 1TB: 600 TBW, 5-year warranty, 82/85 °C. Any other model shows "no rating", never a guess.
- The projection takes the readings from the last 30 days, works out bytes per day, and gives the date the remaining rated TBW runs out. It says "needs a day of readings" until there is a day of history.
Where readings come from
- Inventory is read from
/sysand/proc. - Once an hour (first run 30 seconds after boot) the forge runs
smartctl -j -a /dev/nvmeN. The device name comes from the kernel's own list, never from a request. Readings are stored in a newhardware_readingtable for 90 days, so the page never waits on smartctl. - This host has no smartctl, so the page says so. An admin can paste
smartctl -j -aoutput under "Add a reading" to store a reading, which is how today's numbers could go in. Until a reading exists, the NVMe line says it has no reading yet.
Tests (test/hardware.test.ts) cover recorded 980 PRO logs for healthy, warm-but-fine, worn, hot and failing drives. They also check that a missing field is refused rather than read as zero, and that the projection matches a hand calculation (1 TB/day from 550 GB reaches 600 TB in 599.45 days). The PERC inventory and the rendered rows have tests too.
Left out
- Only NVMe SMART is read. SATA/SAS SMART and the RAID controller's own view (storcli) are not.
- There is no alerting channel yet; alerts show on the page only.
- No chart of wear over time.
To try it
Open /admin/hardware on the preview as an admin and paste a smartctl -j -a /dev/nvme0 output. How it works is written up in docs/features/hardware.md.
The check failed because no commit on the pull had a "Why now:" line, which the gate needs when a pull adds src/web/pages/hardware.ts. I added an empty commit carrying that line and pushed it. No code changed.
I didn't re-run the tests, since only a commit message changed. bun scripts/gate.ts printed nothing, which is how it reported clean last time too. I haven't seen the forge's check re-run on the new push.
The Why now line says the owner has NVMe wear warnings at work and needs the forge to say plainly, without false positives, whether the vxrail's drives are being worn out.
The pull description is unchanged from my previous message.
Hardware health (#10352) now fixes three of the four failing checks. The fourth, "each page stays inside its first-view budget", I could not find, so the pushed commit may still fail it.
I fixed these three:
- Design audit:
/admin/hardwareand/admin/hardware.jsonnow have entries indocs/design/audit.md. - Hollow-work check:
/admin/hardware.jsonis now linked from the hardware page ("This page as JSON"), so it is no longer a screen no page links to. - Piping spawns:
src/hardware/collect.tsis now listed intest/spawn-pipes.test.ts, with the reason: it reads smartctl's stdout whole under a 10 s kill timer and ignores stderr.
Locally the audit, spawn-pipes, hollow, hardware and server-timing-sweep tests pass, as do tsc --noEmit and design-lint.
I searched the repository for "each page stays inside its first-view budget" and found no test with that name. web-budget, perf-budget-fire and mcp-size all pass here. Nothing in this change touches page weight beyond one added link line. If it fails again on the forge, its log will name the file.
I didn't run scripts/gate.ts or the full suite here.
What changed in the pull: /admin/hardware shows each machine's CPU, memory and disks. It sets the vendor's ratings against real NVMe wear, and raises alerts only on what a drive reports as bad. Hosts without smartctl can paste a smartctl -j -a reading into the "Add a reading" form.
Not done: no new dependencies, and nothing reads disks without smartctl or a pasted reading.
Try it: open /admin/hardware as a site admin.
Parked by the crew: the check stayed red through 2 automatic tries
I added a hardware page at
/admin/hardware(site admins only) that shows each disk with the vendor's ratings beside the drive's own SMART numbers.bun x tsc --noEmit,bun scripts/gate.ts,bun scripts/design-lint.tsand the tests I ran (the new hardware tests plus the admin-gate, route-timing, migration and settings tests) all pass. I did not run the full suite, becausescripts/fast.tsnamed 1,589 test files for this diff. I haven't seen the page on a preview or against a real drive.What the page shows
/admin/hardware.jsonreturns the same data. The page is reachable from Account settings → This forge → Hardware.Alerts Only what the drive reports as bad raises one:
"Rising" compares with the previous stored reading, so a single reading never alerts on a count. A hot, busy or high-unsafe-shutdown drive stays a quiet number.
Ratings and projection
Where readings come from
/sysand/proc.smartctl -j -a /dev/nvmeN. The device name comes from the kernel's own list, never from a request. Readings are stored in a newhardware_readingtable for 90 days, so the page never waits on smartctl.smartctl -j -aoutput under "Add a reading" to store a reading, which is how today's numbers could go in. Until a reading exists, the NVMe line says it has no reading yet.Tests (
test/hardware.test.ts) cover recorded 980 PRO logs for healthy, warm-but-fine, worn, hot and failing drives. They also check that a missing field is refused rather than read as zero, and that the projection matches a hand calculation (1 TB/day from 550 GB reaches 600 TB in 599.45 days). The PERC inventory and the rendered rows have tests too.Left out
To try it Open
/admin/hardwareon the preview as an admin and paste asmartctl -j -a /dev/nvme0output. How it works is written up indocs/features/hardware.md.Closes #10352