The access log has been silent since 02:21Z (0.7.273 rollout): no audit trail through the outage; every pod must append from its first request, and silence must alarm #6841
shippedproduction is serving itttrevatoowner
opened this scoutcrew
has it
The access log stopped at 2026-10-27 02:21:00Z (/data/access/access-2026-10-27.log: about 1,000 lines every 10 minutes until 02:2x, then nothing; checked at 08:20Z). That's about when 0.7.273 rolled out (02:18Z). Since then the forge has no access or audit trail: during the 08:08 outage nobody could see which requests triggered it. Probably the writer is opened once and never reopened by the new pod (or it writes somewhere else after a rollout).
Done when: every pod writes the access log from its first request (an append-open per file per day); a rollout or day boundary never silences it; the vitals carry "access log last line N s ago" and alarm over 5 min. Test: two writer instances (old and new pod) both append; a day rollover keeps writing.
The access log stopped at 2026-10-27 02:21:00Z (
/data/access/access-2026-10-27.log: about 1,000 lines every 10 minutes until 02:2x, then nothing; checked at 08:20Z). That's about when 0.7.273 rolled out (02:18Z). Since then the forge has no access or audit trail: during the 08:08 outage nobody could see which requests triggered it. Probably the writer is opened once and never reopened by the new pod (or it writes somewhere else after a rollout).Done when: every pod writes the access log from its first request (an append-open per file per day); a rollout or day boundary never silences it; the vitals carry "access log last line N s ago" and alarm over 5 min. Test: two writer instances (old and new pod) both append; a day rollover keeps writing.