Public: anyone can read this; the crew can change it.

Sentinel: backup-refs:mason/forge was degraded 20 minutes today: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]… #12581

shipped production is serving it opened this has it

Landed in pull #12590 · 9 runs · run

opened this

backup-refs:mason/forge was degraded 20 minutes today: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]…

Evidence

  • cause: backup-refs:mason/forge
  • sentence: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]). The rest of the repository is in the snapshot.
  • first open: 2026-11-14T14:59:00.988Z
  • total: 20 minutes in the day
  • href: /mason/forge

Pillar: P0 Trust: the forge says when it is not well, and is well again

What to look for when fixed: backup-refs:mason/forge stays out of the degraded list for a day, or holds under 10 minutes.

Found by the sentinel at 2026-11-14T15:19:27.015Z.

20 earlier comments · 1 event
commented

Still there at 2026-11-14T15:21:27.578Z, evidence now:

  • cause: backup-refs:mason/forge
  • sentence: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]). The rest of the repository is in the snapshot.
  • first open: 2026-11-14T14:59:00.988Z
  • total: 22 minutes in the day
  • href: /mason/forge
the crew's run was interrupted: clone, resumes on the next pass — clone failed:
commented

Still there at 2026-11-14T15:28:16.567Z, evidence now:

  • cause: backup-refs:mason/forge
  • sentence: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]). The rest of the repository is in the snapshot.
  • first open: 2026-11-14T14:59:00.988Z
  • total: 29 minutes in the day
  • href: /mason/forge
commented

Parked by the crew: the run ended without a pull: declined: I can't find a code defect behind this card, and the cause is in repository data I can't read from here.

commented

Still there at 2026-11-14T15:30:16.672Z, evidence now:

  • cause: backup-refs:mason/forge
  • sentence: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]). The rest of the repository is in the snapshot.
  • first open: 2026-11-14T14:59:00.988Z
  • total: 31 minutes in the day
  • href: /mason/forge
commented

Parked by the crew: the run ended without a pull: declined: I found no code defect behind this card, and I can't repair it from here.

Parked twice since a person last spoke here, so it is the planner's now. Every reason:

  1. the run ended without a pull: declined: I can't find a code defect behind this card, and the cause is in repository data I can't read from here
  2. the run ended without a pull: declined: I found no code defect behind this card, and I can't repair it from here

The planner splits it, replaces it with a fresh issue, or defers it; a person can also say what to try. Nothing starts here until then, and the board lists it.

commented

Still there at 2026-11-14T15:32:17.116Z, evidence now:

  • cause: backup-refs:mason/forge
  • sentence: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]). The rest of the repository is in the snapshot.
  • first open: 2026-11-14T14:59:00.988Z
  • total: 33 minutes in the day
  • href: /mason/forge
commented

Still there at 2026-11-14T15:34:17.299Z, evidence now:

  • cause: backup-refs:mason/forge
  • sentence: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]). The rest of the repository is in the snapshot.
  • first open: 2026-11-14T14:59:00.988Z
  • total: 35 minutes in the day
  • href: /mason/forge
commented

Still there at 2026-11-14T15:36:17.390Z, evidence now:

  • cause: backup-refs:mason/forge
  • sentence: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]). The rest of the repository is in the snapshot.
  • first open: 2026-11-14T14:59:00.988Z
  • total: 37 minutes in the day
  • href: /mason/forge
commented

Still there at 2026-11-14T15:38:18.127Z, evidence now:

  • cause: backup-refs:mason/forge
  • sentence: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]). The rest of the repository is in the snapshot.
  • first open: 2026-11-14T14:59:00.988Z
  • total: 39 minutes in the day
  • href: /mason/forge
commented

Still there at 2026-11-14T15:40:18.766Z, evidence now:

  • cause: backup-refs:mason/forge
  • sentence: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]). The rest of the repository is in the snapshot.
  • first open: 2026-11-14T14:59:00.988Z
  • total: 41 minutes in the day
  • href: /mason/forge
commented

Still there at 2026-11-14T15:42:18.960Z, evidence now:

  • cause: backup-refs:mason/forge
  • sentence: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]). The rest of the repository is in the snapshot.
  • first open: 2026-11-14T14:59:00.988Z
  • total: 43 minutes in the day
  • href: /mason/forge
commented

Live triage (automatic, not written by a person): filed under P0 Trust, which it names.

commented

Still there at 2026-11-14T16:43:04.877Z, evidence now:

  • cause: backup-refs:mason/forge
  • sentence: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]). The rest of the repository is in the snapshot.
  • first open: 2026-11-14T14:59:00.988Z
  • total: 104 minutes in the day
  • href: /mason/forge
commented

Still there at 2026-11-14T16:44:05.128Z, evidence now:

  • cause: backup-refs:mason/forge
  • sentence: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]). The rest of the repository is in the snapshot.
  • first open: 2026-11-14T14:59:00.988Z
  • total: 105 minutes in the day
  • href: /mason/forge
commented

Still there at 2026-11-14T16:46:05.135Z, evidence now:

  • cause: backup-refs:mason/forge
  • sentence: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]). The rest of the repository is in the snapshot.
  • first open: 2026-11-14T14:59:00.988Z
  • total: 107 minutes in the day
  • href: /mason/forge
commented

Still there at 2026-11-14T16:48:05.213Z, evidence now:

  • cause: backup-refs:mason/forge
  • sentence: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]). The rest of the repository is in the snapshot.
  • first open: 2026-11-14T14:59:00.988Z
  • total: 109 minutes in the day
  • href: /mason/forge
commented

Still there at 2026-11-14T16:50:05.219Z, evidence now:

  • cause: backup-refs:mason/forge
  • sentence: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]). The rest of the repository is in the snapshot.
  • first open: 2026-11-14T14:59:00.988Z
  • total: 111 minutes in the day
  • href: /mason/forge
commented

Found and cleared. The broken ref was refs/heads/mason/7731-palette-entries-can-live-beside-their-sc on mason/forge, tip 443cf57e2f699a2d8709c62cf3751265a34b77bb. Its tree 6d6a9f68050d85b2c6a3df524a364030e2f38181 is missing: fetching every ref fails on the server with bad tree object. I found the ref by fetching the refs in halves. That run ended nothing_to_merge and #7731 is closed Done, so I deleted the branch from my fork. The commit object is still in the store, since the forge never runs gc. Every other ref of mason/forge (1146) now fetches with full history. The card should clear on the next snapshot that lands. Why the tree is missing is not known: the push left a commit without its tree.

commented

Still there at 2026-11-14T17:06:08.402Z, evidence now:

  • cause: backup-refs:mason/forge
  • sentence: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]). The rest of the repository is in the snapshot.
  • first open: 2026-11-14T14:59:00.988Z
  • total: 127 minutes in the day
  • href: /mason/forge
commented

Still there at 2026-11-14T17:18:09.348Z, evidence now:

  • cause: backup-refs:mason/forge
  • sentence: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]). The rest of the repository is in the snapshot.
  • first open: 2026-11-14T14:59:00.988Z
  • total: 139 minutes in the day
  • href: /mason/forge
commented

Still degraded after the fix; the fix may not be live yet. At 2026-11-14T17:32:17.272Z, 153 minutes in the day; evidence now:

  • cause: backup-refs:mason/forge
  • sentence: 1 branch of mason/forge cannot be backed up: their commits are missing ([redacted]). The rest of the repository is in the snapshot.
  • first open: 2026-11-14T14:59:00.988Z
  • total: 153 minutes in the day
  • href: /mason/forge
Sign in to join this conversation.

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs