feat(builder): repair existing templates, a weekly check that queues repairs, and eval
Validate game templates / validate (push) Successful in 36s
Validate game templates / boot-test (push) Successful in 1m11s
Validate game templates / publish (push) Skipped

The builder could only create games; nothing noticed when a live template
stopped working, and there was no way to tell whether a builder change made
it better or worse.

- repair -id ID: boots an existing template as nodes run it; if it fails,
  the same investigation as for a draft runs and the model changes only what
  the diagnosis says. The worker handles "repair" jobs (gameId from the
  panel) and opens "fix(games): repair ..." on draft/<id>; a template that
  boots as it is reports "nothing to repair" and opens no PR. The boot and
  fix loop is now shared by builds and repairs.
- validate.yml runs weekly and boot-tests every game; failed games are sent
  to the panel (X-GBR-Event: games_boot_failed) to queue repairs.
- eval -ids all: builds games that already have a template from scratch,
  blind, and scores each draft against it (boots, same image and app, ports,
  ready line, settings found) into eval.md and eval.json.
- research skips our own sites, so neither research nor eval reads our
  templates back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
gregandClaude Opus 5.5 committed 2026-10-26 00:42:34 -04:00
1 parent f11c5eaf2f
commit f1271341ef
12 files changed
+535 -20

No files matched your search

+17
View File
@@ -42,6 +42,23 @@ example is an invented server). `TestBuilderIsGameAgnostic` fails the build if o
The worker only connects out (to the panel, Gitea, the model and the search services).
## Keeping games working
- **Repair** (`gbrgames repair -id ID`): boots an existing template as nodes run it. If it boots, there is
nothing to do. If not, the same investigation as for a draft runs, the model changes only what the
diagnosis says, and the worker opens `fix(games): repair …` on `draft/<id>` (replacing any open draft of
that game). The panel queues repairs from a game's admin page, or by itself (next point).
- **Weekly check**: `validate.yml` boot-tests every game each Monday. Each game that no longer boots is sent
to the panel (`X-GBR-Event: games_boot_failed`), which queues a repair job for it.
- **Eval** (`gbrgames eval -ids all`): builds games that already have a template from scratch, blind (the
builder never reads `games/`, and research skips our own sites), and scores each draft against the merged
template: boots, same image and Steam app, same ports, ready line, settings found. Run it before and after
changing the builder or the model. It takes as long as the builds; on the builder VM:
```bash
docker exec gbr-builder sh -c 'cd /work && rm -rf eval && git clone -q https://www.gitea.gbreed.com/gbreed-public/gbr-games.git eval && cd eval && gbrgames eval -ids all -out eval.md && cat eval.md'
```
## Requirements
A Docker host with the isolated `gbr-boottest` network (`RUNNER.md`, step 1), so servers booted by the