feat(builder): repair existing templates, a weekly check that queues repairs, and eval
The builder could only create games; nothing noticed when a live template stopped working, and there was no way to tell whether a builder change made it better or worse. - repair -id ID: boots an existing template as nodes run it; if it fails, the same investigation as for a draft runs and the model changes only what the diagnosis says. The worker handles "repair" jobs (gameId from the panel) and opens "fix(games): repair ..." on draft/<id>; a template that boots as it is reports "nothing to repair" and opens no PR. The boot and fix loop is now shared by builds and repairs. - validate.yml runs weekly and boot-tests every game; failed games are sent to the panel (X-GBR-Event: games_boot_failed) to queue repairs. - eval -ids all: builds games that already have a template from scratch, blind, and scores each draft against it (boots, same image and app, ports, ready line, settings found) into eval.md and eval.json. - research skips our own sites, so neither research nor eval reads our templates back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
00cc61aba2
commit
17f6477b71
12 files changed
+535
-20
No files matched your search
+17
@@ -42,6 +42,23 @@ example is an invented server). `TestBuilderIsGameAgnostic` fails the build if o
|
||||
|
||||
The worker only connects out (to the panel, Gitea, the model and the search services).
|
||||
|
||||
## Keeping games working
|
||||
|
||||
- **Repair** (`gbrgames repair -id ID`): boots an existing template as nodes run it. If it boots, there is
|
||||
nothing to do. If not, the same investigation as for a draft runs, the model changes only what the
|
||||
diagnosis says, and the worker opens `fix(games): repair …` on `draft/<id>` (replacing any open draft of
|
||||
that game). The panel queues repairs from a game's admin page, or by itself (next point).
|
||||
- **Weekly check**: `validate.yml` boot-tests every game each Monday. Each game that no longer boots is sent
|
||||
to the panel (`X-GBR-Event: games_boot_failed`), which queues a repair job for it.
|
||||
- **Eval** (`gbrgames eval -ids all`): builds games that already have a template from scratch, blind (the
|
||||
builder never reads `games/`, and research skips our own sites), and scores each draft against the merged
|
||||
template: boots, same image and Steam app, same ports, ready line, settings found. Run it before and after
|
||||
changing the builder or the model. It takes as long as the builds; on the builder VM:
|
||||
|
||||
```bash
|
||||
docker exec gbr-builder sh -c 'cd /work && rm -rf eval && git clone -q https://www.gitea.gbreed.com/gbreed-public/gbr-games.git eval && cd eval && gbrgames eval -ids all -out eval.md && cat eval.md'
|
||||
```
|
||||
|
||||
## Requirements
|
||||
|
||||
A Docker host with the isolated `gbr-boottest` network (`RUNNER.md`, step 1), so servers booted by the
|
||||
|
||||
Reference in new issue
Block a user