feat(builder): research, build and a worker for the AI builder #6

Merged
greg merged 3 commits from feat/builder-worker into main 2026-10-25 20:22:46 +00:00
Owner

The AI builder as code in this repo instead of n8n workflows: a self-hosted model does the drafting, real boots check the facts, and every result is a pull request that CI validates and boot-tests like any other. See BUILDER.md.

Commands

  • research -name N: SearXNG search → reads the best pages with a reader → the model lists the server's variants (images, Steam app, startup, ports, notes, sources) → every image (docker manifest inspect) and Steam app (anonymous steamcmd +app_info_print, on the isolated network) is checked before an admin sees it.
  • build -name N -variant @v.json: first boot of each candidate image, skipping images that need root (imageUnusable) → draft → the draft deployed with the node-agent's BootTest (the same check as CI) → crashes, error lines and saves still written outside /server go back to the same model conversation, three boots at most.
  • worker: polls the panel's builder queue (outbound only), runs research/build in a fresh clone, streams output as the job log (heartbeat; the panel can cancel), opens draft/<id>-<job> as a PR from the bot account; WIP: if it never booted.

Also: Dockerfile.builder and .gitea/workflows/builder-image.yml (publishes gbr-builder:latest when cmd/ changes). All configuration is environment variables; deployment files are kept outside this public repo. draft's loop now lives in a drafter that keeps the conversation, so build can feed boot failures back.

Checked for real: research -name Valheim found Steam app 896660 ("Valheim Dedicated Server", verified), three images (all exist), ports 2456–2457/udp, and -crossplay. build with that variant rejected lloesche and community-valheim-tools within seconds (both need root) and continued with cm2network/steamcmd. The full build result follows in a comment.

🤖 Generated with Claude Code

The AI builder as code in this repo instead of n8n workflows: a self-hosted model does the drafting, real boots check the facts, and every result is a pull request that CI validates and boot-tests like any other. See **BUILDER.md**. **Commands** - `research -name N`: SearXNG search → reads the best pages with a reader → the model lists the server's variants (images, Steam app, startup, ports, notes, sources) → every image (`docker manifest inspect`) and Steam app (anonymous `steamcmd +app_info_print`, on the isolated network) is checked before an admin sees it. - `build -name N -variant @v.json`: first boot of each candidate image, skipping images that need root (`imageUnusable`) → `draft` → the draft deployed with the node-agent's `BootTest` (the same check as CI) → crashes, error lines and saves still written outside `/server` go back to the same model conversation, three boots at most. - `worker`: polls the panel's builder queue (outbound only), runs research/build in a fresh clone, streams output as the job log (heartbeat; the panel can cancel), opens `draft/<id>-<job>` as a PR from the bot account; `WIP:` if it never booted. **Also:** `Dockerfile.builder` and `.gitea/workflows/builder-image.yml` (publishes `gbr-builder:latest` when `cmd/` changes). All configuration is environment variables; deployment files are kept outside this public repo. `draft`'s loop now lives in a `drafter` that keeps the conversation, so `build` can feed boot failures back. **Checked for real:** `research -name Valheim` found Steam app 896660 ("Valheim Dedicated Server", verified), three images (all exist), ports 2456–2457/udp, and `-crossplay`. `build` with that variant rejected lloesche and community-valheim-tools within seconds (both need root) and continued with cm2network/steamcmd. The full build result follows in a comment. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Member

Automated AI Review

Here's some quick feedback on your PR:

>_Automated AI Review_ Here's some quick feedback on your PR:
greg added 1 commit 2026-10-25 16:51:32 +00:00
feat(builder): research, build and a worker for the AI builder
Validate game templates / validate (push) Successful in 19s
Validate game templates / boot-test (push) Skipped
Validate game templates / publish (push) Skipped
c75190d8c4
- research: searches SearXNG, reads the best pages through the reader, has the
  model list the server's variants, then checks every Docker image and Steam
  app it named for real before an admin sees it.
- build: first boot of the variant's images (skipping ones that need root),
  a draft by the model, then the draft deployed with the node-agent's
  BootTest like CI does; crashes, error lines and saves outside /server go
  back to the model, three boots at most.
- worker: polls the panel's builder queue (outbound only), runs research or
  build in a fresh clone, streams the output as the job log, and opens the
  result as a pull request from the bot account.
- Dockerfile.builder, a workflow publishing gbr-builder:latest, and
  BUILDER.md. Everything about where it runs comes from environment
  variables; deployment files live outside this public repo.

draft now keeps its conversation in a drafter, so build can send boot
failures back to the same model session. The boot-test runner comment no
longer names its VM.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
greg force-pushed feat/builder-worker from dcb07cfed7 to c75190d8c4 2026-10-25 16:51:32 +00:00 Compare
greg changed title from feat(builder): research, build and a worker for the AI builder on our own hardware to feat(builder): research, build and a worker for the AI builder 2026-10-25 16:52:00 +00:00
Member

Automated AI Review

Here's some quick feedback on your PR:

  • Security: In openPR (worker.go), the Gitea bot token is passed as a git -c http.extraHeader=Authorization: token <TOKEN> CLI argument, exposing it in the process table (/proc/*/cmdline, ps) to any local user.
  • Logic: In build() (build.go), res.Problems = nil after a successful first-boot selection silently discards the diagnostic reasons why earlier candidate images failed, losing useful context for the PR reviewer.
  • Reliability: In the worker's main loop, a failed POST /jobs/{id}/result is only logged and the worker moves on; the expensive build result is lost with no retry or local persistence, so the panel never learns the outcome.
>_Automated AI Review_ Here's some quick feedback on your PR: - **Security**: In `openPR` (worker.go), the Gitea bot token is passed as a `git -c http.extraHeader=Authorization: token <TOKEN>` CLI argument, exposing it in the process table (`/proc/*/cmdline`, `ps`) to any local user. - **Logic**: In `build()` (build.go), `res.Problems = nil` after a successful first-boot selection silently discards the diagnostic reasons why earlier candidate images failed, losing useful context for the PR reviewer. - **Reliability**: In the worker's main loop, a failed `POST /jobs/{id}/result` is only logged and the worker moves on; the expensive build result is lost with no retry or local persistence, so the panel never learns the outcome.
Author
Owner

End-to-end build: Valheim (research → build, run by hand, not through the worker yet)

  1. First boot: lloesche/valheim-server and community-valheim-tools/valheim-server were rejected after 3 s each (they need root, groupmod: Permission denied). It went on with cm2network/steamcmd and Steam app 896660.
  2. Draft: round 1 was rejected by the text check (saves written to ~/.config/unity3d/IronGate/Valheim, outside the volume). Round 2 passed the text checks.
  3. Boot 1: passed the node-agent BootTest (Loading: Done). But the second boot showed the server still writing worlds outside /server, and that went back to the model. This is the case text checks alone missed last time.
  4. Round 3 → boot 2: passed the BootTest, and the files stayed inside /server (it set HOME=/server). Done in 3 model rounds and 2 boots.

The result is a template with crossplay on by default, 2456–2457/udp, a measured readyPattern, and settings for name, password, world, crossplay and port.

What a reviewer would still have to catch (why it ends in a PR):

  • -name ${SERVER_NAME} inside an unquoted $ARGS gets split into words: "GBRHosting - My Server" reaches Valheim as just "GBRHosting".
  • GamePort is editable, but defaultPorts stays 2456–2457, so changing it would break joins.

Both look like good next deterministic checks (word-split startup arguments; port settings that don't match defaultPorts).

**End-to-end build: Valheim** (`research` → `build`, run by hand, not through the worker yet) 1. **First boot:** `lloesche/valheim-server` and `community-valheim-tools/valheim-server` were rejected after 3 s each (they need root, `groupmod: Permission denied`). It went on with `cm2network/steamcmd` and Steam app 896660. 2. **Draft:** round 1 was rejected by the text check (saves written to `~/.config/unity3d/IronGate/Valheim`, outside the volume). Round 2 passed the text checks. 3. **Boot 1:** passed the node-agent BootTest (`Loading: Done`). But the second boot showed the server **still** writing worlds outside `/server`, and that went back to the model. This is the case text checks alone missed last time. 4. **Round 3 → boot 2:** passed the BootTest, and the files stayed inside `/server` (it set `HOME=/server`). Done in 3 model rounds and 2 boots. The result is a template with crossplay on by default, 2456–2457/udp, a measured `readyPattern`, and settings for name, password, world, crossplay and port. **What a reviewer would still have to catch** (why it ends in a PR): - `-name ${SERVER_NAME}` inside an unquoted `$ARGS` gets split into words: "GBRHosting - My Server" reaches Valheim as just "GBRHosting". - `GamePort` is editable, but `defaultPorts` stays 2456–2457, so changing it would break joins. Both look like good next deterministic checks (word-split startup arguments; port settings that don't match `defaultPorts`).
Member

Automated AI Review

Here's some quick feedback on your PR:

>_Automated AI Review_ Here's some quick feedback on your PR:
greg requested review from git_ai_bot 2026-10-25 18:26:56 +00:00
greg left a comment
Author
Owner

🤖 AI Code Review

⚠️ The review feedback was empty or could not be generated.


Automated review by AI Git Bot

## 🤖 AI Code Review ⚠️ *The review feedback was empty or could not be generated.* --- *Automated review by AI Git Bot*
greg added 1 commit 2026-10-25 19:43:30 +00:00
feat(draft): catch split text settings and owner-editable game ports
Validate game templates / validate (push) Successful in 19s
Validate game templates / boot-test (push) Skipped
Validate game templates / publish (push) Skipped
c2b22d3955
Two problems a draft can boot fine with but break once an owner changes a
setting, both in the builder's Valheim draft:

- splitSettings: text settings used unquoted on the command line, directly
  or through ARGS="-name ${SERVER_NAME}"; ./server $ARGS, so "GBRHosting -
  My Server" reaches the game as several arguments. Suggests quoting or a
  bash array.
- portSettings: game-port settings whose default isn't in defaultPorts, or
  that owners can change although nodes always publish the same container
  port. Internal ports (RCON, REST) and {PUBLIC_PORT_n} values are left out.

Both run on drafts only; AUTHORING.md gets the matching rules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
git_ai_bot left a comment
Member

🤖 AI Code Review

1. Blocking issues

None that I can identify from the visible code. The logic is sound and the new checks (splitSettings, portSettings) are well-designed and well-tested.

One thing I cannot verify and would like you to confirm: the worker.go file (listed in the repo tree but not in the diff) — specifically:

  • That it cleans up the work directory between jobs (the newDrafter check "%s already exists" would fail on a re-run if the folder isn't removed).
  • That a panel-side cancellation actually kills the running docker subprocess (otherwise a cancelled job leaves a container running with the Docker socket mounted).

If those are handled, no blockers.

2. Non-blocking suggestions

AUTHORING.md is read from CWD (os.ReadFile("AUTHORING.md") in newDrafter). This works in the worker (WORKDIR /work) and in the documented go run examples, but breaks if someone invokes the binary from a different directory with -games /path/to/games. You already have -games and -schemas flags; consider a -rules flag or resolving it relative to the binary's directory. This is pre-existing behaviour, not a regression, so it's a low-priority cleanup.

isSteamCMDImage hardcodes cm2network/steamcmd. If the team ever publishes a self-hosted SteamCMD image (e.g. gbr-builder itself could pull one), the build would treat it as a plain image and skip the Steam-app path. A small env var override or a comment noting the assumption would save a future debugging session.

scanShell doesn't handle $(…) / backtick command substitution. If a startup command contains $(cat /server/config), the $ triggers the variable-expansion branch, varName fails to match (, and the character is silently skipped. The net effect is that any ${SETTING} inside a command substitution won't be flagged by splitSettings. This is an acceptable limitation for a lint check, but a brief comment on scanShell noting what it intentionally doesn't cover (subshells, $(…), process substitution) would help future maintainers understand the scope.

bootCheck invokes itself via os.Executable() to produce the boot-test payload. This works but is a bit unusual — a direct buildBootTestPayload(gamesDir, id) function call would avoid spawning a process and make the code path easier to unit-test. Not urgent, but worth considering.

portSettings — the internalPort regex is an unanchored substring match. A hypothetical setting named GameHttpPort would be silently excluded. Very unlikely in practice, but anchoring to word boundaries or to the end of the key ((?i)(rcon|rest|api|admin|telnet|web|http|metrics)_?(port|$)) would be more precise.

builder-image.yml — the workflow has no timeout-minutes. A hung docker build (e.g. registry unreachable) would sit in the queue indefinitely. A 15-minute timeout is cheap insurance.

3. Tests

Coverage for the new pure-logic functions is good (bootFailure, outsideDirs, splitSettings, portSettings, variantNotes). A few gaps worth filling:

  • scanShell / unwrapShell: no direct unit tests. The splitSettings tests cover integration, but a table-driven test for scanShell with tricky inputs (nested ${A:-${B}}, A="x $C y" where C is a text setting, bash -c 'nested "quotes"') would pin down the parser's behaviour and make it safer to extend later.
  • unwrapShell: the negative case (a command that looks like sh -c '…' but contains a single quote inside) isn't tested. The !strings.Contains(m[1], "'") guard means it falls through — confirm that's the intended fallback with a test.
  • build() image-selection loop: the path where all images are imageUnusable and the path where a SteamCMD image has no StartupHint are only exercised end-to-end. A small table test on a helper (if you extract the selection logic) would guard against regressions.

4. Overall assessment

This is a well-structured, well-documented feature. The separation of concerns (research → draft → boot → PR) is clean, the new lint checks (splitSettings, portSettings) are correctly scoped and well-tested, and the drafter refactor makes the model conversation reusable without duplicating state. The CI workflow is minimal and appropriate. The main risk area is the hand-rolled shell scanner, but its scope is narrow and the tests cover the important cases. No correctness or security blockers in the visible code; ship it with the minor cleanups above tracked as follow-ups.


Automated review by AI Git Bot

## 🤖 AI Code Review ## 1. Blocking issues None that I can identify from the visible code. The logic is sound and the new checks (`splitSettings`, `portSettings`) are well-designed and well-tested. One thing I **cannot verify** and would like you to confirm: the `worker.go` file (listed in the repo tree but not in the diff) — specifically: - That it cleans up the work directory between jobs (the `newDrafter` check `"%s already exists"` would fail on a re-run if the folder isn't removed). - That a panel-side cancellation actually kills the running `docker` subprocess (otherwise a cancelled job leaves a container running with the Docker socket mounted). If those are handled, no blockers. ## 2. Non-blocking suggestions **`AUTHORING.md` is read from CWD** (`os.ReadFile("AUTHORING.md")` in `newDrafter`). This works in the worker (WORKDIR `/work`) and in the documented `go run` examples, but breaks if someone invokes the binary from a different directory with `-games /path/to/games`. You already have `-games` and `-schemas` flags; consider a `-rules` flag or resolving it relative to the binary's directory. This is pre-existing behaviour, not a regression, so it's a low-priority cleanup. **`isSteamCMDImage` hardcodes `cm2network/steamcmd`.** If the team ever publishes a self-hosted SteamCMD image (e.g. `gbr-builder` itself could pull one), the build would treat it as a plain image and skip the Steam-app path. A small `env var` override or a comment noting the assumption would save a future debugging session. **`scanShell` doesn't handle `$(…)` / backtick command substitution.** If a startup command contains `$(cat /server/config)`, the `$` triggers the variable-expansion branch, `varName` fails to match `(`, and the character is silently skipped. The net effect is that any `${SETTING}` *inside* a command substitution won't be flagged by `splitSettings`. This is an acceptable limitation for a lint check, but a brief comment on `scanShell` noting what it intentionally doesn't cover (subshells, `$(…)`, process substitution) would help future maintainers understand the scope. **`bootCheck` invokes itself via `os.Executable()` to produce the boot-test payload.** This works but is a bit unusual — a direct `buildBootTestPayload(gamesDir, id)` function call would avoid spawning a process and make the code path easier to unit-test. Not urgent, but worth considering. **`portSettings` — the `internalPort` regex is an unanchored substring match.** A hypothetical setting named `GameHttpPort` would be silently excluded. Very unlikely in practice, but anchoring to word boundaries or to the end of the key (`(?i)(rcon|rest|api|admin|telnet|web|http|metrics)_?(port|$)`) would be more precise. **`builder-image.yml`** — the workflow has no `timeout-minutes`. A hung `docker build` (e.g. registry unreachable) would sit in the queue indefinitely. A 15-minute timeout is cheap insurance. ## 3. Tests Coverage for the new pure-logic functions is good (`bootFailure`, `outsideDirs`, `splitSettings`, `portSettings`, `variantNotes`). A few gaps worth filling: - **`scanShell` / `unwrapShell`:** no direct unit tests. The `splitSettings` tests cover integration, but a table-driven test for `scanShell` with tricky inputs (nested `${A:-${B}}`, `A="x $C y"` where `C` is a text setting, `bash -c 'nested "quotes"'`) would pin down the parser's behaviour and make it safer to extend later. - **`unwrapShell`:** the negative case (a command that *looks* like `sh -c '…'` but contains a single quote inside) isn't tested. The `!strings.Contains(m[1], "'")` guard means it falls through — confirm that's the intended fallback with a test. - **`build()` image-selection loop:** the path where all images are `imageUnusable` and the path where a SteamCMD image has no `StartupHint` are only exercised end-to-end. A small table test on a helper (if you extract the selection logic) would guard against regressions. ## 4. Overall assessment This is a well-structured, well-documented feature. The separation of concerns (research → draft → boot → PR) is clean, the new lint checks (`splitSettings`, `portSettings`) are correctly scoped and well-tested, and the `drafter` refactor makes the model conversation reusable without duplicating state. The CI workflow is minimal and appropriate. The main risk area is the hand-rolled shell scanner, but its scope is narrow and the tests cover the important cases. No correctness or security blockers in the visible code; ship it with the minor cleanups above tracked as follow-ups. --- *Automated review by AI Git Bot*
greg added 1 commit 2026-10-25 19:52:19 +00:00
fix(builder): drafts don't use CI secrets, and a missing one is fed back
Validate game templates / validate (push) Successful in 20s
Validate game templates / boot-test (push) Skipped
Validate game templates / publish (push) Skipped
5dfc0cc794
The second Valheim build stopped before its boot test: told that
test.yaml's password "looks like a credential; map it under 'secrets'",
the model mapped it to a CI secret nobody created, and boottest-payload's
missing-secret exit ended the build.

- The validator now says to remove a test password first (boot tests fill
  password settings with a throwaway value); secrets only when the server
  can't start without a real one.
- Drafts with test.yaml 'secrets' fail the draft checks.
- build turns boottest-payload's missing-secret exit into feedback for the
  model instead of an error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
git_ai_bot left a comment
Member

🤖 AI Code Review

1. Blocking issues

None found in the visible code. The logic across build.go, draft.go, draftchecks.go, and research.go is sound, and the tests cover the key paths well.

One file I cannot verify: worker.go (present in the repo tree, called from main.go, but not shown in the diff). Please confirm:

  • It cleans up WORK_DIR (or uses a fresh subdirectory) between jobs, since newDrafter hard-fails if games/<id>/ already exists.
  • A panel-side job cancellation kills the running docker run subprocess (otherwise a cancelled build leaves a container holding the Docker socket).
  • The GITEA_TOKEN is never written to a file or passed as a CLI arg (visible in ps).

If those are handled, no blockers.

2. Non-blocking suggestions

builder-image.yml has no timeout-minutes. A hung docker build (registry unreachable, go mod download stuck) occupies the runner indefinitely. Add timeout-minutes: 20 to the job.

AUTHORING.md is read from CWD (os.ReadFile("AUTHORING.md") in newDrafter). Works in the Docker image (WORKDIR /work) and in the documented go run examples, but breaks if you invoke the binary from a different directory with -games /path/to/games. Consider resolving it relative to the executable (os.Executable() → dir → AUTHORING.md) or adding a -rules flag. Low priority since the worker always runs from /work.

internalPort regex is an unanchored substring match ((?i)rcon|rest|api|admin|telnet|web|http|metrics). A hypothetical setting named `WebPort


Automated review by AI Git Bot

## 🤖 AI Code Review ## 1. Blocking issues None found in the visible code. The logic across `build.go`, `draft.go`, `draftchecks.go`, and `research.go` is sound, and the tests cover the key paths well. One file I **cannot verify**: `worker.go` (present in the repo tree, called from `main.go`, but not shown in the diff). Please confirm: - It cleans up `WORK_DIR` (or uses a fresh subdirectory) between jobs, since `newDrafter` hard-fails if `games/<id>/` already exists. - A panel-side job cancellation kills the running `docker run` subprocess (otherwise a cancelled build leaves a container holding the Docker socket). - The `GITEA_TOKEN` is never written to a file or passed as a CLI arg (visible in `ps`). If those are handled, no blockers. ## 2. Non-blocking suggestions **`builder-image.yml` has no `timeout-minutes`.** A hung `docker build` (registry unreachable, `go mod download` stuck) occupies the runner indefinitely. Add `timeout-minutes: 20` to the job. **`AUTHORING.md` is read from CWD** (`os.ReadFile("AUTHORING.md")` in `newDrafter`). Works in the Docker image (WORKDIR `/work`) and in the documented `go run` examples, but breaks if you invoke the binary from a different directory with `-games /path/to/games`. Consider resolving it relative to the executable (`os.Executable()` → dir → `AUTHORING.md`) or adding a `-rules` flag. Low priority since the worker always runs from `/work`. **`internalPort` regex is an unanchored substring match** (`(?i)rcon|rest|api|admin|telnet|web|http|metrics`). A hypothetical setting named `WebPort --- *Automated review by AI Git Bot*
greg merged commit 929e7d68d0 into main 2026-10-25 20:22:46 +00:00
greg deleted branch feat/builder-worker 2026-10-25 20:22:46 +00:00
Sign in to join this conversation.
No Reviewers
No labels
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: gbreed-public/gbr-games#6