Merge branch 'main' into ci/no-go-cache
This commit is contained in:
commit
e8216809d4
4 files changed
+853
-1
No files matched your search
+102
@@ -0,0 +1,102 @@
|
||||
# Writing a game template
|
||||
|
||||
House rules for `games/<id>/game.yaml` and `test.yaml`. They apply to people and to the AI builder alike;
|
||||
the builder reads this file as its instructions. Most rules exist because breaking them broke a real server.
|
||||
`gbrgames validate` and `gbrgames inspect-image -game <id>` enforce what they can.
|
||||
|
||||
## How a server actually runs
|
||||
|
||||
Know this before writing anything:
|
||||
|
||||
- The node-agent creates one Docker volume per server and mounts it at **both `/server` and `/data`**.
|
||||
Game files, saves and config live there. Anything written elsewhere is lost when the container is
|
||||
recreated, which happens on every start.
|
||||
- The container runs as **`1000:1000`** unless the template sets `containerUser`. Never as root.
|
||||
- Every key in `defaultConfig` (merged with the owner's changes) reaches the container as an
|
||||
**environment variable**, named by the agent's rule: insert `_` before an upper-case run that follows a
|
||||
lower-case letter, then upper-case everything. `ServerName` → `SERVER_NAME`, `MaxPlayers` →
|
||||
`MAX_PLAYERS`, `RCONPort` → `RCONPORT` (no underscore: no lower-case letter before `P`),
|
||||
`bIsPvP` → `B_IS_PV_P`.
|
||||
- If `configFilePath` is set, the same settings are also written to that file in `configFileFormat`
|
||||
before each start.
|
||||
- `defaultStartupCommand`, when set, runs as `bash -c "… cd /server && <command>"`; otherwise the
|
||||
image's own entrypoint runs.
|
||||
- `installMethod: STEAMCMD` installs `installConfig.appId` into `/server` with anonymous SteamCMD before
|
||||
the server starts.
|
||||
- Nodes reach a server's RCON, REST and query ports **over the Docker network**, never through
|
||||
published ports.
|
||||
|
||||
## Rules
|
||||
|
||||
### Passwords and secrets
|
||||
- Admin, RCON and REST passwords: `defaultConfig` value `""`, and the `configSchema` property gets
|
||||
`"x-generate": {"type": "password"}`. The panel then gives every server its own password.
|
||||
**Never put a placeholder or default password in a template.** Shared defaults are readable by anyone
|
||||
from the public game list.
|
||||
- Join passwords stay `""`, with `"x-savedValue": {"key": "GAME_JOIN_PASSWORD", "label": "Game join password", "secret": true}`.
|
||||
- No real credentials, tokens or IDs anywhere in this repo. A value a server can't start without (a
|
||||
player ID, a license token) goes in `test.yaml` under `secrets`, mapped to a CI secret name.
|
||||
|
||||
### Ports
|
||||
- `defaultPorts` lists **only what players connect to** (game and query ports). RCON, REST and admin
|
||||
ports are **never** published.
|
||||
- Use `"7777/udp"` or `"8211:8211/udp"`. Every port needs its protocol.
|
||||
|
||||
### The container user
|
||||
- Run `gbrgames inspect-image -game <id>`. If the image is built around its own user (it declares
|
||||
`USER`, or locks a home folder to a uid other than 1000), set `containerUser` to that `uid:gid`.
|
||||
Example: ARK's image needs `25000:25000`.
|
||||
|
||||
### The startup command
|
||||
- Read settings under the names the agent gives them (see above). `${RCON_PORT}` when the setting is
|
||||
`RCONPort` silently ignores the owner's value; it arrives as `RCONPORT`.
|
||||
- Use `${VAR:-default}` for anything that may be unset.
|
||||
- If the image installs the game itself on start, make sure a failed install is retried rather than
|
||||
starting a server that isn't there (see ARK's template).
|
||||
|
||||
### Memory
|
||||
- `requiredMemoryMb` is the container's hard limit. A JVM or engine heap must stay about **1 GB below**
|
||||
it: Minecraft's `MEMORY` is `3G` for `4096` MB. A heap equal to the limit is killed on start.
|
||||
|
||||
### Settings (`defaultConfig` and `configSchema`)
|
||||
- Take keys and default values from what the server really writes: run
|
||||
`gbrgames first-boot -game <id>` and read the settings in its report.
|
||||
- Every key in `defaultConfig` has a `configSchema` property, and the other way round.
|
||||
- Every property has a `title`, a short `description` for a player, a `type`, a `default` equal to the
|
||||
`defaultConfig` value, and a `category`: one of `Security & Access`, `Gameplay & Rules`,
|
||||
`World & Environment`, `System & General`. Mark the few that matter when deploying with
|
||||
`"showOnDeploy": true`.
|
||||
- An EULA or other agreement is a boolean with `"x-consent": {"label": "...", "url": "..."}` and
|
||||
default `false`. Never `true`; the owner accepts it themselves.
|
||||
- Keys may only use letters, digits, `_`, `-` and `.`.
|
||||
- Placeholders the panel fills in: `{SERVER_NAME}`, `{PUBLIC_IP}`, `{PUBLIC_HOST}`,
|
||||
`{PUBLIC_PORT_<port>}`, `{saved:KEY}`. Server names default to `GBRHosting - {SERVER_NAME}`.
|
||||
|
||||
### Being ready
|
||||
- `readyPattern`: a regex that matches the line the server prints once players can join. Take it from a
|
||||
first boot (its report suggests one), and put that real line in `test.yaml` as `readyExample`;
|
||||
validation checks the pattern matches it.
|
||||
- `failPatterns`: lines that mean the server will never come up (e.g. an unaccepted EULA).
|
||||
- No reliable ready line: leave `readyPattern` out, and the boot test checks the server stays up
|
||||
(`settleSec` in `test.yaml`, default 120 s).
|
||||
|
||||
### `test.yaml`
|
||||
- `config`: only what the server can't start without in CI (accepting the EULA, for example).
|
||||
- `secrets`: required values that can't be public, mapped to CI secret names.
|
||||
- `timeoutSec` covers install plus start; SteamCMD games need 2400 or more.
|
||||
|
||||
### Commands and players (`agnosticCommandConfig`)
|
||||
- Kick, ban, save and broadcast go through RCON or REST using the server's own password key
|
||||
(`passwordKey` / `rconPasswordKey` pointing at the `x-generate` setting). If a game has neither, mark
|
||||
the action `"type": "UNSUPPORTED"`; never invent one.
|
||||
- `playerJoinRegex` / `playerLeaveRegex` capture the player name from real log lines.
|
||||
|
||||
## Before opening a PR
|
||||
|
||||
```bash
|
||||
go run ./cmd/gbrgames validate
|
||||
go run ./cmd/gbrgames inspect-image -game <id>
|
||||
go run ./cmd/gbrgames first-boot -game <id> # on a machine with Docker and the gbr-boottest network
|
||||
```
|
||||
|
||||
CI then boot-tests the changed game with the node-agent's real deploy code before anything reaches the panel.
|
||||
@@ -0,0 +1,661 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// draft writes a first template for a new game with an AI model: it gives the model AUTHORING.md as its
|
||||
// rules, the facts read from the image (inspect-image) and, when available, a first-boot report, plus two
|
||||
// working templates as examples. The answer is written to games/<id>/ and checked with the same validator
|
||||
// and image checks as any template; problems go back to the model to fix, a few rounds at most.
|
||||
|
||||
type chatMessage struct {
|
||||
Role string `json:"role"`
|
||||
Content string `json:"content"`
|
||||
}
|
||||
|
||||
type model interface {
|
||||
chat(messages []chatMessage, schema map[string]any) (string, error)
|
||||
name() string
|
||||
}
|
||||
|
||||
func runDraft(args []string) error {
|
||||
fs := flag.NewFlagSet("draft", flag.ExitOnError)
|
||||
gamesDir := fs.String("games", "games", "directory holding one folder per game")
|
||||
schemaDir := fs.String("schemas", "schema", "directory holding the JSON Schemas")
|
||||
gameName := fs.String("name", "", "the game's name, e.g. \"Valheim\"")
|
||||
id := fs.String("id", "", "folder/id for the new game (default: from -name)")
|
||||
image := fs.String("image", "", "Docker image the server runs in")
|
||||
steamApp := fs.String("steam-app", "", "SteamCMD app id, if the server is installed with SteamCMD")
|
||||
notes := fs.String("notes", "", "anything else the model should know (variant, crossplay plugin...)")
|
||||
reportPath := fs.String("first-boot", "", "a first-boot report.json for this image")
|
||||
provider := fs.String("provider", "ollama", "ollama or gemini")
|
||||
modelName := fs.String("model", "", "model name (default qwen3.8:27b for ollama, gemini-3.5-flash for gemini)")
|
||||
attempts := fs.Int("attempts", 3, "fix rounds after the first draft")
|
||||
noPull := fs.Bool("no-pull", false, "use the local image instead of pulling it")
|
||||
fs.Parse(args)
|
||||
|
||||
if *gameName == "" || *image == "" {
|
||||
return errors.New("draft needs -name and -image")
|
||||
}
|
||||
if *id == "" {
|
||||
*id = strings.Trim(regexp.MustCompile(`[^a-z0-9]+`).ReplaceAllString(strings.ToLower(*gameName), "-"), "-")
|
||||
}
|
||||
dir := filepath.Join(*gamesDir, *id)
|
||||
if _, err := os.Stat(dir); err == nil {
|
||||
return fmt.Errorf("%s already exists; drafts only create new games", dir)
|
||||
}
|
||||
|
||||
m, err := newModel(*provider, *modelName)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rules, err := os.ReadFile("AUTHORING.md")
|
||||
if err != nil {
|
||||
return fmt.Errorf("AUTHORING.md (the model's rules): %w", err)
|
||||
}
|
||||
gameSchema, err := compileSchema(filepath.Join(*schemaDir, "game.schema.json"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
testSchema, err := compileSchema(filepath.Join(*schemaDir, "test.schema.json"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
answerSchema, err := draftAnswerSchema(*schemaDir)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if !*noPull {
|
||||
fmt.Printf("[draft] pulling %s\n", *image)
|
||||
if out, err := runCmd("docker", "pull", "-q", *image); err != nil {
|
||||
return fmt.Errorf("docker pull: %v: %s", err, out)
|
||||
}
|
||||
}
|
||||
facts, err := inspectImage(*image, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var report *bootReport
|
||||
if *reportPath != "" {
|
||||
raw, err := os.ReadFile(*reportPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
report = &bootReport{}
|
||||
if err := json.Unmarshal(raw, report); err != nil {
|
||||
return fmt.Errorf("%s: %w", *reportPath, err)
|
||||
}
|
||||
if why := imageUnusable(report); why != "" {
|
||||
return errors.New(why)
|
||||
}
|
||||
}
|
||||
|
||||
messages := []chatMessage{
|
||||
{Role: "system", Content: draftSystemPrompt(string(rules))},
|
||||
{Role: "user", Content: draftRequest(*gameName, *id, *image, *steamApp, *notes, facts, report, exampleTemplates(*gamesDir, 2))},
|
||||
}
|
||||
|
||||
for round := 0; round <= *attempts; round++ {
|
||||
fmt.Printf("[draft] round %d: asking %s\n", round+1, m.name())
|
||||
start := time.Now()
|
||||
answer, err := m.chat(messages, answerSchema)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("[draft] answer in %.0fs\n", time.Since(start).Seconds())
|
||||
messages = append(messages, chatMessage{Role: "assistant", Content: answer})
|
||||
|
||||
problems := writeDraft(dir, *id, answer, report)
|
||||
if len(problems) == 0 {
|
||||
problems = validateGame(dir, *id, gameSchema, testSchema)
|
||||
}
|
||||
if len(problems) == 0 {
|
||||
if g, err := loadMap(filepath.Join(dir, "game.yaml")); err == nil {
|
||||
imageProblems, _ := checkTemplate(g, facts)
|
||||
problems = append(problems, imageProblems...)
|
||||
t, _ := loadMap(filepath.Join(dir, "test.yaml"))
|
||||
problems = append(problems, draftCompleteness(g, t)...)
|
||||
problems = append(problems, unreadSettings(g, facts)...)
|
||||
problems = append(problems, savesOutsideVolumes(g, report)...)
|
||||
}
|
||||
}
|
||||
if len(problems) == 0 {
|
||||
fmt.Printf("[draft] %s passes validation and the image checks\n", dir)
|
||||
fmt.Println("[draft] next: review it, then boot-test it (first-boot / CI) before opening a PR")
|
||||
return nil
|
||||
}
|
||||
fmt.Printf("[draft] %d problem(s):\n", len(problems))
|
||||
for _, p := range problems {
|
||||
fmt.Println(" - " + p)
|
||||
}
|
||||
messages = append(messages, chatMessage{Role: "user", Content: "Your template has these problems. Fix all of them and " +
|
||||
"answer with the complete corrected JSON object (both \"game\" and \"test\"):\n- " + strings.Join(problems, "\n- ")})
|
||||
}
|
||||
return fmt.Errorf("draft still has problems after %d fix rounds; it is in %s for a person to finish", *attempts, dir)
|
||||
}
|
||||
|
||||
func draftSystemPrompt(rules string) string {
|
||||
return "You write game server templates for GBR Hosting. Follow these house rules exactly; they describe " +
|
||||
"how servers really run, and breaking them has broken real servers:\n\n" + rules + "\n\n" +
|
||||
"Answer with one JSON object and nothing else: {\"game\": <the game.yaml content as JSON>, " +
|
||||
"\"test\": <the test.yaml content as JSON>}.\n\n" +
|
||||
"The template must be COMPLETE enough for a player to run and configure the server:\n" +
|
||||
"- defaultPorts: every port players or the server browser connect to (from the image's exposed ports " +
|
||||
"and the game's documentation), each with its protocol. Never RCON/REST/admin or web/status ports.\n" +
|
||||
"- defaultConfig and configSchema: every setting the image reads (the facts' envRead and env lists, the " +
|
||||
"first-boot settings) that an owner would want to change: server name, world, password, player limit, " +
|
||||
"difficulty and gameplay options. Each with title, description, type, default and category.\n" +
|
||||
"- requiredMemoryMb: what the game really needs, with headroom.\n" +
|
||||
"- readyPattern when a first-boot report shows the ready line; otherwise set test.settleSec.\n" +
|
||||
"Base every value on the facts given (image facts, first-boot report) and on what you know for certain " +
|
||||
"about this game and image. Do not invent settings an image does not read; say what you are unsure of " +
|
||||
"in the setting's description instead of leaving the template empty."
|
||||
}
|
||||
|
||||
// draftCompleteness is the bar a draft must clear on top of validation: an almost empty template is
|
||||
// valid but useless, and a cautious model will happily produce one.
|
||||
func draftCompleteness(game, test map[string]any) []string {
|
||||
var problems []string
|
||||
if ports, _ := game["defaultPorts"].([]any); len(ports) == 0 {
|
||||
problems = append(problems, "defaultPorts is empty: list the ports players connect to")
|
||||
}
|
||||
props := schemaProperties(game)
|
||||
defaults, _ := game["defaultConfig"].(map[string]any)
|
||||
if len(defaults) < 3 {
|
||||
problems = append(problems, fmt.Sprintf("defaultConfig has only %d settings: add every setting the image reads that an owner would change (name, world, password, players, gameplay)", len(defaults)))
|
||||
}
|
||||
var undescribed []string
|
||||
for _, key := range sortedKeys(defaults) {
|
||||
if _, ok := props[key]; !ok {
|
||||
undescribed = append(undescribed, key)
|
||||
}
|
||||
}
|
||||
if len(undescribed) > 0 {
|
||||
problems = append(problems, "these defaultConfig settings have no configSchema.properties entry (each needs title, description, "+
|
||||
"type, default and category, under configSchema: {type: object, properties: {...}}): "+strings.Join(undescribed, ", "))
|
||||
}
|
||||
for _, key := range sortedKeys(props) {
|
||||
p, _ := props[key].(map[string]any)
|
||||
for _, field := range []string{"title", "description", "type", "category"} {
|
||||
if v, ok := p[field]; !ok || v == "" {
|
||||
problems = append(problems, fmt.Sprintf("setting %s has no %s", key, field))
|
||||
}
|
||||
}
|
||||
}
|
||||
if n, ok := game["requiredMemoryMb"].(json.Number); !ok {
|
||||
problems = append(problems, "requiredMemoryMb is missing")
|
||||
} else if v, _ := n.Int64(); v < 1024 {
|
||||
problems = append(problems, "requiredMemoryMb is below 1024")
|
||||
}
|
||||
if _, ok := game["readyPattern"]; !ok {
|
||||
if _, ok := test["settleSec"]; !ok {
|
||||
problems = append(problems, "no readyPattern and no test.settleSec: boot tests need one of them")
|
||||
}
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
// unreadSettings finds settings that reach the container under a variable name the image never reads,
|
||||
// e.g. ServerPassword -> SERVER_PASSWORD when the image reads SERVER_PASS. Only for images configured
|
||||
// through environment variables (no config file) whose startup scripts the inspector could read.
|
||||
func unreadSettings(game map[string]any, facts *imageFacts) []string {
|
||||
if path, _ := game["configFilePath"].(string); path != "" || len(facts.EnvRead) == 0 {
|
||||
return nil
|
||||
}
|
||||
read := map[string]bool{}
|
||||
for _, v := range facts.EnvRead {
|
||||
read[v] = true
|
||||
}
|
||||
for v := range facts.Env {
|
||||
read[v] = true
|
||||
}
|
||||
startup, _ := game["defaultStartupCommand"].(string)
|
||||
defaults, _ := game["defaultConfig"].(map[string]any)
|
||||
var problems []string
|
||||
for _, key := range sortedKeys(defaults) {
|
||||
env := envName(key)
|
||||
if read[env] || strings.Contains(startup, env) {
|
||||
continue
|
||||
}
|
||||
msg := fmt.Sprintf("setting %s reaches the container as %s, which the image never reads", key, env)
|
||||
if near := closestVar(env, facts.EnvRead); near != "" {
|
||||
msg += fmt.Sprintf(" (did you mean %s? name the key so it becomes that)", near)
|
||||
}
|
||||
problems = append(problems, msg)
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
// closestVar picks the variable that shares the longest common prefix with name (SERVER_PASSWORD ->
|
||||
// SERVER_PASS), if it's a real resemblance.
|
||||
func closestVar(name string, vars []string) string {
|
||||
best, bestLen := "", 0
|
||||
for _, v := range vars {
|
||||
n := 0
|
||||
for n < len(v) && n < len(name) && v[n] == name[n] {
|
||||
n++
|
||||
}
|
||||
if n > bestLen {
|
||||
best, bestLen = v, n
|
||||
}
|
||||
}
|
||||
if bestLen < 5 {
|
||||
return ""
|
||||
}
|
||||
return best
|
||||
}
|
||||
|
||||
// Files the Steam client, Proton and the OS write for themselves: not the game's data
|
||||
var outsideNoise = []string{"/Steam/logs/", "/.steam/", "/steamcmd/", "/.cache/", "/tmp/", "/.local/share/umu"}
|
||||
|
||||
// savesOutsideVolumes turns first boot's "written outside the volumes" into a hard check: anything the
|
||||
// game itself wrote there (Valheim's worlds and admin lists in ~/.config/unity3d/...) is lost every time the
|
||||
// node recreates the container. A template passes once its startup command or settings point the game at
|
||||
// /server or /data; a boot test then shows whether that really worked.
|
||||
func savesOutsideVolumes(game map[string]any, report *bootReport) []string {
|
||||
if report == nil {
|
||||
return nil
|
||||
}
|
||||
dirs := map[string]bool{}
|
||||
for _, c := range report.ContainerChanges {
|
||||
_, p, _ := strings.Cut(c, " ")
|
||||
noise := false
|
||||
for _, n := range outsideNoise {
|
||||
if strings.Contains(p, n) {
|
||||
noise = true
|
||||
}
|
||||
}
|
||||
if !noise && p != "" {
|
||||
dirs[path.Dir(p)] = true
|
||||
}
|
||||
}
|
||||
if len(dirs) == 0 {
|
||||
return nil
|
||||
}
|
||||
startup, _ := game["defaultStartupCommand"].(string)
|
||||
redirected := strings.Contains(startup, "/server") || strings.Contains(startup, "/data")
|
||||
defaults, _ := game["defaultConfig"].(map[string]any)
|
||||
for _, v := range defaults {
|
||||
if s, ok := v.(string); ok && (strings.HasPrefix(s, "/server") || strings.HasPrefix(s, "/data")) {
|
||||
redirected = true
|
||||
}
|
||||
}
|
||||
if redirected {
|
||||
return nil
|
||||
}
|
||||
return []string{fmt.Sprintf("first boot: the server wrote its own files outside the server volume, in %s. Those are lost "+
|
||||
"whenever the node recreates the container (every start). Make the game keep them under /server, e.g. with "+
|
||||
"its save-directory launch option or by pointing that folder at /server, and include it in backupPaths",
|
||||
strings.Join(sortedKeysBool(dirs), ", "))}
|
||||
}
|
||||
|
||||
// imageUnusable recognises a first boot where the image died within seconds because it needs root (or
|
||||
// another privilege nodes never grant): drafting a template for it can't produce a working server.
|
||||
func imageUnusable(r *bootReport) string {
|
||||
if r == nil || r.ExitCode == nil || *r.ExitCode == 0 || r.RanSeconds > 60 {
|
||||
return ""
|
||||
}
|
||||
var denied []string
|
||||
for _, l := range r.ErrorLines {
|
||||
if strings.Contains(strings.ToLower(l.Text), "permission denied") {
|
||||
denied = append(denied, l.Text)
|
||||
}
|
||||
}
|
||||
if len(denied) == 0 {
|
||||
return ""
|
||||
}
|
||||
return fmt.Sprintf("first boot: the image exited (code %d) after %.0fs as %s with permission errors, e.g. %q. It needs root, "+
|
||||
"which nodes never allow; use another image for this game, or cm2network/steamcmd with -steam-app",
|
||||
*r.ExitCode, r.RanSeconds, r.User, denied[0])
|
||||
}
|
||||
|
||||
func draftRequest(gameName, id, image, steamApp, notes string, facts *imageFacts, report *bootReport, examples string) string {
|
||||
var b strings.Builder
|
||||
fmt.Fprintf(&b, "Write the template for **%s**.\n- id: %s\n- defaultDockerImage: %s\n", gameName, id, image)
|
||||
if steamApp != "" {
|
||||
fmt.Fprintf(&b, "- installMethod STEAMCMD with appId %s\n", steamApp)
|
||||
}
|
||||
if notes != "" {
|
||||
fmt.Fprintf(&b, "- notes: %s\n", notes)
|
||||
}
|
||||
|
||||
f := *facts
|
||||
f.Scripts = map[string]string{}
|
||||
for p, s := range facts.Scripts {
|
||||
if len(s) > 8000 {
|
||||
s = s[:8000] + "\n...(truncated)"
|
||||
}
|
||||
f.Scripts[p] = s
|
||||
}
|
||||
fj, _ := json.MarshalIndent(f, "", " ")
|
||||
fmt.Fprintf(&b, "\n## Facts read from the image (inspect-image)\n```json\n%s\n```\n", fj)
|
||||
|
||||
if report != nil {
|
||||
fmt.Fprintf(&b, "\n## First boot with no settings\nOutcome: %s\n", report.Outcome)
|
||||
if report.SuggestedReady != "" {
|
||||
fmt.Fprintf(&b, "Suggested readyPattern: `%s` (readyExample: `%s`)\n", report.SuggestedReady, report.SuggestedExample)
|
||||
}
|
||||
for _, l := range report.ReadyCandidates {
|
||||
fmt.Fprintf(&b, "ready-looking line: %s\n", l.Text)
|
||||
}
|
||||
for _, l := range report.ErrorLines {
|
||||
fmt.Fprintf(&b, "error-looking line: %s\n", l.Text)
|
||||
}
|
||||
fmt.Fprintf(&b, "Config files the server wrote: %s\n", strings.Join(report.ConfigFiles, ", "))
|
||||
settings := report.Settings
|
||||
if len(settings) > 300 {
|
||||
settings = settings[:300]
|
||||
}
|
||||
b.WriteString("Settings it wrote, with defaults (file | section | key = value):\n")
|
||||
for _, s := range settings {
|
||||
fmt.Fprintf(&b, "%s | %s | %s = %s\n", s.File, s.Section, s.Key, s.Value)
|
||||
}
|
||||
} else {
|
||||
b.WriteString("\n(No first-boot report: keep readyPattern out unless the facts show the ready line.)\n")
|
||||
}
|
||||
b.WriteString("\n## Working templates from this repo, for format and style (not content)\n")
|
||||
b.WriteString(examples)
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// exampleTemplates picks the shortest templates in the repo as examples: real, validated, and small
|
||||
// enough to leave the model room for the facts.
|
||||
func exampleTemplates(gamesDir string, n int) string {
|
||||
entries, _ := os.ReadDir(gamesDir)
|
||||
type ex struct {
|
||||
id string
|
||||
body string
|
||||
}
|
||||
var all []ex
|
||||
for _, e := range entries {
|
||||
raw, err := os.ReadFile(filepath.Join(gamesDir, e.Name(), "game.yaml"))
|
||||
if err == nil {
|
||||
all = append(all, ex{e.Name(), string(raw)})
|
||||
}
|
||||
}
|
||||
sort.Slice(all, func(i, j int) bool { return len(all[i].body) < len(all[j].body) })
|
||||
var b strings.Builder
|
||||
for i := 0; i < n && i < len(all); i++ {
|
||||
fmt.Fprintf(&b, "\n### games/%s/game.yaml\n```yaml\n%s```\n", all[i].id, all[i].body)
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// writeDraft turns the model's JSON answer into games/<id>/game.yaml and test.yaml, using the importer's
|
||||
// writer so drafts look like every other template.
|
||||
func writeDraft(dir, id, answer string, report *bootReport) []string {
|
||||
answer = strings.TrimSpace(answer)
|
||||
if i := strings.Index(answer, "{"); i > 0 {
|
||||
answer = answer[i:] // some models still wrap JSON in prose or code fences
|
||||
}
|
||||
if i := strings.LastIndex(answer, "}"); i >= 0 {
|
||||
answer = answer[:i+1]
|
||||
}
|
||||
var parsed struct {
|
||||
Game map[string]any `json:"game"`
|
||||
Test map[string]any `json:"test"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(answer), &parsed); err != nil {
|
||||
return []string{"the answer is not valid JSON: " + err.Error()}
|
||||
}
|
||||
if parsed.Game == nil {
|
||||
return []string{`the answer has no "game" object`}
|
||||
}
|
||||
parsed.Game["id"] = id
|
||||
if parsed.Test == nil {
|
||||
parsed.Test = map[string]any{}
|
||||
}
|
||||
// The ready line is a fact from the first boot, already checked against the real log line; a model
|
||||
// that leaves it out (or answers null) gets the measured one rather than another guessing round
|
||||
if p, _ := parsed.Game["readyPattern"].(string); p == "" && report != nil && report.SuggestedReady != "" {
|
||||
parsed.Game["readyPattern"] = report.SuggestedReady
|
||||
parsed.Test["readyExample"] = report.SuggestedExample
|
||||
}
|
||||
if err := os.MkdirAll(dir, 0o755); err != nil {
|
||||
return []string{err.Error()}
|
||||
}
|
||||
var problems []string
|
||||
known := map[string]bool{}
|
||||
for _, f := range gameFields {
|
||||
known[f] = true
|
||||
}
|
||||
for k := range parsed.Game {
|
||||
if !known[k] {
|
||||
problems = append(problems, fmt.Sprintf("%q is not a template field; remove it or use one from the schema", k))
|
||||
}
|
||||
}
|
||||
if err := writeGame(filepath.Join(dir, "game.yaml"), parsed.Game); err != nil {
|
||||
return []string{"could not write game.yaml: " + err.Error()}
|
||||
}
|
||||
node, err := toNode(normalize(parsed.Test))
|
||||
if err != nil {
|
||||
return []string{err.Error()}
|
||||
}
|
||||
if err := writeYAML(filepath.Join(dir, "test.yaml"), testHeader, node); err != nil {
|
||||
return []string{"could not write test.yaml: " + err.Error()}
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
// draftAnswerSchema is the shape Ollama is asked to produce: {"game": ..., "test": ...}, using the repo's
|
||||
// schemas minus the parts structured-output engines don't support.
|
||||
func draftAnswerSchema(schemaDir string) (map[string]any, error) {
|
||||
load := func(name string) (map[string]any, error) {
|
||||
raw, err := os.ReadFile(filepath.Join(schemaDir, name))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var s map[string]any
|
||||
if err := json.Unmarshal(raw, &s); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, k := range []string{"$schema", "$id", "allOf", "title", "description"} {
|
||||
delete(s, k)
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
game, err := load("game.schema.json")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
test, err := load("test.schema.json")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tightenForDraft(game)
|
||||
return map[string]any{
|
||||
"type": "object",
|
||||
"required": []string{"game", "test"},
|
||||
"properties": map[string]any{"game": game, "test": test},
|
||||
"additionalProperties": false,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// tightenForDraft makes the fields a working template needs required and non-null in the format Ollama
|
||||
// enforces. The repo schema leaves them optional (hand-written templates may omit some), and a constrained
|
||||
// model takes that literally: it answered null for installConfig, defaultConfig and configSchema, the
|
||||
// writer dropped them, and no amount of "add installConfig" could change that.
|
||||
func tightenForDraft(game map[string]any) {
|
||||
props, _ := game["properties"].(map[string]any)
|
||||
nonNull := func(name, typ string) {
|
||||
if p, ok := props[name].(map[string]any); ok {
|
||||
p["type"] = typ
|
||||
}
|
||||
}
|
||||
nonNull("defaultStartupCommand", "string")
|
||||
nonNull("requiredMemoryMb", "integer")
|
||||
nonNull("defaultPorts", "array")
|
||||
if p, ok := props["defaultPorts"].(map[string]any); ok {
|
||||
p["minItems"] = 1
|
||||
}
|
||||
props["installConfig"] = map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"appId": map[string]any{"type": "string", "description": "Steam app id, for installMethod STEAMCMD"},
|
||||
"script": map[string]any{"type": "string", "description": "install script, for installMethod BASH_SCRIPT"},
|
||||
},
|
||||
}
|
||||
props["defaultConfig"] = map[string]any{"type": "object", "minProperties": 3}
|
||||
props["configSchema"] = map[string]any{
|
||||
"type": "object",
|
||||
"required": []string{"type", "properties"},
|
||||
"properties": map[string]any{
|
||||
"type": map[string]any{"enum": []string{"object"}},
|
||||
"properties": map[string]any{
|
||||
"type": "object",
|
||||
"minProperties": 3,
|
||||
"additionalProperties": map[string]any{
|
||||
"type": "object",
|
||||
"required": []string{"title", "description", "type", "default", "category"},
|
||||
"properties": map[string]any{
|
||||
"title": map[string]any{"type": "string"},
|
||||
"description": map[string]any{"type": "string"},
|
||||
"type": map[string]any{"enum": []string{"string", "integer", "number", "boolean"}},
|
||||
"category": map[string]any{"enum": []string{"Security & Access", "Gameplay & Rules", "World & Environment", "System & General"}},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
game["required"] = []string{"id", "name", "defaultDockerImage", "defaultStartupCommand", "requiredMemoryMb",
|
||||
"defaultPorts", "installMethod", "installConfig", "defaultConfig", "configSchema"}
|
||||
}
|
||||
|
||||
func newModel(provider, name string) (model, error) {
|
||||
switch provider {
|
||||
case "ollama":
|
||||
if name == "" {
|
||||
name = "qwen3.8:27b"
|
||||
}
|
||||
url := os.Getenv("OLLAMA_URL")
|
||||
if url == "" {
|
||||
url = "http://localhost:11434"
|
||||
}
|
||||
return &ollamaModel{url: strings.TrimRight(url, "/"), model: name}, nil
|
||||
case "gemini":
|
||||
if name == "" {
|
||||
name = "gemini-3.5-flash"
|
||||
}
|
||||
key := os.Getenv("GEMINI_API_KEY")
|
||||
if key == "" {
|
||||
return nil, errors.New("set GEMINI_API_KEY to use -provider gemini")
|
||||
}
|
||||
return &geminiModel{key: key, model: name}, nil
|
||||
}
|
||||
return nil, fmt.Errorf("unknown provider %q (ollama or gemini)", provider)
|
||||
}
|
||||
|
||||
type ollamaModel struct{ url, model string }
|
||||
|
||||
func (o *ollamaModel) name() string { return "ollama " + o.model + " at " + o.url }
|
||||
|
||||
func (o *ollamaModel) chat(messages []chatMessage, schema map[string]any) (string, error) {
|
||||
body := map[string]any{
|
||||
"model": o.model, "messages": messages, "stream": false, "format": schema,
|
||||
// Ollama silently truncates input beyond its default context
|
||||
"options": map[string]any{"num_ctx": 32768, "temperature": 0.2},
|
||||
}
|
||||
var resp struct {
|
||||
Message struct {
|
||||
Content string `json:"content"`
|
||||
} `json:"message"`
|
||||
Error string `json:"error"`
|
||||
}
|
||||
if err := postJSON(o.url+"/api/chat", body, &resp, 30*time.Minute); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if resp.Error != "" {
|
||||
return "", errors.New("ollama: " + resp.Error)
|
||||
}
|
||||
return resp.Message.Content, nil
|
||||
}
|
||||
|
||||
type geminiModel struct{ key, model string }
|
||||
|
||||
func (g *geminiModel) name() string { return "gemini " + g.model }
|
||||
|
||||
func (g *geminiModel) chat(messages []chatMessage, _ map[string]any) (string, error) {
|
||||
var system string
|
||||
var contents []map[string]any
|
||||
for _, m := range messages {
|
||||
if m.Role == "system" {
|
||||
system = m.Content
|
||||
continue
|
||||
}
|
||||
role := "user"
|
||||
if m.Role == "assistant" {
|
||||
role = "model"
|
||||
}
|
||||
contents = append(contents, map[string]any{"role": role, "parts": []map[string]string{{"text": m.Content}}})
|
||||
}
|
||||
body := map[string]any{
|
||||
"systemInstruction": map[string]any{"parts": []map[string]string{{"text": system}}},
|
||||
"contents": contents,
|
||||
"generationConfig": map[string]any{"responseMimeType": "application/json", "temperature": 0.2},
|
||||
}
|
||||
var resp struct {
|
||||
Candidates []struct {
|
||||
Content struct {
|
||||
Parts []struct {
|
||||
Text string `json:"text"`
|
||||
} `json:"parts"`
|
||||
} `json:"content"`
|
||||
} `json:"candidates"`
|
||||
Error *struct {
|
||||
Message string `json:"message"`
|
||||
} `json:"error"`
|
||||
}
|
||||
url := "https://generativelanguage.googleapis.com/v1beta/models/" + g.model + ":generateContent?key=" + g.key
|
||||
if err := postJSON(url, body, &resp, 10*time.Minute); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if resp.Error != nil {
|
||||
return "", errors.New("gemini: " + resp.Error.Message)
|
||||
}
|
||||
if len(resp.Candidates) == 0 || len(resp.Candidates[0].Content.Parts) == 0 {
|
||||
return "", errors.New("gemini returned no answer")
|
||||
}
|
||||
return resp.Candidates[0].Content.Parts[0].Text, nil
|
||||
}
|
||||
|
||||
func postJSON(url string, body, out any, timeout time.Duration) error {
|
||||
payload, err := json.Marshal(body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
client := &http.Client{Timeout: timeout}
|
||||
resp, err := client.Post(url, "application/json", bytes.NewReader(payload))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
raw, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := json.Unmarshal(raw, out); err != nil {
|
||||
return fmt.Errorf("HTTP %d, unreadable answer: %.300s", resp.StatusCode, raw)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func runCmd(name string, args ...string) (string, error) {
|
||||
out, err := exec.Command(name, args...).CombinedOutput()
|
||||
return strings.TrimSpace(string(out)), err
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestDraftCompletenessNamesMissingSchemaEntries(t *testing.T) {
|
||||
// The Valheim draft that went round in circles: settings in defaultConfig, no configSchema
|
||||
game := map[string]any{
|
||||
"defaultPorts": []any{"2456/udp"},
|
||||
"requiredMemoryMb": json.Number("4096"),
|
||||
"defaultConfig": map[string]any{"ServerName": "x", "ServerPassword": "", "MaxPlayers": json.Number("8")},
|
||||
}
|
||||
problems := strings.Join(draftCompleteness(game, map[string]any{"settleSec": json.Number("120")}), "\n")
|
||||
if !strings.Contains(problems, "no configSchema.properties entry") || !strings.Contains(problems, "MaxPlayers, ServerName, ServerPassword") {
|
||||
t.Errorf("want the missing schema entries named, got:\n%s", problems)
|
||||
}
|
||||
|
||||
game["configSchema"] = map[string]any{"properties": map[string]any{
|
||||
"ServerName": map[string]any{"title": "Name", "description": "d", "type": "string", "category": "System & General"},
|
||||
"ServerPassword": map[string]any{"title": "Password", "description": "d", "type": "string", "category": "Security & Access"},
|
||||
"MaxPlayers": map[string]any{"title": "Players", "description": "d", "type": "integer", "category": "System & General"},
|
||||
}}
|
||||
if p := draftCompleteness(game, map[string]any{"settleSec": json.Number("120")}); len(p) != 0 {
|
||||
t.Errorf("complete draft still has problems: %v", p)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUnreadSettings(t *testing.T) {
|
||||
facts := &imageFacts{EnvRead: []string{"SERVER_NAME", "SERVER_PASS", "SERVER_PUBLIC"}, Env: map[string]string{"TZ": "UTC"}}
|
||||
game := map[string]any{"defaultConfig": map[string]any{"ServerName": "x", "ServerPassword": "", "Tz": "UTC"}}
|
||||
problems := unreadSettings(game, facts)
|
||||
if len(problems) != 1 || !strings.Contains(problems[0], "SERVER_PASSWORD") || !strings.Contains(problems[0], "did you mean SERVER_PASS") {
|
||||
t.Errorf("got %v", problems)
|
||||
}
|
||||
|
||||
// Settings written to a config file aren't environment-only, so nothing to check
|
||||
game["configFilePath"] = "server.cfg"
|
||||
if p := unreadSettings(game, facts); p != nil {
|
||||
t.Errorf("config-file game: %v", p)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSavesOutsideVolumes(t *testing.T) {
|
||||
// Valheim's real first boot: its own files in ~/.config, plus Steam client logs (noise)
|
||||
valheim := &bootReport{ContainerChanges: []string{
|
||||
"A /home/steam/.config/unity3d/IronGate/Valheim/adminlist.txt",
|
||||
"A /home/steam/.config/unity3d/IronGate/Valheim/bannedlist.txt",
|
||||
"A /home/steam/Steam/logs/connection_log_2456.txt",
|
||||
}}
|
||||
game := map[string]any{"defaultStartupCommand": "./valheim_server.x86_64 -nographics -batchmode -world X"}
|
||||
problems := savesOutsideVolumes(game, valheim)
|
||||
if len(problems) != 1 || !strings.Contains(problems[0], "/home/steam/.config/unity3d/IronGate/Valheim") || strings.Contains(problems[0], "Steam/logs") {
|
||||
t.Fatalf("got %v", problems)
|
||||
}
|
||||
|
||||
// Fixed with a save-directory option under /server
|
||||
game["defaultStartupCommand"] = "./valheim_server.x86_64 -nographics -batchmode -world X -savedir /server/saves"
|
||||
if p := savesOutsideVolumes(game, valheim); p != nil {
|
||||
t.Errorf("redirected template still flagged: %v", p)
|
||||
}
|
||||
|
||||
// ARK's real first boot: only Proton's cache, which is not game data
|
||||
ark := &bootReport{ContainerChanges: []string{"A /home/gameserver/.cache/umu-protonfixes/protonfixes_test.log"}}
|
||||
if p := savesOutsideVolumes(map[string]any{}, ark); p != nil {
|
||||
t.Errorf("ARK proton cache flagged: %v", p)
|
||||
}
|
||||
}
|
||||
|
||||
func TestImageUnusable(t *testing.T) {
|
||||
code := 2
|
||||
r := &bootReport{User: "1000:1000", ExitCode: &code, RanSeconds: 2.4, ErrorLines: []logLine{
|
||||
{Text: "ln: failed to create symbolic link '/etc/localtime': Permission denied"},
|
||||
}}
|
||||
if why := imageUnusable(r); !strings.Contains(why, "needs root") {
|
||||
t.Errorf("want a needs-root verdict, got %q", why)
|
||||
}
|
||||
|
||||
// A server that ran for minutes and then stopped is not this case
|
||||
r.RanSeconds = 600
|
||||
if why := imageUnusable(r); why != "" {
|
||||
t.Errorf("long run flagged: %q", why)
|
||||
}
|
||||
}
|
||||
@@ -6,6 +6,7 @@
|
||||
// go run ./cmd/gbrgames boottest-payload ID # build the deploy message the node-agent's BootTest runs
|
||||
// go run ./cmd/gbrgames inspect-image -game ID # read the image's facts and check the template against them
|
||||
// go run ./cmd/gbrgames first-boot -game ID # start the server once, untouched; record its log and files
|
||||
// go run ./cmd/gbrgames draft -name N -image I # have an AI model write a new game's template from those facts
|
||||
package main
|
||||
|
||||
import (
|
||||
@@ -31,6 +32,8 @@ func main() {
|
||||
err = runInspectImage(os.Args[2:])
|
||||
case "first-boot":
|
||||
err = runFirstBoot(os.Args[2:])
|
||||
case "draft":
|
||||
err = runDraft(os.Args[2:])
|
||||
default:
|
||||
usage()
|
||||
}
|
||||
@@ -41,6 +44,6 @@ func main() {
|
||||
}
|
||||
|
||||
func usage() {
|
||||
fmt.Fprintln(os.Stderr, "usage: gbrgames validate [-games DIR] | import [-games DIR] [-all] EXPORT.json | export [-repo R] [-commit SHA] [-out FILE] | boottest-payload [-out FILE] ID | inspect-image [-game ID] [-json] [-no-pull] [IMAGE] | first-boot [-game ID | -image IMAGE] [-env K=V] [-max D] [-quiet D] [-out DIR]")
|
||||
fmt.Fprintln(os.Stderr, "usage: gbrgames validate [-games DIR] | import [-games DIR] [-all] EXPORT.json | export [-repo R] [-commit SHA] [-out FILE] | boottest-payload [-out FILE] ID | inspect-image [-game ID] [-json] [-no-pull] [IMAGE] | first-boot [-game ID | -image IMAGE] [-env K=V] [-max D] [-quiet D] [-out DIR] | draft -name NAME -image IMAGE [-steam-app ID] [-first-boot REPORT] [-provider ollama|gemini] [-model M]")
|
||||
os.Exit(2)
|
||||
}
|
||||
Reference in new issue
Block a user