From ab0e59ad719ab1e2f3c58117a27480e10832f3a8 Mon Sep 17 00:00:00 2001 From: Bryan Frimin Date: Tue, 21 Apr 2026 18:20:07 +0200 Subject: [PATCH] Generate cfg/dev.yaml via a Make target Committing a fully-materialised cfg/dev.yaml hid the dev configuration surface and blocked the OAuth2 signing-key inlining change: the new config requires a per-dev private key that must not be committed. Replace the checked-in file with a dev-config Make target that shells out to probod-bootstrap with dev-safe defaults and a stable RSA signing key stashed under cfg/.dev-oauth2-signing-key.pem on first run. The recipe sources cfg/dev.env when present so devs can override any setting without editing the Makefile; cfg/dev.env.example ships the full list of overridable knobs. cfg/dev.yaml, cfg/dev.env, and the signing key are all gitignored. Update README, CONTRIBUTING, and contrib/claude/config.md to describe the new workflow. Signed-off-by: Bryan Frimin --- .gitignore | 2 + CONTRIBUTING.md | 13 +++- GNUmakefile | 28 ++++++++ README.md | 5 +- cfg/dev.env.example | 81 ++++++++++++++++++++++ cfg/dev.yaml | 143 --------------------------------------- contrib/claude/config.md | 6 +- 7 files changed, 130 insertions(+), 148 deletions(-) create mode 100644 cfg/dev.env.example delete mode 100644 cfg/dev.yaml diff --git a/.gitignore b/.gitignore index 99f639ee4..c0e586faa 100644 --- a/.gitignore +++ b/.gitignore @@ -20,3 +20,5 @@ __generated__/ pkg/server/api/*/v1/types/types.go pkg/server/api/*/v1/schema.graphql cfg/dev_local.yaml +cfg/dev.yaml +cfg/dev.env diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a8340d529..5404a9355 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -84,7 +84,18 @@ Signed-off-by: John Doe make stack-up ``` -6. Start the development servers: +6. Generate the local dev config (writes `cfg/dev.yaml`): + + ```bash + # Optional: override any dev default (secrets, OAuth clients, LLM keys). + # cp cfg/dev.env.example cfg/dev.env && $EDITOR cfg/dev.env + + make dev-config + ``` + + The target stashes a dev-only RSA signing key under `cfg/.dev-oauth2-signing-key.pem` so tokens survive probod restarts, and sources `cfg/dev.env` if present so you can override defaults without editing the Makefile. `cfg/dev.yaml`, `cfg/dev.env`, and the signing key are all gitignored. Re-run the target to regenerate. + +7. Start the development servers: ```bash # In one terminal - start the API server diff --git a/GNUmakefile b/GNUmakefile index 88d634974..1bbea9112 100644 --- a/GNUmakefile +++ b/GNUmakefile @@ -151,6 +151,34 @@ coverage-combined: coverage-report test-e2e-coverage ## Generate combined covera .PHONY: build build: bin/probod bin/prb bin/probod-bootstrap +CFG_DEV_OAUTH2_KEY = cfg/.dev-oauth2-signing-key.pem +CFG_DEV_ENV = cfg/dev.env + +.PHONY: dev-config +dev-config: cfg/dev.yaml ## Generate cfg/dev.yaml via probod-bootstrap (rerun after deleting the file) + +$(CFG_DEV_OAUTH2_KEY): + @$(MKDIR) $(@D) + $(OPENSSL) genrsa -out $@ 2048 + +cfg/dev.yaml: bin/probod-bootstrap $(CFG_DEV_OAUTH2_KEY) + @$(MKDIR) $(@D) + @set -a; \ + PROBOD_ENCRYPTION_KEY="thisisnotasecretAAAAAAAAAAAAAAAAAAAAAAAAAAA="; \ + AUTH_COOKIE_SECRET="this-is-a-secure-secret-for-cookie-signing-at-least-32-bytes"; \ + AUTH_PASSWORD_PEPPER="this-is-a-secure-pepper-for-password-hashing-at-least-32-bytes"; \ + AUTH_COOKIE_SECURE=false; \ + OAUTH2_SERVER_SIGNING_KEY="$$($(CAT) $(CFG_DEV_OAUTH2_KEY))"; \ + API_CORS_ALLOWED_ORIGINS="http://localhost:8080,http://localhost:5173,http://localhost:5174"; \ + AWS_ACCESS_KEY_ID=probod; \ + AWS_SECRET_ACCESS_KEY=thisisnotasecret; \ + AWS_ENDPOINT=http://127.0.0.1:8333; \ + OPENAI_API_KEY=thisisnotasecret; \ + ACME_DIRECTORY=https://localhost:14000/dir; \ + if [ -f $(CFG_DEV_ENV) ]; then . $(CFG_DEV_ENV); fi; \ + set +a; \ + ./bin/probod-bootstrap -output $@ + .PHONY: sbom-docker sbom-docker: docker-build $(SYFT) docker:$(DOCKER_IMAGE_NAME):$(DOCKER_TAG_NAME) -o cyclonedx-json \ diff --git a/README.md b/README.md index a2925bc98..7274a68a8 100644 --- a/README.md +++ b/README.md @@ -48,6 +48,9 @@ Probo is designed to be accessible, transparent, and community-driven. # Build the project make build + # Generate the local dev config (writes cfg/dev.yaml) + make dev-config + # Start the application using development settings bin/probod -cfg-file cfg/dev.yaml ``` @@ -65,7 +68,7 @@ To test the custom domains feature locally, add the CNAME target to your hosts f 127.0.0.1 custom.getprobo.com ``` -This allows you to test custom trust center domains on your local machine. The CNAME target can be configured in `cfg/dev.yaml` under `custom-domains.cname-target`. +This allows you to test custom trust center domains on your local machine. The generated `cfg/dev.yaml` sets the CNAME target via `custom-domains.cname-target`; change `CUSTOM_DOMAINS_CNAME_TARGET` before running `make dev-config` to override it. For detailed setup instructions, see our [Contributing Guide](CONTRIBUTING.md). diff --git a/cfg/dev.env.example b/cfg/dev.env.example new file mode 100644 index 000000000..6ba15fcd0 --- /dev/null +++ b/cfg/dev.env.example @@ -0,0 +1,81 @@ +# Dev overrides for `make dev-config`. +# +# Copy this file to cfg/dev.env and uncomment any variable you want to +# override. The file is sourced as a shell snippet, so quote values that +# contain spaces. cfg/dev.env is gitignored; this .example file is the +# canonical list of overridable knobs. +# +# See pkg/bootstrap/builder.go for the full env-var surface; the values +# set below in the Makefile's `cfg/dev.yaml` recipe are the dev defaults +# that this file can override. + +# ── Base URL & ports ────────────────────────────────────────────────── +# PROBOD_BASE_URL=http://localhost:8080 +# API_ADDR=:8080 +# API_CORS_ALLOWED_ORIGINS=http://localhost:8080,http://localhost:5173,http://localhost:5174 +# TRUST_CENTER_HTTP_ADDR=:80 +# TRUST_CENTER_HTTPS_ADDR=:443 + +# ── Secrets (rotate before prod) ────────────────────────────────────── +# PROBOD_ENCRYPTION_KEY=thisisnotasecretAAAAAAAAAAAAAAAAAAAAAAAAAAA= +# AUTH_COOKIE_SECRET=this-is-a-secure-secret-for-cookie-signing-at-least-32-bytes +# AUTH_PASSWORD_PEPPER=this-is-a-secure-pepper-for-password-hashing-at-least-32-bytes + +# ── Cookie ──────────────────────────────────────────────────────────── +# AUTH_COOKIE_DOMAIN=localhost +# AUTH_COOKIE_SECURE=false + +# ── Postgres ────────────────────────────────────────────────────────── +# PG_ADDR=localhost:5432 +# PG_USERNAME=postgres +# PG_PASSWORD=postgres +# PG_DATABASE=probod +# PG_POOL_SIZE=100 + +# ── Object storage (SeaweedFS via compose) ──────────────────────────── +# AWS_REGION=us-east-1 +# AWS_BUCKET=probod +# AWS_ACCESS_KEY_ID=probod +# AWS_SECRET_ACCESS_KEY=thisisnotasecret +# AWS_ENDPOINT=http://127.0.0.1:8333 + +# ── Mailer (Mailpit via compose) ────────────────────────────────────── +# SMTP_ADDR=localhost:1025 +# MAILER_SENDER_EMAIL=no-reply@notification.getprobo.com +# MAILER_SENDER_NAME=Probo + +# ── LLM providers ───────────────────────────────────────────────────── +# OPENAI_API_KEY=sk-... +# ANTHROPIC_API_KEY=sk-ant-... +# LLM_DEFAULT_PROVIDER=openai +# LLM_DEFAULT_MODEL_NAME=gpt-4o + +# ── OIDC sign-in providers ──────────────────────────────────────────── +# AUTH_GOOGLE_CLIENT_ID= +# AUTH_GOOGLE_CLIENT_SECRET= +# AUTH_MICROSOFT_CLIENT_ID= +# AUTH_MICROSOFT_CLIENT_SECRET= + +# ── Connectors (set _CLIENT_ID to enable; _CLIENT_SECRET is required) ─ +# CONNECTOR_SLACK_CLIENT_ID= +# CONNECTOR_SLACK_CLIENT_SECRET= +# CONNECTOR_SLACK_SIGNING_SECRET= +# CONNECTOR_HUBSPOT_CLIENT_ID= +# CONNECTOR_HUBSPOT_CLIENT_SECRET= +# CONNECTOR_GITHUB_CLIENT_ID= +# CONNECTOR_GITHUB_CLIENT_SECRET= +# CONNECTOR_NOTION_CLIENT_ID= +# CONNECTOR_NOTION_CLIENT_SECRET= +# CONNECTOR_SENTRY_CLIENT_ID= +# CONNECTOR_SENTRY_CLIENT_SECRET= +# CONNECTOR_INTERCOM_CLIENT_ID= +# CONNECTOR_INTERCOM_CLIENT_SECRET= +# CONNECTOR_DOCUSIGN_CLIENT_ID= +# CONNECTOR_DOCUSIGN_CLIENT_SECRET= +# CONNECTOR_BREX_CLIENT_ID= +# CONNECTOR_BREX_CLIENT_SECRET= + +# ── Custom domains (Pebble ACME via compose) ────────────────────────── +# CUSTOM_DOMAINS_CNAME_TARGET=custom.getprobo.com +# ACME_DIRECTORY=https://localhost:14000/dir +# ACME_EMAIL=admin@getprobo.com diff --git a/cfg/dev.yaml b/cfg/dev.yaml deleted file mode 100644 index d0720d97e..000000000 --- a/cfg/dev.yaml +++ /dev/null @@ -1,143 +0,0 @@ -unit: - metrics: - addr: "localhost:8081" - tracing: - addr: "localhost:4317" - max-batch-size: 512 - batch-timeout: 1 - export-timeout: 1 - max-queue-size: 2048 - -probod: - base-url: "http://localhost:8080" - encryption-key: "thisisnotasecretAAAAAAAAAAAAAAAAAAAAAAAAAAA=" - chrome-dp-addr: "localhost:9222" - - trust-center: - http-addr: ":8085" - https-addr: ":8443" - - api: - addr: "localhost:8080" - cors: - allowed-origins: - - "http://localhost:8080" - - "http://localhost:5173" - - "http://localhost:5174" - - pg: - addr: "localhost:5432" - username: "postgres" - password: "postgres" - database: "probod" - pool-size: 100 - - auth: - disable-signup: false - invitation-confirmation-token-validity: 3600 - cookie: - name: "SSID" - domain: "localhost" - secret: "this-is-a-secure-secret-for-cookie-signing-at-least-32-bytes" - duration: 24 - secure: false - password: - pepper: "this-is-a-secure-pepper-for-password-hashing-at-least-32-bytes" - iterations: 1000000 - - aws: - region: "us-east-1" - bucket: "probod" - access-key-id: "probod" - secret-access-key: "thisisnotasecret" - endpoint: "http://127.0.0.1:8333" - - notifications: - mailer: - sender-name: "Probo" - sender-email: "no-reply@notification.getprobo.com" - smtp: - addr: "localhost:1025" - tls-required: false - mailer-interval: 60 - slack: - sender-interval: 60 - - llm: - providers: - openai: - type: openai - api-key: "thisisnotasecret" - defaults: - provider: openai - model-name: gpt-4o - temperature: 0.1 - max-tokens: 4096 - - evidence-describer: - interval: 10 - stale-after: 300 - max-concurrency: 10 - - custom-domains: - renewal-interval: 3600 - provision-interval: 30 - cname-target: "custom.getprobo.com" - acme: - directory: "https://localhost:14000/dir" - email: "admin@getprobo.com" - key-type: "EC256" - - connectors: - - provider: "SLACK" - protocol: "oauth2" - config: - client-id: "your-slack-client-id" - client-secret: "your-slack-client-secret" - settings: - signing-secret: "your-slack-signing-secret" - - provider: "GOOGLE_WORKSPACE" - protocol: "oauth2" - config: - client-id: "your-google-client-id.apps.googleusercontent.com" - client-secret: "your-google-client-secret" - - provider: "LINEAR" - protocol: "oauth2" - config: - client-id: "your-linear-client-id" - client-secret: "your-linear-client-secret" - - provider: "BREX" - protocol: "oauth2" - config: - client-id: "your-brex-client-id" - client-secret: "your-brex-client-secret" - - provider: "HUBSPOT" - protocol: "oauth2" - config: - client-id: "your-hubspot-client-id" - client-secret: "your-hubspot-client-secret" - - provider: "DOCUSIGN" - protocol: "oauth2" - config: - client-id: "your-docusign-client-id" - client-secret: "your-docusign-client-secret" - - provider: "NOTION" - protocol: "oauth2" - config: - client-id: "your-notion-client-id" - client-secret: "your-notion-client-secret" - - provider: "GITHUB" - protocol: "oauth2" - config: - client-id: "your-github-client-id" - client-secret: "your-github-client-secret" - - provider: "SENTRY" - protocol: "oauth2" - config: - client-id: "your-sentry-client-id" - client-secret: "your-sentry-client-secret" - - provider: "INTERCOM" - protocol: "oauth2" - config: - client-id: "your-intercom-client-id" - client-secret: "your-intercom-client-secret" diff --git a/contrib/claude/config.md b/contrib/claude/config.md index 897578aa3..e20de2741 100644 --- a/contrib/claude/config.md +++ b/contrib/claude/config.md @@ -10,7 +10,7 @@ When a configuration field is added, renamed, or removed in the Go config struct | 2 | `pkg/probod/probod.go` `New()` | Default values for new fields | | 3 | `pkg/bootstrap/builder.go` | Env-var → struct mapping (`Build()` method) | | 4 | `pkg/bootstrap/builder.go` | Required-env validation (`validateRequired()`) | -| 5 | `cfg/dev.yaml` | Local development config | +| 5 | `GNUmakefile` (`dev-config` target) | Env vars fed to `probod-bootstrap` to regenerate `cfg/dev.yaml` (file itself is gitignored) | | 6 | `e2e/internal/testutil/testutil.go` | E2E env-var map fed to `bootstrap.NewBuilder` | | 7 | `contrib/lima/provision.sh` | Sandbox env vars passed to `probod-bootstrap` | | 8 | `contrib/helm/charts/probo/values.yaml` | Helm default values | @@ -27,7 +27,7 @@ Go struct (pkg/probod/) │ ├─► bootstrap builder.go (env var → struct) │ │ - │ ├─► cfg/dev.yaml (static YAML, local dev) + │ ├─► GNUmakefile dev-config (env vars → probod-bootstrap → cfg/dev.yaml) │ ├─► e2e/internal/testutil/ (env map → bootstrap.Build, tests) │ ├─► contrib/lima/provision.sh (env vars → probod-bootstrap) │ └─► Helm chart @@ -44,7 +44,7 @@ Go struct (pkg/probod/) 1. **Never add a Go config field without updating every file in the checklist.** 2. **Env var naming** — follow the existing convention in `builder.go`: `SECTION_FIELD_NAME` (e.g. `AUTH_COOKIE_DOMAIN`, `CUSTOM_DOMAINS_RENEWAL_INTERVAL`). 3. **Secrets** go through `secret.yaml` and are referenced via `secretKeyRef` in `deployment.yaml`. Non-secret values are set inline. -4. **`cfg/dev.yaml`** uses safe, non-production defaults (plaintext passwords, `localhost`, `secure: false`). +4. **`make dev-config`** writes `cfg/dev.yaml` via `probod-bootstrap` with safe, non-production defaults (plaintext passwords, `localhost`, `secure: false`). The generated file and the per-dev OAuth2 signing key (`cfg/.dev-oauth2-signing-key.pem`) are both gitignored. The recipe sources `cfg/dev.env` if present so devs can override any env var without editing the `GNUmakefile`; keep `cfg/dev.env.example` in sync when you add or rename env vars. 5. **`e2e/internal/testutil/testutil.go`** builds the e2e config through `bootstrap.NewBuilder` with a test-only env-var map (different ports, `probod_test` DB, shorter intervals). Any new field whose test value differs from the bootstrap default must be added to that map. 6. **`provision.sh`** only sets env vars that differ from `builder.go` defaults (e.g. `PROBOD_BASE_URL`, `AUTH_COOKIE_DOMAIN`, `AUTH_COOKIE_SECURE`). If the new field's default is acceptable in the sandbox, no env var is needed. 7. **Helm `values.yaml`** exposes the field under the appropriate `probo.*` key with a sensible default. `values-production.yaml.example` includes it only when the production value differs or the user must set it.