diff --git a/AGENTS.md b/AGENTS.md index fd7505dea..b77c7a83e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -10,6 +10,7 @@ Detailed guides for specific subsystems live in `contrib/claude/`: - [`contrib/claude/go-worker.md`](contrib/claude/go-worker.md) — Go worker pattern (poll-based, bounded concurrency, FOR UPDATE SKIP LOCKED) - [`contrib/claude/gid.md`](contrib/claude/gid.md) — Global identifiers (GID layout, TenantID, entity type registry) - [`contrib/claude/coredata.md`](contrib/claude/coredata.md) — Data access layer (Scoper, SQL patterns, filters, order fields, migrations) +- [`contrib/claude/logging.md`](contrib/claude/logging.md) — Structured logging (PII-free rules, field helpers, logger wiring) - [`contrib/claude/graphql.md`](contrib/claude/graphql.md) — Go GraphQL backend (gqlgen, @goModel, connection types, cursor pagination) - [`contrib/claude/mcp.md`](contrib/claude/mcp.md) — MCP API patterns (specification.yaml, mcpgen, resolvers, type helpers) - [`contrib/claude/cli.md`](contrib/claude/cli.md) — CLI command patterns (cobra, huh prompts, pagination, output formatting) diff --git a/contrib/claude/coredata.md b/contrib/claude/coredata.md index fee15e0aa..82eb19e51 100644 --- a/contrib/claude/coredata.md +++ b/contrib/claude/coredata.md @@ -204,6 +204,94 @@ Each entity gets a unique `uint16` constant in `entity_type_reg.go`. **Never reu **Avoid default values.** Columns should not have `DEFAULT` clauses. When adding a non-nullable column to an existing table, use a `DEFAULT` to backfill existing rows, then drop it in the same migration. +## Sensitive data protection + +Every column that stores a secret, credential, or private key **must** be protected at rest in the application layer. Never store sensitive values as plaintext in the database. There are three protection strategies depending on the data's nature. + +### Strategy 1 — SHA-256 hash (high-entropy tokens) + +Use for values generated by the application with guaranteed entropy: bearer tokens, API keys, SCIM tokens, one-time tokens, SAML relay state tokens. These values are random and never chosen by a human, so a fast non-reversible hash is sufficient. + +- Package: `pkg/crypto/hash` → `hash.SHA256Hex([]byte) string` +- DB column type: `BYTEA` (store the raw hash bytes) or `TEXT` (store hex-encoded hash) +- Go field name: `Hashed*` (e.g. `HashedToken`, `HashedValue`) +- Lookup: compute SHA-256 of the presented token, then `WHERE hashed_token = @hashed_token` +- The plaintext token is returned to the caller **once** at creation time and never stored + +Existing examples: `Token.HashedValue`, `SCIMConfiguration.HashedToken`. + +```go +// At creation time — hash before insert +hashedValue := hash.SHA256Hex([]byte(rawToken)) +token.HashedValue = []byte(hashedValue) + +// At verification time — hash the presented value, then query +hashedValue := hash.SHA256Hex([]byte(presentedToken)) +token.LoadByHashedValueForUpdate(ctx, conn, []byte(hashedValue)) +``` + +### Strategy 2 — PBKDF2 (human-chosen passwords) + +Use for values chosen by humans with low or unpredictable entropy: passwords, passphrases, PINs. PBKDF2 with HMAC-SHA256 pepper provides brute-force resistance. + +- Package: `pkg/crypto/passwdhash` +- DB column type: `BYTEA NOT NULL` +- Go field name: `HashedPassword` +- Hash on write: `profile.HashPassword([]byte(password))` +- Compare on read: `profile.ComparePasswordAndHash([]byte(password), storedHash)` +- Parameters: minimum 600 000 iterations, 32-byte salt, 32-byte pepper + +Existing example: `Identity.HashedPassword`. + +```go +// At registration / password change +hashed, err := passwdProfile.HashPassword([]byte(plainPassword)) +identity.HashedPassword = hashed + +// At login +ok, err := passwdProfile.ComparePasswordAndHash([]byte(inputPassword), identity.HashedPassword) +``` + +### Strategy 3 — AES-256-GCM encryption (secrets that must be read back) + +Use for values the application needs to decrypt later: OAuth `access_token` / `refresh_token`, `client_secret`, API keys for third-party services, TLS private keys, webhook signing secrets. + +- Package: `pkg/crypto/cipher` +- DB column type: `BYTEA NOT NULL` +- Go field name: `Encrypted*` (e.g. `EncryptedConnection`, `EncryptedSigningSecret`) +- Encrypt on write: `cipher.Encrypt(plaintext, encryptionKey)` +- Decrypt on read: `cipher.Decrypt(ciphertext, encryptionKey)` +- The `cipher.EncryptionKey` is a 32-byte key loaded from configuration — never stored in the database + +Existing examples: `Connector.EncryptedConnection`, `WebhookSubscription.EncryptedSigningSecret`, `CustomDomain.EncryptedSSLPrivateKey`. + +```go +// On insert / update — encrypt before writing +connection, _ := json.Marshal(c.Connection) +encrypted, err := cipher.Encrypt(connection, encryptionKey) +c.EncryptedConnection = encrypted + +// On load — decrypt after reading +plaintext, err := cipher.Decrypt(c.EncryptedConnection, encryptionKey) +json.Unmarshal(plaintext, &c.Connection) +``` + +### Decision table + +| Data kind | Entropy source | Needs decryption? | Strategy | Go field prefix | Package | +|-----------|---------------|-------------------|----------|----------------|---------| +| Bearer / API / SCIM / one-time tokens | Application CSPRNG | No — compare by hash | SHA-256 | `Hashed*` | `pkg/crypto/hash` | +| Passwords, passphrases | Human | No — compare with constant-time check | PBKDF2 | `HashedPassword` | `pkg/crypto/passwdhash` | +| OAuth tokens, client secrets, private keys, signing secrets | External provider or application | Yes — must read back | AES-256-GCM | `Encrypted*` | `pkg/crypto/cipher` | + +### Rules + +- **Never store a plaintext secret** in a `TEXT` or `VARCHAR` column. If a column holds a secret, it must be `BYTEA` with one of the three strategies above. +- **Never log sensitive values.** Do not pass raw tokens, passwords, or decrypted secrets to `slog` or `fmt.Errorf` messages. +- **Name columns and fields consistently.** Use `hashed_` prefix for hashed values and `encrypted_` prefix for encrypted values. The Go struct field must mirror this (e.g. `HashedToken`, `EncryptedConnection`). +- **Return plaintext tokens once.** For SHA-256-hashed tokens, return the raw token to the caller at creation time only. After that, the application only ever sees the hash. +- **Migration columns.** When adding a new sensitive column, always use `BYTEA`. Never add `DEFAULT` on sensitive columns. + ## New entity checklist 1. **Entity file** (`entity.go`) — struct with `db` tags, slice type alias, `LoadByID`, `Insert`, `Update`, `Delete`, `CursorKey`, `AuthorizationAttributes` diff --git a/contrib/claude/go-style.md b/contrib/claude/go-style.md index d14d53122..b0ed92f79 100644 --- a/contrib/claude/go-style.md +++ b/contrib/claude/go-style.md @@ -177,7 +177,7 @@ var trustCenterIDKey = &ctxKey{name: "trust_center_id"} ## Logging -`go.gearno.de/kit/log` — named, context-aware structured logging with typed fields. **Never log PII, PHI, or other sensitive data** (e.g. emails, names, passwords, tokens, health records). Log opaque identifiers (IDs, request IDs) instead. +`go.gearno.de/kit/log` — named, context-aware structured logging with typed fields. **Never log PII, PHI, or other sensitive data** (e.g. emails, names, passwords, tokens, health records). Log opaque identifiers (IDs, request IDs) instead. See [`contrib/claude/logging.md`](logging.md) for the full guide (allowed/forbidden data, field helpers, wiring patterns). ```go l.InfoCtx( diff --git a/contrib/claude/logging.md b/contrib/claude/logging.md new file mode 100644 index 000000000..29450b318 --- /dev/null +++ b/contrib/claude/logging.md @@ -0,0 +1,134 @@ +# Logging — Structured, PII-free Observability + +Library: `go.gearno.de/kit/log` — named, context-aware structured logger with typed fields. + +## Golden rule + +**Only log opaque identifiers (IDs, correlation IDs, request IDs). Never log PII, PHI, credentials, or any data that can identify or harm a natural person.** + +## What must never appear in logs + +| Category | Examples | Why | +|----------|----------|-----| +| **PII** (Personally Identifiable Information) | Email addresses, full names, phone numbers, IP addresses of end-users, postal addresses, dates of birth | GDPR / privacy — log the entity ID instead | +| **PHI** (Protected Health Information) | Medical records, health status, insurance IDs | HIPAA / privacy | +| **Credentials & secrets** | Passwords, API keys, tokens (access, refresh, bearer, SCIM), signing secrets, private keys, client secrets | Security — a leaked log line becomes a breach | +| **User-generated content** | Form input, document text, chat messages, file contents | May embed PII/PHI; can also be weaponized for log injection | +| **Financial data** | Credit card numbers, bank accounts, billing addresses | PCI-DSS | + +## What is safe to log + +| Safe | Example field | +|------|--------------| +| Entity IDs (GIDs, UUIDs) | `log.String("identity_id", identity.ID.String())` | +| Correlation / request IDs | `log.String("correlation_id", correlationID)` | +| Operation names and types | `log.String("graphql_operation_name", operationName)` | +| URL paths (without query strings containing secrets) | `log.String("path", r.URL.Path)` | +| Domain names (public, not user-chosen) | `log.String("domain", domain)` | +| Counts, sizes, durations | `log.Int("count", n)`, `log.Duration("elapsed", d)` | +| Error messages from internal code | `log.Error(err)` | +| State transitions / enum values | `log.String("state", string(newState))` | +| Timestamps | `log.Time("expires_at", cert.ExpiresAt)` | + +## Logger wiring and progressive enrichment + +Loggers are constructor-injected and progressively enriched with `.Named()` / `.With()` as they flow deeper into the call stack. Every layer that has meaningful context **must** derive a child logger and attach attributes — this builds up a rich, filterable log trail without repeating fields on every call site. + +### Deriving loggers + +- **`.Named(subsystem)`** — adds a dot-separated name prefix to every log line. Use at service/component boundaries. +- **`.With(fields...)`** — returns a new logger with permanent structured fields. Use when an identifier or attribute is known for the lifetime of that scope (a request, a job iteration, a connection). + +Always assign the derived logger to a new variable or field — never mutate the parent: + +```go +// Service constructor — name the subsystem +func NewRenewer(logger *log.Logger, ...) *Renewer { + return &Renewer{ + logger: logger.Named("renewer"), + } +} + +// Worker iteration — attach the entity being processed +func (r *Renewer) renewDomain(ctx context.Context, domain CustomDomain) { + logger := r.logger.With( + log.String("domain", domain.Domain), + log.String("custom_domain_id", domain.ID.String()), + ) + + logger.InfoCtx(ctx, "starting certificate renewal") + // ... all subsequent logs in this function carry domain + custom_domain_id + logger.InfoCtx(ctx, "certificate renewed successfully") +} +``` + +### Enrichment chain + +The root logger created in `probod` flows through the system, gaining context at each layer: + +``` +probod (root) + → .Named("http.server") + → .Named("api") + → .With(log.String("correlation_id", id)) + → .Named("certmanager") + → .With(log.String("domain", d)) +``` + +A log line emitted at the leaf carries **all** ancestor attributes automatically. This means: +- You never need to repeat `correlation_id` or `organization_id` at inner call sites +- Filtering by any attribute in the chain works across the entire request span +- Adding a new attribute at one layer enriches every log line below it + +### Where to derive + +| Boundary | Derive with | Typical attributes | +|----------|------------|-------------------| +| Service/component constructor | `.Named("subsystem")` | — | +| HTTP middleware / request entry | `.With(...)` | `correlation_id`, `identity_id`, `path` | +| Worker job iteration | `.With(...)` | entity ID being processed | +| Agent / tool execution | `.Named("agent").With(...)` | `agent`, tool name | +| Retry / loop body | `.With(...)` | `attempt`, iteration key | + +### Fallback + +In HTTP handlers where no constructor-injected logger is available (e.g. panic recovery), use `httpserver.LoggerFromContext(ctx)` — it returns the request-scoped logger with all middleware-attached attributes. + +## Structured field helpers + +Use typed field constructors — never `fmt.Sprintf` into a log message for structured data: + +| Helper | Use for | +|--------|---------| +| `log.String(key, val)` | String values (IDs, domains, operation names) | +| `log.Int(key, val)` | Integer counts | +| `log.Int64(key, val)` | Large integers (byte sizes, database counts) | +| `log.Bool(key, val)` | Flags | +| `log.Float64(key, val)` | Floating-point metrics | +| `log.Duration(key, val)` | `time.Duration` values | +| `log.Time(key, val)` | `time.Time` values | +| `log.Error(err)` | Error values (key is automatically `"error"`) | +| `log.Any(key, val)` | Last resort — prefer a typed helper when one exists | + +## Context-aware methods + +Always prefer the `*Ctx` variants to propagate trace/request context: + +```go +logger.InfoCtx(ctx, "message", log.String("key", "value")) +logger.WarnCtx(ctx, "message", log.String("key", "value")) +logger.ErrorCtx(ctx, "message", log.Error(err)) +``` + +Use the non-context `Info` / `Error` only at process startup/shutdown where no request context exists. + +## Rules + +- **Always derive, never repeat.** When you enter a new scope that has a meaningful identifier (a request, a job, an entity), derive a child logger with `.Named()` or `.With()` and use it for all subsequent calls in that scope. Never pass the same attribute as an inline field on every log call — attach it once on the derived logger. +- **Log IDs, not values.** Instead of `log.String("email", user.Email)`, write `log.String("identity_id", user.ID.String())`. +- **Treat error descriptions from external sources as untrusted.** OAuth `error_description`, OIDC provider messages, and SAML responses may contain user data or be provider-controlled. Log a sanitized error code, not the full description. +- **Guard `log.Any`.** The `log.Any` helper serializes arbitrary values. Never pass structs that may contain sensitive fields. Prefer explicit field selection. +- **Never log raw HTTP bodies or query strings.** Query strings may carry `code`, `token`, or `state` parameters. Log `r.URL.Path` only. +- **Never log `fmt.Errorf` messages that embed secrets.** If an error wraps sensitive context (e.g. a decryption failure message that echoes input), strip it before logging. +- **GraphQL errors are semi-public.** `log.Any("errors", resp.Errors)` is acceptable because GraphQL errors are already filtered for the client, but never add raw input variables to log context. +- **Keep log messages static.** The message string should be a fixed human-readable sentence. Dynamic data goes in structured fields, not in `fmt.Sprintf` message templates.