Three follow-ups from review of the registry consolidation:
- Build Vercel's authorization URL with net/url instead of a
hand-rolled "{integration_slug}" placeholder resolved by
strings.ReplaceAll. The slug is escaped via url.PathEscape in a
per-provider Registration.BuildAuthURL closure, and the unused
AuthURLParams plumbing on Registration and OAuth2Connector is
removed (OAuth2Connector now carries a typed IntegrationSlug).
- Drop the SettingsInput union type and the per-provider
MarshalSettings closures. The create resolvers now build the typed
coredata.*ConnectorSettings directly from the gqlgen input, the
same way the OAuth callback path already does, so there is no
shared catch-all DTO and no stringly-typed boundary.
- Restore ConnectorProviders() to a plain ordered slice literal; the
intermediate map + slices.Sort added nondeterminism and a sort for
no benefit.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
The console previously dispatched per-provider logic through a fan of
init()-side-effect maps (driver names, OAuth2 metadata, probe URLs,
display names, settings switches), spread across pkg/connector,
pkg/accessreview/drivers and the console v1 resolvers. Adding a new
provider required edits in every one of those places and a corresponding
switch arm in CreateConnectorRequest. The same per-provider knowledge
also leaked into Helm templates as hand-rolled environment-variable
blocks per connector.
This commit collapses the dispatch surface into a single typed
*provider.Registry. The registry is constructed once by
NewBuiltinRegistry at probod startup and threaded as an explicit
dependency into every consumer (accessreview service, console v1
resolver, OAuth2 wiring). There is no package-level state. Each
provider lives in one file under pkg/connector/provider/ that exposes
a private xxxRegistration() *Registration constructor; NewBuiltinRegistry
enumerates them.
CreateConnectorRequest loses its per-provider settings fields and
takes a single RawSettings json.RawMessage produced by the
per-provider MarshalSettings closure. The 1Password SCIM bridge URL
is validated at create time (http(s) scheme + non-empty host) so a
malformed value fails fast at the resolver boundary. The Helm chart
gains probo.connectorEnv and probo.connectorSecretEntries templates
so adding a connector requires zero Helm changes. Access-review name
resolution moves into the same Registration value to keep one
authoritative dispatch table.
Tests cover every Registration (DisplayName, NewDriver wired),
Register error paths (nil, empty Provider, empty DisplayName,
duplicate), All / ProviderDisplayName / ProviderOAuth2Scopes /
ProbeURL hit and miss paths, the ApplyOAuth2Defaults templating and
PKCE branches, and ConnectorSettings[T] round-trip plus malformed-JSON
error path. The pre-refactor ApplyProviderDefaults test in
pkg/connector is replaced by the equivalent in
pkg/connector/provider.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Apply five style rules: convert iota string enums to typed
string constants, replace errors.As with errors.AsType,
merge three-group imports into two groups, fix multiline
parameter/argument formatting, and replace fmt.Sprintf URL
construction with net/url.
Signed-off-by: Émile Ré <emile@probo.com>
Introduce a hierarchical risk assessment model with six entity types:
- Risk Assessment: top-level container scoped to an organization
- Risk Assessment Scope: sub-container for scoping threat modeling
exercises within an assessment
- Risk Assessment Node: DFD elements typed as ENTITY, BOUNDARY,
ASSET, or DATA within a scope
- Risk Assessment Process: directed data flows between two nodes
- Risk Assessment Threat: descriptive threats attached to a process
with a free-text category (e.g. Confidentiality, Integrity)
- Risk Scenario: thin join linking a threat to a risk from the
register, carrying only a name and description
Risk scoring (likelihood, impact, treatment) remains on the existing
Risk entity. Threats are purely descriptive. Risk Scenarios connect
the threat model to the risk register without duplicating scores.
Backend: migration with PG enum for node types, coredata structs,
service layer with full CRUD and validation, GraphQL schema with
18 mutations and paginated connections, authorization actions and
policies, and base_resolvers.go Node dispatch for all entity types.
Frontend: Risk Assessments list page with create dialog, detail page
showing scopes as cards with nodes/processes/threats tables, inline
create/edit/delete actions on all entities, and a Scenarios tab on
the Risk detail page linking threats to risks. Existing RiskGraph.ts
hook file removed in favor of colocated queries in page files.
E2E tests cover CRUD for all entity types, RBAC, and tenant
isolation.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
Firecrawl is a tool used by agents (tracker mapping, third-party
assessor), so its configuration belongs under AgentsConfig rather than
as a standalone Config field. Adds AgentToolsConfig to hold agent tool
credentials and updates all config propagation consumers.
Signed-off-by: Émile Ré <emile@probo.com>
Firecrawl has a single public API at https://api.firecrawl.dev/v2.
The endpoint was configurable but never varied across environments,
so hardcode it as a package-level const and remove the Endpoint
field from FirecrawlConfig and all downstream wiring (bootstrap,
Helm chart, probod, vetting, cookiebanner).
Signed-off-by: Émile Ré <emile@probo.com>
SearXNG was a fallback search backend that added complexity without
being used in practice. All search-dependent features (web search,
government DB checks, vetting orchestrator, tracker mapping) now use
Firecrawl exclusively. Removes the SEARCH_ENDPOINT config plumbing
from probodconfig, bootstrap, Helm charts, and all callers.
Signed-off-by: Émile Ré <emile@probo.com>
Group firecrawl-endpoint and firecrawl-api-key under a nested firecrawl
config key. Add env var mappings (FIRECRAWL_ENDPOINT, FIRECRAWL_API_KEY,
SEARCH_ENDPOINT, AGENT_TRACKER_MAPPING_*) to the bootstrap builder with
test coverage. Wire the new values through the Helm chart (values,
deployment, secret, production example).
Signed-off-by: Émile Ré <emile@probo.com>
Firecrawl provides higher quality search results than SearXNG.
When configured (firecrawl-endpoint + firecrawl-api-key), the
tracker-mapping agent and search toolset prefer it over the
SearXNG backend. Also improves the tracker identification prompt
with multi-strategy search queries that leverage domain signals
and adapt to tracker type.
Signed-off-by: Émile Ré <emile@probo.com>
When both pattern matching and domain matching fail to identify a
tracker, an opt-in LLM agent can now attempt identification using
internal database searches and optional web search. The agent returns
structured output (third party name, category, description, confidence)
and the worker auto-creates CommonThirdParty records when needed.
The feature is gated behind the `llm.tracker-mapping.provider` config
field; when unset the worker behaves exactly as before.
Signed-off-by: Émile Ré <emile@probo.com>
Move tracker_mapping_worker.go from pkg/probo to pkg/cookiebanner and
rename worker.go to pattern_analysis_worker.go to reflect the worker
name.
Signed-off-by: Émile Ré <emile@probo.com>
Poll-based worker that maps org-scoped tracker patterns to the
common knowledge base via pattern matching and domain-based
attribution. Populates initiator_domain on detected trackers
at report time. Resolves org-scoped vendors through the common
third party link.
Signed-off-by: Émile Ré <emile@getprobo.com>
Signed-off-by: Émile Ré <emile@probo.com>
Renames the user-facing 'vendor' concept to 'third party' across the
entire codebase. The shared common_third_parties reference table is
unchanged.
Migration. Renames the vendor_category enum, the vendors and
vendor_<entity> tables (contacts, services, compliance_reports,
business_associate_agreements, data_privacy_agreements,
risk_assessments) and their vendor_id columns, the asset_vendors /
data_vendors / processing_activity_vendors junction tables,
generated_documents.vendors_document_id, the webhook_event_type
'vendor:<verb>' values, and the snapshots_type 'VENDORS' value.
Backend. Renames coredata models and SQL queries, probo services,
GraphQL / MCP API surface, console / trust / webhook resolvers and
types, the CLI (prb vendor* -> prb third-party*; pkg/cmd/vendormgmt
-> pkg/cmd/thirdpartymgmt), the document generator, vetting agent
prompts, and the common-third-parties-import command.
Frontend, packages, n8n, e2e. Renames apps/console pages, components,
hooks, routes, dialogs, and tabs; the shared @probo/vendors package
(now @probo/third-parties); the @probo/ui Vendors atoms (now
ThirdParties, VendorLogo -> ThirdPartyLogo); the n8n community node
actions/vendor folder (now actions/thirdParty); and the e2e Go test
suite (console and MCP). Filesystem and URL paths use kebab-case
(third-parties), GraphQL fields and TypeScript identifiers use
camelCase (thirdParty / thirdParties), Go types use PascalCase
(ThirdParty), and human-facing text uses 'third party' with a space.
Co-authored-by: Bryan Frimin <bryan@getprobo.com>
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
Fetch favicons at import time instead of calling Google's favicon
service per page load. Logos are stored as public files in S3 and
served through the existing /api/files/v1/{id} endpoint.
Signed-off-by: Émile Ré <emile@getprobo.com>
The CreateVendorDialog previously loaded the entire @probo/vendors
JSON bundle client-side and used MiniSearch for fuzzy search. This
replaces it with a GraphQL query against the common_third_parties
database table, searched server-side via ILIKE filtering.
Backend: adds CommonThirdParty GraphQL type, a pkg/thirdparty
service, and a commonThirdParties(name) root query. Frontend:
splits into CommonThirdPartyCombobox (display) and an @inline
fragment read on selection via readInlineData.
Signed-off-by: Émile Ré <emile@getprobo.com>
Resolve the visitor's IP to a country code via the geoloc service and
map it to the applicable privacy regulation (GDPR, UK GDPR, FADP, CCPA,
PIPEDA, LGPD, LFPDPPP, POPIA, PDPA, PIPL, PIPA, APPI, DPDP, PDPL).
The regulation and its implied consent mode (OPT_IN / OPT_OUT) are
injected into the GET /config response so the SDK can adapt its behavior.
Also makes geoloc.Service self-contained: LookupCountry and IsPopulated
now manage their own DB connections instead of requiring callers to pass
a pg.Querier.
Signed-off-by: Émile Ré <emile@getprobo.com>
Introduce a geoloc package that stores CIDR-to-country mappings in
PostgreSQL using the native cidr type with a GiST index for fast
containment lookups. Data comes from the ipverse/country-ip-blocks
dataset added as a git submodule.
A standalone geoloc-import command reads the TXT files from disk
and bulk-loads them via COPY. probod wires the service and logs a
warning when the table is empty.
Signed-off-by: Émile Ré <emile@getprobo.com>
The worker now operates on TrackerPattern/DetectedTrackers
instead of CookiePattern/Cookies, with TrackerType included
in merge group keys to prevent cross-type merging.
Signed-off-by: Émile Ré <emile@getprobo.com>
The HTTP middleware and proxy-protocol listeners both pinned trust
to exact IPs, which forced re-applying terraform every time AWS
rotated an ALB or NLB ENI. Trusted-proxies entries now accept CIDR
ranges in addition to plain IPs, so callers can trust whole subnets
(where the load balancer ENIs always live) and stop chasing
rotating IPs.
The HTTP middleware splits parsed entries into IPs and IPNets and
checks both. The proxy-protocol listeners switch from
TrustProxyHeaderFrom (IP-only, REJECT) to ConnStrictWhiteListPolicy
(IP or CIDR, REJECT) which preserves the existing reject-on-unknown
semantics.
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
Background worker polls cookie_banners with pattern_analysis_requested_at
set, groups EXACT patterns sharing a common prefix, and merges groups of
3+ into a PREFIX pattern. Detection sets the flag when new EXACT patterns
are created. The worker relinks cookies, removes orphaned patterns, and
updates the draft version via ensureDraftVersionForBanner.
Signed-off-by: Émile Ré <emile@getprobo.com>
probod-bootstrap only needs the config struct definitions for
YAML marshaling but transitively pulled in ~40 heavy runtime
dependencies via pkg/probod. Move all config types and their
methods to a new pkg/probodconfig package and re-export them
from pkg/probod via type aliases for backward compatibility.
Signed-off-by: Émile Ré <emile@getprobo.com>
Move PDF generation from synchronous publish flow to a background polling
job. Published versions with file_id IS NULL are picked up by the job,
which generates the PDF, uploads to S3, and links the file. Export PDF
now serves stored files for published versions (with optional signature
page and watermark) and generates on the fly for drafts.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
Introduces a BRANDING boolean config (default true) propagated through
the standard config pipeline. Cookie banners now initialize their
show_branding column from this config instead of hardcoding true.
Signed-off-by: Émile Ré <emile@getprobo.com>
The OAuth2/OIDC server accepted its signing key via a file path
(key-file), while every other PEM key in the probod config (SAML
private key, ACME account key) is embedded inline. Switch the
field to a private-key string so the convention is uniform.
The signing key is operator-supplied material that must outlive
any process restart, so the bootstrap builder now treats
OAUTH2_SERVER_SIGNING_KEY as required and refuses to start
without one; silently minting a fresh key per boot would break
token validation across rollouts. The OAUTH2_SERVER_* env vars
otherwise flow through builder.Build like the existing SAML
block so the new OAuth2Server section is populated end-to-end.
Rework the e2e harness to render its config via bootstrap at
test setup, which removes the static
e2e/console/testdata/config.yaml and the previously generated
test-only PEM file. A per-run RSA key is minted via
bootstrap.GenerateOAuth2SigningKey (kept public for test
tooling) and injected through the builder env map. CI now
passes ACME_ROOT_CA inline instead of mutating a YAML on disk.
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
Implement a full OAuth2 2.0 and OpenID Connect 1.0 authorization
server with support for authorization code flow (with PKCE),
refresh token rotation, device authorization grant, dynamic
client registration, token introspection, and token revocation.
Includes database schema, coredata layer, service logic, HTTP
handlers, OIDC discovery endpoint, and JWKS publishing.
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
Strip forwarded headers (Forwarded, X-Forwarded-For, X-Real-Ip)
from requests originating from untrusted proxies at the HTTP
server level, reusing the existing proxy-protocol trusted-proxies
config. The clientip package is now a pure extraction helper;
context plumbing and middleware wrappers are removed.
Signed-off-by: Émile Ré <emile@getprobo.com>
Nest membership ID, role, and state into a membership sub-object
in the user webhook payload. Also emit user:updated webhook when
the membership role is changed. Add X-Probo-Webhook-Host header
to webhook HTTP calls. Skip delete webhook when membership is
not found in SCIM user deletion.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
Replace hand-rolled polling loops, semaphores, and WaitGroups
in all 7 background workers with go.gearno.de/kit/worker. Each
worker now implements Handler[T] (Claim/Process) and optionally
StaleRecoverer, gaining automatic Prometheus metrics and
OpenTelemetry tracing. Bumps kit from v0.3.0 to v0.5.0.
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
Compute the OAuth2 redirect URI from the base URL using the
CallbackPath constant and apply provider defaults before
registering each connector.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
All other OAuth2 properties (redirect URI, auth URL, token URL,
scopes, extra params, token endpoint auth) now come from the
connector package provider definitions at wiring time.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Replace the monolithic agents config with a cleaner structure:
- llm: holds provider credentials and default model settings
- probo-agent: LLM overrides for the probo agent
- evidence-describer: worker config (interval, stale-after,
max-concurrency) alongside LLM overrides
This makes worker tuning configurable via YAML and env vars
instead of being hardcoded in Go, and separates provider
credentials from per-consumer model settings.
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
Add AccessReview field to server.Config and api.Config,
pass through to console and MCP NewMux. Create the
service in probod and run its background workers.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Add API key connector protocol, OAuth2 client credentials
grant, token refresh config, provider info endpoint,
ConnectorProviders helper, and bootstrap configs for all
OAuth providers. Move OAuth2 state decode near type.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Introduce a background worker that automatically generates
compliance-focused descriptions for uploaded evidence files
using configurable LLM providers. Descriptions are surfaced
across all interfaces: GraphQL API, MCP API, CLI, and the
console UI.
Key changes:
- Multi-provider LLM config with per-agent settings (pointer
types for Temperature/MaxTokens to preserve zero values)
- Evidence description worker with bounded concurrency
- EvidenceDescriptionStatus typed enum with PostgreSQL enum type
- New `prb evidence` CLI commands (list, view, delete)
- Evidence description displayed in console table and preview
- Migration only marks evidences without files as completed
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
Before requesting a certificate from the ACME provider, verify
that CAA DNS records for the domain permit issuance by the
configured CA. This avoids wasting ACME attempts on domains
whose CAA policy would reject the request.
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
Implements OpenID Connect authentication flow with PKCE, JWT verification, and enterprise-only account restrictions. Adds OIDC service with JWKS caching and state management, HTTP handlers for login/callback flows, GraphQL query for available providers, and sign-in UI integration.
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
Use baseurl.Parse to construct the HTTPS redirect URL in the
trust center HTTP handler, breaking the taint chain from raw
request headers. Apply path.Clean to the slug-based redirect
in stripTrustPrefix to normalize path traversal sequences.
Addresses CodeQL go/unvalidated-url-redirection (CWE-601).
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
Adds a visibility enum to files with PRIVATE (default) and PUBLIC states.
PUBLIC files are accessible via an unauthenticated /api/files/v1/{fileID}
endpoint that redirects to a presigned S3 URL. Introduces pkg/file service
to manage file operations. Logo uploads (trust centers, organizations,
frameworks, references) are marked PUBLIC; other files are PRIVATE.
Includes database migration and backfill for existing logos.
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
Expand mixed inline/multiline function calls so each argument
is on its own line, matching the one-argument-per-line rule.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
