The tracker-mapping, common-pattern enrichment, and third-party
disambiguation agents were all built from one shared
TrackerAgentsConfig fed by a single tracker-mapping config slot. That
forced a single AgentTimeout to be reused and patched per worker, and
two unrelated max-turns fields to share one struct.
Split the in-code config into TrackerMappingAgentConfig,
TrackerEnrichmentAgentConfig, and DisambiguationAgentConfig, each with
its own timeout and max-turns, and add dedicated tracker-enrichment and
third-party-disambiguation provider slots (the latter resolving next to
third-party-vetter). Enrichment and disambiguation fall back to the
tracker-mapping slot when their own provider is unset, preserving
single-config deployments.
Drop the shared pkg/agentsbuild package and duplicate its small wiring
into probod and proboctl so the two executables stay decoupled. Wire
the new env vars, builder test coverage, and Helm values.
Signed-off-by: Émile Ré <emile@probo.com>
Move agent-run orchestration from the legacy supervisor path into the new
agentrun worker/service package and wire it through coredata, server,
policies, and GraphQL resolvers.
This consolidates run lifecycle handling around lease-aware workers and
aligns API surface with the new agent-run domain model so reviewers can
follow one coherent execution path.
Signed-off-by: Bryan Frimin <bryan@probo.com>
Google Workspace SMTP relay (smtp-relay.gmail.com) rejects generic
EHLO identifiers such as "localhost", which is Go net/smtp's default.
Operators can now set SMTP_HELLO_NAME to a valid hostname so that
c.Hello() is called before StartTLS, satisfying strict relay policies.
Fixes https://github.com/getprobo/probo/issues/1284
Signed-off-by: Bryan Frimin <bryan@probo.com>
`jo -a` is waiting on stdin; adding `< /dev/null` allows to
generate an empty array without waiting on stdin.D
Signed-off-by: Ludovic Vielle <ludovic@probo.com>
Queue vetting on third_parties with PENDING, PROCESSING,
COMPLETED, and FAILED states. Expose enqueue and status through
GraphQL, MCP, CLI, and n8n, validate vet requests, tune the
worker via config, and poll the detail page while vetting runs.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
The tracker-mapping and common-pattern enrichment agents ran with
4 and 3 max turns. Those budgets are smaller than the tool-call
rounds their prompts authorize (two/one DB searches plus up to
three web searches) plus the forced structured-output synthesis
turn, so the run loop tripped MaxTurnsExceededError before it
could emit JSON.
Raise the budgets to 10 across every layer that drives them: the
bootstrap env-var defaults, the Helm chart values and production
example, and the in-code fallback. Update the bootstrap test
defaults to match.
Revert the prompt instructions that told the model to reserve a
turn for its final answer; the model cannot reliably count its
remaining turns, so the fix belongs in the turn budget rather
than the prompt text.
Signed-off-by: Émile Ré <emile@probo.com>
The tracker-mapping worker clears mapping_requested_at at claim time, so
a crash or hard failure between Process phases left the pattern dequeued,
unmapped, and with nothing to re-trigger it. Only an incidental sibling
remap could rescue it, so a lone pattern could stay stranded forever.
Implement the worker.StaleRecoverer interface, mirroring the enrichment
worker. ResetStaleMappings re-arms rows that were claimed but never
assigned a catalog row (common_tracker_pattern_id IS NULL) once idle past
a configurable window; a successful Process always assigns one via the
unmatched fallback, so the predicate cleanly detects interrupted runs and
self-heals after a single pass. ClearMappingRequestedAt now bumps
updated_at so the stale clock starts at claim time and the sweep never
recycles an in-flight claim.
Plumb a StaleAfter knob (default 600s) through the config struct, builder
env var, probod wiring, and Helm templates.
Signed-off-by: Émile Ré <emile@probo.com>
The tracker mapping, common-pattern enrichment, and third-party
disambiguation agents default to a small max-tokens budget on the
premise that their final output is tiny structured JSON. On
reasoning models such as the GPT-5 family, reasoning tokens count
against max_tokens, so a small budget is consumed by reasoning and
the JSON is truncated, surfacing as "unexpected end of JSON input".
Raise the defaults to 4096 (1024 -> 4096 for tracker mapping, 512
-> 4096 for disambiguation) to leave headroom for the reasoning
phase. Update the bootstrap builder default, its test, and the
production values example to match.
Signed-off-by: Émile Ré <emile@probo.com>
The tracker-mapping and common-pattern-enrichment workers ran with the
kit/worker defaults (interval 10s, max-concurrency 5 each) and dropped
the resolved per-agent max-tokens/temperature, so up to ten LLM
pipelines could run unbounded on one OpenAI client. The mapping worker
also held a FOR UPDATE transaction across the LLM and Firecrawl calls
while its DB search tools acquired a second pooled connection, risking
pool exhaustion under concurrency.
Plumb max-tokens, temperature, agent timeout, and per-worker max-turns
through TrackerAgentsConfig and DisambiguationConfig into all three
agent builders, replacing the hard-coded constants with config-fed
fields and package fallbacks. Expose worker interval, concurrency,
stale-after, agent timeout, and max-turns as config (env, Helm values,
deployment template) mirroring the evidence-describer pattern, and
apply them at registration.
Refactor Process into deterministic-read, agent (no transaction), and
persist phases so neither the mapping agent nor disambiguation runs
inside an open transaction, removing the row locks held across network
latency and the nested-connection pressure.
Signed-off-by: Émile Ré <emile@probo.com>
Address the highest-impact Kubernetes chart issues from #1245.
Align PG_ADDR with postgresql.host/port, wire
OAUTH2_SERVER_SIGNING_KEY, add early validation for required
base64 and PEM secrets, and isolate the main service/deployment
with component labels so Chrome pods are not selected by server
traffic.
Update chart values and documentation to clarify required secret
formats, managed PostgreSQL prerequisites, ACME account key
persistence, and current Azure Blob compatibility caveats for S3
proxy deployments.
Signed-off-by: Cursor Agent <cursoragent@cursor.com>
Manual moves of a non-extension TrackerPattern lacking a ThirdPartyID
now request mapping, which the tracker-mapping worker resolves with a
four-stage pipeline: exact common_third_party_id link, heuristic
ranking, agent disambiguation, and finally CreateFromCommon. Existing
fuzzy-matched org rows are tagged with common_third_party_id so the
next promotion takes the O(1) exact-link path.
The matching primitives live in pkg/thirdparty (RankCandidates,
LinkToCommon, CreateFromCommon, ScoredCandidate, threshold constants)
so the disambiguation agent and the heuristic share one candidate
type. Cookiebanner orchestrates them; cookie-banner-specific concerns
(pattern -> common-pattern -> common-party navigation, the EXTENSION
gate, and structured logs) stay in the worker.
Signed-off-by: Émile Ré <emile@probo.com>
The authorize/Authorize helpers (GraphQL and MCP) already return the
*coredata.Scope resolved from the resource's organization_id attribute,
but several resolvers discarded it and rebuilt the scope with
coredata.NewScopeFromObjectID(...) right after. NewScopeFromObjectID
only reads the tenant encoded in the GID, while the authorizer derives
the scope from loaded resource attributes, so the two silently drift if
the resource lookup ever changes.
Capture scope from authorize and feed it straight to the service/coredata
layer. For the LinkX/UnlinkX MCP tools, move the per-case Authorize
inside the switch and drop the shared scope so each case owns its own
authorization result. Document the rule in contrib/claude/authorization.md
and add a matching .cursor/rules/go-authorize-scope.mdc, including the
narrow exception for global-catalog authorize calls (e.g. identity-scoped
ActionCommonThirdPartyList) where downstream services take no scope.
Signed-off-by: Émile Ré <emile@probo.com>
The repo guide already prescribed the seven-rules format but did
not rule out the Conventional-Commits alternative, and recent
history has drifted into mixing both styles. Several existing
commits use a "fix(scope): ..." prefix that the project does not
consume for any tooling (no changelog generator, no semantic
release, no commit-lint), so the prefix only adds noise and hurts
log readability.
Add an alwaysApply Cursor rule that explicitly bans Conventional
Commits and restates the seven-rules format with concrete good and
bad examples. Update contrib/claude/commit.md with the same
prohibition so the documentation and the rule agree, and
cross-link both from the existing signing rule so an agent reading
git-commit-signing.mdc lands on the style rule too.
Signed-off-by: Émile Ré <emile@probo.com>
Describe AuthorizeBatch semantics (all-or-nothing, single-entity-type,
single-organization, batch attribute requirement, audit logging), the
authz.NewBatchAuthorizeFunc / WithBatch* helpers, the MCP
Resolver.AuthorizeBatch entry point, and the new batch-style
AuthorizationAttributes implementation contract (non-empty,
deduplicated, same-entity-type input; only found rows returned).
Signed-off-by: Bryan Frimin <bryan@probo.com>
url.JoinPath does not percent-encode slashes or reserved characters in
its arguments, so user-supplied values (group IDs, slugs, team IDs) must
be wrapped with url.PathEscape to prevent path traversal. Update cursor
rule and contrib guide to codify this as a mandatory practice.
Signed-off-by: Émile Ré <emile@probo.com>
Remove dead 23505 checks on single-GID primary keys (oauth2_consent,
risk_assessment, risk_assessment_scenario, risk_assessment_scope).
Add missing constraints to membership_profile and
statement_of_applicability. Document composite-PK vs GID-PK rule in
cursor rules and contrib guide.
Signed-off-by: Émile Ré <emile@probo.com>
Firecrawl is a tool used by agents (tracker mapping, third-party
assessor), so its configuration belongs under AgentsConfig rather than
as a standalone Config field. Adds AgentToolsConfig to hold agent tool
credentials and updates all config propagation consumers.
Signed-off-by: Émile Ré <emile@probo.com>
Replace the cross-entity JOIN in
DetectedTrackers.LoadCommonThirdPartyIDByDomainMatch with two
idiomatic coredata calls: LoadInitiatorDomainsByTrackerPatternID
on DetectedTrackers, then a new CommonThirdPartyDomains.Load with
a CommonThirdPartyDomainFilter. Each entity now queries only its
own table, and the caller orchestrates the lookup.
Document the Load vs LoadAll naming convention and the no
cross-entity JOINs rule in contrib/claude/coredata.md.
Signed-off-by: Émile Ré <emile@probo.com>
Upsert methods now RETURNING all struct columns and scan the result
back into the pointer receiver, keeping the caller in sync with the
actual DB state (id, created_at, etc. from the existing row on
conflict). Insert detection compares the saved original ID with the
returned ID instead of relying on the PostgreSQL-internal xmax column.
Signed-off-by: Émile Ré <emile@probo.com>
Checking only pgErr.Code == "23505" is insufficient when a
table has multiple unique constraints. Always verify
pgErr.ConstraintName as well to avoid mapping unrelated
violations to the wrong sentinel error.
Signed-off-by: Émile Ré <emile@probo.com>
Generate the category list from coredata.ThirdPartyCategories() at
runtime instead of hardcoding it in the prompt text. Type the
TrackerIdentification.Category field as coredata.ThirdPartyCategory so
JSON unmarshaling validates values automatically.
Also documents the .txt.tmpl template file naming convention.
Signed-off-by: Émile Ré <emile@probo.com>
Firecrawl has a single public API at https://api.firecrawl.dev/v2.
The endpoint was configurable but never varied across environments,
so hardcode it as a package-level const and remove the Endpoint
field from FirecrawlConfig and all downstream wiring (bootstrap,
Helm chart, probod, vetting, cookiebanner).
Signed-off-by: Émile Ré <emile@probo.com>
SearXNG was a fallback search backend that added complexity without
being used in practice. All search-dependent features (web search,
government DB checks, vetting orchestrator, tracker mapping) now use
Firecrawl exclusively. Removes the SEARCH_ENDPOINT config plumbing
from probodconfig, bootstrap, Helm charts, and all callers.
Signed-off-by: Émile Ré <emile@probo.com>
Update go-style guide and cursor rule to clarify that even a single
argument spanning multiple lines must break after the opening
parenthesis. Fix six violations across the branch.
Signed-off-by: Émile Ré <emile@probo.com>
Group firecrawl-endpoint and firecrawl-api-key under a nested firecrawl
config key. Add env var mappings (FIRECRAWL_ENDPOINT, FIRECRAWL_API_KEY,
SEARCH_ENDPOINT, AGENT_TRACKER_MAPPING_*) to the bootstrap builder with
test coverage. Wire the new values through the Helm chart (values,
deployment, secret, production example).
Signed-off-by: Émile Ré <emile@probo.com>
- Drop Helm wiring for the 9 new access-review connectors
- Drop e2e TODO about access-review connector coverage gap
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
- Require CLIENT_SECRET for new access-review connectors
- Use Heroku account UUID as ExternalID
- Bump GitHub orgs picker to per_page=100
- Drop Snyk, Ramp, Lever, Deel access-review providers
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
- Strip email local-part from cassette guard failure
- Drop dead src/components/connectors vite include
- Cover all 13 access-review providers in builder test
- Wire 13 access-review connectors in Helm chart
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
- Wire post-OAuth pickers and 2-auto callback handling
- Add 13 vendor logo components for new connectors
- Wire access-review connectors into bootstrap config
- Track e2e gap for new access-review connectors
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
The register/document model has fully replaced the snapshot system.
Delete every snapshot-scoped row and strip the application code that
referenced them: SnapshotID/SourceID struct fields, snapshot_id IS NULL
filters, snapshot columns from SELECT/INSERT statements and named args,
and the eight migrate-*-snapshots-to-documents one-shot tools.
The remaining snapshot_id / source_id columns, the snapshots and
controls_snapshots tables, the snapshots_type enum, and the
snapshot-scoped indexes are now unused; they are dropped in a follow-up
schema migration so this change can roll back cleanly without leaving
orphaned data.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
Renames the user-facing 'vendor' concept to 'third party' across the
entire codebase. The shared common_third_parties reference table is
unchanged.
Migration. Renames the vendor_category enum, the vendors and
vendor_<entity> tables (contacts, services, compliance_reports,
business_associate_agreements, data_privacy_agreements,
risk_assessments) and their vendor_id columns, the asset_vendors /
data_vendors / processing_activity_vendors junction tables,
generated_documents.vendors_document_id, the webhook_event_type
'vendor:<verb>' values, and the snapshots_type 'VENDORS' value.
Backend. Renames coredata models and SQL queries, probo services,
GraphQL / MCP API surface, console / trust / webhook resolvers and
types, the CLI (prb vendor* -> prb third-party*; pkg/cmd/vendormgmt
-> pkg/cmd/thirdpartymgmt), the document generator, vetting agent
prompts, and the common-third-parties-import command.
Frontend, packages, n8n, e2e. Renames apps/console pages, components,
hooks, routes, dialogs, and tabs; the shared @probo/vendors package
(now @probo/third-parties); the @probo/ui Vendors atoms (now
ThirdParties, VendorLogo -> ThirdPartyLogo); the n8n community node
actions/vendor folder (now actions/thirdParty); and the e2e Go test
suite (console and MCP). Filesystem and URL paths use kebab-case
(third-parties), GraphQL fields and TypeScript identifiers use
camelCase (thirdParty / thirdParties), Go types use PascalCase
(ThirdParty), and human-facing text uses 'third party' with a space.
Co-authored-by: Bryan Frimin <bryan@getprobo.com>
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
