The source headers, LICENSE files, and license metadata had drifted
apart. Align the entire project to MIT:
- Convert every source-file header to the MIT text across all comment
styles (Go, TS, TSX, JS, MJS, SQL, CSS, GraphQL, shell), including
SPDX-License-Identifier tags
- Set the root and cookie-banner LICENSE files to the MIT text with a
"MIT License" title line
- Switch the package.json license fields, Docker image label, and
cookie-banner README to MIT
- Update docs and the genmodels header generator accordingly
- Normalize copyright lines to a single format
(Copyright (c) <year(s)> Probo Inc <hello@probo.com>.): unify the
hello@getprobo.com and hello@probo.inc emails to hello@probo.com and
the comma-separated years to a hyphenated range
Genuine third-party references are intentionally left untouched: the
Lucide icon attributions (Lucide is ISC) and the trivy dependency
license allowlist.
Signed-off-by: Sacha Al Himdani <sacha@probo.com>
probod-bootstrap was writing empty strings and stub blocks such as
`esign: {}` into generated YAML. The post-marshal prune pass caused
part of that by stripping empty leaf strings while leaving empty
parent maps behind.
Drop the prune round-trip in WriteConfig and rely on struct-level
omitzero/omitempty tags plus custom IsZero() helpers on probodconfig.
Only include LLM providers when an API key is set, use a nil map for
extra API headers, and extend the dev-config Makefile recipe with the
local dev defaults already documented in .env.example.
Config loading is unchanged: omitted keys still decode to Go zero
values.
Signed-off-by: Ludovic Vielle <ludovic@probo.com>
Introduce a poll-based worker that fills the global common_third_parties
catalog (URLs, headquarter address, legal name, certifications, logo)
so each tenant no longer starts from sparse, name-only rows. Enrichment
is requested at row creation by ResolveOrCreateCommonThirdParty; curated
seed rows are not enqueued, to avoid a re-seed storm.
The pipeline uses two specialized agents plus a deterministic logo step.
Agent A (company profile) resolves legal name, headquarter address, and
the canonical website over web search; its website and legal name feed
Agent B and the logo step. Agent B (compliance docs) resolves the legal
document URLs, trust/security/status pages, and certifications using the
browser read-only toolset (gated on ChromeDPAddr) plus web search. The
logo step restores pkg/webinspect as a pure deterministic package and
stores the discovered icon in S3, linked via logo_file_id.
Each agent returns per-field value/confidence/source_url. The worker
writes a column only when confidence clears a configurable threshold and
the field is not externally owned (seed or human), and always records
full per-field provenance in a new enrichment JSONB column so re-runs
fill only gaps and human edits are never clobbered. New bookkeeping
columns (enrichment_requested_at, enrichment, enrichment_attempts) back
the claim queue and stale recovery; agents run outside transactions and
results persist in one final transaction.
The worker is opt-in: it no-ops unless its agent provider is configured.
Signed-off-by: Émile Ré <emile@probo.com>
The tracker-mapping worker had been reduced to catalog resolution only,
which removed not just the auto-creation of an org ThirdParty but also
the auto-linking of an existing one. Only the creation needed to go: it
raced the load-then-create check and produced duplicate vendors.
Restore the full org ThirdParty resolution (exact common-id link,
sibling direct-link, high-confidence heuristic, and the disambiguation
agent) and remove only the CreateFromCommon branch and its
categorisation gate. When nothing matches, the worker now leaves
third_party_id unset rather than creating a vendor; creation happens
exclusively through the explicit ImportFromCommon action. Drop the
now-dead CreateFromCommon helper and rename match.go to common_match.go.
Fix a latent test bug surfaced by actually running the DB-backed suite
(skipped in CI without Postgres): the heuristic-match candidate lacked
Level 1, so the level-filtered candidate loader excluded it and the old
fallback create masked the miss.
Signed-off-by: Émile Ré <emile@probo.com>
The tracker-mapping worker no longer auto-creates or auto-links org
third parties, so the heuristic ranker, the disambiguation agent, and
the catalog-to-org seeding helpers are dead code.
Delete pkg/thirdparty/match.go (RankCandidates, ScoredCandidate,
LinkToCommon, CreateFromCommon, suffix stripping, score thresholds) and
disambiguation_agent.go, along with their tests. Drop the
ThirdPartyDisambiguation agent slot and the worker's
DisambiguationAgentTimeout from probodconfig, the builder env wiring, and
the builder tests.
Remove the matching helm surface too: the thirdPartyDisambiguation agent
values, the disambiguationAgentTimeout worker tuning, and the
AGENT_THIRD_PARTY_DISAMBIGUATION_* / TRACKER_MAPPING_DISAMBIGUATION_AGENT
_TIMEOUT environment mappings, with a chart changelog note. The probod
config is built from env lookups with defaults, so a lingering value in
an older deployment is simply ignored.
Signed-off-by: Émile Ré <emile@probo.com>
The tracker-mapping, common-pattern enrichment, and third-party
disambiguation agents were all built from one shared
TrackerAgentsConfig fed by a single tracker-mapping config slot. That
forced a single AgentTimeout to be reused and patched per worker, and
two unrelated max-turns fields to share one struct.
Split the in-code config into TrackerMappingAgentConfig,
TrackerEnrichmentAgentConfig, and DisambiguationAgentConfig, each with
its own timeout and max-turns, and add dedicated tracker-enrichment and
third-party-disambiguation provider slots (the latter resolving next to
third-party-vetter). Enrichment and disambiguation fall back to the
tracker-mapping slot when their own provider is unset, preserving
single-config deployments.
Drop the shared pkg/agentsbuild package and duplicate its small wiring
into probod and proboctl so the two executables stay decoupled. Wire
the new env vars, builder test coverage, and Helm values.
Signed-off-by: Émile Ré <emile@probo.com>
Queue vetting on third_parties with PENDING, PROCESSING,
COMPLETED, and FAILED states. Expose enqueue and status through
GraphQL, MCP, CLI, and n8n, validate vet requests, tune the
worker via config, and poll the detail page while vetting runs.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
ResolveAgent aliased the default config's Temperature and MaxTokens
pointers into every agent that left them unset, so all resolved
agents shared one backing value. A mutation through any of those
pointers would corrupt the default and every other agent. It also
dereferenced the default unconditionally even though it can be nil.
Allocate a fresh pointer holding a copy of the default value, and
guard against a nil default so each resolved agent owns independent
state.
Signed-off-by: Émile Ré <emile@probo.com>
The tracker-mapping worker clears mapping_requested_at at claim time, so
a crash or hard failure between Process phases left the pattern dequeued,
unmapped, and with nothing to re-trigger it. Only an incidental sibling
remap could rescue it, so a lone pattern could stay stranded forever.
Implement the worker.StaleRecoverer interface, mirroring the enrichment
worker. ResetStaleMappings re-arms rows that were claimed but never
assigned a catalog row (common_tracker_pattern_id IS NULL) once idle past
a configurable window; a successful Process always assigns one via the
unmatched fallback, so the predicate cleanly detects interrupted runs and
self-heals after a single pass. ClearMappingRequestedAt now bumps
updated_at so the stale clock starts at claim time and the sweep never
recycles an in-flight claim.
Plumb a StaleAfter knob (default 600s) through the config struct, builder
env var, probod wiring, and Helm templates.
Signed-off-by: Émile Ré <emile@probo.com>
The tracker-mapping and common-pattern-enrichment workers ran with the
kit/worker defaults (interval 10s, max-concurrency 5 each) and dropped
the resolved per-agent max-tokens/temperature, so up to ten LLM
pipelines could run unbounded on one OpenAI client. The mapping worker
also held a FOR UPDATE transaction across the LLM and Firecrawl calls
while its DB search tools acquired a second pooled connection, risking
pool exhaustion under concurrency.
Plumb max-tokens, temperature, agent timeout, and per-worker max-turns
through TrackerAgentsConfig and DisambiguationConfig into all three
agent builders, replacing the hard-coded constants with config-fed
fields and package fallbacks. Expose worker interval, concurrency,
stale-after, agent timeout, and max-turns as config (env, Helm values,
deployment template) mirroring the evidence-describer pattern, and
apply them at registration.
Refactor Process into deterministic-read, agent (no transaction), and
persist phases so neither the mapping agent nor disambiguation runs
inside an open transaction, removing the row locks held across network
latency and the nested-connection pressure.
Signed-off-by: Émile Ré <emile@probo.com>
Firecrawl is a tool used by agents (tracker mapping, third-party
assessor), so its configuration belongs under AgentsConfig rather than
as a standalone Config field. Adds AgentToolsConfig to hold agent tool
credentials and updates all config propagation consumers.
Signed-off-by: Émile Ré <emile@probo.com>
When both pattern matching and domain matching fail to identify a
tracker, an opt-in LLM agent can now attempt identification using
internal database searches and optional web search. The agent returns
structured output (third party name, category, description, confidence)
and the worker auto-creates CommonThirdParty records when needed.
The feature is gated behind the `llm.tracker-mapping.provider` config
field; when unset the worker behaves exactly as before.
Signed-off-by: Émile Ré <emile@probo.com>
Renames the user-facing 'vendor' concept to 'third party' across the
entire codebase. The shared common_third_parties reference table is
unchanged.
Migration. Renames the vendor_category enum, the vendors and
vendor_<entity> tables (contacts, services, compliance_reports,
business_associate_agreements, data_privacy_agreements,
risk_assessments) and their vendor_id columns, the asset_vendors /
data_vendors / processing_activity_vendors junction tables,
generated_documents.vendors_document_id, the webhook_event_type
'vendor:<verb>' values, and the snapshots_type 'VENDORS' value.
Backend. Renames coredata models and SQL queries, probo services,
GraphQL / MCP API surface, console / trust / webhook resolvers and
types, the CLI (prb vendor* -> prb third-party*; pkg/cmd/vendormgmt
-> pkg/cmd/thirdpartymgmt), the document generator, vetting agent
prompts, and the common-third-parties-import command.
Frontend, packages, n8n, e2e. Renames apps/console pages, components,
hooks, routes, dialogs, and tabs; the shared @probo/vendors package
(now @probo/third-parties); the @probo/ui Vendors atoms (now
ThirdParties, VendorLogo -> ThirdPartyLogo); the n8n community node
actions/vendor folder (now actions/thirdParty); and the e2e Go test
suite (console and MCP). Filesystem and URL paths use kebab-case
(third-parties), GraphQL fields and TypeScript identifiers use
camelCase (thirdParty / thirdParties), Go types use PascalCase
(ThirdParty), and human-facing text uses 'third party' with a space.
Co-authored-by: Bryan Frimin <bryan@getprobo.com>
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
probod-bootstrap only needs the config struct definitions for
YAML marshaling but transitively pulled in ~40 heavy runtime
dependencies via pkg/probod. Move all config types and their
methods to a new pkg/probodconfig package and re-export them
from pkg/probod via type aliases for backward compatibility.
Signed-off-by: Émile Ré <emile@getprobo.com>
