When the common-third-party enrichment worker discovers owned domains
for a vendor, tracker patterns that were detected and left unmatched
before those domains were known had nothing to re-trigger them.
Add RequestMappingForUnmappedByInitiatorDomains, a global re-arm that
stamps mapping_requested_at on still-unmapped patterns whose detected
trackers share one of the new domains, and call it from the enrichment
worker's persist step for newly-inserted domains only. The mapping
worker then re-resolves them through its existing domain-overlap path.
Targeting is limited to patterns with no resolved vendor (no org third
party and an absent or unlinked catalog row), so a pattern already
attributed to this or any other vendor is never disturbed.
Signed-off-by: Émile Ré <emile@probo.com>
Add a domain-discovery step to the enrichment pipeline so the catalog's
domain set, previously written only by the curated seed, grows
automatically. A focused agent enumerates the registrable domains a
vendor owns and operates - marketing, product and sub-brand, app, API,
and CDN/asset domains - from links seen while browsing and from web
search, anchored on the website resolved earlier in the run.
A deterministic ownership gate reduces the candidates to eTLD+1 and
keeps only those that clear a strict confidence floor and match the
vendor by domain label. Shared tracker-delivery and CDN infrastructure
is dropped unless the vendor itself is that provider, in which case its
own brand-matching domain passes a stricter exact-label check. The
survivors are upserted into common_third_party_domains in the run's
final transaction and recorded in the enrichment payload, feeding the
tracker-mapping domain step and disambiguation.
Signed-off-by: Émile Ré <emile@probo.com>
Give the company-profile agent (Agent A) the read-only browser toolset
and build it per-run, so it can read footer, imprint, about, and legal
pages and follow a product domain to the corporate one to resolve the
legal name and headquarters address rather than failing cold.
Make the website the hard precondition: when Agent A cannot resolve a
canonical website, skip the compliance-docs agent and logo step instead
of running them blind, which previously produced inconsistent
cross-domain document URLs. Fall back to the catalog display name for
the legal name when nothing better is found, recorded with a distinct
provenance status so a later real find overwrites it.
Rewrite both enrichment prompts in the project's role/task/instructions
XML style, add a domain-consistency rule for document URLs and a
tool-budget directive, and document the prompt style as a rule and guide.
Fix the find_links_matching browser tool, which double-encoded its
pattern and made JSON.parse fail on every keyword, starving any agent
that used it until it hit the turn cap. Salvage output when an agent
exhausts its turn budget while still exploring with a pending structured
output by forcing one final synthesis turn instead of failing outright.
Signed-off-by: Émile Ré <emile@probo.com>
Introduce a poll-based worker that fills the global common_third_parties
catalog (URLs, headquarter address, legal name, certifications, logo)
so each tenant no longer starts from sparse, name-only rows. Enrichment
is requested at row creation by ResolveOrCreateCommonThirdParty; curated
seed rows are not enqueued, to avoid a re-seed storm.
The pipeline uses two specialized agents plus a deterministic logo step.
Agent A (company profile) resolves legal name, headquarter address, and
the canonical website over web search; its website and legal name feed
Agent B and the logo step. Agent B (compliance docs) resolves the legal
document URLs, trust/security/status pages, and certifications using the
browser read-only toolset (gated on ChromeDPAddr) plus web search. The
logo step restores pkg/webinspect as a pure deterministic package and
stores the discovered icon in S3, linked via logo_file_id.
Each agent returns per-field value/confidence/source_url. The worker
writes a column only when confidence clears a configurable threshold and
the field is not externally owned (seed or human), and always records
full per-field provenance in a new enrichment JSONB column so re-runs
fill only gaps and human edits are never clobbered. New bookkeeping
columns (enrichment_requested_at, enrichment, enrichment_attempts) back
the claim queue and stale recovery; agents run outside transactions and
results persist in one final transaction.
The worker is opt-in: it no-ops unless its agent provider is configured.
Signed-off-by: Émile Ré <emile@probo.com>
Use the rebased filemanager method name in the regression test after
main split the S3 URL helpers into their own file.
Signed-off-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ludovic <ludovic@probo.com>
Use an ASCII-only fallback for Content-Disposition filename and
RFC 5987 path escaping for filename* so presigned S3 responses keep
spaces and Unicode filenames interoperable.
Signed-off-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ludovic <ludovic@probo.com>
Trust GraphQL and MCP still exposed presigned URL strings for
trust-center logos while console and connect already serve stable
File.downloadUrl paths. Phase 1 migrates the seven public logo
fields on trust GraphQL and the trust-center file references on MCP
to the shared File type; trust GraphQL NDA stays on fileUrl for a
follow-up.
Trust resolvers load public files through filemanager and map them
with types.NewFile. The trust app Relay queries and components now
read logo.downloadUrl. MCP specification, resolvers, and helpers
are updated in sync, including NDA on MCP where callers already
have file access.
filemanager is split into focused files and its URL surface is
narrowed to GenerateFileURL(file) for stable app URLs and
GeneratePresignedURL for S3 redirects. GetPublicFile remains the
DB entry point when only a file ID is known.
Add trust and MCP e2e coverage for public logo download URLs.
Signed-off-by: Ludovic Vielle <ludovic@probo.com>
Replace Organization.logoUrl and horizontalLogoUrl with nested File
objects whose downloadUrl points at /api/files/v1/public/{id}, matching
the Console migration.
Org logos are FileVisibilityPublic and served without HTTP auth, so
Connect File.downloadUrl is built eagerly in NewFile with no field-level
authorize. Logo loading moves to iam.OrganizationService.LogoFile and
HorizontalLogoFile; the old URL generators are removed.
Sync IAM Relay components and n8n organization operations. Add an e2e
test for Connect multipart logo upload and ExecuteConnectWithFile.
Signed-off-by: Ludovic Vielle <ludovic@probo.com>
Replace the bare `getprobo.com` domain and the `www.getprobo.com`
marketing host with `probo.com` / `www.probo.com` across the codebase.
Functional subdomains (app, console, notification, custom, test,
cookie-banner, compliance) keep their existing `getprobo.com` hosts,
and changelog entries are left untouched.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
The Select content class listed overflow-y-auto twice. Drop the
redundant copy; the remaining one keeps scrolling behavior unchanged.
Signed-off-by: Émile Ré <emile@probo.com>
The tracker policy document deduped third parties by name and kept the
first-seen row wholesale. Org third parties are appended before catalog
common vendors so user-editable data wins, but an org row may leave the
description or privacy policy URL empty. In that case the early return
discarded the later common-vendor row entirely, dropping metadata that
would have completed the document.
Track the kept row by name and backfill only its empty fields from later
duplicates, so org data still wins while common-vendor metadata is no
longer lost.
Signed-off-by: Émile Ré <emile@probo.com>
Stack the tracker type badge above the pattern name and drop the
separate Type column so the now-wider row reads more compactly; the
edit row spans the reduced column count. Tint script-sourced rows light
blue (excluded styling still wins when both apply).
Make the move-to-category select stand out when the tracker sits in a
real category by switching it to the highlighted pill variant, and align
its ghost-variant padding so the label does not shift horizontally
between categorised and uncategorised rows.
Signed-off-by: Émile Ré <emile@probo.com>
Moving a tracker to a category used to auto-create an org third party in
the mapping worker, so the move was gated behind a confirmation that
warned about it. The worker now only links to an existing vendor and
never creates one, making the prompt inaccurate.
Remove the confirm dialog and move directly on selection, folding the
same-category no-op guard into handleMove.
Signed-off-by: Émile Ré <emile@probo.com>
The tracker-mapping worker had been reduced to catalog resolution only,
which removed not just the auto-creation of an org ThirdParty but also
the auto-linking of an existing one. Only the creation needed to go: it
raced the load-then-create check and produced duplicate vendors.
Restore the full org ThirdParty resolution (exact common-id link,
sibling direct-link, high-confidence heuristic, and the disambiguation
agent) and remove only the CreateFromCommon branch and its
categorisation gate. When nothing matches, the worker now leaves
third_party_id unset rather than creating a vendor; creation happens
exclusively through the explicit ImportFromCommon action. Drop the
now-dead CreateFromCommon helper and rename match.go to common_match.go.
Fix a latent test bug surfaced by actually running the DB-backed suite
(skipped in CI without Postgres): the heuristic-match candidate lacked
Level 1, so the level-filtered candidate loader excluded it and the old
fallback create masked the miss.
Signed-off-by: Émile Ré <emile@probo.com>
A tracker pattern resolved only to a catalog vendor now shows an "Import
to third parties" action in its row menu. It calls
importThirdPartyFromCommon for the pattern's common third party and, on
success, links the imported org vendor onto the row so it stops showing
the "Common catalog" badge.
Sibling patterns of the same vendor are backfilled server-side and pick
up the link on the next fetch of the list, so the store update only
needs to reflect the clicked row.
Signed-off-by: Émile Ré <emile@probo.com>
Expose the explicit import action over the console API. The mutation
takes an organization and a common third party, authorizes as a
third-party create, and delegates to ThirdPartyService.ImportFromCommon,
returning the org ThirdParty edge plus a created flag so the client can
tell a fresh import from a re-import.
Add an end-to-end test covering the two behaviours that matter: the
first import seeds the org vendor from the catalog and backfills the
linked tracker pattern's third_party_id, and a second import is
idempotent, returning the same row with created=false.
The gqlgen-generated types and execution code are build artifacts (not
tracked), so only the schema and the resolver change here.
Signed-off-by: Émile Ré <emile@probo.com>
Add the partial unique index that enforces one org ThirdParty per catalog
vendor per organization, on (organization_id, common_third_party_id)
where common_third_party_id is not null. Manually created vendors (null
common id) stay unconstrained.
Existing data may already hold duplicates left by the old auto-create
race, which would make the index build fail, so a preceding DO block
merges them first. It maps each duplicate onto the earliest-created
survivor and, driven by pg_constraint / pg_index introspection, repoints
every foreign key that references third_parties(id) onto the survivor,
removing link rows that would collide on a referencing table's unique
key before repointing, then deletes the extras.
The dedupe is destructive and introspection-driven; validate it against a
production dump before deploying.
Signed-off-by: Émile Ré <emile@probo.com>
The tracker-mapping worker no longer auto-creates or auto-links org
third parties, so the heuristic ranker, the disambiguation agent, and
the catalog-to-org seeding helpers are dead code.
Delete pkg/thirdparty/match.go (RankCandidates, ScoredCandidate,
LinkToCommon, CreateFromCommon, suffix stripping, score thresholds) and
disambiguation_agent.go, along with their tests. Drop the
ThirdPartyDisambiguation agent slot and the worker's
DisambiguationAgentTimeout from probodconfig, the builder env wiring, and
the builder tests.
Remove the matching helm surface too: the thirdPartyDisambiguation agent
values, the disambiguationAgentTimeout worker tuning, and the
AGENT_THIRD_PARTY_DISAMBIGUATION_* / TRACKER_MAPPING_DISAMBIGUATION_AGENT
_TIMEOUT environment mappings, with a chart changelog note. The probod
config is built from env lookups with defaults, so a lingering value in
an older deployment is simply ignored.
Signed-off-by: Émile Ré <emile@probo.com>
The generated tracker policy listed only org ThirdParty rows linked to
the banner's patterns. Now that the mapping worker no longer auto-creates
those rows, a banner whose vendors have not been imported would render an
empty third-party section.
Build the list from the same union the banner's linkedThirdParties
resolver exposes: org ThirdParty rows plus the CommonThirdParty catalog
entries reached through patterns that carry no org link. Dedupe by name
with org rows taking precedence, so the policy stays complete and
accurate whether or not a vendor has been imported into the register.
Signed-off-by: Émile Ré <emile@probo.com>
With the tracker-mapping worker no longer materializing org third
parties, add the deliberate action that does: ThirdPartyService.Import
FromCommon seeds an org ThirdParty from a CommonThirdParty catalog entry
or returns the one the organization already imported, making it
idempotent on the (organization_id, common_third_party_id) pair.
On both the create and reuse paths it backfills tracker_patterns.third_
party_id for the organization's unlinked patterns whose catalog row
resolves to the same common third party, via the new TrackerPatterns.Link
ThirdPartyByCommonThirdPartyID. Patterns that previously surfaced the
catalog entry then surface the managed org vendor in the trackers UI and
the tracker-policy document. Only unlinked patterns are touched, so the
backfill is idempotent and picks up newly detected patterns on re-import.
End-to-end coverage (idempotency and pattern backfill) lands with the
GraphQL mutation in a following commit.
Signed-off-by: Émile Ré <emile@probo.com>
The tracker-mapping worker materialized a per-org ThirdParty for every
categorized tracker, linking or creating one through heuristic and
disambiguation-agent matching. Concurrent mapping of two patterns for
the same common third party raced the load-then-create check and left
duplicate org third parties with the same name.
Reduce the worker to catalog resolution only: it resolves the shared
common_tracker_pattern_id / common_third_party_id link and leaves
third_party_id untouched, preserving any link set elsewhere. Org third
parties will instead be created through an explicit per-vendor import
action added in a later commit.
Remove resolveOrgThirdParty, prepareOrgThirdParty, the sibling
direct-link signal, and the disambiguation-agent wiring (including its
constructor parameter and buildTrackerAgents return), and update the
worker tests to assert the catalog link is resolved while third_party_id
is preserved.
Signed-off-by: Émile Ré <emile@probo.com>
The publish flow ignored a document's stored default approvers and only
requested approval when approver_ids were passed in the call, so a major
publish with no approver_ids silently published directly without routing
through the approval flow — there was no way to tell "caller forgot
approvers" (null) from "caller wants no approval" (empty).
Make approver_ids an explicit choice, enforced once in the service so it
covers every caller (console, MCP, n8n):
- major publish: approver_ids must be set; an empty list publishes
directly, a non-empty list requests approval.
- minor publish: approver_ids must be omitted (approvers are ignored).
Validate this in PublishDocumentRequest.Validate(), update the console
publish dialog and the n8n publish node to honour the contract, document
it in the MCP tool spec, and cover it with e2e tests.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
Replace presigned URL string fields (logoUrl, fileUrl, ndaFileName,
etc.) with nested File references resolved through /api/files/v1/.
Update console Relay queries and e2e coverage accordingly.
Route NDA upload through filemanager.PutFile and return stable IAM
org logo URLs for consistency with the files API.
Signed-off-by: Ludovic Vielle <ludovic@probo.com>
Drivers that cannot determine account status leave Active nil.
Rather than surfacing NULL in the database, treat an absent signal
as active (the account appeared in the source listing).
- Migration: NOT NULL DEFAULT TRUE on the active column
- Insert/upsert SQL: COALESCE(@active, TRUE) on write
- Fix new(true) in test (not valid Go; use &activeTrue)
Signed-off-by: Bryan Frimin <bryan@probo.com>
Track whether an account is active (enabled) or disabled at the
source system. The field is nullable so existing entries without
this data remain valid.
- DB migration adds active BOOLEAN column to access_entries
- Coredata read/write/upsert/filter wiring for the new column
- Review engine propagates Active from source accounts
- GraphQL schema exposes active on AccessEntry and AccessEntryFilter
- MCP spec, types, and resolvers expose active and fix missing
account_type filter that was wired in GraphQL but not MCP
- CLI list command adds --active filter flag and ACTIVE output column
- Console campaign detail table shows Active/Disabled status badge
- E2e and unit tests updated to cover the new field
Signed-off-by: Bryan Frimin <bryan@probo.com>
When a domain has multiple TXT records (SPF, DKIM, etc.), the UDP
response can exceed 512 bytes and the server sets the TC bit. The
verifier was not handling this case, so any truncated response that
omitted the probo-verification record would silently fail as a
mismatch.
Fix by checking resp.Truncated after the UDP exchange and retrying
over TCP when set. Also pass the caller's context instead of
context.Background(), and simplify message construction with
dns.NewMsg.
Closes#1335
Signed-off-by: Bryan Frimin <bryan@probo.com>
The ResetProgressFunc callback streamed per-phase messages from inside
the reset transaction, so a rollback could leave already-printed
progress describing work that never committed. Remove the callback
entirely; the final result summary printed by proboctl still conveys
the outcome without the misleading intermediate output.
Signed-off-by: Émile Ré <emile@probo.com>
When IP geolocation does not resolve a country, or resolves one with no
known cookie-consent regulation (common on localhost and unmapped
regions), the banner previously fell back to OPT_OUT with no recorded
regulation. Apply GDPR (OPT_IN) as the safe default in that case so the
strictest consent model wins when origin is unknown.
To keep consent records auditable, stamp each one with a regulation
source of DETECTED (resolved from geolocation) or DEFAULT (fell back to
GDPR). The shared cookiebanner.ResolveRegulation helper centralizes the
decision for both the config and consent endpoints, and the new value is
exposed through GraphQL, MCP, the CLI, the n8n node, and the console
consent-records views.
Signed-off-by: Émile Ré <emile@probo.com>
