Replace the immediate per-document approval email and the manual
"send signing notifications" action with a single debounced worker that
batches pending requests per recipient and organization.
The worker (go.gearno.de/kit/worker) polls on an interval (default 5m)
and claims one (organization, recipient) group at a time, sending one
consolidated signing email and/or one approval email per recipient/org
that lists every document awaiting their signature or approval. The
claim is a conditional UPDATE that doubles as concurrency-safe dedup, so
several workers never email the same group twice.
Each request is notified once it has been pending past the debounce
delay (default 15m), then reminded at 1x, 2x and 3x the reminder
interval (default 1 day) after the previous email, after which it stops.
New last_notified_at and notification_count columns on signatures and
approval decisions drive the debounce, the widening reminder cadence and
the four-email cap.
Email copy lists each document with its title, type and a deep link to
the employee page. Removed the inline approval-on-publish email, the
SendSigningNotifications service method/mutation/MCP tool, its IAM action,
and the related console UI and n8n operation.
Signed-off-by: Sacha Al Himdani <sacha@probo.com>
Map membership, profile, and inactive-user failures from
OpenOIDCChildSessionForOrganization to a generic 404 instead
of 500 so org-scoped OIDC callbacks do not reveal tenant
access details.
Signed-off-by: Bryan Frimin <bryan@probo.com>
OIDC login dropped organization_id before the provider redirect, so
callbacks with an existing matching root session never created an org
child session. Persist organization_id in OIDC state, open the child
session on callback, and forward the parameter from the sign-in UI.
Signed-off-by: Bryan Frimin <bryan@probo.com>
The common tracker pattern "show" only printed attempt counters, so the
enrichment payload — run status, model, agent attribution, and per-field
outcomes — was invisible, and a partial run was indistinguishable from a
complete one. Render that payload the same way the common third party
"show" already does.
Surface the last enrichment attempt timestamp in both catalog list views
so an operator can spot stale or never-attempted rows at a glance.
Signed-off-by: Émile Ré <emile@probo.com>
The migration dropped the DEFAULT on
common_tracker_patterns.enrichment_attempts, so the column is now
NOT NULL with no default and inserts must set it explicitly. The e2e
seedCommonTrackerPattern helper still did a raw insert without it,
which broke TestThirdParty_ImportFromCommon and
TestTrackerPattern_CommonTrackerPatternID with a 23502 violation. Pass
0 for the seeded rows.
Signed-off-by: Émile Ré <emile@probo.com>
Three crash- and migration-recovery gaps in the unified enrichment
model left rows stuck or misclassified:
- Upsert re-armed a blank, newly-linked tracker pattern without
clearing its prior enrichment payload. A crash between the worker's
claim and persist then left the row with a stale payload, so the
stale-recovery sweep (which only catches rows with a null payload)
skipped it forever. Clear enrichment on re-arm so the row reads as
not-yet-completed again, and pin the behavior with a test.
- The migration added last_enrichment_attempt_at to
common_third_parties without seeding it. Rows with prior attempts
kept a NULL clock and could never satisfy the stale-reset predicate.
Backfill from updated_at, the historical claim-time proxy.
- The migration switched the tracker-pattern enriched-state source to
the enrichment payload without backfilling rows previously marked by
enriched_at, making already-enriched rows read as unenriched.
Seed a provenance sentinel for rows that carried the old done-flag.
Signed-off-by: Émile Ré <emile@probo.com>
Let an operator create or update a catalog third party and queue it for
the async enrichment worker in one step, so a minimal name/category row
gets its profile, compliance documents, owned domains, and logo filled
in without a separate reenrich call.
Enrichment is armed explicitly via RequestEnrichmentByIDs rather than
through the receiver: the coredata Upsert sets enrichment_requested_at
from the receiver only on insert and leaves it untouched on conflict,
so this is the only path that re-arms an existing row uniformly while
also resetting the attempt budget for a fresh run.
Signed-off-by: Émile Ré <emile@probo.com>
Make common_tracker_patterns and common_third_parties share one
enrichment-tracking model and fix the misleading proboctl status.
Both tables now carry the enrichment JSONB provenance payload, an
enrichment_attempts counter, and a last_enrichment_attempt_at clock.
On common_tracker_patterns the enriched_at done-flag is renamed to
last_enrichment_attempt_at and stamped at claim time, so it is truthful
to "attempt" rather than "success". A row is considered to have been
through the workflow when it carries an enrichment payload, not when a
timestamp is set, which lets stale recovery key off the payload being
absent with budget remaining, exactly like common_third_parties.
The claim path reads the attempt counter and timestamp back via
RETURNING so the in-memory receiver matches the database clock instead
of a separate app-side time.Now.
The enricher builds a per-field provenance payload (description and
third-party outcomes plus the mapping attribution) and persists it via
UpdateEnrichment, named to mirror the common-third-party sibling. The
common pattern enrichment worker gains a max-attempts ceiling so a
permanently failing row stops looping.
proboctl now shows "enriched" only when every field the last run
recorded an outcome for resolved a value, otherwise "partial (X/Y)",
replacing the misleading "enriched (no description)" label.
Signed-off-by: Émile Ré <emile@probo.com>
A token without admin.directory.customer.readonly makes
Customers.Get return 403 forever. Treat that as terminal so
the source-name worker keeps the generic name and stops
retrying.
Signed-off-by: Bryan Frimin <bryan@probo.com>
Unbounded LoadAll* loaders materialised an entire result set in one
query with no ceiling. A table that is small in development can grow
without bound in production, so these loaders were a latent memory
and query-time hazard.
Remove the LoadAll* methods from pkg/coredata and walk the cursor-
paginated LoadBy* siblings instead through a shared page.LoadAll
helper. The helper advances a MaxCursorSize forward cursor until the
result set is exhausted and concatenates the pages. It caps a single
call at MaxLoadAllPages (20) batches of 500 rows and errors past that
rather than materialising an unbounded set, so a runaway caller fails
loudly instead of exhausting memory.
Callers that genuinely need every row now express that explicitly,
and the coredata load-naming rule and docs are updated to discourage
new unbounded loaders.
Signed-off-by: Sacha Al Himdani <sacha@probo.com>
The MCP Profile schema declares additional_email_addresses as a
required, non-nullable array, but NewProfile passed the mail.Addrs
value through unchanged. A profile with no extra emails has a nil
slice, which marshals to JSON null and fails tool output validation
with "type: null, want array".
Default the nil slice to an empty mail.Addrs so the marshalled output
always honors the strict array schema.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
Long role strings in the access review table broke row layout when
drivers joined many roles into one comma-separated value. Expose
roles as a string array in GraphQL by splitting the stored role at
the API layer, and render the first three roles as badges with a
"+X more" popover for the rest.
Closes ENG-459.
Signed-off-by: Ludovic Vielle <ludovic@probo.com>
Employee document signatures recorded an acknowledgment with no
cryptographic proof, unlike document approvals which already create
and accept an electronic signature on every decision.
Mirror the approval flow on the sign path: generate the signed
document PDF, create-and-accept an esign record, and persist its id
on the document_version_signatures row through a new
electronic_signature_id column. Capture the signer IP and user agent
in the resolver, and re-check the published/archived preconditions
inside the transaction so the seal cannot race document state.
Make the consent wording a single backend source of truth shared by
the text that is sealed and the text shown in the UI. Define
DocumentSignatureConsentText and DocumentApprovalConsentText in the
probo service package and the NDA copy in the trust service, each
owned by the flow that uses it, and stop esign from appending the
generic clause to caller-provided consent text so approvals no
longer seal a duplicated sentence.
Expose the resolved consent text through GraphQL on
EmployeeDocumentVersion and DocumentVersionApprovalDecision, and have
the signing, approval, and NDA pages render it from the API instead
of hard-coded strings, mirroring how the NDA page already worked.
Align the wording with the actual interaction: the buttons read
"Review and sign" and "Review and approve", the clauses reference
those actions, and the inaccurate "typing my full name" phrasing is
removed everywhere.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
Denials were invisible in the audit trail and evaluator explainability
(policy_id, reason) was discarded before reaching logs. Emit a structured
authz decision line on every evaluation in evaluateMultiInTx — allow,
deny, no_match, and assumption errors — using the existing authorizer
logger with opaque IDs only.
Add decision_log.go with DecisionRecord and logDecision. Surface
PolicyID and Reason on EvaluationResult for logging. Audit log
behavior is unchanged (allow-only). Document the convention in
authorization.md.
Signed-off-by: Ludovic Vielle <ludovic@probo.com>
Expose campaign sources as first-class nodes, paginate fetch attempts
instead of denormalized status fields, and bind entries to their
campaign snapshot. Update GraphQL, MCP, CLI, console, and e2e coverage
to match.
Signed-off-by: Bryan Frimin <bryan@probo.com>
Source category was never surfaced in the product and added noise to
snapshots and APIs. Remove the enum, columns, and service fields so
campaign sources track only the identity fields reviewers need.
Signed-off-by: Bryan Frimin <bryan@probo.com>
Fresh installs already had csv_data on campaign sources from the
initial migration; add a follow-up migration to backfill existing
rows from the live source snapshot.
Signed-off-by: Bryan Frimin <bryan@probo.com>
Bad API keys and expired OAuth tokens showed Connected because
probes ran only for OAuth2 and many providers had no ProbeURL.
Add a registry ProbeConnection dispatcher with static, dynamic,
and custom probes so all 41 providers are checked on demand.
Signed-off-by: Bryan Frimin <bryan@probo.com>
Cursor's Admin API exposes two independent removal signals: the isRemoved
boolean and a role value of "removed". They are not always consistent —
a member can carry role "removed" while isRemoved is still false, a known
gap documented on the Cursor community forum.
Previously Active was derived from isRemoved alone, so a member with
role "removed" but isRemoved=false was incorrectly reported as active.
Now either signal is sufficient to mark the account inactive.
Add a cassette entry and test case covering the inconsistent state
(role "removed", isRemoved false) to prevent regression.
Signed-off-by: Bryan Frimin <bryan@probo.com>
Decouple each campaign from the live access-review sources it was started
with by introducing a per-campaign source snapshot table
(access_review_campaign_sources). The snapshot captures the source name,
category, and connector at start time, so a review remains coherent even
after the underlying source is edited or deleted. Fetch tracking becomes
an append-only log (access_review_campaign_source_fetch_attempts) that
preserves every attempt with its own status and error rather than
overwriting a single row.
Rename the shared access-review tables and enums to use a consistent
access_review_ prefix throughout:
access_entries → access_review_entries
access_sources → access_review_sources
access_source_category → access_review_source_category
access_entry_* → access_review_entry_*
The same rename propagates to every coredata type, service, GraphQL
schema, MCP specification, CLI command, frontend component, and e2e test.
The accessreview package gains dedicated actions.go and policies.go files
for its own IAM policy set, mirroring the agentrun package pattern.
Signed-off-by: Bryan Frimin <bryan@probo.com>
The enrichment agent now reads full cookie-database and vendor pages
through the browser. The facts it gathers (setter, purpose, data
stored) are not protectable, but a source page's written description
is original expression. Instruct the agent to state the facts in its
own words and never copy a sentence verbatim, so stored descriptions
stay clear of third-party prose.
Signed-off-by: Émile Ré <emile@probo.com>
The tracker-mapping and common-pattern enrichment agents only had
web search, which returns title/url/snippet, so they could never
open a cookie-database or cookie-policy page to read which vendor
actually sets a tracker. This mis-attributed setters whose snippet
is misleading (e.g. _li_* read as LinkedIn rather than LiveIntent).
Wire the read-only headless-browser toolset into both agents, gated
on a configured Chrome endpoint, mirroring the common-third-party
enrichment worker: agent construction moves into the run path so each
run can carry a per-run browser that is closed when the run returns.
The prompts now direct the agent to open a promising result and read
the named setter from the full page text. Both agents stay unchanged
when no Chrome endpoint is configured.
Signed-off-by: Émile Ré <emile@probo.com>
Brand assets served at /api/files/v1/static had no cache headers.
Introduce brand.Assets to own the embedded filesystem, content-hash
ETags, and HTTP serving. Responses now carry Cache-Control and ETag
so clients can cache and revalidate; stable email URLs stay
revalidatable (max-age=3600, no immutable).
Replace hardcoded Default*Path constants with StaticPathPrefix,
logical filename constants, and StaticPath(). NewAssets validates
required assets at startup so a rename fails fast instead of 404ing
in sent emails. The files handler keeps routing and 404 rendering;
ServeAssets sets cache headers and serves the file.
Signed-off-by: Ludovic Vielle <ludovic@probo.com>
Address PR review feedback: rename the shared string-helper package
from strutil to stringsx to avoid the discouraged util suffix and the
collision with the standard strings package, updating all import paths
and call sites.
Replace the manual wg.Add/wg.Done bookkeeping in the enrichment worker
with wg.Go, which is less error-prone.
Signed-off-by: Émile Ré <emile@probo.com>
The alphanumeric-normalisation helper was duplicated verbatim in the
common-third-party owned-domains resolver and the cookiebanner tracker
mapping worker. Hoist it into a new dependency-free strutil package so
both call sites share one implementation and one test.
Signed-off-by: Émile Ré <emile@probo.com>
The common-third-party enrichment pipeline ran Agent B (compliance
docs), Agent C (owned domains), and the deterministic logo step
sequentially even though, once Agent A resolves the website, the
three depend only on that website and not on each other. Fan them
out across goroutines under a WaitGroup so wall time is the slowest
of the three rather than their sum. Each step builds its own per-run
browser and writes only into its own locals; the shared LLM, HTTP,
and FileManager clients are safe for concurrent use and the database
is untouched until persist. Results merge in a fixed order so
runErrors and log output stay deterministic.
Also replace the single-sentence confidence guidance in the three
agent prompts with an explicit, calibrated rubric tied to evidence
strength, and remind the model that a downstream threshold gates
persistence so it should neither inflate nor deflate its estimates.
Signed-off-by: Émile Ré <emile@probo.com>
Reject oversized logo responses instead of silently truncating them,
which could persist corrupt image bytes as a valid logo.
Tighten ownership substring matching with a length-ratio guard so a
short label root no longer attributes unrelated domains to a vendor.
Render the worker confidence threshold when set to zero by testing
against nil, so an explicit "accept all" value is not dropped by Helm's
falsy-numeric truthiness.
Sanitize and bound per-agent error text before persisting it to the
enrichment metadata column to avoid leaking unbounded internal detail.
Signed-off-by: Émile Ré <emile@probo.com>
Columns must not carry DEFAULT clauses. The DEFAULT only backfills
existing rows when the non-nullable column is added, so drop it in the
same migration; inserts already supply enrichment_attempts explicitly.
Signed-off-by: Émile Ré <emile@probo.com>
Add `proboctl common-third-party reenrich` to re-arm the async
enrichment worker for selected catalog rows, and `stats` to summarize
the catalog by enrichment state and last run status. Rows are selected
verbatim via --id/--slug or across the catalog via
--category/--keyword/--state/--status, gated by --dry-run and --yes.
Extend `list` with --state/--status filters and STATE/STATUS columns,
and `show` with enrichment state, attempts, last run status, error,
per-field provenance, and discovered domains.
Back these with CommonThirdPartyFilter state/status/IDs filters plus
CommonThirdParties.LoadAllIDs and RequestEnrichmentByIDs. The latter
stamps enrichment_requested_at and resets the attempt counter while
preserving the existing payload, so the worker merge keeps curated and
human-edited provenance.
Also simplify exactLabelMatch to use slices.Contains.
Signed-off-by: Émile Ré <emile@probo.com>
When the common-third-party enrichment worker discovers owned domains
for a vendor, tracker patterns that were detected and left unmatched
before those domains were known had nothing to re-trigger them.
Add RequestMappingForUnmappedByInitiatorDomains, a global re-arm that
stamps mapping_requested_at on still-unmapped patterns whose detected
trackers share one of the new domains, and call it from the enrichment
worker's persist step for newly-inserted domains only. The mapping
worker then re-resolves them through its existing domain-overlap path.
Targeting is limited to patterns with no resolved vendor (no org third
party and an absent or unlinked catalog row), so a pattern already
attributed to this or any other vendor is never disturbed.
Signed-off-by: Émile Ré <emile@probo.com>
Add a domain-discovery step to the enrichment pipeline so the catalog's
domain set, previously written only by the curated seed, grows
automatically. A focused agent enumerates the registrable domains a
vendor owns and operates - marketing, product and sub-brand, app, API,
and CDN/asset domains - from links seen while browsing and from web
search, anchored on the website resolved earlier in the run.
A deterministic ownership gate reduces the candidates to eTLD+1 and
keeps only those that clear a strict confidence floor and match the
vendor by domain label. Shared tracker-delivery and CDN infrastructure
is dropped unless the vendor itself is that provider, in which case its
own brand-matching domain passes a stricter exact-label check. The
survivors are upserted into common_third_party_domains in the run's
final transaction and recorded in the enrichment payload, feeding the
tracker-mapping domain step and disambiguation.
Signed-off-by: Émile Ré <emile@probo.com>
Give the company-profile agent (Agent A) the read-only browser toolset
and build it per-run, so it can read footer, imprint, about, and legal
pages and follow a product domain to the corporate one to resolve the
legal name and headquarters address rather than failing cold.
Make the website the hard precondition: when Agent A cannot resolve a
canonical website, skip the compliance-docs agent and logo step instead
of running them blind, which previously produced inconsistent
cross-domain document URLs. Fall back to the catalog display name for
the legal name when nothing better is found, recorded with a distinct
provenance status so a later real find overwrites it.
Rewrite both enrichment prompts in the project's role/task/instructions
XML style, add a domain-consistency rule for document URLs and a
tool-budget directive, and document the prompt style as a rule and guide.
Fix the find_links_matching browser tool, which double-encoded its
pattern and made JSON.parse fail on every keyword, starving any agent
that used it until it hit the turn cap. Salvage output when an agent
exhausts its turn budget while still exploring with a pending structured
output by forcing one final synthesis turn instead of failing outright.
Signed-off-by: Émile Ré <emile@probo.com>
Introduce a poll-based worker that fills the global common_third_parties
catalog (URLs, headquarter address, legal name, certifications, logo)
so each tenant no longer starts from sparse, name-only rows. Enrichment
is requested at row creation by ResolveOrCreateCommonThirdParty; curated
seed rows are not enqueued, to avoid a re-seed storm.
The pipeline uses two specialized agents plus a deterministic logo step.
Agent A (company profile) resolves legal name, headquarter address, and
the canonical website over web search; its website and legal name feed
Agent B and the logo step. Agent B (compliance docs) resolves the legal
document URLs, trust/security/status pages, and certifications using the
browser read-only toolset (gated on ChromeDPAddr) plus web search. The
logo step restores pkg/webinspect as a pure deterministic package and
stores the discovered icon in S3, linked via logo_file_id.
Each agent returns per-field value/confidence/source_url. The worker
writes a column only when confidence clears a configurable threshold and
the field is not externally owned (seed or human), and always records
full per-field provenance in a new enrichment JSONB column so re-runs
fill only gaps and human edits are never clobbered. New bookkeeping
columns (enrichment_requested_at, enrichment, enrichment_attempts) back
the claim queue and stale recovery; agents run outside transactions and
results persist in one final transaction.
The worker is opt-in: it no-ops unless its agent provider is configured.
Signed-off-by: Émile Ré <emile@probo.com>
Use the rebased filemanager method name in the regression test after
main split the S3 URL helpers into their own file.
Signed-off-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ludovic <ludovic@probo.com>
Use an ASCII-only fallback for Content-Disposition filename and
RFC 5987 path escaping for filename* so presigned S3 responses keep
spaces and Unicode filenames interoperable.
Signed-off-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ludovic <ludovic@probo.com>
Trust GraphQL and MCP still exposed presigned URL strings for
trust-center logos while console and connect already serve stable
File.downloadUrl paths. Phase 1 migrates the seven public logo
fields on trust GraphQL and the trust-center file references on MCP
to the shared File type; trust GraphQL NDA stays on fileUrl for a
follow-up.
Trust resolvers load public files through filemanager and map them
with types.NewFile. The trust app Relay queries and components now
read logo.downloadUrl. MCP specification, resolvers, and helpers
are updated in sync, including NDA on MCP where callers already
have file access.
filemanager is split into focused files and its URL surface is
narrowed to GenerateFileURL(file) for stable app URLs and
GeneratePresignedURL for S3 redirects. GetPublicFile remains the
DB entry point when only a file ID is known.
Add trust and MCP e2e coverage for public logo download URLs.
Signed-off-by: Ludovic Vielle <ludovic@probo.com>
Replace Organization.logoUrl and horizontalLogoUrl with nested File
objects whose downloadUrl points at /api/files/v1/public/{id}, matching
the Console migration.
Org logos are FileVisibilityPublic and served without HTTP auth, so
Connect File.downloadUrl is built eagerly in NewFile with no field-level
authorize. Logo loading moves to iam.OrganizationService.LogoFile and
HorizontalLogoFile; the old URL generators are removed.
Sync IAM Relay components and n8n organization operations. Add an e2e
test for Connect multipart logo upload and ExecuteConnectWithFile.
Signed-off-by: Ludovic Vielle <ludovic@probo.com>
Replace the bare `getprobo.com` domain and the `www.getprobo.com`
marketing host with `probo.com` / `www.probo.com` across the codebase.
Functional subdomains (app, console, notification, custom, test,
cookie-banner, compliance) keep their existing `getprobo.com` hosts,
and changelog entries are left untouched.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
The tracker policy document deduped third parties by name and kept the
first-seen row wholesale. Org third parties are appended before catalog
common vendors so user-editable data wins, but an org row may leave the
description or privacy policy URL empty. In that case the early return
discarded the later common-vendor row entirely, dropping metadata that
would have completed the document.
Track the kept row by name and backfill only its empty fields from later
duplicates, so org data still wins while common-vendor metadata is no
longer lost.
Signed-off-by: Émile Ré <emile@probo.com>
The tracker-mapping worker had been reduced to catalog resolution only,
which removed not just the auto-creation of an org ThirdParty but also
the auto-linking of an existing one. Only the creation needed to go: it
raced the load-then-create check and produced duplicate vendors.
Restore the full org ThirdParty resolution (exact common-id link,
sibling direct-link, high-confidence heuristic, and the disambiguation
agent) and remove only the CreateFromCommon branch and its
categorisation gate. When nothing matches, the worker now leaves
third_party_id unset rather than creating a vendor; creation happens
exclusively through the explicit ImportFromCommon action. Drop the
now-dead CreateFromCommon helper and rename match.go to common_match.go.
Fix a latent test bug surfaced by actually running the DB-backed suite
(skipped in CI without Postgres): the heuristic-match candidate lacked
Level 1, so the level-filtered candidate loader excluded it and the old
fallback create masked the miss.
Signed-off-by: Émile Ré <emile@probo.com>
