Expose the generated cookie policy Document on the CookieBanner
GraphQL type through a nullable policyDocument field and resolver,
and surface a link to it from the banner configuration header next
to the origin and ID. The link is hidden until a banner version is
published and the document exists.
Rework the tracker table rows: drop the Source column in favour of
a tracker Type badge, and move each tracker's description inline
beneath its name (and into the add/edit row inputs) instead of a
separate Description column.
Signed-off-by: Émile Ré <emile@probo.com>
Other generated policy documents number their top-level sections.
Apply the same convention to the cookie and tracking technologies
policy by prefixing each "##" heading with a running counter. A
counter is used rather than literal numbers so the sequence stays
contiguous when the optional "Third parties that set trackers"
section is omitted.
Signed-off-by: Émile Ré <emile@probo.com>
The themed banner and headless cookie list exposed only the tracker
name, description, and duration, so trackers that share a name but
differ in technology were indistinguishable to visitors.
Add the tracker type to the headless CookieItem model and render it as a
labeled "Type: <value>" detail under the tracker name, with an explicit
aria-label for assistive technology. Tracker type names are Web platform
API names (Cookie, Local storage, IndexedDB, ...), so they are kept
canonical while only the surrounding label is localized.
Signed-off-by: Émile Ré <emile@probo.com>
PublishTrackerPolicy ran template execution, markdown parsing and JSON
marshaling inside the WithTx callback, holding a write transaction open
across CPU-bound work. Split it into a read phase (WithConn) that
gathers data and renders the document, followed by a write phase that
persists the document and version, mirroring PublishThirdPartyList.
Also sort the generated third-party rows before returning. LoadByIDs
has no ORDER BY, so the policy document could otherwise be emitted in a
different row order on each regeneration.
Signed-off-by: Émile Ré <emile@probo.com>
Trackers sharing a display name can differ in type, so the generated
cookie and tracking technologies policy was ambiguous without it. Carry
the tracker type through the banner version snapshot and surface it as a
dedicated column in the policy table.
Stop the snapshot from dropping non-cookie trackers so storage, IndexedDB
and cache technologies appear in the policy and served banner config with
their real type. Duration now reflects the type when no max-age applies:
session storage clears with the tab, the remaining storage technologies
persist. Legacy snapshots predate the field and only ever held cookies,
so GetSnapshot backfills an empty type as COOKIE, keeping the non-null
GraphQL enum and policy output valid without a migration.
Signed-off-by: Émile Ré <emile@probo.com>
An organization can run several websites, each with its own banner and
generated policy. Add the banner origin to the document title and the
rendered heading so the policies are distinguishable in document lists
and the trust center.
Signed-off-by: Émile Ré <emile@probo.com>
Add a poll-based tracker policy worker that claims banners flagged for
policy generation, clears the flag, and rebuilds the cookie and
tracking technologies policy from the latest published snapshot. A
banner that lost its published version is skipped rather than failed.
Wire the worker into probod with crash propagation and graceful
shutdown alongside the other tracker workers.
Signed-off-by: Émile Ré <emile@probo.com>
Add a markdown "Cookie and Tracking Technologies Policy" template in
the policy-writer tone that covers cookies and other tracking
technologies, with GDPR, UK GDPR, FADP, and CCPA/CPRA privacy-rights
framing. Convert it to ProseMirror JSON and add PublishTrackerPolicy,
which builds the document from a banner's latest published version
snapshot and its tracker third parties. The document is GENERATED and
private in the trust center, linked to the banner via
policy_document_id.
Add CookieItem.HumanizedDuration so server-rendered lifetimes match the
banner's humanizeDuration helper, and reword the publish trigger and
backfill comment to reflect the broader tracker scope.
Signed-off-by: Émile Ré <emile@probo.com>
Add policy_document_id and policy_generation_requested_at columns to
cookie_banners and backfill banners that already have a published
version, so existing live banners get a policy on the worker's first
pass.
Flag the banner for policy generation inside the publish transaction so
generation is requested only when a cookie banner version is published,
not on draft edits. Fold policy_document_id into the existing Update so
the upcoming worker can persist the generated document id with the
scope it already holds.
Signed-off-by: Émile Ré <emile@probo.com>
Provider 4xx on an instance-name lookup is rarely actionable -- it
usually reflects a configuration state (deleted org, revoked token)
rather than a system error worth paging on. Downgrade Probo's own
log entry to Warn. The kit/worker entry for the same failure is
still Error pending the retry-limit work, which will mark sources
synced after N attempts and stop the duplicate log line.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
When a GitHub access-source connector has no organization configured
yet (user finished OAuth but abandoned the picker), the source-name
worker called https://api.github.com/orgs/ and got a 404 every 10
seconds, flooding logs. All other picker resolvers (sentry, gitlab,
bitbucket, heroku, asana, netlify, clickup, vercel) short-circuit to
("", nil) for empty settings -- this aligns github with them so the
worker falls into its existing "empty instance name -> mark synced
with generic name" branch instead of retrying forever.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Queue vetting on third_parties with PENDING, PROCESSING,
COMPLETED, and FAILED states. Expose enqueue and status through
GraphQL, MCP, CLI, and n8n, validate vet requests, tune the
worker via config, and poll the detail page while vetting runs.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
The tracker-mapping and common-pattern enrichment agents ran with
4 and 3 max turns. Those budgets are smaller than the tool-call
rounds their prompts authorize (two/one DB searches plus up to
three web searches) plus the forced structured-output synthesis
turn, so the run loop tripped MaxTurnsExceededError before it
could emit JSON.
Raise the budgets to 10 across every layer that drives them: the
bootstrap env-var defaults, the Helm chart values and production
example, and the in-code fallback. Update the bootstrap test
defaults to match.
Revert the prompt instructions that told the model to reserve a
turn for its final answer; the model cannot reliably count its
remaining turns, so the fix belongs in the turn budget rather
than the prompt text.
Signed-off-by: Émile Ré <emile@probo.com>
The client no longer strips sampling parameters per model, so the
sanitization test and its capturing provider helper assert behavior
that was intentionally removed. Delete them to restore a green build.
Signed-off-by: Émile Ré <emile@probo.com>
ResolveAgent aliased the default config's Temperature and MaxTokens
pointers into every agent that left them unset, so all resolved
agents shared one backing value. A mutation through any of those
pointers would corrupt the default and every other agent. It also
dereferenced the default unconditionally even though it can be nil.
Allocate a fresh pointer holding a copy of the default value, and
guard against a nil default so each resolved agent owns independent
state.
Signed-off-by: Émile Ré <emile@probo.com>
Remove sanitizeRequest and its call sites so the client no longer
strips sampling parameters based on the model registry. Unsupported
knobs are now avoided by setting the config carefully per model
rather than silently mutating outgoing requests.
Signed-off-by: Émile Ré <emile@probo.com>
The tracker-mapping worker clears mapping_requested_at at claim time, so
a crash or hard failure between Process phases left the pattern dequeued,
unmapped, and with nothing to re-trigger it. Only an incidental sibling
remap could rescue it, so a lone pattern could stay stranded forever.
Implement the worker.StaleRecoverer interface, mirroring the enrichment
worker. ResetStaleMappings re-arms rows that were claimed but never
assigned a catalog row (common_tracker_pattern_id IS NULL) once idle past
a configurable window; a successful Process always assigns one via the
unmatched fallback, so the predicate cleanly detects interrupted runs and
self-heals after a single pass. ClearMappingRequestedAt now bumps
updated_at so the stale clock starts at claim time and the sweep never
recycles an in-flight claim.
Plumb a StaleAfter knob (default 600s) through the config struct, builder
env var, probod wiring, and Helm templates.
Signed-off-by: Émile Ré <emile@probo.com>
The identification agent attributed probo_distinct_id to Mixpanel
purely on the shared distinct_id token, and the enrichment agent
returned no description for the glob ph_phc_*_posthog because it
searched the literal "*" string and found nothing.
Tighten the identification prompt so attribution requires a perfect
pattern match or a meaningful prefix that belongs to the vendor; a
generic token behind a different prefix is not a match. Teach the
enrichment prompt to strip wildcard and variable parts before
searching, and to treat a vendor name embedded in the key as
corroboration so clearly-named trackers still get a description.
Signed-off-by: Émile Ré <emile@probo.com>
The deterministic tracker-mapping heuristics group patterns by shared
initiator domain, but tag managers, customer-data platforms, and
generic CDNs (Google Tag Manager, Segment, cloudfront.net, ...)
initiate trackers for many unrelated vendors. Grouping on such a
domain mis-attributes one vendor's tracker to another.
Add uri.FilterSharedInfrastructureDomains backed by a curated eTLD+1
denylist and apply it once in resolveDeterministic, so sibling
grouping, catalog domain matching, and the sibling re-enqueue cascade
all ignore shared-infrastructure hosts. Vendor-specific domains such
as google-analytics.com are intentionally kept as a same-vendor
signal. The agent path is unchanged: it still sees observed domains,
now with a prompt caveat about shared infrastructure.
Update the two sibling tests that used googletagmanager.com as the
initiator domain to a vendor domain, since that host is now stripped
before grouping.
Signed-off-by: Émile Ré <emile@probo.com>
Address review feedback on the agent-driven tracker catalog path:
- Return initiator-domain load failures instead of swallowing them,
so the worker retries rather than running the agent on partial
context.
- In the resolver, treat only ErrResourceNotFound as a catalog miss
and propagate genuine name/slug lookup errors.
- Insert the new vendor inside a savepoint and, on the slug
unique-violation race, reload and return the winning row instead of
aborting the caller's transaction.
- Stop seeding common_third_party_domains from observed initiator
domains. They are a co-occurrence signal, not verified ownership,
and writing them into the global cross-tenant catalog pollutes the
domain-based matcher. The curated seed owns that data.
- Warn the mapping agent that observed domains may belong to shared
CDNs, tag managers, or hosting infrastructure rather than the
vendor, so it does not attribute on that basis alone.
- Extract a shared tracker-identification prompt helper and move the
common-pattern identification prompt next to the enrichment agent.
Signed-off-by: Émile Ré <emile@probo.com>
resolveOrCreateCommonThirdParty lived as a package-level helper in the
tracker mapping worker, but the common pattern enrichment worker now
reuses it. Homing shared catalog logic in a mapping-named file made the
enrichment worker quietly depend on the mapping worker's file, and it is
not a mapping concern.
Move it to pkg/thirdparty as exported ResolveOrCreateCommonThirdParty,
decoupled from cookiebanner's TrackerMappingAgentResult (it now takes a
name and category) to avoid an import cycle. It stays a transaction-
scoped free function so both workers compose it into their own tx for
atomicity rather than receiving a service that owns its own connection.
Relocate the catalog dedup DB test alongside it.
Signed-off-by: Émile Ré <emile@probo.com>
The enrichment worker no longer invents a description when a tracker's
purpose cannot be substantiated; it records an empty description and
marks the row enriched so the stale-recovery loop does not retry it.
Vendor identification is the mapping pipeline's job, so the enricher
reuses the existing tracker-mapping agent to attribute a third party
for an unlinked common pattern before describing it. A confident
catalog match seeds the enrichment prompt and links the pattern, but
the enricher never creates or overrides an attribution.
When a blank, unlinked catalog row later gains a third party through
the mapping pipeline's upsert, enrichment is re-armed so the now-known
vendor gets a second, better-informed description attempt.
Signed-off-by: Émile Ré <emile@probo.com>
The tracker mapping, common-pattern enrichment, and third-party
disambiguation agents default to a small max-tokens budget on the
premise that their final output is tiny structured JSON. On
reasoning models such as the GPT-5 family, reasoning tokens count
against max_tokens, so a small budget is consumed by reasoning and
the JSON is truncated, surfacing as "unexpected end of JSON input".
Raise the defaults to 4096 (1024 -> 4096 for tracker mapping, 512
-> 4096 for disambiguation) to leave headroom for the reasoning
phase. Update the bootstrap builder default, its test, and the
production values example to match.
Signed-off-by: Émile Ré <emile@probo.com>
The tracker-mapping worker runs its LLM and web-search phases
between short transactions and holds no row lock across them. The
pattern-analysis worker can merge a pattern into a glob and delete
it in that window, so the final UpdateMapping then fails with
ErrResourceNotFound and the task errors out spuriously.
A vanished pattern has nothing left to map, so treat the concurrent
delete as a no-op: log it and return nil instead of failing.
Signed-off-by: Émile Ré <emile@probo.com>
The pattern-analysis worker created the merged glob blank and re-armed
mapping, discarding the org ThirdParty and description already resolved
on the exacts it absorbed. That forced a full re-map (LLM/web-search)
and opened a window where an in-flight exact could vanish mid-mapping.
Seed the glob from the merged exacts when they unanimously agree on a
single third party, carrying its description too, while still re-arming
mapping so the glob derives its own catalog row. With the third party
pre-set, the mapping worker skips the expensive org/disambiguation
resolution. Conflicting or unresolved groups stay blank as before.
The catalog link is deliberately not inherited: it is keyed on the
exact pattern string, not the glob template, so the mapping worker
resolves the right row itself.
Signed-off-by: Émile Ré <emile@probo.com>
The common-pattern enrichment and tracker-mapping agents run on
reasoning models such as gpt-5-nano, which reject an explicit
temperature and fail the whole request with a 400 ("Unsupported
value: 'temperature' does not support 0.1 with this model"). The
model registry already records this capability, but nothing
consulted it before dispatch, and dated provider snapshots like
gpt-5-nano-2025-08-07 did not resolve in the registry.
Resolve dated snapshots to their undated base model in registry
Lookup, and sanitize each chat completion request in the LLM
client by omitting the sampling knobs the target model does not
accept (temperature, top_p, frequency/presence penalties, stop).
Unknown models are left untouched, so models absent from the
registry keep their current behavior.
Signed-off-by: Émile Ré <emile@probo.com>
The tracker-mapping and common-pattern-enrichment workers ran with the
kit/worker defaults (interval 10s, max-concurrency 5 each) and dropped
the resolved per-agent max-tokens/temperature, so up to ten LLM
pipelines could run unbounded on one OpenAI client. The mapping worker
also held a FOR UPDATE transaction across the LLM and Firecrawl calls
while its DB search tools acquired a second pooled connection, risking
pool exhaustion under concurrency.
Plumb max-tokens, temperature, agent timeout, and per-worker max-turns
through TrackerAgentsConfig and DisambiguationConfig into all three
agent builders, replacing the hard-coded constants with config-fed
fields and package fallbacks. Expose worker interval, concurrency,
stale-after, agent timeout, and max-turns as config (env, Helm values,
deployment template) mirroring the evidence-describer pattern, and
apply them at registration.
Refactor Process into deterministic-read, agent (no transaction), and
persist phases so neither the mapping agent nor disambiguation runs
inside an open transaction, removing the row locks held across network
latency and the nested-connection pressure.
Signed-off-by: Émile Ré <emile@probo.com>
SCIM DELETE returned 500 when a profile was still referenced
elsewhere in the org, which disabled the identity-provider bridge
after repeated sync failures. Fall back to deactivation when delete
is blocked, log the conflict without failing sync, and still
attempt delete for excluded users even when inactive.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
Allow documents to be archived or unarchived directly from the
list row actions, matching the detail-page behavior. Remove the
row from the active or archived connection after the status change
so filtered lists update immediately.
Signed-off-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
Address the highest-impact Kubernetes chart issues from #1245.
Align PG_ADDR with postgresql.host/port, wire
OAUTH2_SERVER_SIGNING_KEY, add early validation for required
base64 and PEM secrets, and isolate the main service/deployment
with component labels so Chrome pods are not selected by server
traffic.
Update chart values and documentation to clarify required secret
formats, managed PostgreSQL prerequisites, ACME account key
persistence, and current Azure Blob compatibility caveats for S3
proxy deployments.
Signed-off-by: Cursor Agent <cursoragent@cursor.com>
Allow campaigns to continue when a source fetch fails by keeping
that failure on the source fetch record only.
The source fetch worker now logs the failure after persisting it and
returns success so campaign execution is not interrupted by source-level
fetch errors.
Signed-off-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
Treat a single source fetch failure as tolerated so campaigns can
continue fetching and transition normally.
The worker now records failed fetches and only propagates a process
error once the failed source count exceeds one. This keeps the first
failed source visible on the source fetch while preventing the
campaign-level run from being marked failed too early.
Signed-off-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Bryan FRIMIN <bryan@frimin.fr>
Turn the category column into an inline Select so a tracker pattern or
resource can be recategorised in place. On the pattern row the move is
gated by a confirm that surfaces the existing behaviour of promoting or
linking a third party. The remaining row actions (include/exclude and
delete) move into a single action dropdown, leaving only Edit inline.
Add a Max Age column to the pattern table and a Category column to the
resource table, and lay the edit forms out with the name above the
inputs so the description can use the full row width. Shrink the action
column to its content so the freed space goes to the data columns.
Signed-off-by: Émile Ré <emile@probo.com>
The agent returned a single confidence that conflated two unrelated
judgments: whether an artifact is a meaningful web tracker and which
vendor set it. The prompt's tracker-worthiness skepticism drove the
number down for extension state like __darkreader__wasEnabledForHost,
pushing it below the gate and dropping the attribution entirely, so a
clearly-named vendor never reached the catalog.
Rename the agent field to ThirdPartyConfidence and scope it to the
attribution alone. The identify gate now checks that a vendor is named
with sufficient confidence; on success the catalog row is stored at a
fixed agent confidence like the other heuristic signals, and on failure
the unmatched fallback still records the pattern with no third party.
The stored pattern confidence was only used for ordering and as agent
context, never as a gate, so a separate LLM-provided number is dropped
rather than split out.
Signed-off-by: Émile Ré <emile@probo.com>
