The tracker-mapping worker had been reduced to catalog resolution only,
which removed not just the auto-creation of an org ThirdParty but also
the auto-linking of an existing one. Only the creation needed to go: it
raced the load-then-create check and produced duplicate vendors.
Restore the full org ThirdParty resolution (exact common-id link,
sibling direct-link, high-confidence heuristic, and the disambiguation
agent) and remove only the CreateFromCommon branch and its
categorisation gate. When nothing matches, the worker now leaves
third_party_id unset rather than creating a vendor; creation happens
exclusively through the explicit ImportFromCommon action. Drop the
now-dead CreateFromCommon helper and rename match.go to common_match.go.
Fix a latent test bug surfaced by actually running the DB-backed suite
(skipped in CI without Postgres): the heuristic-match candidate lacked
Level 1, so the level-filtered candidate loader excluded it and the old
fallback create masked the miss.
Signed-off-by: Émile Ré <emile@probo.com>
When IP geolocation does not resolve a country, or resolves one with no
known cookie-consent regulation (common on localhost and unmapped
regions), the banner previously fell back to OPT_OUT with no recorded
regulation. Apply GDPR (OPT_IN) as the safe default in that case so the
strictest consent model wins when origin is unknown.
To keep consent records auditable, stamp each one with a regulation
source of DETECTED (resolved from geolocation) or DEFAULT (fell back to
GDPR). The shared cookiebanner.ResolveRegulation helper centralizes the
decision for both the config and consent endpoints, and the new value is
exposed through GraphQL, MCP, the CLI, the n8n node, and the console
consent-records views.
Signed-off-by: Émile Ré <emile@probo.com>
Expose a non-destructive re-trigger that re-arms tracker policy
generation for a banner that already has a published version, so the
tracker-policy worker regenerates the document after iterating on the
generator. RegenerateTrackerPolicy returns a conflict when nothing has
been published yet.
Wire it across all API surfaces per the api-surface rule: the console
GraphQL mutation and resolver, the MCP tool, the prb cookie-banner
regenerate-policy command, and the n8n operation, with console e2e
coverage for the success and no-published-version paths.
Signed-off-by: Émile Ré <emile@probo.com>
Add a markdown "Cookie and Tracking Technologies Policy" template in
the policy-writer tone that covers cookies and other tracking
technologies, with GDPR, UK GDPR, FADP, and CCPA/CPRA privacy-rights
framing. Convert it to ProseMirror JSON and add PublishTrackerPolicy,
which builds the document from a banner's latest published version
snapshot and its tracker third parties. The document is GENERATED and
private in the trust center, linked to the banner via
policy_document_id.
Add CookieItem.HumanizedDuration so server-rendered lifetimes match the
banner's humanizeDuration helper, and reword the publish trigger and
backfill comment to reflect the broader tracker scope.
Signed-off-by: Émile Ré <emile@probo.com>
Add policy_document_id and policy_generation_requested_at columns to
cookie_banners and backfill banners that already have a published
version, so existing live banners get a policy on the worker's first
pass.
Flag the banner for policy generation inside the publish transaction so
generation is requested only when a cookie banner version is published,
not on draft edits. Fold policy_document_id into the existing Update so
the upcoming worker can persist the generated document id with the
scope it already holds.
Signed-off-by: Émile Ré <emile@probo.com>
A tracker pattern's source ratchets PRE_EXISTING -> EXTENSION -> SCRIPT
as stronger detections arrive, but that promotion was never reflected
back to the mapping pipeline. The detection that promotes the source
also brings a fresh initiator domain that matchByDomain and
matchBySiblingOrigin can use, and an EXTENSION -> SCRIPT promotion lifts
the creationAllowed gate that blocks org third-party creation. Yet the
pattern's mapping_requested_at was already cleared after its first pass,
so the worker never revisited it.
Re-arm mapping_requested_at via SetMappingRequested at each
source-promotion site (reportDetectedTracker plus the glob-merge and
adoption paths in the pattern-analysis worker). Update's SET clause does
not cover mapping_requested_at, so assigning the field before Update
would be a silent no-op; SetMappingRequested only writes when the column
is NULL, keeping already-queued patterns from being double-enqueued.
Signed-off-by: Émile Ré <emile@probo.com>
Replace the two-step ID-materializing pattern (fetch IDs in Go, pass
as ANY(@ids)) with an IN-subquery that keeps the filtering entirely
in the database and eliminates an extra round trip. Remove the now
unused LoadIDsByCommonThirdPartyID and its service wrapper. Update
the coredata rule to clarify that subqueries for filtering are OK.
Signed-off-by: Émile Ré <emile@probo.com>
Manual moves of a non-extension TrackerPattern lacking a ThirdPartyID
now request mapping, which the tracker-mapping worker resolves with a
four-stage pipeline: exact common_third_party_id link, heuristic
ranking, agent disambiguation, and finally CreateFromCommon. Existing
fuzzy-matched org rows are tagged with common_third_party_id so the
next promotion takes the O(1) exact-link path.
The matching primitives live in pkg/thirdparty (RankCandidates,
LinkToCommon, CreateFromCommon, ScoredCandidate, threshold constants)
so the disambiguation agent and the heuristic share one candidate
type. Cookiebanner orchestrates them; cookie-banner-specific concerns
(pattern -> common-pattern -> common-party navigation, the EXTENSION
gate, and structured logs) stay in the worker.
Signed-off-by: Émile Ré <emile@probo.com>
Surface the third-party links carried by tracker patterns (org-scoped
ThirdParty via third_party_id, or global CommonThirdParty via
common_tracker_pattern_id) requires three new batch loaders and two
filter dimensions, all kept inside their owning entity tables to honour
the no-cross-entity-JOIN rule.
* CommonTrackerPatterns gains LoadByIDs and the ID-only
LoadIDsByCommonThirdPartyID helper, which lets callers translate a
common third party into a set of common_tracker_pattern_id values
without ever JOINing against tracker_patterns.
* CommonThirdParties gains LoadByIDs.
* TrackerPatterns gains LoadDistinctThirdPartyIDsByCookieBannerID and
LoadDistinctCommonTrackerPatternIDsByCookieBannerID, used by the
upcoming CookieBanner.linkedThirdParties resolver to enumerate the
third parties referenced in a banner.
* TrackerPatternFilter gains thirdPartyID and commonTrackerPatternIDs
filter dimensions; the GraphQL layer will dispatch a single
thirdPartyId argument to the right one based on the GID entity-type
prefix.
Service-layer wrappers (cookiebanner.GetCommonTrackerPatternsByIDs,
cookiebanner.LoadCommonTrackerPatternIDsByCommonThirdPartyID,
cookiebanner.LoadDistinctThirdPartyIDsByCookieBannerID,
cookiebanner.LoadDistinctCommonTrackerPatternIDsByCookieBannerID, and
thirdparty.GetCommonThirdPartiesByIDs) expose the new loaders to the
console resolvers and dataloaders that follow.
Signed-off-by: Émile Ré <emile@probo.com>
Every PromoteSource caller already loaded the tracker pattern under
the same transaction, so a dedicated single-column UPDATE only
duplicated machinery and forced callers to learn a second mutation
verb. Add `source = @source` to Update's SET clause, mutate
Source/UpdatedAt on the receiver, and call Update at the three
promotion sites (worker merge loop, worker adoption loop, and
reportDetectedTracker). The shouldPromoteSource gate still ranks the
candidate against the loaded value; Update is now the single write
path that can advance source, with a doc comment spelling out the
load-first contract.
Re-cast the coredata tests around Update: WritesSource pins the
round-trip from receiver to DB, NotFoundForMissingRow preserves the
ErrResourceNotFound contract callers rely on. The old
OnlyTouchesSourceAndUpdatedAt test was a property of the narrow
PromoteSource UPDATE and no longer applies — Update intentionally
rewrites the full editable column set from the receiver.
Signed-off-by: Émile Ré <emile@probo.com>
The cookie detector tags every detection with a source (script,
pre-existing, http) but the storage detector did not, so storage
rows always landed in detected_trackers with source NULL even though
the SDK already distinguishes wrapper writes from pre-load scans.
Plumb a "script"/"pre-existing" source from the storage detector
through the report endpoint into DetectedStorageItem, so the column
gets populated for localStorage, sessionStorage, indexedDB and
cacheStorage entries. No schema change is needed: detected_trackers
already accepts CookieSource values regardless of tracker_type, and
the existing row renders the badge as soon as it is non-null.
Signed-off-by: Émile Ré <emile@probo.com>
Replace duplicated LoadConsentCategoriesByCookieBannerID,
CountConsentCategoriesByCookieBannerID, and
LoadAllConsentCategoriesByCookieBannerID with a single
CookieCategoryFilter in pkg/coredata. The filter uses the
standard CASE WHEN idiom to optionally exclude a kind,
eliminating branching in the service layer.
Signed-off-by: Émile Ré <emile@probo.com>
Poll-based worker that maps org-scoped tracker patterns to the
common knowledge base via pattern matching and domain-based
attribution. Populates initiator_domain on detected trackers
at report time. Resolves org-scoped vendors through the common
third party link.
Signed-off-by: Émile Ré <emile@getprobo.com>
Signed-off-by: Émile Ré <emile@probo.com>
The consent mode is now determined dynamically by the visitor's
country and its applicable regulation. The configured consent_mode
column is dropped from cookie_banners and added to
cookie_consent_records to persist the geo-derived mode at
consent-recording time. When no regulation matches, the default
is OPT_OUT.
Signed-off-by: Émile Ré <emile@getprobo.com>
Replace per-row UPDATE inside the detected-tracker loop with a single
bulk UPDATE ... WHERE id = ANY(...) after the loop. The old approach
locked pattern rows in request-dependent order, causing deadlocks
under concurrent ReportDetectedTrackers calls.
Signed-off-by: Émile Ré <emile@getprobo.com>
The method uses ON CONFLICT ... DO UPDATE, so the name now
matches the actual behaviour. TrackerPattern.InsertIfNotExists
keeps its name because it uses ON CONFLICT ... DO NOTHING.
Signed-off-by: Émile Ré <emile@getprobo.com>
When third-party JS sets a cookie or writes to local/sessionStorage
inside a customer page, the SDK now walks the synchronous call stack
to find the first non-extension, non-Probo, non-first-party http(s)
URL. That origin+path is sent as initiator_url on the report payload,
persisted in a new nullable column on detected_trackers, and preserved
across upserts via COALESCE.
This unlocks per-vendor attribution for cookies and storage writes
without needing pattern name matching, so future categorisation logic
can simply look up the initiator URL in the existing tracker_resources
table and inherit that vendor's category.
GraphQL/MCP exposure is intentionally deferred -- the column is captured
now, surfaced later.
Signed-off-by: Émile Ré <emile@getprobo.com>
Wire resource ingestion and add full CRUD + list/count service methods
for the new tracker_resources table.
- reportDetectedResource splits the URL into origin/path and upserts
into tracker_resources with the uncategorised category.
- CreateTrackerResource, GetTrackerResource, UpdateTrackerResource,
DeleteTrackerResource, MoveTrackerResourceToCategory mirror the
tracker-pattern service surface.
- ListTrackerResourcesForCategory, CountTrackerResourcesForCategory,
ListUncategorisedTrackerResources, CountUncategorisedTrackerResources
provide paginated access.
- Request structs with Validate() and dedicated error sentinels.
Signed-off-by: Émile Ré <emile@getprobo.com>
Move resource tracking (scripts, iframes) out of the pattern-based
tracker_patterns/detected_trackers machinery into its own
tracker_resources table keyed by (banner, type, origin, path).
- Add migration that creates the tracker_resource_type enum, the
tracker_resources table with a unique index, drops existing
SCRIPT/IFRAME rows (not yet in production), and recreates the
tracker_type enum without those values.
- Add TrackerResource coredata model with full CRUD, Upsert (bumps
last_detected_at on conflict), list/count/move operations, filter,
and order field support.
- Register TrackerResourceEntityType (91) in the entity type registry.
- Drop TrackerTypeScript/TrackerTypeIframe from TrackerType enum.
- Update handler to use TrackerResourceType for resource detection.
- Temporarily stub out resource ingestion in ReportDetectedTrackers
pending the service-layer wiring in the next commit.
- Drop SCRIPT/IFRAME from the GraphQL TrackerType enum.
Signed-off-by: Émile Ré <emile@getprobo.com>
Replace prefix-only merge logic with token-template analysis that
discovers sandwich patterns (e.g. ph_phc_*_posthog). The worker now
emits GLOB patterns, adoption uses globMatch, and validation enforces
exactly one wildcard for GLOB patterns.
Signed-off-by: Émile Ré <emile@getprobo.com>
The displayName field was always predictable from pattern + matchType
and allowing edits added unnecessary complexity. Remove displayName
from UpdateTrackerPatternInput across all surfaces (GraphQL, MCP, CLI,
n8n) and make the frontend show it as non-editable text.
Signed-off-by: Émile Ré <emile@getprobo.com>
Strip query params and send origin+pathname so the backend can
distinguish resources served from the same domain but different paths
(e.g. gtm.js vs recaptcha/api.js on googletagmanager.com).
Signed-off-by: Émile Ré <emile@getprobo.com>
In OPT_OUT mode the button_opt_out text was mapped to
button_customize, which opens the preference panel. Map it
to button_reject_all instead so the button performs a
one-click reject for all OPT_OUT regulations.
Signed-off-by: Émile Ré <emile@getprobo.com>
Store the visitor's resolved country code alongside regulation in
cookie_consent_records so downstream surfaces can expose it.
Signed-off-by: Émile Ré <emile@getprobo.com>
The server now resolves regulation-specific translations
(opt-out notice for CCPA, simple notice when no regulation
applies) and remaps text keys before returning the config.
The client hides buttons whose text is empty, so the banner
layout adapts without client-side consent-mode logic.
Signed-off-by: Émile Ré <emile@getprobo.com>
Move cookie banner types (CookieItem, Category, Regulation, BannerConfig,
etc.) into a dedicated types.ts file. Add a coredata.Regulation type with
parsing, JSON marshaling, and database scanning methods. Hardcode the
geoloc-import data directory since the submodule path is fixed.
Signed-off-by: Émile Ré <emile@getprobo.com>
Add a regulation column to cookie_consent_records so each consent
captures which privacy law was in effect. Thread the value from the
handler's geoloc resolution through the service into the DB insert.
On the SDK side, add a Regulation union type to BannerConfig and expose
it via a getter on CookieBannerClient and in the probo-ready event
detail so themed-banner consumers can adapt their UI per regulation.
Signed-off-by: Émile Ré <emile@getprobo.com>
- Fix TotalCount resolver to dispatch by parent type instead
of always using the uncategorised banner counter
- Sync MCP tracker_type enum with canonical TrackerType values
- Add validation for UpdateTrackerPatternRequest
- Validate tracker_type on CreateTrackerPatternRequest
- Set LastMatchedAt when creating pattern from detection
- Use COALESCE for SOURCE cursor pagination with NULLs
- Make source nullable in CLI tracker-pattern list
Signed-off-by: Émile Ré <emile@getprobo.com>
Delete coredata.CookiePattern and all associated CRUD methods,
rename shared types (CookiePatternOrderField, CookiePatternFilter,
CookiePatternMatchType) to TrackerPattern equivalents, and migrate
all API surfaces (GraphQL, MCP, CLI, n8n) to tracker_pattern naming.
The worker was already migrated in the base branch; this commit
completes the removal by dropping the old GraphQL schema/resolvers,
service methods, CLI commands, and n8n operations that operated on
the legacy cookie_patterns table.
Signed-off-by: Émile Ré <emile@getprobo.com>
Non-not-found errors from LoadByBannerIDTypeAndPattern were
silently swallowed, risking partial state on DB failures.
Signed-off-by: Émile Ré <emile@getprobo.com>
CookiePattern Create/Update/Delete/Move now also write the
corresponding TrackerPattern (TrackerType=COOKIE) so the
tracker-pattern-based snapshot picks up the changes and
draft versions are created correctly.
Signed-off-by: Émile Ré <emile@getprobo.com>
The compat endpoint now writes to the unified tracker tables
instead of the legacy cookies + cookie_patterns tables.
Signed-off-by: Émile Ré <emile@getprobo.com>
- TrackerType enum (cookie, local_storage, session_storage, indexed_db, script, iframe)
- TrackerPattern model with EXACT + PREFIX matching for all types
- DetectedTracker model with upsert on conflict
- ReportDetectedTrackers service method handling cookies, storage, and resources
- POST /detected-trackers endpoint on cookie-banner v1 API
- buildSnapshot() now reads from tracker_patterns (cookie type only)
- Entity types registered (89, 90)
Signed-off-by: Émile Ré <emile@getprobo.com>
Exclude the UNCATEGORISED category at the SQL level so
the admin cookie/display/translations pages only see
consent-relevant categories. Removes dead client-side
UNCATEGORISED filters that are no longer needed.
Signed-off-by: Émile Ré <emile@getprobo.com>
Backend for the cookie banner detection page: a new
uncategorisedPatterns connection on CookieBanner with
sortable (NAME, LAST_MATCHED_AT, UPDATED_AT, SOURCE)
and filterable (text ILIKE on name/description, source
enum) paginated results. COALESCE handles NULL-first
ordering for last_matched_at.
Signed-off-by: Émile Ré <emile@getprobo.com>
Track when cookies are last detected (last_detected_at on cookies)
and when patterns last matched a detected cookie
(last_matched_at on cookie_patterns). The cookie timestamp is
refreshed on every detection report; the pattern timestamp is
computed as MAX(last_detected_at) during pattern analysis.
Signed-off-by: Émile Ré <emile@getprobo.com>
Instead of loading all categories and filtering out
UNCATEGORISED in Go, add LoadConsentCategoriesByCookieBannerID
which excludes it in the query. This avoids fetching data we
immediately discard and makes the intent explicit at each call
site.
Signed-off-by: Émile Ré <emile@getprobo.com>
Translations are no longer part of the version snapshot, so the
banner_description text must be validated at write time to ensure
the {{cookie_policy_link}} placeholder is present. Without it
the cookie policy URL silently disappears from the rendered banner.
Signed-off-by: Émile Ré <emile@getprobo.com>
