Renames the user-facing 'vendor' concept to 'third party' across the
entire codebase. The shared common_third_parties reference table is
unchanged.
Migration. Renames the vendor_category enum, the vendors and
vendor_<entity> tables (contacts, services, compliance_reports,
business_associate_agreements, data_privacy_agreements,
risk_assessments) and their vendor_id columns, the asset_vendors /
data_vendors / processing_activity_vendors junction tables,
generated_documents.vendors_document_id, the webhook_event_type
'vendor:<verb>' values, and the snapshots_type 'VENDORS' value.
Backend. Renames coredata models and SQL queries, probo services,
GraphQL / MCP API surface, console / trust / webhook resolvers and
types, the CLI (prb vendor* -> prb third-party*; pkg/cmd/vendormgmt
-> pkg/cmd/thirdpartymgmt), the document generator, vetting agent
prompts, and the common-third-parties-import command.
Frontend, packages, n8n, e2e. Renames apps/console pages, components,
hooks, routes, dialogs, and tabs; the shared @probo/vendors package
(now @probo/third-parties); the @probo/ui Vendors atoms (now
ThirdParties, VendorLogo -> ThirdPartyLogo); the n8n community node
actions/vendor folder (now actions/thirdParty); and the e2e Go test
suite (console and MCP). Filesystem and URL paths use kebab-case
(third-parties), GraphQL fields and TypeScript identifiers use
camelCase (thirdParty / thirdParties), Go types use PascalCase
(ThirdParty), and human-facing text uses 'third party' with a space.
Co-authored-by: Bryan Frimin <bryan@getprobo.com>
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
Add a single info log line in each authn middleware once an identity is
resolved, so every authenticated request emits a record that ties the
request back to its user and credential:
- Cookie session middleware logs "session authenticated" with
identity_id and session_id.
- API key middleware logs "api key authenticated" with identity_id and
api_key_id.
- OAuth2 access token middleware logs "access token authenticated"
with identity_id and access_token_id.
The credential IDs are row identifiers (GIDs), not the secret token
values, so they're safe to log and let operators correlate a request
back to the specific credential used. The log lines use the
request-scoped logger from httpserver.LoggerFromContext so they
inherit http_request_id and any other middleware-attached attributes.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
These tables were superseded by tracker_patterns and
detected_trackers in migration 20260505T090000Z, which
backfilled all data. The old tables are no longer referenced
by any Go code, GraphQL schema, MCP specification, or CLI
command.
The cookie_source and cookie_pattern_match_type enum types
are retained because tracker_patterns still uses them.
Signed-off-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Émile Ré <nemile.re@gmail.com>
Extract X-SDK-Version from request headers in a dedicated
middleware and store it in context. All ErrorCtx calls in the
cookie banner handler now include the sdk_version field.
Signed-off-by: Émile Ré <emile@getprobo.com>
The consent mode is now determined dynamically by the visitor's
country and its applicable regulation. The configured consent_mode
column is dropped from cookie_banners and added to
cookie_consent_records to persist the geo-derived mode at
consent-recording time. When no regulation matches, the default
is OPT_OUT.
Signed-off-by: Émile Ré <emile@getprobo.com>
Replace per-row UPDATE inside the detected-tracker loop with a single
bulk UPDATE ... WHERE id = ANY(...) after the loop. The old approach
locked pattern rows in request-dependent order, causing deadlocks
under concurrent ReportDetectedTrackers calls.
Signed-off-by: Émile Ré <emile@getprobo.com>
When a cookie name has no separator and looks variable (e.g. a long
hex string), heuristicTemplate returned "*" which created a catch-all
glob pattern matching every tracker of the same type and duration
bucket. Skip these separator-less names entirely so they remain as
individual exact-match patterns for human triage.
Signed-off-by: Émile Ré <emile@getprobo.com>
- relay: key uploadables map by actual variable name instead of iteration
index so order-mismatch between Object.keys passes can't desync the
multipart map from form field names
- mcp/v1: drop dead commented middleware line
- DurationPicker: tighten parse regex to require PT prefix for M/H and P
for D/W, and reject NaN in stringify so cleared inputs don't produce
invalid duration strings
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
- Preserve SSRF protection by wrapping the existing transport
instead of replacing it with a bare http.Transport
- Strip DSN from url.Parse error to avoid leaking credentials
- Gate CommonThirdPartyCombobox on search length to prevent
showing stale results when input is shortened
- Handle multi-value and uppercase sizes attributes in
parseSizeAttr for correct icon-size ranking
- Match rel tokens containing "icon" (e.g. "shortcut icon")
instead of requiring an exact match
- Limit HTML response body to 10 MiB before parsing
- Reject sslmode=prefer explicitly in both import tools
Signed-off-by: Émile Ré <emile@getprobo.com>
Replace the Google Favicons API with HTML head tag parsing
to find higher-quality logos (SVG, apple-touch-icon, large
PNG icons, msapplication-TileImage). The new pkg/webinspect
package parses a website's DOM tree and is extensible for
future resource extraction (footer links, etc.).
Signed-off-by: Émile Ré <emile@getprobo.com>
The description field was never surfaced in the UI and added no value.
Drop it from the database, Go structs, GraphQL schema, import tool,
frontend fragment, and vendor seed data.
Signed-off-by: Émile Ré <emile@getprobo.com>
Fetch favicons at import time instead of calling Google's favicon
service per page load. Logos are stored as public files in S3 and
served through the existing /api/files/v1/{id} endpoint.
Signed-off-by: Émile Ré <emile@getprobo.com>
The CreateVendorDialog previously loaded the entire @probo/vendors
JSON bundle client-side and used MiniSearch for fuzzy search. This
replaces it with a GraphQL query against the common_third_parties
database table, searched server-side via ILIKE filtering.
Backend: adds CommonThirdParty GraphQL type, a pkg/thirdparty
service, and a commonThirdParties(name) root query. Frontend:
splits into CommonThirdPartyCombobox (display) and an @inline
fragment read on selection via readInlineData.
Signed-off-by: Émile Ré <emile@getprobo.com>
Introduce a globally-shared, non-tenant-scoped common_third_parties
table that mirrors the public subset of vendor metadata, plus a
one-shot cmd/common-third-parties-import CLI that seeds it from
packages/vendors/data.json. The catalog will back future flows (e.g.
vendor autocomplete) so each tenant no longer needs to duplicate the
same baseline data.
The importer is idempotent via ON CONFLICT (lower(name)) DO UPDATE
and prints inserted/updated counts. GIDs use gid.NilTenant since the
table is not tenant-scoped; uniqueness still comes from the entity
type plus 14 bytes of timestamp/random suffix.
Signed-off-by: Émile Ré <emile@getprobo.com>
tracker_patterns rows were created with entity type 88 (removed
CookiePatternEntityType) instead of 89 (TrackerPatternEntityType), and
detected_trackers rows migrated from the cookies table carried entity
type 85 (removed CookieEntityType) instead of 90 (DetectedTrackerEntityType).
Signed-off-by: Émile Ré <emile@getprobo.com>
When IP geolocation returned no matching CIDR block, LookupCountryByIP
returned an empty string with nil error. The handler took the address of
that empty string, producing a non-nil pointer to "", which was inserted
into the database. Guard against this by returning nil when the resolved
country code is empty, and backfill existing rows with a migration.
Signed-off-by: Émile Ré <emile@getprobo.com>
fmt.Sprintf interprets the literal % characters in the LIKE escape
clause as format verbs, corrupting the query and causing a 500 on
the /report endpoint. Reorder tracker type / source filters in the
trackers page.
Signed-off-by: Émile Ré <emile@getprobo.com>
The pattern analysis worker now recognises UUID-like, hash-like,
and long numeric tokens as variable parts and replaces them with
wildcards heuristically, even from a single observation. This
prevents site-specific identifiers from being treated as static
text while meaningful suffixes (window_id, posthog, …) get
incorrectly wildcarded.
Also upgrades globMatch and the FindMatchingPattern SQL query
to support multiple wildcards in a single pattern.
Signed-off-by: Émile Ré <emile@getprobo.com>
The trackers page now lets users filter by tracker type
(Cookie, localStorage, sessionStorage, IndexedDB, Cache
Storage) in addition to the existing source filter. Each
tracker type and cookie source badge uses a distinct color
for quick visual scanning.
Signed-off-by: Émile Ré <emile@getprobo.com>
The method uses ON CONFLICT ... DO UPDATE, so the name now
matches the actual behaviour. TrackerPattern.InsertIfNotExists
keeps its name because it uses ON CONFLICT ... DO NOTHING.
Signed-off-by: Émile Ré <emile@getprobo.com>
- Detectors: keep batched entries in `pending` until the POST succeeds
and guard against concurrent flushes, so transient network errors no
longer silently drop detection reports.
- Worker: add stable tie-breakers to the merge-candidate sort so the
greedy assignment produces deterministic groups across runs.
- Handler: skip resource entries with an empty URL (zero-value `uri.URI`
when the `url` field is missing) before persisting them.
- Third-party detector: allow same-origin service worker scripts through
`processResource` -- service workers are always same-origin by spec,
so the previous filter made `wrapServiceWorker` unreachable.
- Resource row edit: bump the description cell `colSpan` to 3 so the
edit row spans all five table columns.
- Resolver: handle `ErrSameResourceCategoryMove` explicitly so the no-op
move returns a validation error instead of an internal one.
Signed-off-by: Émile Ré <emile@getprobo.com>
A registered service worker is a URL-shaped artifact (origin+path of
the worker script), so it goes in tracker_resources as a new
SERVICE_WORKER resource type. A Cache Storage bucket is an opaque
named string with no URL, so it goes in detected_trackers as a new
CACHE_STORAGE tracker type.
Frontend:
- StorageDetector wraps caches.open() and enumerates caches.keys()
on start to surface pre-existing buckets that pre-date the SDK
load (service workers commonly populate caches eagerly on
install).
- ThirdPartyDetector wraps navigator.serviceWorker.register() and
enumerates getRegistrations() on start.
Both wrappers degrade silently on insecure contexts where these APIs
are unavailable.
Signed-off-by: Émile Ré <emile@getprobo.com>
ThirdPartyDetector previously only saw <script src> and <iframe src>
because it scanned the DOM and watched mutations. Add a single
PerformanceObserver({type:'resource', buffered:true}) that picks up
everything the browser actually loaded:
- tracking pixels (<img>, <picture>, srcset)
- cross-origin stylesheets and web fonts
- fetch / XHR / sendBeacon / ping calls (SDK call-homes)
- video, audio, embed, object media
initiatorType is mapped to six new tracker_resource_type enum values
(IMAGE, STYLESHEET, FONT, BEACON, FETCH, MEDIA) and the existing
upsert path in tracker_resources picks them up unchanged.
Closes a real gap with headless cookie scanners: most SDKs phone home
via beacons after their script is gone, and the DOM scan never saw it.
Signed-off-by: Émile Ré <emile@getprobo.com>
When third-party JS sets a cookie or writes to local/sessionStorage
inside a customer page, the SDK now walks the synchronous call stack
to find the first non-extension, non-Probo, non-first-party http(s)
URL. That origin+path is sent as initiator_url on the report payload,
persisted in a new nullable column on detected_trackers, and preserved
across upserts via COALESCE.
This unlocks per-vendor attribution for cookies and storage writes
without needing pattern name matching, so future categorisation logic
can simply look up the initiator URL in the existing tracker_resources
table and inherit that vendor's category.
GraphQL/MCP exposure is intentionally deferred -- the column is captured
now, surfaced later.
Signed-off-by: Émile Ré <emile@getprobo.com>
Add the full GraphQL surface for the new tracker_resources table:
- TrackerResourceType enum (SCRIPT, IFRAME), TrackerResource node type
with connection/edge/order/filter, fields on CookieBanner
(uncategorisedTrackerResources) and CookieCategory (trackerResources).
- Mutations: createTrackerResource, updateTrackerResource,
deleteTrackerResource, moveTrackerResourceToCategory with
inputs and payloads.
- Resolvers for all mutations, connection fields, field resolvers
(cookieCategory, permission), and totalCount.
- IAM actions: core:tracker-resource:{get,list,create,update,delete}.
Signed-off-by: Émile Ré <emile@getprobo.com>
Wire resource ingestion and add full CRUD + list/count service methods
for the new tracker_resources table.
- reportDetectedResource splits the URL into origin/path and upserts
into tracker_resources with the uncategorised category.
- CreateTrackerResource, GetTrackerResource, UpdateTrackerResource,
DeleteTrackerResource, MoveTrackerResourceToCategory mirror the
tracker-pattern service surface.
- ListTrackerResourcesForCategory, CountTrackerResourcesForCategory,
ListUncategorisedTrackerResources, CountUncategorisedTrackerResources
provide paginated access.
- Request structs with Validate() and dedicated error sentinels.
Signed-off-by: Émile Ré <emile@getprobo.com>
Move resource tracking (scripts, iframes) out of the pattern-based
tracker_patterns/detected_trackers machinery into its own
tracker_resources table keyed by (banner, type, origin, path).
- Add migration that creates the tracker_resource_type enum, the
tracker_resources table with a unique index, drops existing
SCRIPT/IFRAME rows (not yet in production), and recreates the
tracker_type enum without those values.
- Add TrackerResource coredata model with full CRUD, Upsert (bumps
last_detected_at on conflict), list/count/move operations, filter,
and order field support.
- Register TrackerResourceEntityType (91) in the entity type registry.
- Drop TrackerTypeScript/TrackerTypeIframe from TrackerType enum.
- Update handler to use TrackerResourceType for resource detection.
- Temporarily stub out resource ingestion in ReportDetectedTrackers
pending the service-layer wiring in the next commit.
- Drop SCRIPT/IFRAME from the GraphQL TrackerType enum.
Signed-off-by: Émile Ré <emile@getprobo.com>
Rewrite worker unit tests: TestTemplateCandidates, TestGlobMatch,
TestSplitTokens, and updated TestFindMergeGroups with sandwich pattern
cases. Update e2e test to use GLOB instead of PREFIX.
Signed-off-by: Émile Ré <emile@getprobo.com>
Replace prefix-only merge logic with token-template analysis that
discovers sandwich patterns (e.g. ph_phc_*_posthog). The worker now
emits GLOB patterns, adoption uses globMatch, and validation enforces
exactly one wildcard for GLOB patterns.
Signed-off-by: Émile Ré <emile@getprobo.com>
Introduces a wildcard-based match type that supports prefix, suffix,
and sandwich patterns (e.g. ph_phc_*_posthog). The SQL matching uses
starts_with/ends_with on the parts split at '*', avoiding LIKE and
its underscore escaping issues. Existing PREFIX rows are migrated to
GLOB with a trailing '*'.
Signed-off-by: Émile Ré <emile@getprobo.com>
The displayName field was always predictable from pattern + matchType
and allowing edits added unnecessary complexity. Remove displayName
from UpdateTrackerPatternInput across all surfaces (GraphQL, MCP, CLI,
n8n) and make the frontend show it as non-editable text.
Signed-off-by: Émile Ré <emile@getprobo.com>
Progressive enhancement for Chromium browsers: listen on the
CookieStore change event to catch cookies set by Set-Cookie HTTP
response headers, which the document.cookie setter hook cannot see.
Adds a new "http" cookie source through the full stack.
Signed-off-by: Émile Ré <emile@getprobo.com>
Strip query params and send origin+pathname so the backend can
distinguish resources served from the same domain but different paths
(e.g. gtm.js vs recaptcha/api.js on googletagmanager.com).
Signed-off-by: Émile Ré <emile@getprobo.com>
Trackers sharing a prefix but with materially different lifetimes
(e.g. session vs 1-year) were incorrectly merged into a single
prefix pattern. Port the snap table from cookie-utils.ts into Go
and use it to bucket durations so only trackers that display the
same human-readable lifetime can merge. Update the unique index
to include COALESCE(max_age_seconds, -1) so prefix patterns with
different durations can coexist.
Signed-off-by: Émile Ré <emile@getprobo.com>
Address review feedback:
- Move ErrSuspendForCheckpoint from checkpoint.go to errors.go
next to the rest of the agent error declarations; drop the
colon in the error string so it matches the existing
`agent run <event>` style used by the supervisor sentinels.
- Replace the inline `outerCtx := ctx; ctx = context.WithoutCancel(ctx)`
pattern with a small `suspendShield` helper in context.go used
by coreLoop, resumeWithOpts, and resumeNested. Reads more
cleanly and stops surfacing the WithoutCancel mechanism at
every call site.
- Trim the doc comments on Run, RunStreamed, Resume, Restore, the
ErrSuspendForCheckpoint declaration, and the saveCtx comment in
restoreNestedSuspended down to the contract bullet.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Two additions:
- agent_test.go's "context cancellation triggers graceful suspend"
now also asserts the input messages land in the suspension
checkpoint — verifies the embedded-Checkpoint path that fires
when no Checkpointer is configured.
- cancel_test.go gets a third subtest that parks the LLM provider
inside ChatCompletion via a release channel, cancels ctx while
the call is in flight, then confirms the LLM call still saw a
non-cancelled ctx and the just-completed turn lands in the
persisted checkpoint. Proves the framework's WithoutCancel
shielding works end-to-end at the unit level.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
