Introduce a geoloc package that stores CIDR-to-country mappings in
PostgreSQL using the native cidr type with a GiST index for fast
containment lookups. Data comes from the ipverse/country-ip-blocks
dataset added as a git submodule.
A standalone geoloc-import command reads the TXT files from disk
and bulk-loads them via COPY. probod wires the service and logs a
warning when the table is empty.
Signed-off-by: Émile Ré <emile@getprobo.com>
The application always sets status explicitly when creating a campaign,
so the database does not need to provide a default.
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
Source fetch failures stay surfaced on the source fetch (status and
last error) instead of failing the entire campaign, so reviewers can
proceed on the sources that succeeded.
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
Microsoft 365's native SCIM endpoint is unreliable, so mirror the
Google Workspace bridge over Microsoft Graph: a new MICROSOFT_365
OAuth2 connector, a SCIM bridge provider listing /v1.0/users with
$select pagination, and an access review driver that derives admin
status from /directoryRoles members. Refactor the bridge runner to
share OAuth2 plumbing across providers and surface the new bridge
type, scopes, UI card, and bootstrap env wiring.
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
- Fix TotalCount resolver to dispatch by parent type instead
of always using the uncategorised banner counter
- Sync MCP tracker_type enum with canonical TrackerType values
- Add validation for UpdateTrackerPatternRequest
- Validate tracker_type on CreateTrackerPatternRequest
- Set LastMatchedAt when creating pattern from detection
- Use COALESCE for SOURCE cursor pagination with NULLs
- Make source nullable in CLI tracker-pattern list
Signed-off-by: Émile Ré <emile@getprobo.com>
Delete coredata.CookiePattern and all associated CRUD methods,
rename shared types (CookiePatternOrderField, CookiePatternFilter,
CookiePatternMatchType) to TrackerPattern equivalents, and migrate
all API surfaces (GraphQL, MCP, CLI, n8n) to tracker_pattern naming.
The worker was already migrated in the base branch; this commit
completes the removal by dropping the old GraphQL schema/resolvers,
service methods, CLI commands, and n8n operations that operated on
the legacy cookie_patterns table.
Signed-off-by: Émile Ré <emile@getprobo.com>
The worker now operates on TrackerPattern/DetectedTrackers
instead of CookiePattern/Cookies, with TrackerType included
in merge group keys to prevent cross-type merging.
Signed-off-by: Émile Ré <emile@getprobo.com>
- TrackerType enum (cookie, local_storage, session_storage, indexed_db, script, iframe)
- TrackerPattern model with EXACT + PREFIX matching for all types
- DetectedTracker model with upsert on conflict
- ReportDetectedTrackers service method handling cookies, storage, and resources
- POST /detected-trackers endpoint on cookie-banner v1 API
- buildSnapshot() now reads from tracker_patterns (cookie type only)
- Entity types registered (89, 90)
Signed-off-by: Émile Ré <emile@getprobo.com>
Introduce a unified detection schema that supports cookies,
localStorage, sessionStorage, IndexedDB, scripts, and iframes
as tracker types. Existing cookie_patterns and cookies data is
backfilled into the new tables to prepare for the organic
privacy scanner feature.
Signed-off-by: Émile Ré <emile@getprobo.com>
Exclude the UNCATEGORISED category at the SQL level so
the admin cookie/display/translations pages only see
consent-relevant categories. Removes dead client-side
UNCATEGORISED filters that are no longer needed.
Signed-off-by: Émile Ré <emile@getprobo.com>
Backend for the cookie banner detection page: a new
uncategorisedPatterns connection on CookieBanner with
sortable (NAME, LAST_MATCHED_AT, UPDATED_AT, SOURCE)
and filterable (text ILIKE on name/description, source
enum) paginated results. COALESCE handles NULL-first
ordering for last_matched_at.
Signed-off-by: Émile Ré <emile@getprobo.com>
Track when cookies are last detected (last_detected_at on cookies)
and when patterns last matched a detected cookie
(last_matched_at on cookie_patterns). The cookie timestamp is
refreshed on every detection report; the pattern timestamp is
computed as MAX(last_detected_at) during pattern analysis.
Signed-off-by: Émile Ré <emile@getprobo.com>
Instead of loading all categories and filtering out
UNCATEGORISED in Go, add LoadConsentCategoriesByCookieBannerID
which excludes it in the query. This avoids fetching data we
immediately discard and makes the intent explicit at each call
site.
Signed-off-by: Émile Ré <emile@getprobo.com>
Replace the old snapshot-based system for risks with the publish
document system, mirroring the prior vendor / processing activity / DPIA
/ TIA migration. Includes the GraphQL mutation, MCP tool, CLI command,
n8n operation, frontend publish dialog, e2e tests, and a prosemirror
register template covering name, description, category, treatment,
owner, inherent and residual scoring, and notes.
The risk register lives as a generated DocumentTypeRegister document on
the organization, reused across publishes (the major version bumps on
every republish). Approvers can be passed in to create a draft pending
approval; otherwise the version is published immediately. The frontend
Risks page exposes a Publish button and a Document link button when the
document exists, and pre-fills the previous default approvers.
Risks was the last remaining snapshot type, so this commit also removes
the entire snapshot system: drop snapshotId from the Risk GraphQL type
and RiskFilter; remove RiskSnapshotter, Risks.Snapshot,
InsertRiskSnapshots, and the SnapshotID/SourceID fields on Risk; delete
Snapshot, ControlSnapshot, SnapshotsType, SnapshotOrderField,
Snapshottable, the SnapshotService, the Snapshot console resolvers and
GraphQL schema, the Snapshot MCP types and operations
(list/get/take/listControlSnapshots), the snapshot CLI (prb snapshot),
the snapshot frontend pages, routes, banner, LinkedSnapshotsCard,
SnapshotGraph, snapshot helpers, and the snapshot n8n resource and
control link/unlink snapshot operations. The snapshot_id columns remain
in the database but are now filtered out with snapshot_id IS NULL.
Add Get/Upsert/Clear GeneratedDocumentID methods on Risk backed by a new
risks_document_id column on generated_documents, matching the
ProcessingActivity/Finding/Vendor pattern. The migration command
migrate-risk-snapshots-to-documents uses raw SQL queries instead of the
Go snapshot types, since those are gone.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
Translation changes are cosmetic, not consent-contract changes,
so they should not trigger a version bump. Translations are now
loaded live from the database at serve time instead of being
frozen in the snapshot.
Signed-off-by: Émile Ré <emile@getprobo.com>
Adds an `excluded` boolean column to the cookie_patterns table
so operators can mark patterns to be omitted from the consent
banner without deleting them. Includes the migration, struct
field, updated SQL queries, and filter support.
Signed-off-by: Émile Ré <emile@getprobo.com>
The adoptUncategorisedPatterns method loaded all patterns for a
banner then filtered in Go. This adds a CookiePatternFilter
(match_type + cookie_category_id) and wires it into
LoadAllByCookieBannerID so the two targeted loads only fetch
the rows they need.
Signed-off-by: Émile Ré <emile@getprobo.com>
Replace the ad-hoc patternAnalysisTask struct with coredata.CookieBanner
as the worker type parameter, matching the pattern used by other workers
(esign, accessreview). SQL methods move back to coredata on *CookieBanner.
Signed-off-by: Émile Ré <emile@getprobo.com>
- Fix DurationInput fallback unit from "minutes" to "seconds" and add
seconds as a selectable unit to prevent silent duration inflation
- Use parseFloat instead of parseInt for duration input to preserve
fractional values
- Scope prefix merge groups by category ID to prevent cross-category
merging
- Relink cookies and delete exact patterns even when prefix pattern
already exists
- Prefer exact matches and longest prefix in pattern selection query
- Fix wrong error type in GetCookiePattern (ErrCookiePatternNotFound)
- Handle singular/plural in humanizeSeconds fallback branch
Signed-off-by: Émile Ré <emile@getprobo.com>
Replace the free-form duration TEXT column with a nullable
max_age_seconds INTEGER on both cookies and cookie_patterns
tables. The SDK detector now sends raw seconds instead of
humanized strings, eliminating locale-dependent comparisons
in the pattern merge worker. Humanization happens at display
time in the widget and console UI.
Signed-off-by: Émile Ré <emile@getprobo.com>
Set pattern_analysis_requested_at on all existing banners so the worker
runs once per banner on release and merges any existing prefix groups.
Signed-off-by: Émile Ré <emile@getprobo.com>
Background worker polls cookie_banners with pattern_analysis_requested_at
set, groups EXACT patterns sharing a common prefix, and merges groups of
3+ into a PREFIX pattern. Detection sets the flag when new EXACT patterns
are created. The worker relinks cookies, removes orphaned patterns, and
updates the draft version via ensureDraftVersionForBanner.
Signed-off-by: Émile Ré <emile@getprobo.com>
Introduce a cookie_patterns table that groups cookies sharing a common
prefix (e.g. phc_*) into a single manageable row. Every cookie now
belongs to a pattern (EXACT or PREFIX match type). Category, description,
and display metadata move from cookies to patterns, making patterns the
unit of management and display in the console and published snapshots.
Signed-off-by: Émile Ré <emile@getprobo.com>
Cookies set by browser extensions are not the website operator's
compliance responsibility. This adds stack-trace inspection to
filter out extension-originated document.cookie writes, and
annotates pre-existing cookies with a source field so operators
can triage them separately.
Introduces a CookieSource enum (SCRIPT / PRE_EXISTING) across
the full stack: PostgreSQL, coredata, service, HTTP handler, and
GraphQL schema. On conflict, source is upgraded from PRE_EXISTING
to SCRIPT when a page script is later observed setting the cookie.
Signed-off-by: Émile Ré <emile@getprobo.com>
Replace the old snapshot-based system for vendors with the publish
document system, mirroring the prior processing activity / DPIA / TIA
migration. Includes the GraphQL mutation, MCP tool, CLI command, n8n
operation, frontend publish dialog, e2e tests, and a prosemirror
register template covering vendor profile fields plus per-vendor
sections for services, contacts, risk assessments, compliance reports,
BAA and DPA agreements.
The vendor register lives as a generated DocumentTypeRegister document
on the organization, reused across publishes (the major version bumps
on every republish). Approvers can be passed in to create a draft
pending approval; otherwise the version is published immediately. The
frontend Vendors page exposes a Publish button and a Document link
button when the document exists, and pre-fills the previous default
approvers.
Remove snapshot mode entirely from vendors and their sub-entities: drop
snapshotId/sourceId from GraphQL Vendor type and VendorFilter; remove
SnapshotsTypeVendors from the snapshot registry and delete
Vendors.Snapshot, VendorSnapshotter interface and all
*.InsertVendorSnapshots methods on contacts, services, risk
assessments, compliance reports, BAA and DPA. Drop the snapshot routes
and banner from the frontend. The snapshot_id columns remain in the
database but are now filtered out with snapshot_id IS NULL.
Add Get/Upsert/Clear GeneratedDocumentID methods on Vendor backed by a
new vendors_document_id column on generated_documents, matching the
ProcessingActivity/Finding/Obligation pattern.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
Session validity was decoupled from credential rotation: changing
a password (logged-in flow) or completing a forgot-password reset
left every existing iam_sessions row valid until its idle TTL.
A user who saw their account compromised on another device had
no way to actually evict that device by rotating the password.
Inside the same DB transaction as the password update, expire the
identity's other active sessions:
- ChangePassword keeps the caller's current session and revokes
every other session for the identity, so the user is not
logged out of the browser they just used.
- ResetPassword has no caller session (the user is anonymous,
authenticated only by a stateless token) and revokes all of
the identity's active sessions.
The session middleware already rejects rows with expire_reason
set, so revoked sessions are kicked out on the next request
without any middleware change.
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
Replace the old PDF/snapshot-based exports for processing activities,
Data Protection Impact Assessments and Transfer Impact Assessments with
the publish document system. Includes GraphQL mutations, MCP tools, CLI
commands, n8n operations, frontend publish dialogs, e2e tests, and
prosemirror register templates that mirror the previous PDF layouts.
Each register lives as a generated DocumentTypeRegister document on the
organization, reused across publishes (the major version bumps on every
republish). Approvers can be passed in to create a draft pending
approval; otherwise the version is published immediately. The frontend
ProcessingActivities page exposes a Publish dropdown per register and a
Document link button per active tab, pre-fills the previous default
approvers, and navigates to the published document on success.
Remove snapshot mode entirely from these three entities: drop snapshotId
and sourceId from GraphQL schemas, types, filters, resolvers, MCP spec,
frontend routes and pages; remove SnapshotsTypeProcessingActivities from
the snapshot registry and delete the ProcessingActivities.Snapshot,
ProcessingActivitySnapshotter interface and *.InsertProcessingActivitySnapshots
methods. The snapshot_id columns remain in the database but are now
filtered out with snapshot_id IS NULL.
Add Get/Upsert/Clear GeneratedDocumentID methods on each entity type
(ProcessingActivity, DataProtectionImpactAssessment,
TransferImpactAssessment) backed by new columns in the generated_documents
table, matching the Finding/Obligation pattern.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
Batch-load CookieBanner and CookieCategory entities via
dataloadgen instead of making individual service calls in
GraphQL resolvers, matching the existing dataloader pattern
used for organizations, frameworks, etc.
Signed-off-by: Émile Ré <emile@getprobo.com>
Display record attributes and parsed consent data with
per-category consent state and cookies from the banner
version snapshot. The page lives outside the config layout
with its own breadcrumb navigation.
Signed-off-by: Émile Ré <emile@getprobo.com>
Replace the opaque cookieBannerVersionId filter with an
integer version filter. The SQL filter now resolves the
version number via a subquery against cookie_banner_versions.
Also fix the CookieBannerVersion resolver on consent records
to load the full version from the database instead of
returning a stub with only the ID set (which caused the
version to always display as 0).
Signed-off-by: Émile Ré <emile@getprobo.com>
Exposes the cookie consent record audit trail through a new
"Consent Records" tab on the cookie banner configuration page.
The full stack includes: extended coredata filter (visitor ID,
banner version), GraphQL schema/types/resolvers, and a React
page with SortableTable (size 50) and three compliance filters
(action, visitor ID, banner version).
Signed-off-by: Émile Ré <emile@getprobo.com>
Replace the old snapshot-based approach with the new publish document
system for findings and obligations. Includes GraphQL mutations, MCP
tools, CLI commands, e2e tests, frontend publish dialogs, and
snapshot-to-document migration tools.
Remove snapshot mode entirely from findings and obligations: drop
snapshotId from GraphQL schemas, filters, resolvers, MCP spec, frontend
routes, pages, and helpers. The snapshot_id column remains in the
database but is now filtered out with snapshot_id IS NULL.
Remove auditor's ability to publish SoA.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
Add the profile state attribute (ACTIVE/INACTIVE) to the MCP
Profile schema so listUsers and getUser tools expose it, and
add a state filter to listUsers.
Rename excludeContractEnded to contractEnded across the entire
stack (MCP, GraphQL, CLI, frontend). The new boolean is two-way:
true returns only users with ended contracts, false returns only
users with active or no contract, and null returns all.
Signed-off-by: Émile Ré <emile@getprobo.com>
Update writes every mutable column of agent_runs except checkpoint.
That exclusion is intentional: PGCheckpointer.Save and ClearCheckpoint
are the only paths that touch the column, so a status commit cannot
overwrite an in-flight checkpoint saved between Load and Update.
Surface the rule on the Update method so future readers do not patch
in a checkpoint write thinking it was an oversight.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Each subtest now inserts its own PENDING run and runs under
t.Parallel(); shared state across subtests was the only reason they
had to stay sequential. Also adds a round-trip test that exercises
the approval-state fields (PendingToolCalls, PendingApprovals,
ApprovalInput, AllToolCalls, InnerCheckpoints, CompletedCalls) to
catch regressions where Save/Load drops nested or approval payloads.
The nonexistent-run case now uses a valid GID in the same tenant so
it reaches the row-not-found branch instead of short-circuiting on
the tenant-scope guard.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Each Save and Load now derives tenant_id from the run GID and pins it
in the WHERE clause. A caller that supplies an ID from another tenant
fails closed instead of silently reading or overwriting cross-tenant
checkpoint data. Also rejects oversize checkpoints on load as a
read-side guard against a tampered or migrated row exceeding
MaxCheckpointBytes.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
The transactional variant is no longer reachable: supervisor paths
persist checkpoints through PGCheckpointer (WithConn, not WithTx),
and Update deliberately excludes the checkpoint column to avoid
racing a concurrent checkpoint save. Nothing else calls it.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
The supervisor was a hand-rolled polling, semaphore, and wait-group
loop predating the project's adoption of the shared worker kit. Two
sibling workers in pkg/probo already use the kit, and go-worker.md
documents it as the project convention.
This commit introduces agentRunHandler, which implements
worker.Handler[coredata.AgentRun] and worker.StaleRecoverer, and
reduces AgentRunSupervisor to a thin wrapper that owns the handler
plus a worker.Worker and bridges ctx cancellation into a handler-
level shutdown broadcast via context.AfterFunc. The agent stop
channel is now closed by a per-Process forwarder goroutine when the
broadcast fires, so in-flight runs checkpoint at the next turn
boundary and drain through wg.Wait before Run returns.
The stop_requested column, struct field, supporting SQL, and the
LoadRunningStopRequestedIDs function are removed end-to-end. None
of it was ever wired to an external surface; it existed purely to
let the supervisor find runs the operator wanted to halt. With the
kit handling the polling cadence and the AfterFunc bridging
shutdown, per-row flagging is dead weight.
The supervisor's public API (NewAgentRunSupervisor, Run, the With*
option helpers, and the error sentinels) stays intact so probod.go
needs no change. The integration test now triggers stop by
cancelling the supervisor context, which is the actual production
path through SIGTERM rather than a synthetic DB flag. Prometheus
counters and OTel spans labelled worker="agent-run-supervisor"
come for free.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Create agent_runs table with lease-based concurrency control.
AgentRun entity follows standard coredata patterns with Scoper,
StrictNamedArgs, and cursor pagination. PGCheckpointStore implements
agent.CheckpointStore backed by the checkpoint JSONB column with
version validation and 10 MiB size guard. Register AgentRunEntityType
as entity type 75.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Include a `bid` field in the consent cookie so it explicitly
identifies which cookie banner it belongs to, making validation
direct instead of relying on the visitor ID as an implicit
discriminator. Existing cookies without `bid` self-heal on the
next load by falling through to the API fetch.
Signed-off-by: Émile Ré <emile@getprobo.com>
Introduce a required cookie_policy_url alongside the existing
privacy_policy_url (now optional) so banners can link directly to a
dedicated cookie policy — a compliance best practice recommended by
CNIL, ICO, and the EDPB. Existing rows are seeded from their current
privacy_policy_url value.
Both {{cookie_policy_link}} and {{privacy_policy_link}} placeholders
are supported independently in banner description translations.
Signed-off-by: Émile Ré <emile@getprobo.com>
Origin is a fundamental identity property of a banner tied to consent
records for a specific site. Changing it would break the audit trail
and violate GDPR consent specificity requirements.
Signed-off-by: Émile Ré <emile@getprobo.com>
When DNS verification, CAA checks, or HTTP challenge completion
fail during certificate provisioning, the error is now stored on
the custom domain record and exposed via GraphQL. The console
displays it in both the domain card and domain detail dialog so
users can diagnose configuration issues without checking logs.
Previously these failures returned an error that was only logged
server-side, leaving users with no visibility into why their
domain was stuck in a pending state.
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
