Commit Graph

129 Commits

Author SHA1 Message Date
Émile Ré
f0fe70fe1c Restore url.PathEscape on user-supplied path segments in url.JoinPath calls
url.JoinPath does not percent-encode slashes or reserved characters in
its arguments, so user-supplied values (group IDs, slugs, team IDs) must
be wrapped with url.PathEscape to prevent path traversal. Update cursor
rule and contrib guide to codify this as a mandatory practice.

Signed-off-by: Émile Ré <emile@probo.com>
2026-05-20 12:57:26 +04:00
Émile Ré
34c25c2727 Refine PgError constraint checks and document PK rule
Remove dead 23505 checks on single-GID primary keys (oauth2_consent,
risk_assessment, risk_assessment_scenario, risk_assessment_scope).
Add missing constraints to membership_profile and
statement_of_applicability. Document composite-PK vs GID-PK rule in
cursor rules and contrib guide.

Signed-off-by: Émile Ré <emile@probo.com>
2026-05-20 10:14:45 +04:00
Émile Ré
2040931405 Move Firecrawl API key from top-level config into Agents.Tools
Firecrawl is a tool used by agents (tracker mapping, third-party
assessor), so its configuration belongs under AgentsConfig rather than
as a standalone Config field. Adds AgentToolsConfig to hold agent tool
credentials and updates all config propagation consumers.

Signed-off-by: Émile Ré <emile@probo.com>
2026-05-19 12:32:41 +04:00
Émile Ré
c4147e6801 Extract agent and tools from tracker mapping worker into dedicated files
Split tracker_mapping_worker.go: agent construction, prompts, and
structured output type move to tracker_mapping_agent.go; each tool gets
its own *_tool.go file. Update naming conventions (worker, agent, tool
file patterns, AgentResult suffix, RunTyped preference).

Signed-off-by: Émile Ré <emile@probo.com>
2026-05-19 12:06:15 +04:00
Émile Ré
10adf2bd4b Refactor domain-match lookup to idiomatic Load + filter
Replace the cross-entity JOIN in
DetectedTrackers.LoadCommonThirdPartyIDByDomainMatch with two
idiomatic coredata calls: LoadInitiatorDomainsByTrackerPatternID
on DetectedTrackers, then a new CommonThirdPartyDomains.Load with
a CommonThirdPartyDomainFilter. Each entity now queries only its
own table, and the caller orchestrates the lookup.

Document the Load vs LoadAll naming convention and the no
cross-entity JOINs rule in contrib/claude/coredata.md.

Signed-off-by: Émile Ré <emile@probo.com>
2026-05-19 11:52:57 +04:00
Émile Ré
370b593217 Replace xmax upsert trick with RETURNING full row
Upsert methods now RETURNING all struct columns and scan the result
back into the pointer receiver, keeping the caller in sync with the
actual DB state (id, created_at, etc. from the existing row on
conflict). Insert detection compares the saved original ID with the
returned ID instead of relying on the PostgreSQL-internal xmax column.

Signed-off-by: Émile Ré <emile@probo.com>
2026-05-19 11:37:26 +04:00
Émile Ré
0c89a4b241 Drop RowsAffected check from Delete methods
Deletes are idempotent — zero affected rows is not an error.

Signed-off-by: Émile Ré <emile@probo.com>
2026-05-19 11:08:01 +04:00
Émile Ré
f1357a7e20 Check constraint name in PgError unique violations
Checking only pgErr.Code == "23505" is insufficient when a
table has multiple unique constraints. Always verify
pgErr.ConstraintName as well to avoid mapping unrelated
violations to the wrong sentinel error.

Signed-off-by: Émile Ré <emile@probo.com>
2026-05-19 11:03:37 +04:00
Émile Ré
f81168bc18 Build tracker identification prompt categories dynamically
Generate the category list from coredata.ThirdPartyCategories() at
runtime instead of hardcoding it in the prompt text. Type the
TrackerIdentification.Category field as coredata.ThirdPartyCategory so
JSON unmarshaling validates values automatically.

Also documents the .txt.tmpl template file naming convention.

Signed-off-by: Émile Ré <emile@probo.com>
2026-05-19 10:59:16 +04:00
Émile Ré
76d42ef2db Hardcode Firecrawl API endpoint, drop FIRECRAWL_ENDPOINT config
Firecrawl has a single public API at https://api.firecrawl.dev/v2.
The endpoint was configurable but never varied across environments,
so hardcode it as a package-level const and remove the Endpoint
field from FirecrawlConfig and all downstream wiring (bootstrap,
Helm chart, probod, vetting, cookiebanner).

Signed-off-by: Émile Ré <emile@probo.com>
2026-05-19 10:28:05 +04:00
Émile Ré
a67ac462d7 Remove SearXNG search backend, use Firecrawl exclusively
SearXNG was a fallback search backend that added complexity without
being used in practice. All search-dependent features (web search,
government DB checks, vetting orchestrator, tracker mapping) now use
Firecrawl exclusively. Removes the SEARCH_ENDPOINT config plumbing
from probodconfig, bootstrap, Helm charts, and all callers.

Signed-off-by: Émile Ré <emile@probo.com>
2026-05-19 10:01:01 +04:00
Émile Ré
091be2653a Enforce multiline rule for single multiline arguments
Update go-style guide and cursor rule to clarify that even a single
argument spanning multiple lines must break after the opening
parenthesis. Fix six violations across the branch.

Signed-off-by: Émile Ré <emile@probo.com>
2026-05-19 09:47:52 +04:00
Émile Ré
c162e6ba20 Update go-style guide
Signed-off-by: Émile Ré <emile@probo.com>
2026-05-19 09:37:14 +04:00
Émile Ré
e87b426801 Propagate firecrawl and tracker-mapping config to bootstrap and Helm
Group firecrawl-endpoint and firecrawl-api-key under a nested firecrawl
config key. Add env var mappings (FIRECRAWL_ENDPOINT, FIRECRAWL_API_KEY,
SEARCH_ENDPOINT, AGENT_TRACKER_MAPPING_*) to the bootstrap builder with
test coverage. Wire the new values through the Helm chart (values,
deployment, secret, production example).

Signed-off-by: Émile Ré <emile@probo.com>
2026-05-18 19:05:14 +04:00
Émile Ré
d4e56b40e6 Add proboctl release track
Signed-off-by: Émile Ré <emile@probo.com>
2026-05-18 19:05:09 +04:00
Aurélien Sibiril
0d6484a0ef Drop Helm wiring for the 9 new access-review connectors → Drop e2e TODO about access-review connector coverage gap
- Drop Helm wiring for the 9 new access-review connectors
- Drop e2e TODO about access-review connector coverage gap

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
2026-05-17 17:22:51 +02:00
Aurélien Sibiril
7d0dec82f6 Guard Helm connector env vars on clientId / clientSecret → Address review feedback on access-review drivers
- Guard Helm connector env vars on clientId / clientSecret
- Decode Vercel pagination cursor as *int64
- Drop Monday probe URL — no valid GET endpoint
- Address review feedback on access-review drivers

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
2026-05-17 17:22:51 +02:00
Aurélien Sibiril
c2db47e698 Require CLIENT_SECRET for new access-review connectors → Drop Snyk, Ramp, Lever, Deel access-review providers
- Require CLIENT_SECRET for new access-review connectors
- Use Heroku account UUID as ExternalID
- Bump GitHub orgs picker to per_page=100
- Drop Snyk, Ramp, Lever, Deel access-review providers

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
2026-05-17 17:22:50 +02:00
Aurélien Sibiril
7504099507 Strip email local-part from cassette guard failure → Wire 13 access-review connectors in Helm chart
- Strip email local-part from cassette guard failure
- Drop dead src/components/connectors vite include
- Cover all 13 access-review providers in builder test
- Wire 13 access-review connectors in Helm chart

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
2026-05-17 17:22:48 +02:00
Aurélien Sibiril
8f6ecd9f81 Wire post-OAuth pickers and 2-auto callback handling → Track e2e gap for new access-review connectors
- Wire post-OAuth pickers and 2-auto callback handling
- Add 13 vendor logo components for new connectors
- Wire access-review connectors into bootstrap config
- Track e2e gap for new access-review connectors

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
2026-05-17 17:22:48 +02:00
Sacha Al Himdani
3ff66757ad Remove deprecated snapshot system
The register/document model has fully replaced the snapshot system.
Delete every snapshot-scoped row and strip the application code that
referenced them: SnapshotID/SourceID struct fields, snapshot_id IS NULL
filters, snapshot columns from SELECT/INSERT statements and named args,
and the eight migrate-*-snapshots-to-documents one-shot tools.

The remaining snapshot_id / source_id columns, the snapshots and
controls_snapshots tables, the snapshots_type enum, and the
snapshot-scoped indexes are now unused; they are dropped in a follow-up
schema migration so this change can roll back cleanly without leaving
orphaned data.

Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
2026-05-15 23:26:02 +02:00
Sacha Al Himdani
eecbe4c46c Rename vendors to third parties
Renames the user-facing 'vendor' concept to 'third party' across the
entire codebase. The shared common_third_parties reference table is
unchanged.

Migration. Renames the vendor_category enum, the vendors and
vendor_<entity> tables (contacts, services, compliance_reports,
business_associate_agreements, data_privacy_agreements,
risk_assessments) and their vendor_id columns, the asset_vendors /
data_vendors / processing_activity_vendors junction tables,
generated_documents.vendors_document_id, the webhook_event_type
'vendor:<verb>' values, and the snapshots_type 'VENDORS' value.

Backend. Renames coredata models and SQL queries, probo services,
GraphQL / MCP API surface, console / trust / webhook resolvers and
types, the CLI (prb vendor* -> prb third-party*; pkg/cmd/vendormgmt
-> pkg/cmd/thirdpartymgmt), the document generator, vetting agent
prompts, and the common-third-parties-import command.

Frontend, packages, n8n, e2e. Renames apps/console pages, components,
hooks, routes, dialogs, and tabs; the shared @probo/vendors package
(now @probo/third-parties); the @probo/ui Vendors atoms (now
ThirdParties, VendorLogo -> ThirdPartyLogo); the n8n community node
actions/vendor folder (now actions/thirdParty); and the e2e Go test
suite (console and MCP). Filesystem and URL paths use kebab-case
(third-parties), GraphQL fields and TypeScript identifiers use
camelCase (thirdParty / thirdParties), Go types use PascalCase
(ThirdParty), and human-facing text uses 'third party' with a space.

Co-authored-by: Bryan Frimin <bryan@getprobo.com>
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
2026-05-13 21:21:39 +02:00
Sacha Al Himdani
d88ae3288b Add pg pool tuning options from kit v0.10.0
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
2026-05-12 10:04:13 +02:00
Émile Ré
4aebf65b89 ui: stop importing phosphor icons with bare-name aliases
Add a rule to contrib/claude/ui.md requiring the Icon-suffixed exports
(e.g. EyeIcon) instead of aliased bare names (Eye as IconEye), and fix
TrackerResourceRow to follow it.

Signed-off-by: Émile Ré <emile@getprobo.com>
2026-05-11 14:56:34 +04:00
Aurélien Sibiril
1505bf0b7e Refine agent cancellation guideline
Lead with the observable contract (ctx.Done = graceful suspend,
return is *SuspendedError, framework shields downstream calls) and
mention agent.ErrSuspendForCheckpoint as the recommended cancel
cause for graceful-stop intent. Drop the leak of the WithoutCancel
mechanism — readers need the contract, not the strategy.

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
2026-05-08 12:49:03 +02:00
Aurélien Sibiril
c8432ad2b6 Document agent ctx-cancel suspend contract
Add a Cancellation semantics section explaining ctx.Done() =
graceful suspend, the WithoutCancel shadow inside coreLoop, the
implication for context.WithTimeout deadlines, the absence of an
in-process hard-abort, and the supervisor-side mapping of SIGTERM
shutdown onto cancelRun(ErrSuspendForCheckpoint).

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
2026-05-08 12:49:03 +02:00
Émile Ré
ba918c01ab Upgrade go to 1.26.3
Signed-off-by: Émile Ré <emile@getprobo.com>
2026-05-08 13:46:14 +04:00
Émile Ré
8bd5454ad7 Update release guides
Signed-off-by: Émile Ré <emile@getprobo.com>
2026-05-07 16:49:58 +04:00
Bryan Frimin
ac526fb54d Split release into per-track packages
Each shippable artifact (prb, probod server group, probod-bootstrap,
@probo/n8n-nodes-probo, @probo/cookie-banner) now has its own version
file, its own CHANGELOG.md, its own annotated-tag scheme of the form
<track>/v<version>, and its own GitHub Actions release workflow. The
unified release.yaml is removed; the unified CHANGELOG.md becomes a
short index pointing at each per-track file, with the prior history
preserved in CHANGELOG.archive.md.

Probod's CHANGELOG carries the post-split monorepo releases (0.174.0
through 0.181.0) so the server-group history stays continuous and the
probod docker image keeps its existing version line.

contrib/claude/release.md is split into contrib/claude/release/ with
one entrypoint per track plus a README that drives the agent: detect
which tracks have user-facing commits since their last tag and skip
tracks with no relevant changes, so a release request never tags an
unchanged track. The cookie-banner and n8n-node entrypoints add an
explicit npm run build step after the version bump (build.mjs bakes
package.json's version into __SDK_VERSION__) so compile errors and
package-lock.json updates are caught before tagging.

Signed-off-by: Bryan Frimin <bryan@getprobo.com>
2026-05-06 16:04:36 +02:00
Émile Ré
3b4aa7f971 Move cookie management to display page
Consolidate the separate Cookies tab into the Display page so
categories with full cookie CRUD, reordering, and theme preview
live together. Delete the now-redundant CookieBannerCookiesPage,
its loader, route, and nav tab.

Add row actions (edit, move-to-category, exclude, delete) to the
detection page. The move-to-category dropdown uses an
interaction-triggered preloaded query following the
useQueryLoader pattern. Document this pattern in the
react-components guide.

Signed-off-by: Émile Ré <emile@getprobo.com>
2026-05-05 11:06:55 +04:00
Émile Ré
c0d96e662e Fix harcoded sandbox name
Signed-off-by: Émile Ré <emile@getprobo.com>
2026-04-29 12:48:45 +04:00
Émile Ré
36c3d7649f Remove unused dependency
Signed-off-by: Émile Ré <emile@getprobo.com>
2026-04-29 12:48:45 +04:00
Émile Ré
5e78592024 Add sandbox boot logs cmd
Signed-off-by: Émile Ré <emile@getprobo.com>
2026-04-29 12:48:44 +04:00
Émile Ré
f0ebfe67fa Review fixes
Signed-off-by: Émile Ré <emile@getprobo.com>
2026-04-29 12:48:44 +04:00
Émile Ré
d31bd6ec76 Use make sandbox-* targets in sandbox doc
Signed-off-by: Émile Ré <emile@getprobo.com>
2026-04-29 12:48:44 +04:00
Émile Ré
072e9bc4f9 Wait for healthy stack before start probod
Signed-off-by: Émile Ré <emile@getprobo.com>
2026-04-29 12:48:44 +04:00
Émile Ré
dbc5827357 Fix node_modules install + optimize go install by mounting go mod folder into VM
Signed-off-by: Émile Ré <emile@getprobo.com>
2026-04-29 12:48:44 +04:00
Émile Ré
195fa74109 Unify systemctl services start and remove make build instructions
Signed-off-by: Émile Ré <emile@getprobo.com>
2026-04-29 12:48:43 +04:00
Émile Ré
cd73bc1d2d Create targets for generate and embed, replace SKIP_APPS with WITH_APPS
Signed-off-by: Émile Ré <emile@getprobo.com>
2026-04-29 12:48:43 +04:00
Émile Ré
ae47a960ef Fix sandbox provision script
Signed-off-by: Émile Ré <emile@getprobo.com>
2026-04-29 12:48:43 +04:00
Émile Ré
4147239fbc Add consent records tab to cookie banner config
Exposes the cookie consent record audit trail through a new
"Consent Records" tab on the cookie banner configuration page.
The full stack includes: extended coredata filter (visitor ID,
banner version), GraphQL schema/types/resolvers, and a React
page with SortableTable (size 50) and three compliance filters
(action, visitor ID, banner version).

Signed-off-by: Émile Ré <emile@getprobo.com>
2026-04-28 15:28:17 +04:00
Émile Ré
011057f52a Release v0.173.0
Signed-off-by: Émile Ré <emile@getprobo.com>
2026-04-24 19:24:28 +04:00
Bryan Frimin
ad2ae1b20b Fix sandbox
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
2026-04-24 17:18:25 +02:00
Bryan Frimin
f82d76b246 Add safe URL construction rules to agent guides
Add a URL and query parameter construction section to
contrib/claude/go-style.md requiring net/url (url.JoinPath,
url.Values, url.Parse) instead of fmt.Sprintf or string
concatenation.

Create contrib/claude/ts-style.md with matching TypeScript
rules requiring URL and URLSearchParams instead of template
literals or string concatenation.

Signed-off-by: Bryan Frimin <bryan@getprobo.com>
2026-04-24 16:12:27 +02:00
Bryan Frimin
990dfa8438 Update kit with new pg config
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
2026-04-22 22:30:23 +02:00
Émile Ré
c70ca719dc Branding config will be used for compliance page and other products in the future
Signed-off-by: Émile Ré <emile@getprobo.com>
2026-04-22 15:55:02 +04:00
Émile Ré
4313930ae9 Add global branding config field to control cookie banner default
Introduces a BRANDING boolean config (default true) propagated through
the standard config pipeline. Cookie banners now initialize their
show_branding column from this config instead of hardcoding true.

Signed-off-by: Émile Ré <emile@getprobo.com>
2026-04-22 15:41:11 +04:00
Sacha Al Himdani
b603d04d8d Assets as document: replace snapshot with publish workflow
Remove assets from the snapshot system and replace with a publish-based
document workflow that generates versioned ProseMirror documents.

- Remove snapshot_id/source_id from asset and asset_vendor models
- Delete AssetFilter (no longer needed without snapshot filtering)
- Add PublishAssetList service, GraphQL mutation, MCP tool, CLI command,
  and n8n operation
- Add asset_list_document_id column to generated_documents table
- Generate ProseMirror documents with asset inventory tables
  (name, type, amount, data types stored, owner, vendors)
- Add AssetListDocument resolver on Organization type
- Update frontend to remove snapshot routes/params and add publish dialog
- Add e2e tests for asset publish (immediate, with approvers, reuse, RBAC)
- Add migration script for converting legacy asset snapshots to documents
- Exclude ASSETS from snapshot type lists and e2e snapshot tests
- Move generated_documents SQL to coredata methods on Datum and Asset
- Clear generated document and SOA references on soft delete and archive

Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
2026-04-21 19:36:17 +02:00
Bryan Frimin
fbc10a8591 Move dev env file to repo root
Use the conventional .env / .env.example location at the repo
root instead of cfg/dev.env / cfg/dev.env.example. .env is what
contributors expect, keeps cfg/ a pure generated-config directory,
and shares the same file if we ever add another dev target that
needs the same overrides.

Signed-off-by: Bryan Frimin <bryan@getprobo.com>
2026-04-21 18:51:34 +02:00
Bryan Frimin
ab0e59ad71 Generate cfg/dev.yaml via a Make target
Committing a fully-materialised cfg/dev.yaml hid the dev
configuration surface and blocked the OAuth2 signing-key inlining
change: the new config requires a per-dev private key that must
not be committed. Replace the checked-in file with a dev-config
Make target that shells out to probod-bootstrap with dev-safe
defaults and a stable RSA signing key stashed under
cfg/.dev-oauth2-signing-key.pem on first run.

The recipe sources cfg/dev.env when present so devs can override
any setting without editing the Makefile; cfg/dev.env.example
ships the full list of overridable knobs. cfg/dev.yaml,
cfg/dev.env, and the signing key are all gitignored.

Update README, CONTRIBUTING, and contrib/claude/config.md to
describe the new workflow.

Signed-off-by: Bryan Frimin <bryan@getprobo.com>
2026-04-21 18:20:07 +02:00