Commit Graph

34 Commits

Author SHA1 Message Date
Bryan Frimin
552246b5d3 Add SMTP_HELLO_NAME to let operators set the EHLO/HELO identity
Google Workspace SMTP relay (smtp-relay.gmail.com) rejects generic
EHLO identifiers such as "localhost", which is Go net/smtp's default.
Operators can now set SMTP_HELLO_NAME to a valid hostname so that
c.Hello() is called before StartTLS, satisfying strict relay policies.

Fixes https://github.com/getprobo/probo/issues/1284

Signed-off-by: Bryan Frimin <bryan@probo.com>
2026-06-05 14:28:41 +02:00
Aurélien Sibiril
dbd920dc24 Add Zendesk access-review connector
Zendesk is a multi-tenant OAuth connector keyed by the customer
subdomain. The customer enters it at connect time; it rides the signed
state to the callback, is re-validated, and is stored on the connector
settings to build the API host.

List staff (agents and admins) via GET /api/v2/users.json with cursor
pagination, mapping role, active/suspended, and 2FA status; end-users
are excluded. The subdomain is validated as a single DNS label at every
trust boundary to close the SSRF vector, and the data client keeps the
SSRF-protected transport.

Zendesk OAuth across customer subdomains requires a Zendesk-approved
global OAuth client; the connector goes live once those credentials are
supplied via bootstrap.

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
2026-06-04 20:55:48 +02:00
Aurélien Sibiril
3aafdad7f5 Wire Datadog connector bootstrap config
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
2026-06-03 17:16:26 +02:00
Sacha Al Himdani
6e7c96732f Add async third-party vetting
Queue vetting on third_parties with PENDING, PROCESSING,
COMPLETED, and FAILED states. Expose enqueue and status through
GraphQL, MCP, CLI, and n8n, validate vet requests, tune the
worker via config, and poll the detail page while vetting runs.

Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
2026-06-02 11:39:51 +02:00
Émile Ré
b9eac20c9c Raise tracker agent max turns to avoid errors
The tracker-mapping and common-pattern enrichment agents ran with
4 and 3 max turns. Those budgets are smaller than the tool-call
rounds their prompts authorize (two/one DB searches plus up to
three web searches) plus the forced structured-output synthesis
turn, so the run loop tripped MaxTurnsExceededError before it
could emit JSON.

Raise the budgets to 10 across every layer that drives them: the
bootstrap env-var defaults, the Helm chart values and production
example, and the in-code fallback. Update the bootstrap test
defaults to match.

Revert the prompt instructions that told the model to reserve a
turn for its final answer; the model cannot reliably count its
remaining turns, so the fix belongs in the turn budget rather
than the prompt text.

Signed-off-by: Émile Ré <emile@probo.com>
2026-06-01 16:40:03 +02:00
Émile Ré
952c427d2a Add stale recovery to tracker mapping worker
The tracker-mapping worker clears mapping_requested_at at claim time, so
a crash or hard failure between Process phases left the pattern dequeued,
unmapped, and with nothing to re-trigger it. Only an incidental sibling
remap could rescue it, so a lone pattern could stay stranded forever.

Implement the worker.StaleRecoverer interface, mirroring the enrichment
worker. ResetStaleMappings re-arms rows that were claimed but never
assigned a catalog row (common_tracker_pattern_id IS NULL) once idle past
a configurable window; a successful Process always assigns one via the
unmatched fallback, so the predicate cleanly detects interrupted runs and
self-heals after a single pass. ClearMappingRequestedAt now bumps
updated_at so the stale clock starts at claim time and the sweep never
recycles an in-flight claim.

Plumb a StaleAfter knob (default 600s) through the config struct, builder
env var, probod wiring, and Helm templates.

Signed-off-by: Émile Ré <emile@probo.com>
2026-06-01 13:17:18 +02:00
Émile Ré
7360c6eb27 Raise default agent token budget for reasoning models
The tracker mapping, common-pattern enrichment, and third-party
disambiguation agents default to a small max-tokens budget on the
premise that their final output is tiny structured JSON. On
reasoning models such as the GPT-5 family, reasoning tokens count
against max_tokens, so a small budget is consumed by reasoning and
the JSON is truncated, surfacing as "unexpected end of JSON input".

Raise the defaults to 4096 (1024 -> 4096 for tracker mapping, 512
-> 4096 for disambiguation) to leave headroom for the reasoning
phase. Update the bootstrap builder default, its test, and the
production values example to match.

Signed-off-by: Émile Ré <emile@probo.com>
2026-06-01 13:17:17 +02:00
Émile Ré
55302d18f0 Tune tracker workers and bound agent budgets
The tracker-mapping and common-pattern-enrichment workers ran with the
kit/worker defaults (interval 10s, max-concurrency 5 each) and dropped
the resolved per-agent max-tokens/temperature, so up to ten LLM
pipelines could run unbounded on one OpenAI client. The mapping worker
also held a FOR UPDATE transaction across the LLM and Firecrawl calls
while its DB search tools acquired a second pooled connection, risking
pool exhaustion under concurrency.

Plumb max-tokens, temperature, agent timeout, and per-worker max-turns
through TrackerAgentsConfig and DisambiguationConfig into all three
agent builders, replacing the hard-coded constants with config-fed
fields and package fallbacks. Expose worker interval, concurrency,
stale-after, agent timeout, and max-turns as config (env, Helm values,
deployment template) mirroring the evidence-describer pattern, and
apply them at registration.

Refactor Process into deterministic-read, agent (no transaction), and
persist phases so neither the mapping agent nor disambiguation runs
inside an open transaction, removing the row locks held across network
latency and the nested-connection pressure.

Signed-off-by: Émile Ré <emile@probo.com>
2026-06-01 13:17:16 +02:00
Émile Ré
9156d6a16a Add wsl linter and fix
Signed-off-by: Émile Ré <emile@probo.com>
2026-05-20 09:27:28 +04:00
Émile Ré
2040931405 Move Firecrawl API key from top-level config into Agents.Tools
Firecrawl is a tool used by agents (tracker mapping, third-party
assessor), so its configuration belongs under AgentsConfig rather than
as a standalone Config field. Adds AgentToolsConfig to hold agent tool
credentials and updates all config propagation consumers.

Signed-off-by: Émile Ré <emile@probo.com>
2026-05-19 12:32:41 +04:00
Émile Ré
76d42ef2db Hardcode Firecrawl API endpoint, drop FIRECRAWL_ENDPOINT config
Firecrawl has a single public API at https://api.firecrawl.dev/v2.
The endpoint was configurable but never varied across environments,
so hardcode it as a package-level const and remove the Endpoint
field from FirecrawlConfig and all downstream wiring (bootstrap,
Helm chart, probod, vetting, cookiebanner).

Signed-off-by: Émile Ré <emile@probo.com>
2026-05-19 10:28:05 +04:00
Émile Ré
a67ac462d7 Remove SearXNG search backend, use Firecrawl exclusively
SearXNG was a fallback search backend that added complexity without
being used in practice. All search-dependent features (web search,
government DB checks, vetting orchestrator, tracker mapping) now use
Firecrawl exclusively. Removes the SEARCH_ENDPOINT config plumbing
from probodconfig, bootstrap, Helm charts, and all callers.

Signed-off-by: Émile Ré <emile@probo.com>
2026-05-19 10:01:01 +04:00
Émile Ré
e87b426801 Propagate firecrawl and tracker-mapping config to bootstrap and Helm
Group firecrawl-endpoint and firecrawl-api-key under a nested firecrawl
config key. Add env var mappings (FIRECRAWL_ENDPOINT, FIRECRAWL_API_KEY,
SEARCH_ENDPOINT, AGENT_TRACKER_MAPPING_*) to the bootstrap builder with
test coverage. Wire the new values through the Helm chart (values,
deployment, secret, production example).

Signed-off-by: Émile Ré <emile@probo.com>
2026-05-18 19:05:14 +04:00
Aurélien Sibiril
c2db47e698 Require CLIENT_SECRET for new access-review connectors → Drop Snyk, Ramp, Lever, Deel access-review providers
- Require CLIENT_SECRET for new access-review connectors
- Use Heroku account UUID as ExternalID
- Bump GitHub orgs picker to per_page=100
- Drop Snyk, Ramp, Lever, Deel access-review providers

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
2026-05-17 17:22:50 +02:00
Aurélien Sibiril
7504099507 Strip email local-part from cassette guard failure → Wire 13 access-review connectors in Helm chart
- Strip email local-part from cassette guard failure
- Drop dead src/components/connectors vite include
- Cover all 13 access-review providers in builder test
- Wire 13 access-review connectors in Helm chart

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
2026-05-17 17:22:48 +02:00
Aurélien Sibiril
8f6ecd9f81 Wire post-OAuth pickers and 2-auto callback handling → Track e2e gap for new access-review connectors
- Wire post-OAuth pickers and 2-auto callback handling
- Add 13 vendor logo components for new connectors
- Wire access-review connectors into bootstrap config
- Track e2e gap for new access-review connectors

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
2026-05-17 17:22:48 +02:00
Sacha Al Himdani
d88ae3288b Add pg pool tuning options from kit v0.10.0
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
2026-05-12 10:04:13 +02:00
Émile Ré
3acc3191ea Fix builder test
Signed-off-by: Émile Ré <emile@getprobo.com>
2026-05-08 14:35:52 +04:00
Bryan Frimin
5e55c888c4 Add Microsoft 365 SCIM bridge and access review driver
Microsoft 365's native SCIM endpoint is unreliable, so mirror the
Google Workspace bridge over Microsoft Graph: a new MICROSOFT_365
OAuth2 connector, a SCIM bridge provider listing /v1.0/users with
$select pagination, and an access review driver that derives admin
status from /directoryRoles members. Refactor the bridge runner to
share OAuth2 plumbing across providers and surface the new bridge
type, scopes, UI card, and bootstrap env wiring.

Signed-off-by: Bryan Frimin <bryan@getprobo.com>
2026-05-06 15:41:30 +02:00
Émile Ré
f0ebfe67fa Review fixes
Signed-off-by: Émile Ré <emile@getprobo.com>
2026-04-29 12:48:44 +04:00
Émile Ré
c043a7f1f5 Extract config structs into pkg/probodconfig
probod-bootstrap only needs the config struct definitions for
YAML marshaling but transitively pulled in ~40 heavy runtime
dependencies via pkg/probod. Move all config types and their
methods to a new pkg/probodconfig package and re-export them
from pkg/probod via type aliases for backward compatibility.

Signed-off-by: Émile Ré <emile@getprobo.com>
2026-04-29 12:48:43 +04:00
Bryan Frimin
5bd31bd68f Add Google Workspace connector to bootstrap
The SCIM bridge for Google Workspace already exists but the
bootstrap builder did not register it, preventing deployment
via environment variables.

Signed-off-by: Bryan Frimin <bryan@getprobo.com>
2026-04-24 16:06:35 +02:00
Aurélien Sibiril
509d0c88b1 Add vendor assessment agent
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
2026-04-24 08:43:14 +02:00
Bryan Frimin
990dfa8438 Update kit with new pg config
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
2026-04-22 22:30:23 +02:00
Émile Ré
4313930ae9 Add global branding config field to control cookie banner default
Introduces a BRANDING boolean config (default true) propagated through
the standard config pipeline. Cookie banners now initialize their
show_branding column from this config instead of hardcoding true.

Signed-off-by: Émile Ré <emile@getprobo.com>
2026-04-22 15:41:11 +04:00
Bryan Frimin
c4e81ed092 Inline OAuth2 signing key in config
The OAuth2/OIDC server accepted its signing key via a file path
(key-file), while every other PEM key in the probod config (SAML
private key, ACME account key) is embedded inline. Switch the
field to a private-key string so the convention is uniform.

The signing key is operator-supplied material that must outlive
any process restart, so the bootstrap builder now treats
OAUTH2_SERVER_SIGNING_KEY as required and refuses to start
without one; silently minting a fresh key per boot would break
token validation across rollouts. The OAUTH2_SERVER_* env vars
otherwise flow through builder.Build like the existing SAML
block so the new OAuth2Server section is populated end-to-end.

Rework the e2e harness to render its config via bootstrap at
test setup, which removes the static
e2e/console/testdata/config.yaml and the previously generated
test-only PEM file. A per-run RSA key is minted via
bootstrap.GenerateOAuth2SigningKey (kept public for test
tooling) and injected through the builder env map. CI now
passes ACME_ROOT_CA inline instead of mutating a YAML on disk.

Signed-off-by: Bryan Frimin <bryan@getprobo.com>
2026-04-21 17:56:00 +02:00
Aurélien Sibiril
9fc0cd577b Update bootstrap tests for simplified connector config
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
2026-04-08 11:05:12 +02:00
Bryan Frimin
050154ab6a Refactor LLM config into top-level settings
Replace the monolithic agents config with a cleaner structure:
- llm: holds provider credentials and default model settings
- probo-agent: LLM overrides for the probo agent
- evidence-describer: worker config (interval, stale-after,
  max-concurrency) alongside LLM overrides

This makes worker tuning configurable via YAML and env vars
instead of being hardcoded in Go, and separates provider
credentials from per-consumer model settings.

Signed-off-by: Bryan Frimin <bryan@getprobo.com>
2026-04-03 14:06:30 +02:00
Bryan Frimin
619ec7b882 Add AI-powered evidence description generation
Introduce a background worker that automatically generates
compliance-focused descriptions for uploaded evidence files
using configurable LLM providers. Descriptions are surfaced
across all interfaces: GraphQL API, MCP API, CLI, and the
console UI.

Key changes:
- Multi-provider LLM config with per-agent settings (pointer
  types for Temperature/MaxTokens to preserve zero values)
- Evidence description worker with bounded concurrency
- EvidenceDescriptionStatus typed enum with PostgreSQL enum type
- New `prb evidence` CLI commands (list, view, delete)
- Evidence description displayed in console table and preview
- Migration only marks evidences without files as completed

Signed-off-by: Bryan Frimin <bryan@getprobo.com>
2026-03-26 14:32:41 +01:00
Sacha Al Himdani
8c02c53315 Update copyright headers across all Go files
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
2026-03-25 17:38:32 +01:00
Bryan Frimin
7c77c217c7 Fix Anthropic requests failing due to missing MaxTokens in agent configuration
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
2026-03-13 18:24:02 +01:00
Bryan Frimin
ef76a8d2e1 Remove deadcode
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
2026-03-13 17:18:02 +01:00
Sacha Al Himdani
d5c62a9383 Run go fmt/fix
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
2026-03-13 14:48:13 +01:00
Bryan Frimin
63bf983e59 Refactor entrypoint
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
2026-03-06 12:11:12 +01:00