OpenAI rejects schemas where optional properties are absent from the
required array. Promote all properties to required and mark formerly
optional ones nullable so the model knows it may pass null.
Also upgrade tool error log level from Warn to Error.
Signed-off-by: Bryan Frimin <bryan@probo.com>
Give the company-profile agent (Agent A) the read-only browser toolset
and build it per-run, so it can read footer, imprint, about, and legal
pages and follow a product domain to the corporate one to resolve the
legal name and headquarters address rather than failing cold.
Make the website the hard precondition: when Agent A cannot resolve a
canonical website, skip the compliance-docs agent and logo step instead
of running them blind, which previously produced inconsistent
cross-domain document URLs. Fall back to the catalog display name for
the legal name when nothing better is found, recorded with a distinct
provenance status so a later real find overwrites it.
Rewrite both enrichment prompts in the project's role/task/instructions
XML style, add a domain-consistency rule for document URLs and a
tool-budget directive, and document the prompt style as a rule and guide.
Fix the find_links_matching browser tool, which double-encoded its
pattern and made JSON.parse fail on every keyword, starving any agent
that used it until it hit the turn cap. Salvage output when an agent
exhausts its turn budget while still exploring with a pending structured
output by forcing one final synthesis turn instead of failing outright.
Signed-off-by: Émile Ré <emile@probo.com>
Replace the bare `getprobo.com` domain and the `www.getprobo.com`
marketing host with `probo.com` / `www.probo.com` across the codebase.
Functional subdomains (app, console, notification, custom, test,
cookie-banner, compliance) keep their existing `getprobo.com` hosts,
and changelog entries are left untouched.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
Replace the single-case switch in the agent-run handler with an if,
and add the blank lines wsl_v5 requires around statements in the
approval checkpoint, worker tests, and policy-set registration.
Signed-off-by: Bryan Frimin <bryan@probo.com>
The worker leaned on a lease plus a heartbeat goroutine and a stale
recovery sweep to reclaim runs from crashed workers. That machinery
raced with long LLM and tool calls and conflated graceful stops with
failures. Remove the lease columns, heartbeat, and stale recovery, and
rely on FOR UPDATE SKIP LOCKED for single-claim plus explicit state
transitions: a graceful suspend returns the run to PENDING and a crash
now leaves it RUNNING for manual recovery.
Treat an approval interruption as a known stop that parks the run in
AWAITING_APPROVAL, and add SubmitApproval to merge human decisions into
the checkpoint and requeue the run to PENDING. The decisions must cover
exactly the pending approvals, since a missing one would resume as an
implicit denial. Expose this through the submitAgentRunApproval
mutation.
Signed-off-by: Bryan Frimin <bryan@probo.com>
The nested-restore test only checked that an unresolvable inner agent
left its tool call in the checkpoint for a later retry. It did not
verify what happens to a sibling inner agent that does resolve and
complete during the same restore.
Add a resolvable done-agent alongside the missing inner-agent and
assert that its progress is persisted: its inner checkpoint is dropped
and its result recorded as a completed call, so a later retry replays
only the still-unresolved branch instead of re-running finished work.
Signed-off-by: Bryan Frimin <bryan@probo.com>
The single- and multi-level suspend/restore tests gated their slow
leaf tool on a manual release channel closed 50ms after cancel().
That sleep was a guess at how long the suspend signal takes to reach
the running sub-agent, so the post-tool turn boundary could observe
the release before cancellation and complete the run instead of
checkpointing, making the assertions timing-dependent.
Expose the per-run suspend signal through SuspendSignalFrom in an
export_test shim and have the leaf tools block on it directly. The
tool now returns only once the graceful-suspend signal has actually
propagated to its agent, so suspension is observed deterministically
without sleeps or release channels.
Signed-off-by: Bryan Frimin <bryan@probo.com>
Propagate graceful-suspend signals through detached run contexts and let
only opt-in suspendable tools re-attach cancellation, so AsTool sub-agents
can checkpoint and restore across nested trees while leaf tools keep
running detached.
Add focused agent and worker tests for single and multi-level suspend/
restore flows, plus heartbeat lease-loss and nested-restore error paths to
harden functional behavior under failure conditions.
Signed-off-by: Bryan Frimin <bryan@probo.com>
Move agent-run orchestration from the legacy supervisor path into the new
agentrun worker/service package and wire it through coredata, server,
policies, and GraphQL resolvers.
This consolidates run lifecycle handling around lease-aware workers and
aligns API surface with the new agent-run domain model so reviewers can
follow one coherent execution path.
Signed-off-by: Bryan Frimin <bryan@probo.com>
Queue vetting on third_parties with PENDING, PROCESSING,
COMPLETED, and FAILED states. Expose enqueue and status through
GraphQL, MCP, CLI, and n8n, validate vet requests, tune the
worker via config, and poll the detail page while vetting runs.
Signed-off-by: Sacha Al Himdani <sacha@getprobo.com>
Apply five style rules: convert iota string enums to typed
string constants, replace errors.As with errors.AsType,
merge three-group imports into two groups, fix multiline
parameter/argument formatting, and replace fmt.Sprintf URL
construction with net/url.
Signed-off-by: Émile Ré <emile@probo.com>
Firecrawl has a single public API at https://api.firecrawl.dev/v2.
The endpoint was configurable but never varied across environments,
so hardcode it as a package-level const and remove the Endpoint
field from FirecrawlConfig and all downstream wiring (bootstrap,
Helm chart, probod, vetting, cookiebanner).
Signed-off-by: Émile Ré <emile@probo.com>
The userAgentTransport and 15s timeout were inconsistently
applied: government_db.go lost its timeout, wayback.go used
a bare http.Client without the pooled transport or user-agent
header. A new httpclient.go centralizes the setup so all
search tools share the same configuration.
Signed-off-by: Émile Ré <emile@probo.com>
SearXNG was a fallback search backend that added complexity without
being used in practice. All search-dependent features (web search,
government DB checks, vetting orchestrator, tracker mapping) now use
Firecrawl exclusively. Removes the SEARCH_ENDPOINT config plumbing
from probodconfig, bootstrap, Helm charts, and all callers.
Signed-off-by: Émile Ré <emile@probo.com>
Update go-style guide and cursor rule to clarify that even a single
argument spanning multiple lines must break after the opening
parenthesis. Fix six violations across the branch.
Signed-off-by: Émile Ré <emile@probo.com>
Firecrawl provides higher quality search results than SearXNG.
When configured (firecrawl-endpoint + firecrawl-api-key), the
tracker-mapping agent and search toolset prefer it over the
SearXNG backend. Also improves the tracker identification prompt
with multi-strategy search queries that leverage domain signals
and adapt to tracker type.
Signed-off-by: Émile Ré <emile@probo.com>
Address review feedback:
- Move ErrSuspendForCheckpoint from checkpoint.go to errors.go
next to the rest of the agent error declarations; drop the
colon in the error string so it matches the existing
`agent run <event>` style used by the supervisor sentinels.
- Replace the inline `outerCtx := ctx; ctx = context.WithoutCancel(ctx)`
pattern with a small `suspendShield` helper in context.go used
by coreLoop, resumeWithOpts, and resumeNested. Reads more
cleanly and stops surfacing the WithoutCancel mechanism at
every call site.
- Trim the doc comments on Run, RunStreamed, Resume, Restore, the
ErrSuspendForCheckpoint declaration, and the saveCtx comment in
restoreNestedSuspended down to the contract bullet.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Two additions:
- agent_test.go's "context cancellation triggers graceful suspend"
now also asserts the input messages land in the suspension
checkpoint — verifies the embedded-Checkpoint path that fires
when no Checkpointer is configured.
- cancel_test.go gets a third subtest that parks the LLM provider
inside ChatCompletion via a release channel, cancels ctx while
the call is in flight, then confirms the LLM call still saw a
non-cancelled ctx and the just-completed turn lands in the
persisted checkpoint. Proves the framework's WithoutCancel
shielding works end-to-end at the unit level.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
The sentinel is part of the agent cancellation contract — the only
caller that needs it (the supervisor) imports pkg/agent already, so
keeping it next to SuspendedError prevents the upward dependency
that would arise if any future agent.Run caller wanted to trigger
graceful suspend. Update pkg/probo/agent_run_handler.go to
reference agent.ErrSuspendForCheckpoint.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Rewrite the WithStopSignal-driven test in restore_test.go to use a
cancellable ctx. Update agent_test.go's "context cancellation"
case from asserting "cannot complete" failure to asserting a
SuspendedError. Add cancel_test.go covering both pre-first-turn
cancel (no LLM call, empty checkpoint persisted) and mid-run
cancel from inside a tool (just-completed turn preserved in the
checkpoint, second LLM call suppressed).
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Collapse the dual-mechanism (ctx.Done() = abort + WithStopSignal =
graceful suspend) into a single signal: ctx.Done() now means
graceful suspend. coreLoop shadows the incoming ctx with
context.WithoutCancel(ctx) on entry and uses the shadow for every
downstream call (LLM, tools, hooks, guardrails, save), keeping the
original ctx only for the at-boundary cancellation check.
restoreNestedSuspended applies the same shadow to its
saveProgress closure so partial nested-restore writes survive a
graceful cancel. Resume and resumeNested mirror the pattern so
their pre-loop tool dispatch is non-cancellable while coreLoop
still detects the cancel at its first turn boundary. The dedicated
stop signal API (WithStopSignal / stopSignalFrom) is removed.
There is no longer an in-process hard-abort path; tool authors
who need a deadline must derive it themselves. Document the new
contract on Run, RunStreamed, Resume, and Restore.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
The triggering entry in the parallel-suspend path stored
se.Checkpoint into innerCheckpoints unconditionally, while
the sibling loop already guarded otherSE.Checkpoint != nil.
A nil entry would later cause restoreNestedSuspended to
dereference innerCP.AgentName and panic. Apply the same
guard so a malformed SuspendedError falls through to the
regular result-collection path instead of poisoning the
checkpoint map.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Run hooks already exposed OnRunRestore for the read side of a
suspend/restore cycle. The write side -- every coreLoop or restore
call that persists a checkpoint to the Checkpointer -- had no
corresponding hook, so callers wanting to record metrics, audit
events, or trigger external state transitions on every snapshot had
no insertion point.
Add OnRunSnapshot to RunHooks and emit it after each successful
Checkpointer.Save: the suspend, awaiting-approval, nested-approval,
post-tool-turn, and restore-progress sites. The hook fires only on
durable saves; save failures still log and skip the hook so observers
never see a checkpoint that did not land.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Result.LastAgent is *Agent, which holds only unexported fields. The
default json.Marshal renders it as an empty object, which is misleading
when persisted alongside the agent run.
Tag LastAgent as json:"-" and add explicit lowercase JSON tags to the
remaining fields so the serialized shape is stable for callers that
read the persisted result column.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Run, RunStreamed, and Resume each shipped both a no-options form and a
mirror *WithOpts form taking variadic RunOption. Variadic parameters
are backward-compatible additions, so the wrappers were dead surface.
Make Run, RunStreamed, and Resume directly variadic and update the two
internal callers.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Each Save and Load now derives tenant_id from the run GID and pins it
in the WHERE clause. A caller that supplies an ID from another tenant
fails closed instead of silently reading or overwriting cross-tenant
checkpoint data. Also rejects oversize checkpoints on load as a
read-side guard against a tampered or migrated row exceeding
MaxCheckpointBytes.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Prefix the unknown-tool error with the "cannot" convention and drop
the duplicate wrap around executeSingleTool: that helper already wraps
its generic error path, so the outer wrap produced messages shaped like
"cannot execute tool X: cannot execute tool X: ...".
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
MaxTurns is the only agent bound compared against a counter that is
serialised in the checkpoint (Turns). When config drifts between save
and restore -- typically because a deploy changed WithMaxTurns or a
different build of the agent is registered by name -- cp.Turns can
exceed agent.maxTurns on the resumed run, which previously surfaced
as a warning log and then a MaxTurnsExceededError on the first
iteration of the resumed coreLoop.
Capture MaxTurns in the new AgentConfig on every save, and on
restore clone the registry-resolved agent with WithMaxTurns applied
from the snapshot. The override flows through the outer Restore path
and through both inner-agent resolution sites in
restoreNestedSuspended and restoreAwaitingApproval, so nested
runs get the same treatment. Other loop bounds
(maxEmptyOutputRetries, maxToolDepth) reset per turn / per tool
depth and stay intentionally live so deploys can tune them without
invalidating in-flight checkpoints. Live references (tools, hooks,
LLM client, approval callbacks, guardrails) are not snapshotted for
the same reason.
With the snapshot in place, the "restored agent run has already
reached max turns" warning at the top of continueFromMessages is
structurally unreachable -- the live agent's bound is now the same
value cp.Turns was bounded by at save time -- and is removed.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
executeParallel ignored SuspendedError when checkpoint was nil,
treating it as a normal tool error. Nested suspension propagation
also dropped the in-memory checkpoint when persistence failed,
making runs non-resumable.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Poll-based supervisor that claims PENDING agent runs with FOR UPDATE
SKIP LOCKED, runs them with lease-based heartbeat, and handles
graceful shutdown. On infrastructure stop the row stays RUNNING so
stale recovery resets it to PENDING on restart; Restore picks up
from the last checkpoint. Heartbeat loss cancels execution without
committing a terminal status.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Restore loads a checkpoint from the store, resolves the agent from
a registry, and re-enters coreLoop. Handles suspended, nested
suspended (concurrent inner restore), and awaiting-approval states.
Partial progress is saved when some inner agents complete while
others remain suspended.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
coreLoop now saves incremental checkpoints after each tool-call turn
and checks a cooperative stop signal at turn boundaries. SuspendedError
is handled in finishRun, executeParallel, and executeSingleTool.
Approval-interrupted checkpoints are persisted for both flat and
nested interruptions.
Introduce RunOption, WithCheckpointStore, RunWithOpts, ResumeWithOpts,
and RunStreamedWithOpts so callers can provide checkpoint storage.
Add StreamEventSuspended and OnRunRestore hook.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Introduce Checkpoint, CheckpointStore, SuspendedError, AgentRegistry,
and CompletedCall types. Add cooperative stop signal via context.
Export CompletedCall (was unexported completedCall) so checkpoints
can reference completed tool results. Add JSON tags to ToolResult
and ApprovalResult for checkpoint serialization.
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Inline comments are more targeted than excluding the entire file
from secret scanning. Remove the .trufflehog.yml exclude file and
the --exclude-paths flag from the workflow.
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
Skip empty fingerprints in SystemPromptLeakGuardrail to prevent blank
values from flagging every message. Replace overly broad "sk-" pattern
in SensitiveDataGuardrail with specific LLM provider prefixes
("sk-proj-" for OpenAI, "sk-ant-" for Anthropic) to avoid false
positives on common words like "risk-based" or "task-management".
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
Introduce a pkg/agent/guardrail package with three guardrails that
can be composed into any agent:
- PromptInjectionGuardrail: LLM-based input classifier that detects
prompt injection attempts before the agent processes them.
- SensitiveDataGuardrail: pattern-based output check for leaked
tokens, keys, connection strings, and raw SQL.
- SystemPromptLeakGuardrail: configurable output check that detects
system prompt content in responses using caller-provided
fingerprints.
The classifier prompt is embedded from a plain text file for easy
review and editing.
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
