Drop agent-run lease and add approval resume
The worker leaned on a lease plus a heartbeat goroutine and a stale recovery sweep to reclaim runs from crashed workers. That machinery raced with long LLM and tool calls and conflated graceful stops with failures. Remove the lease columns, heartbeat, and stale recovery, and rely on FOR UPDATE SKIP LOCKED for single-claim plus explicit state transitions: a graceful suspend returns the run to PENDING and a crash now leaves it RUNNING for manual recovery. Treat an approval interruption as a known stop that parks the run in AWAITING_APPROVAL, and add SubmitApproval to merge human decisions into the checkpoint and requeue the run to PENDING. The decisions must cover exactly the pending approvals, since a missing one would resume as an implicit denial. Expose this through the submitAgentRunApproval mutation. Signed-off-by: Bryan Frimin <bryan@probo.com>
This commit is contained in:
25
pkg/coredata/migrations/20260608T090000Z.sql
Normal file
25
pkg/coredata/migrations/20260608T090000Z.sql
Normal file
@@ -0,0 +1,25 @@
|
||||
-- Copyright (c) 2026 Probo Inc <hello@getprobo.com>.
|
||||
--
|
||||
-- Permission to use, copy, modify, and/or distribute this software for any
|
||||
-- purpose with or without fee is hereby granted, provided that the above
|
||||
-- copyright notice and this permission notice appear in all copies.
|
||||
--
|
||||
-- THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH
|
||||
-- REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
|
||||
-- AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT,
|
||||
-- INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
|
||||
-- LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR
|
||||
-- OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
|
||||
-- PERFORMANCE OF THIS SOFTWARE.
|
||||
|
||||
-- Drop the agent-run worker lease. Recovery from a crashed worker is now
|
||||
-- manual (inspect logs, move the row out of RUNNING by hand). Known stops
|
||||
-- transition explicitly: graceful shutdown returns the row to PENDING and
|
||||
-- approval pauses park it in AWAITING_APPROVAL, so nothing relies on a
|
||||
-- timeout to requeue. Re-introduce leasing here when the system matures.
|
||||
|
||||
DROP INDEX IF EXISTS idx_agent_runs_running_lease;
|
||||
|
||||
ALTER TABLE agent_runs
|
||||
DROP COLUMN lease_expires_at,
|
||||
DROP COLUMN lease_generation;
|
||||
Reference in New Issue
Block a user