rohit/probo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add stale recovery to tracker mapping worker

Browse Source 
The tracker-mapping worker clears mapping_requested_at at claim time, so
a crash or hard failure between Process phases left the pattern dequeued,
unmapped, and with nothing to re-trigger it. Only an incidental sibling
remap could rescue it, so a lone pattern could stay stranded forever.

Implement the worker.StaleRecoverer interface, mirroring the enrichment
worker. ResetStaleMappings re-arms rows that were claimed but never
assigned a catalog row (common_tracker_pattern_id IS NULL) once idle past
a configurable window; a successful Process always assigns one via the
unmatched fallback, so the predicate cleanly detects interrupted runs and
self-heals after a single pass. ClearMappingRequestedAt now bumps
updated_at so the stale clock starts at claim time and the sweep never
recycles an in-flight claim.

Plumb a StaleAfter knob (default 600s) through the config struct, builder
env var, probod wiring, and Helm templates.

Signed-off-by: Émile Ré <emile@probo.com>
This commit is contained in:
Émile Ré
2026-06-01 11:43:53 +02:00
parent 587a4f63cd
commit 952c427d2a
10 changed files with 167 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
pkg/bootstrap/builder.go
View File

@@ -249,6 +249,7 @@ func (b *Builder) Build() (*probodconfig.FullConfig, error) {
TrackerMappingWorker: probodconfig.TrackerMappingWorkerConfig{
Interval: b.getEnvIntOrDefault("TRACKER_MAPPING_INTERVAL", 10),
MaxConcurrency: b.getEnvIntOrDefault("TRACKER_MAPPING_MAX_CONCURRENCY", 3),
StaleAfter: b.getEnvIntOrDefault("TRACKER_MAPPING_STALE_AFTER", 600),
AgentTimeout: b.getEnvIntOrDefault("TRACKER_MAPPING_AGENT_TIMEOUT", 45),
AgentMaxTurns: b.getEnvIntOrDefault("TRACKER_MAPPING_AGENT_MAX_TURNS", 4),
},

3
pkg/bootstrap/builder_test.go
View File

@@ -223,6 +223,7 @@ func TestBuilder_Build_Defaults(t *testing.T) {
// Tracker worker tuning — defaults
assert.Equal(t, 10, cfg.Probod.TrackerMappingWorker.Interval)
assert.Equal(t, 3, cfg.Probod.TrackerMappingWorker.MaxConcurrency)
assert.Equal(t, 600, cfg.Probod.TrackerMappingWorker.StaleAfter)
assert.Equal(t, 45, cfg.Probod.TrackerMappingWorker.AgentTimeout)
assert.Equal(t, 4, cfg.Probod.TrackerMappingWorker.AgentMaxTurns)
assert.Equal(t, 10, cfg.Probod.CommonPatternEnrichmentWorker.Interval)
@@ -327,6 +328,7 @@ func TestBuilder_Build_CustomValues(t *testing.T) {
// Tracker worker tuning override
env["TRACKER_MAPPING_INTERVAL"] = "20"
env["TRACKER_MAPPING_MAX_CONCURRENCY"] = "5"
env["TRACKER_MAPPING_STALE_AFTER"] = "1200"
env["TRACKER_MAPPING_AGENT_TIMEOUT"] = "30"
env["TRACKER_MAPPING_AGENT_MAX_TURNS"] = "6"
env["COMMON_PATTERN_ENRICHMENT_INTERVAL"] = "15"
@@ -428,6 +430,7 @@ func TestBuilder_Build_CustomValues(t *testing.T) {
// Tracker worker tuning — overrides
assert.Equal(t, 20, cfg.Probod.TrackerMappingWorker.Interval)
assert.Equal(t, 5, cfg.Probod.TrackerMappingWorker.MaxConcurrency)
assert.Equal(t, 1200, cfg.Probod.TrackerMappingWorker.StaleAfter)
assert.Equal(t, 30, cfg.Probod.TrackerMappingWorker.AgentTimeout)
assert.Equal(t, 6, cfg.Probod.TrackerMappingWorker.AgentMaxTurns)
assert.Equal(t, 15, cfg.Probod.CommonPatternEnrichmentWorker.Interval)