Add stale recovery to tracker mapping worker
The tracker-mapping worker clears mapping_requested_at at claim time, so a crash or hard failure between Process phases left the pattern dequeued, unmapped, and with nothing to re-trigger it. Only an incidental sibling remap could rescue it, so a lone pattern could stay stranded forever. Implement the worker.StaleRecoverer interface, mirroring the enrichment worker. ResetStaleMappings re-arms rows that were claimed but never assigned a catalog row (common_tracker_pattern_id IS NULL) once idle past a configurable window; a successful Process always assigns one via the unmatched fallback, so the predicate cleanly detects interrupted runs and self-heals after a single pass. ClearMappingRequestedAt now bumps updated_at so the stale clock starts at claim time and the sweep never recycles an in-flight claim. Plumb a StaleAfter knob (default 600s) through the config struct, builder env var, probod wiring, and Helm templates. Signed-off-by: Émile Ré <emile@probo.com>
This commit is contained in:
@@ -273,6 +273,10 @@ spec:
|
||||
- name: TRACKER_MAPPING_MAX_CONCURRENCY
|
||||
value: {{ .Values.probo.trackerMappingWorker.maxConcurrency | quote }}
|
||||
{{- end }}
|
||||
{{- if .Values.probo.trackerMappingWorker.staleAfter }}
|
||||
- name: TRACKER_MAPPING_STALE_AFTER
|
||||
value: {{ .Values.probo.trackerMappingWorker.staleAfter | quote }}
|
||||
{{- end }}
|
||||
{{- if .Values.probo.trackerMappingWorker.agentTimeout }}
|
||||
- name: TRACKER_MAPPING_AGENT_TIMEOUT
|
||||
value: {{ .Values.probo.trackerMappingWorker.agentTimeout | quote }}
|
||||
|
||||
@@ -176,11 +176,13 @@ probo:
|
||||
# temperature: "0.1"
|
||||
# maxTokens: "4096"
|
||||
|
||||
# Tracker mapping worker tuning (optional; seconds for interval/agentTimeout).
|
||||
# Tracker mapping worker tuning (optional; seconds for
|
||||
# interval/staleAfter/agentTimeout).
|
||||
# Keep concurrency modest to stay under OpenAI/Firecrawl limits and the DB pool.
|
||||
# trackerMappingWorker:
|
||||
# interval: 10
|
||||
# maxConcurrency: 3
|
||||
# staleAfter: 600
|
||||
# agentTimeout: 45
|
||||
# agentMaxTurns: 4
|
||||
|
||||
|
||||
@@ -276,12 +276,14 @@ probo:
|
||||
temperature: ""
|
||||
maxTokens: ""
|
||||
|
||||
# Tracker mapping background worker tuning (optional). interval and
|
||||
# agentTimeout are in seconds. Keep concurrency modest to stay under
|
||||
# OpenAI/Firecrawl rate limits and the database connection pool.
|
||||
# Tracker mapping background worker tuning (optional). interval,
|
||||
# staleAfter, and agentTimeout are in seconds. Keep concurrency modest
|
||||
# to stay under OpenAI/Firecrawl rate limits and the database
|
||||
# connection pool.
|
||||
trackerMappingWorker:
|
||||
interval: 10
|
||||
maxConcurrency: 3
|
||||
staleAfter: 600
|
||||
agentTimeout: 45
|
||||
agentMaxTurns: 4
|
||||
|
||||
|
||||
Reference in New Issue
Block a user