Fix enrichment re-arm and migration backfill gaps

Three crash- and migration-recovery gaps in the unified enrichment
model left rows stuck or misclassified:

- Upsert re-armed a blank, newly-linked tracker pattern without
  clearing its prior enrichment payload. A crash between the worker's
  claim and persist then left the row with a stale payload, so the
  stale-recovery sweep (which only catches rows with a null payload)
  skipped it forever. Clear enrichment on re-arm so the row reads as
  not-yet-completed again, and pin the behavior with a test.

- The migration added last_enrichment_attempt_at to
  common_third_parties without seeding it. Rows with prior attempts
  kept a NULL clock and could never satisfy the stale-reset predicate.
  Backfill from updated_at, the historical claim-time proxy.

- The migration switched the tracker-pattern enriched-state source to
  the enrichment payload without backfilling rows previously marked by
  enriched_at, making already-enriched rows read as unenriched.
  Seed a provenance sentinel for rows that carried the old done-flag.

Signed-off-by: Émile Ré <emile@probo.com>
This commit is contained in:
Émile Ré
2026-06-16 10:21:05 +02:00
parent 1eed89606c
commit 46fc755b6e
3 changed files with 57 additions and 2 deletions

View File

@@ -270,6 +270,23 @@ func TestCommonTrackerPattern_Upsert_RequeuesBlankRowOnThirdPartyLink(t *testing
assert.Equal(t, party.ID, *reloaded.CommonThirdPartyID, "blank row must gain the linked third party")
assert.NotNil(t, reloaded.EnrichmentRequestedAt, "linking a vendor must re-queue the blank row for enrichment")
assert.Equal(t, 0, reloaded.EnrichmentAttempts, "re-queued row must get a fresh retry budget")
assert.Empty(t, reloaded.Enrichment, "re-armed row must drop the prior payload so it reads as not yet completed")
// The prior payload must be cleared so a crash between the worker's
// claim and persist stays recoverable. Simulate the claim (which bumps
// attempts past zero and stamps the idle clock) without completing, then
// confirm the stale sweep re-queues the row — it only catches rows whose
// payload is still null.
require.NoError(t, client.WithTx(ctx, func(ctx context.Context, tx pg.Tx) error {
return reloaded.ClearEnrichmentRequestedAt(ctx, tx)
}))
require.NoError(t, client.WithConn(ctx, func(ctx context.Context, conn pg.Querier) error {
return coredata.ResetStaleEnrichments(ctx, conn, 0, 3)
}))
afterSweep := loadCommonTrackerPattern(t, ctx, client, blank.ID)
assert.NotNil(t, afterSweep.EnrichmentRequestedAt, "stale recovery must re-queue a re-armed row claimed but never completed")
}
// TestCommonTrackerPattern_Upsert_KeepsDescribedRowTerminal pins the