Fix enrichment re-arm and migration backfill gaps
Three crash- and migration-recovery gaps in the unified enrichment model left rows stuck or misclassified: - Upsert re-armed a blank, newly-linked tracker pattern without clearing its prior enrichment payload. A crash between the worker's claim and persist then left the row with a stale payload, so the stale-recovery sweep (which only catches rows with a null payload) skipped it forever. Clear enrichment on re-arm so the row reads as not-yet-completed again, and pin the behavior with a test. - The migration added last_enrichment_attempt_at to common_third_parties without seeding it. Rows with prior attempts kept a NULL clock and could never satisfy the stale-reset predicate. Backfill from updated_at, the historical claim-time proxy. - The migration switched the tracker-pattern enriched-state source to the enrichment payload without backfilling rows previously marked by enriched_at, making already-enriched rows read as unenriched. Seed a provenance sentinel for rows that carried the old done-flag. Signed-off-by: Émile Ré <emile@probo.com>
This commit is contained in:
@@ -270,6 +270,23 @@ func TestCommonTrackerPattern_Upsert_RequeuesBlankRowOnThirdPartyLink(t *testing
|
||||
assert.Equal(t, party.ID, *reloaded.CommonThirdPartyID, "blank row must gain the linked third party")
|
||||
assert.NotNil(t, reloaded.EnrichmentRequestedAt, "linking a vendor must re-queue the blank row for enrichment")
|
||||
assert.Equal(t, 0, reloaded.EnrichmentAttempts, "re-queued row must get a fresh retry budget")
|
||||
assert.Empty(t, reloaded.Enrichment, "re-armed row must drop the prior payload so it reads as not yet completed")
|
||||
|
||||
// The prior payload must be cleared so a crash between the worker's
|
||||
// claim and persist stays recoverable. Simulate the claim (which bumps
|
||||
// attempts past zero and stamps the idle clock) without completing, then
|
||||
// confirm the stale sweep re-queues the row — it only catches rows whose
|
||||
// payload is still null.
|
||||
require.NoError(t, client.WithTx(ctx, func(ctx context.Context, tx pg.Tx) error {
|
||||
return reloaded.ClearEnrichmentRequestedAt(ctx, tx)
|
||||
}))
|
||||
|
||||
require.NoError(t, client.WithConn(ctx, func(ctx context.Context, conn pg.Querier) error {
|
||||
return coredata.ResetStaleEnrichments(ctx, conn, 0, 3)
|
||||
}))
|
||||
|
||||
afterSweep := loadCommonTrackerPattern(t, ctx, client, blank.ID)
|
||||
assert.NotNil(t, afterSweep.EnrichmentRequestedAt, "stale recovery must re-queue a re-armed row claimed but never completed")
|
||||
}
|
||||
|
||||
// TestCommonTrackerPattern_Upsert_KeepsDescribedRowTerminal pins the
|
||||
|
||||
Reference in New Issue
Block a user