rohit/probo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Unify enrichment tracking and outcome-based status

Browse Source 
Make common_tracker_patterns and common_third_parties share one
enrichment-tracking model and fix the misleading proboctl status.

Both tables now carry the enrichment JSONB provenance payload, an
enrichment_attempts counter, and a last_enrichment_attempt_at clock.
On common_tracker_patterns the enriched_at done-flag is renamed to
last_enrichment_attempt_at and stamped at claim time, so it is truthful
to "attempt" rather than "success". A row is considered to have been
through the workflow when it carries an enrichment payload, not when a
timestamp is set, which lets stale recovery key off the payload being
absent with budget remaining, exactly like common_third_parties.

The claim path reads the attempt counter and timestamp back via
RETURNING so the in-memory receiver matches the database clock instead
of a separate app-side time.Now.

The enricher builds a per-field provenance payload (description and
third-party outcomes plus the mapping attribution) and persists it via
UpdateEnrichment, named to mirror the common-third-party sibling. The
common pattern enrichment worker gains a max-attempts ceiling so a
permanently failing row stops looping.

proboctl now shows "enriched" only when every field the last run
recorded an outcome for resolved a value, otherwise "partial (X/Y)",
replacing the misleading "enriched (no description)" label.

Signed-off-by: Émile Ré <emile@probo.com>
This commit is contained in:
Émile Ré
2026-06-15 16:25:53 +02:00
parent a6ed64caaf
commit ef6cead482
17 changed files with 705 additions and 155 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
pkg/cookiebanner/common_pattern_enricher.go
View File

@@ -16,6 +16,7 @@ package cookiebanner
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
@@ -140,6 +141,8 @@ func (e *CommonPatternEnricher) EnrichPattern(ctx context.Context, cp coredata.C
return fmt.Errorf("cannot research tracker description: %w", err)
}
alreadyLinked := cp.CommonThirdPartyID != nil
return e.pg.WithTx(
ctx,
func(ctx context.Context, tx pg.Tx) error {
@@ -156,7 +159,23 @@ func (e *CommonPatternEnricher) EnrichPattern(ctx context.Context, cp coredata.C
}
}
if err := cp.SetEnriched(ctx, tx, description, thirdPartyID); err != nil {
linked := alreadyLinked || thirdPartyID != nil
meta := buildCommonPatternEnrichmentMetadata(
e.enrichmentCfg.Model,
description,
attribution,
alreadyLinked,
linked,
time.Now(),
)
payload, err := json.Marshal(meta)
if err != nil {
return fmt.Errorf("cannot marshal common tracker pattern enrichment metadata: %w", err)
}
if err := cp.UpdateEnrichment(ctx, tx, description, thirdPartyID, payload); err != nil {
return fmt.Errorf("cannot set common tracker pattern enriched: %w", err)
}
@@ -177,7 +196,8 @@ func (e *CommonPatternEnricher) EnrichPattern(ctx context.Context, cp coredata.C
log.String("common_tracker_pattern_id", cp.ID.String()),
log.String("pattern", cp.Pattern),
log.Bool("described", description != ""),
log.Bool("third_party_linked", thirdPartyID != nil),
log.Bool("third_party_linked", linked),
log.Int("enrichment_attempts", cp.EnrichmentAttempts),
log.Int64("backfilled_tracker_patterns", backfilled),
)