Stop sibling re-enqueue from flooding mapping logs
The tracker-mapping worker re-arms same-banner siblings after a pattern
resolves a vendor. Its predicate only excluded promoted siblings
(third_party_id IS NULL), but since org-party auto-creation was dropped a
pattern can resolve a common third party yet never gain an org
third_party_id. Those siblings, and terminal first-party ones, stayed
eligible forever, so every cascade step re-enqueued and reprocessed them,
amplifying Process runs to O(N^2) per banner. The deadlock fix in the
last release removed the rollbacks that had accidentally throttled the
cascade, so the latent amplification surfaced as an INFO-log flood.
Tighten the re-enqueue to skip siblings already linked to a catalog row
that carries a common third party or marked FIRST_PARTY, dropping
per-banner reprocessing back to O(N). Also demote the two per-run handler
logs ("mapped tracker pattern", "re-enqueued unmapped sibling tracker
patterns") to Debug so routine processing no longer logs at INFO.
Signed-off-by: Émile Ré <emile@probo.com>
This commit is contained in:
@@ -341,7 +341,7 @@ func (h *trackerMappingHandler) Process(ctx context.Context, tp coredata.Tracker
|
|||||||
return fmt.Errorf("cannot update tracker pattern mapping: %w", err)
|
return fmt.Errorf("cannot update tracker pattern mapping: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
h.logger.InfoCtx(
|
h.logger.DebugCtx(
|
||||||
ctx,
|
ctx,
|
||||||
"mapped tracker pattern",
|
"mapped tracker pattern",
|
||||||
log.String("pattern", tp.Pattern),
|
log.String("pattern", tp.Pattern),
|
||||||
@@ -539,7 +539,7 @@ func (h *trackerMappingHandler) reenqueueUnmappedSiblings(
|
|||||||
}
|
}
|
||||||
|
|
||||||
if count > 0 {
|
if count > 0 {
|
||||||
h.logger.InfoCtx(
|
h.logger.DebugCtx(
|
||||||
ctx,
|
ctx,
|
||||||
"re-enqueued unmapped sibling tracker patterns",
|
"re-enqueued unmapped sibling tracker patterns",
|
||||||
log.String("tracker_pattern_id", tp.ID.String()),
|
log.String("tracker_pattern_id", tp.ID.String()),
|
||||||
|
|||||||
@@ -1269,11 +1269,18 @@ WHERE id = @id
|
|||||||
// siblings that were processed earlier and left unmatched can now be
|
// siblings that were processed earlier and left unmatched can now be
|
||||||
// re-evaluated against it.
|
// re-evaluated against it.
|
||||||
//
|
//
|
||||||
// Only unpromoted (third_party_id IS NULL), not-already-queued
|
// Only siblings still genuinely unresolved are touched: not promoted to
|
||||||
// (mapping_requested_at IS NULL), non-extension siblings are touched, so
|
// an org party (third_party_id IS NULL), not already linked to a catalog
|
||||||
// a fully mapped banner re-enqueues nothing. detected_trackers is used
|
// row that carries a common third party, and not marked FIRST_PARTY
|
||||||
// only as a filtering subquery. Returns the number of siblings
|
// (a terminal verdict). third_party_id IS NULL alone is no longer a
|
||||||
// re-enqueued.
|
// sufficient guard: since org-party auto-creation was dropped a pattern
|
||||||
|
// can resolve a common third party yet stay third_party_id IS NULL, and
|
||||||
|
// re-enqueueing those (or first-party siblings) on every cascade step is
|
||||||
|
// what amplified reprocessing to O(N^2) per banner. The siblings must
|
||||||
|
// also be not-already-queued (mapping_requested_at IS NULL) and
|
||||||
|
// non-extension. A fully mapped banner re-enqueues nothing.
|
||||||
|
// common_tracker_patterns and detected_trackers are used only as
|
||||||
|
// filtering subqueries. Returns the number of siblings re-enqueued.
|
||||||
func (tps *TrackerPatterns) RequestMappingForUnmappedSiblings(
|
func (tps *TrackerPatterns) RequestMappingForUnmappedSiblings(
|
||||||
ctx context.Context,
|
ctx context.Context,
|
||||||
tx pg.Tx,
|
tx pg.Tx,
|
||||||
@@ -1306,6 +1313,15 @@ WHERE id IN (
|
|||||||
AND third_party_id IS NULL
|
AND third_party_id IS NULL
|
||||||
AND mapping_requested_at IS NULL
|
AND mapping_requested_at IS NULL
|
||||||
AND (source IS NULL OR source != @extension_source)
|
AND (source IS NULL OR source != @extension_source)
|
||||||
|
AND NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM common_tracker_patterns ctp
|
||||||
|
WHERE ctp.id = tracker_patterns.common_tracker_pattern_id
|
||||||
|
AND (
|
||||||
|
ctp.common_third_party_id IS NOT NULL
|
||||||
|
OR ctp.attribution = 'FIRST_PARTY'
|
||||||
|
)
|
||||||
|
)
|
||||||
AND id IN (
|
AND id IN (
|
||||||
SELECT DISTINCT tracker_pattern_id
|
SELECT DISTINCT tracker_pattern_id
|
||||||
FROM detected_trackers
|
FROM detected_trackers
|
||||||
|
|||||||
Reference in New Issue
Block a user