Fix deadlock in concurrent tracker mapping
The tracker-mapping worker runs many Process calls in parallel. In Phase 4 a single transaction locked the worker's own claimed pattern row via UpdateMapping and then locked sibling rows on the same banner via the re-enqueue. Two workers mapping sibling patterns on one banner each held their own row and waited on the other's, forming a lock cycle that Postgres aborted with deadlock detected (40P01). Split the sibling re-enqueue into its own short transaction that runs after the mapping commits, so the claimed-row lock is released before any sibling row is locked. Also take the sibling UPDATE row locks in a deterministic id order through an ORDER BY id ... FOR UPDATE subquery, so overlapping re-enqueues can no longer invert lock order between themselves. The re-enqueue only flags siblings, so deferring it past the commit is safe and lets reprocessed siblings observe committed data. Signed-off-by: Émile Ré <emile@probo.com>
This commit is contained in:
@@ -1286,26 +1286,37 @@ func (tps *TrackerPatterns) RequestMappingForUnmappedSiblings(
|
||||
return 0, nil
|
||||
}
|
||||
|
||||
// The target rows are locked through an ORDER BY id ... FOR UPDATE
|
||||
// subquery so concurrent re-enqueues over overlapping sibling sets
|
||||
// always acquire their row locks in the same ascending id order. Two
|
||||
// workers mapping sibling patterns on the same banner would otherwise
|
||||
// lock the shared rows in opposite orders and deadlock (40P01).
|
||||
q := `
|
||||
UPDATE tracker_patterns
|
||||
SET
|
||||
mapping_requested_at = NOW(),
|
||||
updated_at = NOW()
|
||||
WHERE
|
||||
%[1]s
|
||||
AND cookie_banner_id = @cookie_banner_id
|
||||
AND id != @exclude_pattern_id
|
||||
AND third_party_id IS NULL
|
||||
AND mapping_requested_at IS NULL
|
||||
AND (source IS NULL OR source != @extension_source)
|
||||
AND id IN (
|
||||
SELECT DISTINCT tracker_pattern_id
|
||||
FROM detected_trackers
|
||||
WHERE %[1]s
|
||||
AND cookie_banner_id = @cookie_banner_id
|
||||
AND initiator_domain = ANY(@domains)
|
||||
AND tracker_pattern_id IS NOT NULL
|
||||
)
|
||||
WHERE id IN (
|
||||
SELECT id
|
||||
FROM tracker_patterns
|
||||
WHERE
|
||||
%[1]s
|
||||
AND cookie_banner_id = @cookie_banner_id
|
||||
AND id != @exclude_pattern_id
|
||||
AND third_party_id IS NULL
|
||||
AND mapping_requested_at IS NULL
|
||||
AND (source IS NULL OR source != @extension_source)
|
||||
AND id IN (
|
||||
SELECT DISTINCT tracker_pattern_id
|
||||
FROM detected_trackers
|
||||
WHERE %[1]s
|
||||
AND cookie_banner_id = @cookie_banner_id
|
||||
AND initiator_domain = ANY(@domains)
|
||||
AND tracker_pattern_id IS NOT NULL
|
||||
)
|
||||
ORDER BY id
|
||||
FOR UPDATE
|
||||
)
|
||||
`
|
||||
|
||||
q = fmt.Sprintf(q, scope.SQLFragment())
|
||||
|
||||
Reference in New Issue
Block a user