rohit/probo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Stop tracker agents from inventing vendors

Browse Source 
The identification agent attributed probo_distinct_id to Mixpanel
purely on the shared distinct_id token, and the enrichment agent
returned no description for the glob ph_phc_*_posthog because it
searched the literal "*" string and found nothing.

Tighten the identification prompt so attribution requires a perfect
pattern match or a meaningful prefix that belongs to the vendor; a
generic token behind a different prefix is not a match. Teach the
enrichment prompt to strip wildcard and variable parts before
searching, and to treat a vendor name embedded in the key as
corroboration so clearly-named trackers still get a description.

Signed-off-by: Émile Ré <emile@probo.com>
This commit is contained in:
Émile Ré
2026-06-01 09:52:43 +02:00
parent 323fe4b5c3
commit 587a4f63cd
2 changed files with 12 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
pkg/cookiebanner/prompts/tracker_enrichment.txt.tmpl
View File

@@ -12,16 +12,18 @@ Return a structured JSON response with:
<instructions>
1. Use the search_third_parties tool to confirm details about the operating company when one is associated with this tracker.
2. Use web_search to find authoritative information about the tracker's purpose. Try up to 3 targeted queries, adapting to the available signals:
- With a recognizable prefix or name: "[name] cookie purpose" (e.g. "_ga cookie purpose").
- For localStorage keys: "[name] localStorage purpose tracking".
- Broaden if needed: "[name] cookie what is it used for".
2. Patterns are often aggregated globs: variable parts (IDs, UUIDs, timestamps, project keys) are collapsed into a "*" wildcard (e.g. "ph_phc_*_posthog", "_gat_UA-*"). Before searching, strip the wildcard and any variable parts and search the fixed prefix/root plus any vendor token embedded in the name — never search the literal "*". For "ph_phc_*_posthog", search "ph_ posthog localStorage purpose", not "ph_phc_*_posthog".
3. Use web_search to find authoritative information about the tracker's purpose. Try up to 3 targeted queries, adapting to the available signals:
- With a recognizable prefix or name: "[prefix] cookie purpose" (e.g. "_ga cookie purpose").
- For localStorage keys: "[prefix] localStorage purpose tracking".
- Broaden if needed: "[prefix] cookie what is it used for".
- Stop once you have a confident, well-sourced answer; do not exhaust all queries if the first succeeds.
- Verify that any result discusses a tracker whose name shares a meaningful prefix with the pattern being described. Discard results about a differently-named tracker.
- Verify that any result discusses a tracker whose name shares a meaningful prefix with the pattern being described. Discard results about a differently-named tracker. A generic token shared with a vendor's terminology (e.g. distinct_id, session, uid) is NOT a match when it sits behind a different, meaningful prefix — the leading prefix attributes the vendor, not a common word elsewhere in the name.
3. Be factual and conservative. Describe only what the evidence supports. Do not speculate about data flows or purposes you cannot substantiate.
4. Be factual and conservative. Describe only what the evidence supports. Do not speculate about data flows or purposes you cannot substantiate. The supplied third party is corroborated when the tracker's meaningful prefix belongs to that vendor, when the vendor's name is embedded in the key (e.g. "posthog" in "ph_phc_*_posthog"), or by a perfect pattern match; in those cases name the vendor and describe its purpose. Only when none of those hold — the vendor rests on a shared generic word alone — withhold the vendor name and describe just what you can substantiate, or return an empty description.
4. Keep the description concise (one to two sentences) and free of marketing language. It should read as a neutral, compliance-grade statement of purpose.
5. Keep the description concise (one to two sentences) and free of marketing language. It should read as a neutral, compliance-grade statement of purpose.
5. If you genuinely cannot substantiate the tracker's purpose from evidence, return an empty description. Do not write a fallback such as "purpose could not be determined", and do not guess a purpose from the name or max-age alone (e.g. do not claim a key is "used for session" just because it has no expiry). An empty description is preferable to an unverified one.
6. If you genuinely cannot substantiate the tracker's purpose from evidence, return an empty description. Do not write a fallback such as "purpose could not be determined", and do not guess a purpose from the name or max-age alone (e.g. do not claim a key is "used for session" just because it has no expiry). An empty description is preferable to an unverified one.
</instructions>

3
pkg/cookiebanner/prompts/tracker_identification.txt.tmpl
View File

@@ -23,6 +23,7 @@ Return a structured JSON response with:
- If the first query returns nothing useful, broaden: "[name] web tracker" or "site:[domain] cookie documentation".
- Stop searching once you get a confident match; do not exhaust all query slots if the first one succeeds.
- When evaluating web search results, verify that the tracker name discussed in the result shares a meaningful prefix with the pattern you are identifying. Trackers with different prefixes are distinct — for example, _hjCookieTest (Hotjar's _hj prefix) must not be confused with a pattern named cookietest (no _hj prefix). If the search result discusses a tracker whose prefix does not match, discard it and continue searching or lower your confidence.
- A generic token shared with a vendor's terminology is NOT a match when it appears as a suffix or substring behind a different, meaningful prefix. The leading prefix is what attributes a vendor, not a common word elsewhere in the name. For example, probo_distinct_id carries the custom prefix probo_, so it must NOT be attributed to Mixpanel merely because Mixpanel uses a distinct_id key — the prefix probo_ does not belong to Mixpanel. Likewise, a key ending in _session or _uid is not attributable to a vendor just because that vendor also uses such a word.
4. Common cookie naming conventions to recognize:
- _ga*, _gid, _gat*: Google Analytics
@@ -47,7 +48,7 @@ Return a structured JSON response with:
7. If you truly cannot identify who set the tracker, set third_party_name to an empty string and third_party_confidence below 0.3.
8. Only attribute a tracker to a company or service when you have concrete evidence: a database match, an unmistakable naming convention (including a vendor name embedded in the key), or a clear web search result. Never guess or invent attributions based on vague similarity or general knowledge. If no evidence supports a match, return an empty third_party_name with third_party_confidence below 0.3.
8. Only attribute a tracker to a company or service when you have concrete evidence: an exact (perfect) match on the pattern in the database, an unmistakable naming convention where the tracker's meaningful prefix belongs to that vendor (including a vendor name embedded in the key), or a clear web search result whose tracker name shares that meaningful prefix. Absent a shared meaningful prefix or a perfect pattern match, do NOT imagine a vendor — never guess or invent attributions based on vague similarity, a shared generic word, or general knowledge. If no evidence supports a match, return an empty third_party_name with third_party_confidence below 0.3.
9. For the category field, use one of: {{.Categories}}.
Most cookies fall under ANALYTICS or MARKETING.