Raise tracker agent max turns to avoid errors

The tracker-mapping and common-pattern enrichment agents ran with
4 and 3 max turns. Those budgets are smaller than the tool-call
rounds their prompts authorize (two/one DB searches plus up to
three web searches) plus the forced structured-output synthesis
turn, so the run loop tripped MaxTurnsExceededError before it
could emit JSON.

Raise the budgets to 10 across every layer that drives them: the
bootstrap env-var defaults, the Helm chart values and production
example, and the in-code fallback. Update the bootstrap test
defaults to match.

Revert the prompt instructions that told the model to reserve a
turn for its final answer; the model cannot reliably count its
remaining turns, so the fix belongs in the turn budget rather
than the prompt text.

Signed-off-by: Émile Ré <emile@probo.com>
This commit is contained in:
Émile Ré
2026-06-01 16:40:03 +02:00
parent a3b330ca29
commit b9eac20c9c
7 changed files with 19 additions and 13 deletions

View File

@@ -184,7 +184,7 @@ probo:
# maxConcurrency: 3
# staleAfter: 600
# agentTimeout: 45
# agentMaxTurns: 4
# agentMaxTurns: 10
# Common-pattern enrichment worker tuning (optional; seconds for
# interval/staleAfter/agentTimeout).
@@ -193,7 +193,7 @@ probo:
# maxConcurrency: 2
# staleAfter: 600
# agentTimeout: 45
# agentMaxTurns: 3
# agentMaxTurns: 10
# OpenTelemetry tracing (optional)
tracing:

View File

@@ -285,7 +285,7 @@ probo:
maxConcurrency: 3
staleAfter: 600
agentTimeout: 45
agentMaxTurns: 4
agentMaxTurns: 10
# Common-pattern enrichment background worker tuning (optional).
# interval, staleAfter, and agentTimeout are in seconds.
@@ -294,7 +294,7 @@ probo:
maxConcurrency: 2
staleAfter: 600
agentTimeout: 45
agentMaxTurns: 3
agentMaxTurns: 10
# Custom domains configuration (optional)
customDomains:

View File

@@ -251,14 +251,14 @@ func (b *Builder) Build() (*probodconfig.FullConfig, error) {
MaxConcurrency: b.getEnvIntOrDefault("TRACKER_MAPPING_MAX_CONCURRENCY", 3),
StaleAfter: b.getEnvIntOrDefault("TRACKER_MAPPING_STALE_AFTER", 600),
AgentTimeout: b.getEnvIntOrDefault("TRACKER_MAPPING_AGENT_TIMEOUT", 45),
AgentMaxTurns: b.getEnvIntOrDefault("TRACKER_MAPPING_AGENT_MAX_TURNS", 4),
AgentMaxTurns: b.getEnvIntOrDefault("TRACKER_MAPPING_AGENT_MAX_TURNS", 10),
},
CommonPatternEnrichmentWorker: probodconfig.CommonPatternEnrichmentWorkerConfig{
Interval: b.getEnvIntOrDefault("COMMON_PATTERN_ENRICHMENT_INTERVAL", 10),
MaxConcurrency: b.getEnvIntOrDefault("COMMON_PATTERN_ENRICHMENT_MAX_CONCURRENCY", 2),
StaleAfter: b.getEnvIntOrDefault("COMMON_PATTERN_ENRICHMENT_STALE_AFTER", 600),
AgentTimeout: b.getEnvIntOrDefault("COMMON_PATTERN_ENRICHMENT_AGENT_TIMEOUT", 45),
AgentMaxTurns: b.getEnvIntOrDefault("COMMON_PATTERN_ENRICHMENT_AGENT_MAX_TURNS", 3),
AgentMaxTurns: b.getEnvIntOrDefault("COMMON_PATTERN_ENRICHMENT_AGENT_MAX_TURNS", 10),
},
Branding: b.getEnvBoolOrDefault("BRANDING", true),
},

View File

@@ -225,12 +225,12 @@ func TestBuilder_Build_Defaults(t *testing.T) {
assert.Equal(t, 3, cfg.Probod.TrackerMappingWorker.MaxConcurrency)
assert.Equal(t, 600, cfg.Probod.TrackerMappingWorker.StaleAfter)
assert.Equal(t, 45, cfg.Probod.TrackerMappingWorker.AgentTimeout)
assert.Equal(t, 4, cfg.Probod.TrackerMappingWorker.AgentMaxTurns)
assert.Equal(t, 10, cfg.Probod.TrackerMappingWorker.AgentMaxTurns)
assert.Equal(t, 10, cfg.Probod.CommonPatternEnrichmentWorker.Interval)
assert.Equal(t, 2, cfg.Probod.CommonPatternEnrichmentWorker.MaxConcurrency)
assert.Equal(t, 600, cfg.Probod.CommonPatternEnrichmentWorker.StaleAfter)
assert.Equal(t, 45, cfg.Probod.CommonPatternEnrichmentWorker.AgentTimeout)
assert.Equal(t, 3, cfg.Probod.CommonPatternEnrichmentWorker.AgentMaxTurns)
assert.Equal(t, 10, cfg.Probod.CommonPatternEnrichmentWorker.AgentMaxTurns)
// Custom domains config
assert.Equal(t, 3600, cfg.Probod.CustomDomains.RenewalInterval)

View File

@@ -18,7 +18,7 @@ Return a structured JSON response with:
- With a recognizable prefix or name: "[prefix] cookie purpose" (e.g. "_ga cookie purpose").
- For localStorage keys: "[prefix] localStorage purpose tracking".
- Broaden if needed: "[prefix] cookie what is it used for".
- Stop once you have a confident, well-sourced answer; do not exhaust all queries if the first succeeds. You have a limited number of turns: always keep at least one turn in reserve to return the final JSON, and never keep calling tools until you run out — if you are running low on turns, stop searching and return your best answer (an empty description is valid).
- Stop once you have a confident, well-sourced answer; do not exhaust all queries if the first succeeds.
- Verify that any result discusses a tracker whose name shares a meaningful prefix with the pattern being described. Discard results about a differently-named tracker. A generic token shared with a vendor's terminology (e.g. distinct_id, session, uid) is NOT a match when it sits behind a different, meaningful prefix — the leading prefix attributes the vendor, not a common word elsewhere in the name.
4. Be factual and conservative. Describe only what the evidence supports. Do not speculate about data flows or purposes you cannot substantiate. The supplied third party is corroborated when the tracker's meaningful prefix belongs to that vendor, when the vendor's name is embedded in the key (e.g. "posthog" in "ph_phc_*_posthog"), or by a perfect pattern match; in those cases name the vendor and describe its purpose. Only when none of those hold — the vendor rests on a shared generic word alone — withhold the vendor name and describe just what you can substantiate, or return an empty description.

View File

@@ -21,7 +21,7 @@ Return a structured JSON response with:
- If observed domains are available and the name is opaque, search by domain instead: "[domain] cookies tracking privacy" (e.g. "clarity.ms cookies tracking privacy").
- For localStorage keys, include the type: "[name] localStorage tracking script".
- If the first query returns nothing useful, broaden: "[name] web tracker" or "site:[domain] cookie documentation".
- Stop searching once you get a confident match; do not exhaust all query slots if the first one succeeds. You have a limited number of turns: always keep at least one turn in reserve to return the final JSON, and never keep calling tools until you run out — if you are running low on turns, stop searching and return your best answer (an empty third_party_name with confidence below 0.3 is valid).
- Stop searching once you get a confident match; do not exhaust all query slots if the first one succeeds.
- When evaluating web search results, verify that the tracker name discussed in the result shares a meaningful prefix with the pattern you are identifying. Trackers with different prefixes are distinct — for example, _hjCookieTest (Hotjar's _hj prefix) must not be confused with a pattern named cookietest (no _hj prefix). If the search result discusses a tracker whose prefix does not match, discard it and continue searching or lower your confidence.
- A generic token shared with a vendor's terminology is NOT a match when it appears as a suffix or substring behind a different, meaningful prefix. The leading prefix is what attributes a vendor, not a common word elsewhere in the name. For example, probo_distinct_id carries the custom prefix probo_, so it must NOT be attributed to Mixpanel merely because Mixpanel uses a distinct_id key — the prefix probo_ does not belong to Mixpanel. Likewise, a key ending in _session or _uid is not attributable to a vendor just because that vendor also uses such a word.

View File

@@ -36,9 +36,15 @@ const (
// defaultMappingMaxTurns and defaultEnrichmentMaxTurns bound the
// agent reasoning loop (LLM call + tool round-trips) when the worker
// config does not supply a value.
defaultMappingMaxTurns = 4
defaultEnrichmentMaxTurns = 3
// config does not supply a value. The budget must exceed the number
// of tool rounds each prompt authorizes (the mapping prompt allows
// two DB searches plus up to three web searches, the enrichment
// prompt one DB search plus up to three web searches) and still
// leave a turn for the forced structured-output synthesis turn;
// otherwise the loop trips MaxTurnsExceededError before emitting
// JSON. Ten matches agent.DefaultMaxTurns and gives ample headroom.
defaultMappingMaxTurns = 10
defaultEnrichmentMaxTurns = 10
// defaultAgentMaxTokens caps the output of the mapping and
// enrichment agents when the agent config carries no max-tokens