Firecrawl is a tool used by agents (tracker mapping, third-party
assessor), so its configuration belongs under AgentsConfig rather than
as a standalone Config field. Adds AgentToolsConfig to hold agent tool
credentials and updates all config propagation consumers.
Signed-off-by: Émile Ré <emile@probo.com>
Firecrawl has a single public API at https://api.firecrawl.dev/v2.
The endpoint was configurable but never varied across environments,
so hardcode it as a package-level const and remove the Endpoint
field from FirecrawlConfig and all downstream wiring (bootstrap,
Helm chart, probod, vetting, cookiebanner).
Signed-off-by: Émile Ré <emile@probo.com>
SearXNG was a fallback search backend that added complexity without
being used in practice. All search-dependent features (web search,
government DB checks, vetting orchestrator, tracker mapping) now use
Firecrawl exclusively. Removes the SEARCH_ENDPOINT config plumbing
from probodconfig, bootstrap, Helm charts, and all callers.
Signed-off-by: Émile Ré <emile@probo.com>
Group firecrawl-endpoint and firecrawl-api-key under a nested firecrawl
config key. Add env var mappings (FIRECRAWL_ENDPOINT, FIRECRAWL_API_KEY,
SEARCH_ENDPOINT, AGENT_TRACKER_MAPPING_*) to the bootstrap builder with
test coverage. Wire the new values through the Helm chart (values,
deployment, secret, production example).
Signed-off-by: Émile Ré <emile@probo.com>
Firecrawl provides higher quality search results than SearXNG.
When configured (firecrawl-endpoint + firecrawl-api-key), the
tracker-mapping agent and search toolset prefer it over the
SearXNG backend. Also improves the tracker identification prompt
with multi-strategy search queries that leverage domain signals
and adapt to tracker type.
Signed-off-by: Émile Ré <emile@probo.com>
When both pattern matching and domain matching fail to identify a
tracker, an opt-in LLM agent can now attempt identification using
internal database searches and optional web search. The agent returns
structured output (third party name, category, description, confidence)
and the worker auto-creates CommonThirdParty records when needed.
The feature is gated behind the `llm.tracker-mapping.provider` config
field; when unset the worker behaves exactly as before.
Signed-off-by: Émile Ré <emile@probo.com>
