Add common third party enricher worker
Introduce a poll-based worker that fills the global common_third_parties catalog (URLs, headquarter address, legal name, certifications, logo) so each tenant no longer starts from sparse, name-only rows. Enrichment is requested at row creation by ResolveOrCreateCommonThirdParty; curated seed rows are not enqueued, to avoid a re-seed storm. The pipeline uses two specialized agents plus a deterministic logo step. Agent A (company profile) resolves legal name, headquarter address, and the canonical website over web search; its website and legal name feed Agent B and the logo step. Agent B (compliance docs) resolves the legal document URLs, trust/security/status pages, and certifications using the browser read-only toolset (gated on ChromeDPAddr) plus web search. The logo step restores pkg/webinspect as a pure deterministic package and stores the discovered icon in S3, linked via logo_file_id. Each agent returns per-field value/confidence/source_url. The worker writes a column only when confidence clears a configurable threshold and the field is not externally owned (seed or human), and always records full per-field provenance in a new enrichment JSONB column so re-runs fill only gaps and human edits are never clobbered. New bookkeeping columns (enrichment_requested_at, enrichment, enrichment_attempts) back the claim queue and stale recovery; agents run outside transactions and results persist in one final transaction. The worker is opt-in: it no-ops unless its agent provider is configured. Signed-off-by: Émile Ré <emile@probo.com>
This commit is contained in:
10
pkg/thirdparty/resolver.go
vendored
10
pkg/thirdparty/resolver.go
vendored
@@ -70,8 +70,14 @@ func ResolveOrCreateCommonThirdParty(
|
||||
Slug: partySlug,
|
||||
Category: category,
|
||||
Certifications: []string{},
|
||||
CreatedAt: now,
|
||||
UpdatedAt: now,
|
||||
// Request enrichment at creation: a freshly resolved catalog row
|
||||
// carries only name/slug/category, so the enrichment worker fills
|
||||
// the rest (URLs, address, certifications, logo). Curated seed
|
||||
// rows are inserted via Upsert without this flag, so a full
|
||||
// re-seed does not trigger an enrichment storm.
|
||||
EnrichmentRequestedAt: &now,
|
||||
CreatedAt: now,
|
||||
UpdatedAt: now,
|
||||
}
|
||||
|
||||
// Insert inside a savepoint so a concurrent transaction that created
|
||||
|
||||
Reference in New Issue
Block a user