The source headers, LICENSE files, and license metadata had drifted
apart. Align the entire project to MIT:
- Convert every source-file header to the MIT text across all comment
styles (Go, TS, TSX, JS, MJS, SQL, CSS, GraphQL, shell), including
SPDX-License-Identifier tags
- Set the root and cookie-banner LICENSE files to the MIT text with a
"MIT License" title line
- Switch the package.json license fields, Docker image label, and
cookie-banner README to MIT
- Update docs and the genmodels header generator accordingly
- Normalize copyright lines to a single format
(Copyright (c) <year(s)> Probo Inc <hello@probo.com>.): unify the
hello@getprobo.com and hello@probo.inc emails to hello@probo.com and
the comma-separated years to a hyphenated range
Genuine third-party references are intentionally left untouched: the
Lucide icon attributions (Lucide is ISC) and the trivy dependency
license allowlist.
Signed-off-by: Sacha Al Himdani <sacha@probo.com>
Add a domain-discovery step to the enrichment pipeline so the catalog's
domain set, previously written only by the curated seed, grows
automatically. A focused agent enumerates the registrable domains a
vendor owns and operates - marketing, product and sub-brand, app, API,
and CDN/asset domains - from links seen while browsing and from web
search, anchored on the website resolved earlier in the run.
A deterministic ownership gate reduces the candidates to eTLD+1 and
keeps only those that clear a strict confidence floor and match the
vendor by domain label. Shared tracker-delivery and CDN infrastructure
is dropped unless the vendor itself is that provider, in which case its
own brand-matching domain passes a stricter exact-label check. The
survivors are upserted into common_third_party_domains in the run's
final transaction and recorded in the enrichment payload, feeding the
tracker-mapping domain step and disambiguation.
Signed-off-by: Émile Ré <emile@probo.com>
Give the company-profile agent (Agent A) the read-only browser toolset
and build it per-run, so it can read footer, imprint, about, and legal
pages and follow a product domain to the corporate one to resolve the
legal name and headquarters address rather than failing cold.
Make the website the hard precondition: when Agent A cannot resolve a
canonical website, skip the compliance-docs agent and logo step instead
of running them blind, which previously produced inconsistent
cross-domain document URLs. Fall back to the catalog display name for
the legal name when nothing better is found, recorded with a distinct
provenance status so a later real find overwrites it.
Rewrite both enrichment prompts in the project's role/task/instructions
XML style, add a domain-consistency rule for document URLs and a
tool-budget directive, and document the prompt style as a rule and guide.
Fix the find_links_matching browser tool, which double-encoded its
pattern and made JSON.parse fail on every keyword, starving any agent
that used it until it hit the turn cap. Salvage output when an agent
exhausts its turn budget while still exploring with a pending structured
output by forcing one final synthesis turn instead of failing outright.
Signed-off-by: Émile Ré <emile@probo.com>
Introduce a poll-based worker that fills the global common_third_parties
catalog (URLs, headquarter address, legal name, certifications, logo)
so each tenant no longer starts from sparse, name-only rows. Enrichment
is requested at row creation by ResolveOrCreateCommonThirdParty; curated
seed rows are not enqueued, to avoid a re-seed storm.
The pipeline uses two specialized agents plus a deterministic logo step.
Agent A (company profile) resolves legal name, headquarter address, and
the canonical website over web search; its website and legal name feed
Agent B and the logo step. Agent B (compliance docs) resolves the legal
document URLs, trust/security/status pages, and certifications using the
browser read-only toolset (gated on ChromeDPAddr) plus web search. The
logo step restores pkg/webinspect as a pure deterministic package and
stores the discovered icon in S3, linked via logo_file_id.
Each agent returns per-field value/confidence/source_url. The worker
writes a column only when confidence clears a configurable threshold and
the field is not externally owned (seed or human), and always records
full per-field provenance in a new enrichment JSONB column so re-runs
fill only gaps and human edits are never clobbered. New bookkeeping
columns (enrichment_requested_at, enrichment, enrichment_attempts) back
the claim queue and stale recovery; agents run outside transactions and
results persist in one final transaction.
The worker is opt-in: it no-ops unless its agent provider is configured.
Signed-off-by: Émile Ré <emile@probo.com>
