The third-party vetting agent runs a suite of HTTP "security" tools on
the internal worker network against a caller-supplied URL that is only
validated for length and charset, not host. Several tools reached
internal, loopback, and link-local addresses:
- analyze_csp used a bare http.Client with no host validation, no
redirect control, and no rebinding-safe transport, reflecting the
target's CSP header back to the caller.
- check_security_headers, fetch_robots_txt, and fetch_sitemap
validated only the initial host, then followed 3xx redirects with an
ordinary client, yielding full-read SSRF via a redirect to an
internal address.
- check_cors validated the URL but still dialed through an ordinary
transport, leaving it exposed to DNS-rebinding TOCTOU.
Route every one of these clients through the house-standard
httpclient.DefaultPooledClient(WithSSRFProtection()), which rejects
dials to loopback, private, CGNAT, link-local, ULA, IPv4-mapped, and
reserved ranges on the resolved peer IP at connect time (defeating DNS
rebinding on every redirect hop) and refuses cross-origin redirects.
download_pdf moves onto the same client, and the now-unused local
netcheck.NewPinnedTransport is removed. analyze_csp also gains an
up-front ValidatePublicURL check for a clean early error and scheme
enforcement.
Signed-off-by: Sacha Al Himdani <sacha@probo.com>
98f08b7439