Stop source-name worker looping on permanent failures
The access-review source-name worker never marked a source synced when name resolution errored, so it re-claimed the source on every poll and retried at vendor-latency cadence. Two permanently-failing sources generated millions of error logs (Brex /v2/company 403 and Cloudflare /accounts 400) and hammered vendor APIs (8.6M 403s to Brex in 30 days) -- a ban risk, all for best-effort display metadata. Generalize the Google-403 special case: name resolvers now classify a non-2xx response through nameStatusError, which wraps ErrTerminalNameResolution for permanent client errors (400, 401, 403, 404) and returns a plain, retryable error for everything else (5xx, network). The worker treats a terminal error as done -- it keeps the generic name and marks the source synced -- while transient failures keep retrying as before. Also fix the Cloudflare name resolver's own bug: it requested per_page=1, but Cloudflare's List Accounts endpoint requires per_page in 5..50 and 400s otherwise (the driver already uses 50). That 400 was the sole cause of the Cloudflare retry storm; bump it to 50. Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
This commit is contained in:
@@ -172,6 +172,23 @@ func (h *sourceNameHandler) Process(ctx context.Context, source coredata.AccessR
|
||||
|
||||
instanceName, err := resolver.ResolveInstanceName(resolveCtx)
|
||||
if err != nil {
|
||||
// A permanent failure (auth/bad-request) cannot be fixed by
|
||||
// retrying: keep the generic name and mark the source synced so the
|
||||
// worker stops re-claiming it every poll. Returning the error here
|
||||
// would leave name_synced_at NULL and re-enqueue the source forever
|
||||
// (a single unauthorized source produced millions of error logs).
|
||||
if errors.Is(err, drivers.ErrTerminalNameResolution) {
|
||||
h.logger.WarnCtx(
|
||||
ctx,
|
||||
"permanent name resolution failure, keeping generic name",
|
||||
log.String("source_id", source.ID.String()),
|
||||
log.String("provider", dbConnector.Provider.String()),
|
||||
log.Error(err),
|
||||
)
|
||||
|
||||
return h.markNameSynced(ctx, &source)
|
||||
}
|
||||
|
||||
h.logger.WarnCtx(
|
||||
ctx,
|
||||
"cannot resolve instance name",
|
||||
|
||||
Reference in New Issue
Block a user