Fix certmanager provisioning retry and metrics
Address several provisioning defects that either stalled the retry budget or crashed the process: - Classify CAA resolver/transport failures apart from a real CAA policy denial. Both shared the "caa records" wording, so a transient resolver error was persisted as customer misconfiguration and retried forever without consuming the retry budget. A new ErrCAANotPermitted sentinel now marks the genuine misconfiguration; other CAA errors are treated as ordinary transient failures. - Honor an explicit Retry-After: 0 (or a past date) as permission for an immediate retry instead of promoting it to the one-hour default cooldown. acme.RateLimit collapses zero, invalid, and absent headers to a zero duration, so the header is now parsed directly to tell an explicit zero apart from a missing one. - Reuse already-registered Prometheus collectors when a second ACMEService shares a registerer. The fixed-name collectors were MustRegistered, so a duplicate registration panicked the process. - Persist provisioning failures on a context detached from the process tick deadline. A timed-out attempt reached persistFailure with an expired context, so the write-back failed and the retry budget never advanced, leaving the certificate indefinitely retriable. - Use pgx.StrictNamedArgs in the certificate FOR UPDATE loaders to match the coredata SQL contract. Signed-off-by: Bryan Frimin <bryan@probo.com>
This commit is contained in:
@@ -34,6 +34,13 @@ const (
|
||||
ProvisioningErrorACMEFailed = "ACME_FAILED"
|
||||
)
|
||||
|
||||
// ErrCAANotPermitted marks a CAA policy that actively forbids issuance by our
|
||||
// CA. This is a customer-side misconfiguration and is intentionally
|
||||
// non-terminal. It must stay distinct from a CAA resolver/transport failure,
|
||||
// which shares the "caa records" wording but is a transient error that has to
|
||||
// consume the normal retry budget instead of retrying forever.
|
||||
var ErrCAANotPermitted = errors.New("caa records do not permit issuance")
|
||||
|
||||
func classifyProvisioningError(err error) string {
|
||||
if err == nil {
|
||||
return ""
|
||||
@@ -47,12 +54,14 @@ func classifyProvisioningError(err error) string {
|
||||
return ProvisioningErrorACMEInvalidOrder
|
||||
}
|
||||
|
||||
if errors.Is(err, ErrCAANotPermitted) {
|
||||
return ProvisioningErrorDNSCAA
|
||||
}
|
||||
|
||||
msg := strings.ToLower(err.Error())
|
||||
switch {
|
||||
case strings.Contains(msg, "cname"):
|
||||
return ProvisioningErrorDNSCNAME
|
||||
case strings.Contains(msg, "caa record"):
|
||||
return ProvisioningErrorDNSCAA
|
||||
case strings.Contains(msg, "status: invalid"), strings.Contains(msg, "order is in unexpected status \"invalid\""):
|
||||
return ProvisioningErrorACMEInvalidOrder
|
||||
default:
|
||||
|
||||
Reference in New Issue
Block a user