baldur.services — Control API & Metrics

The control-API request/response surface for runtime operations, and the metrics helpers. The metrics symbols resolve lazily and require the [prometheus] extra at runtime.

Control API

ControlAPIService

ControlAPIService()

Baldur Control API Service.

Provides a unified, auditable, reversible, and governed control surface to manage reliability behaviors across testing, chaos experimentation, and real production operations.

Usage

service = ControlAPIService()

Execute control action

response = service.execute(ControlRequest( service_name="payment", action="allow", reason="PG recovered", environment="ops" ))

Get current status

status = service.get_status(environment="ops")

Get audit logs

logs = service.get_audit_logs(service_name="payment")

Initialize the Control API Service.

execute

execute(request: ControlRequest) -> ControlResponse

Execute a control API action.

Parameters:

Name	Type	Description	Default
request	ControlRequest	Control request	required

Returns:

Type	Description
ControlResponse	ControlResponse with outcome

get_status

get_status(environment: str = 'ops') -> dict

Get the current status of all services.

Parameters:

Name	Type	Description	Default
environment	str	Current environment context	'ops'

Returns:

Type	Description
dict	Status dictionary with all service states

get_service_status

get_service_status(service_name: str) -> dict

Get the status of a specific service.

Parameters:

Name	Type	Description	Default
service_name	str	Service to check	required

Returns:

Type	Description
dict	Service state dictionary

is_failure_injection_active

is_failure_injection_active(service_name: str) -> bool

Check if failure injection is active for a service.

Parameters:

Name	Type	Description	Default
service_name	str	Service to check	required

Returns:

Type	Description
bool	True if failures should be injected

get_failure_injection_config

get_failure_injection_config(
    service_name: str,
) -> dict | None

Get failure injection configuration for a service.

Parameters:

Name	Type	Description	Default
service_name	str	Service to check	required

Returns:

Type	Description
dict | None	Configuration dict or None

get_metrics

get_metrics() -> dict

Collect comprehensive baldur metrics for trend analysis.

Returns operational metrics for dashboards, AI agents, and monitoring. Unlike status (point-in-time snapshot), metrics provide trend data.

Consumers: - Admin UI: Dashboard visualization - AI Agent: Automated decision making - Prometheus/Grafana: Metrics scraping - External Monitoring: Alerting integration

What the failure-rate and circuit-state fields mean:

All three per-service fields describe this worker process: the serving worker's own call-outcome window and its own breaker view. Responses therefore vary across workers on a multi-worker deployment; fleet-level aggregation is the Prometheus surface's job.

The denominator is calls this worker's circuit breakers admitted and counted over the last five minutes — not calls that executed. An inner stage that exhausts its own budget (a timeout whose task never started, for instance) is a failed call from the caller's perspective, and the window counts it exactly as the breaker does. Conversely, an admitted call whose exception the breaker was configured to ignore is in neither the numerator nor the denominator. Under the default composition the breaker is outermost, so one admission is one protected call; under a retry-over-breaker composition the breaker sits inside the retry loop and one admission is one attempt.

Absence is reported as null, never as zero: unprotected traffic, a disabled breaker, an observe-only mode and retry-only compositions are all invisible to this producer, and a service with no observed admissions renders failure_rate_5m: null. Test-mode traffic is not segregated, matching the breaker's own trip evidence. Two service names differing only in characters a metric label cannot carry share one row, whose rate matches neither; the producer warns when that first happens. Beyond the process's domain cap a newly-seen service is absent from the list rather than folded into another row, and appears once a read frees a slot.

Returns:

Type	Description
dict	Dictionary with comprehensive metrics data

ControlRequest dataclass

ControlRequest(
    service_name: str,
    action: str,
    reason: str,
    environment: str,
    ttl_minutes: int | None = None,
    request_id: str = (lambda: str(uuid.uuid4()))(),
    metadata: dict = dict(),
    actor: str = "system",
    actor_role: str = "automation",
)

Internal representation of a control API request.

ControlResponse dataclass

ControlResponse(
    status: str,
    action_applied: str,
    system_state: str = "",
    effective_until: str | None = None,
    reason_classification: str = "",
    evidence: dict = dict(),
    correlation_id: str = (lambda: str(uuid.uuid4()))(),
    error_code: str = "",
    error_message: str = "",
    risk_level: str = "",
)

Bases: SerializableMixin

Internal representation of a control API response.

Metrics

record_sla_breach

record_sla_breach(domain: str) -> None

Record an SLA breach event.

Parameters:

Name	Type	Description	Default
domain	str	Business domain where breach occurred	required

collect_all_metrics

collect_all_metrics() -> dict

Collect all baldur metrics.

Driven per interval by the per-process DomainGaugeUpdater, and available to operators who prefer to run collection as a Celery task.

One get_statistics() snapshot feeds the three DLQ-family updaters, so a tick pays the repository read — the O(pending) breakdown scan on Redis — once rather than three times, and the pending updater's cross-key consistency check compares two numbers from the same read.

Emits the metric_collection heartbeat only when the DLQ status family was actually written. That family carries the pending total the bundled backlog alerts page on, so the dead-man's switch advances exactly when the paged gauge is fresh: a Redis incident that costs only the per-domain breakdown leaves the paged number current and must not also raise a staleness page, while a stalled status write must not leave the switch green over a frozen paged series.

Returns:

Type	Description
dict	Dictionary with all current metric values