Scorers

Scorers turn outputs into metrics. Use built-in scorers whenever possible so scores stay consistent across evaluations, hooks, and agent workflows.

Core API

Scorer wraps a scoring callable and produces Metric values.
@dn.scorer is a decorator for custom scoring functions.
Built-in scorers live under dreadnode.scorers (security, PII, exfiltration, agentic, etc.).
Composition algebra lets you combine scorers with operators and helpers.

Built-in scorers (Python)

The Python SDK ships with 80+ scorers across categories like security, PII detection, exfiltration, MCP/agentic safety, reasoning, and IDE workflows.

from dreadnode.scorers import contains, detect_pii, system_prompt_leaked

mentions_platform = contains("dreadnode")
pii_risk = detect_pii()
prompt_leak = system_prompt_leaked()

Composition algebra

Combine scorers with operators and helpers:

& / | / ~ for logical composition
+ / - / * for arithmetic composition
>> / // to rename scorers (log all vs log primary)
threshold(), normalize(), invert(), remap_range(), scale(), clip(), weighted_avg()

import dreadnode as dn
from dreadnode.scorers import contains, detect_pii, normalize, weighted_avg

mentions = contains("agent")
quality = normalize(mentions, known_max=1.0)
safety = ~detect_pii()

overall = weighted_avg((quality, 0.6), (safety, 0.4)) >> "overall_score"
combined = (quality & safety) // "quality_and_safety"

Threshold conditions for hooks

Use scorer thresholds in agent hooks and conditions with .above(), .below(), or .as_condition():

from dreadnode.scorers import contains

quality = contains("well-structured")
must_pass = quality.above(0.5)
just_record = quality.as_condition()

Build a custom scorer

import dreadnode as dn

@dn.scorer(name="length_bonus")
def length_bonus(text: str) -> float:
    return 1.0 if len(text) > 120 else 0.0

metric = await length_bonus.score("Short response.")
print(metric.value)