ContextQMD
Back to Openclaw

claw-score

.agents/skills/claw-score/SKILL.md

2026.6.117.4 KB
Original Source

claw-score

Use this skill when working on the OpenClaw maturity scorecard in this repo. This is the openclaw-local version of the maintainer claw-score workflow: it keeps the taxonomy and scorecard concepts, but excludes discrawl and the old committed inventory/ report tree.

Authority

This skill owns the operational workflow for:

  • taxonomy.yaml
  • qa/maturity-scores.yaml
  • docs/concepts/qa-e2e-automation.md
  • qa/scenarios/index.yaml

Keep person-specific, maintainer-private, Discord archive, and discrawl facts out of this repo. If a score needs private evidence, use the redacted qa-evidence.json artifact shape generated by OpenClaw QA workflows.

Source Model

  • taxonomy.yaml is the hand-edited source of truth for surfaces, levels, QA profiles, categories, feature coverage IDs, docs refs, LTS overrides, and completeness-instruction paths.
  • Feature coverageIds are ANDed proof targets, not aliases. A feature may list multiple IDs when each ID proves part of one capability.
  • Coverage IDs use dotted namespace.behavior form, with lowercase alphanumeric/dash segments. Profile, surface, and category IDs may remain dashed or dotted.
  • Keep categories and feature names unique, product-shaped, and broader than raw coverage IDs. Do not promote generic IDs into standalone feature names.
  • Avoid duplicate coverage-ID bundles under different feature names in one category.
  • qa/maturity-scores.yaml is the committed aggregate source for Quality, Completeness, and LTS review state.
  • extensions/qa-lab/src/scorecard-taxonomy.ts exports qaMaturityScoresSchema and readValidatedQaMaturityScoreSources; use those QA Lab utilities to validate score output.
  • Generated public docs are docs/maturity/scorecard.md and docs/maturity/taxonomy.md; both come from pnpm maturity:render. Do not hand-edit generated Markdown to change score results.
  • qa-evidence.json artifacts provide per-run QA scorecard evidence. Release profile artifacts are the source of truth for Coverage. They can enrich generated artifact docs, but they are not committed as inventory.

Commands

Run from the openclaw repo root.

Validate taxonomy YAML structure and the maturity score schema after source edits:

bash
node --import tsx --input-type=module <<'NODE'
import fs from "node:fs";
import YAML from "yaml";
import { readValidatedQaMaturityScoreSources } from "./extensions/qa-lab/src/scorecard-taxonomy.ts";

for (const file of ["taxonomy.yaml", "qa/scenarios/index.yaml"]) {
  YAML.parse(fs.readFileSync(file, "utf8"));
}
readValidatedQaMaturityScoreSources();
NODE

Check docs when touching docs prose:

bash
pnpm check:docs

Run focused QA/profile checks when changing coverage IDs or profile membership:

bash
pnpm openclaw qa coverage --json

Scoring Workflow

When asked to score or refresh a surface:

  1. Read the surface in taxonomy.yaml.
  2. Read the surface completeness rubric under .agents/skills/claw-score/references/completeness/.
  3. Gather public repo evidence from docs, source, tests, and QA scenario metadata.
  4. Prefer existing release profile qa-evidence.json artifacts for executed proof.
  5. Update qa/maturity-scores.yaml only for Quality, Completeness, and LTS review state backed by public or redacted artifact evidence.
  6. Run the schema validation command from this skill.
  7. Run pnpm check:docs if docs prose changed, and focused QA coverage checks if coverage IDs or profile membership changed.

For subjective score changes, make the smallest defensible edit and leave the evidence path in the PR or task summary. Keep manual prose in current docs and keep score data in qa/maturity-scores.yaml.

Default Completeness Process

Completeness is scored against the intended operator-visible workflow for each category, not against test breadth or implementation quality. The completeness reference files under references/completeness/ define the category scope and any surface-specific variation from this default process.

By default, Completeness measures how fully OpenClaw exposes the intended surface capability set to the user, operator, author, or maintainer persona for that surface. Score whether each category delivers the full expected workflow, including setup, normal use, status or inspection, recovery, and important platform, provider, channel, security, or lifecycle variants where they apply.

Treat Surface-Specific Scoring Questions and Surface-Specific Guidance as higher-priority instructions for that surface. The surface instructions may flesh out, narrow, or intentionally conflict with the default ideas here; when they do, follow the surface instructions and make the score rationale reflect that surface-specific instruction. If a reference file does not include surface-specific questions or guidance, apply this default process to the surface's Category Scope.

For each category, ask:

  • Can the intended user or operator complete the category workflow end to end?
  • Are the taxonomy features present as supported capabilities rather than isolated implementation fragments?
  • Are the important lifecycle stages represented: setup, normal operation, status/inspection, recovery, and upgrade or removal where relevant?
  • Are the important environment, provider, platform, channel, or security branches present for this surface?
  • Do the known gaps leave major user-visible capability branches missing?

Default guidance:

  • Favor higher Completeness when the category supports the full operator-visible workflow described by taxonomy and category evidence.
  • Lower Completeness when only the happy path exists, when important variants are undocumented or unimplemented, or when recovery/status paths are missing.
  • Do not lower Completeness because tests are thin; that is Coverage.
  • Do not lower Completeness because implementation quality is fragile; that is Quality.

Default Completeness bands:

  • Clawesome (95-100): complete across expected workflows, variants, and recovery branches, with only minor polish gaps.
  • Stable (80-95): the expected workflow set is broadly present, with only bounded missing branches.
  • Beta (70-80): the main workflow exists, but meaningful branches or recovery paths are still absent.
  • Alpha (50-70): only a partial capability set is present; users can complete some core tasks but not the full expected workflow.
  • Experimental (0-50): the category exposes only fragments of the intended capability.

Score Semantics

  • Coverage: deterministic release validation coverage derived from the release profile qa-evidence.json.scorecard feature fulfillment data.
  • Quality: reliability, maintainability, operator safety, and regression confidence for the category.
  • Completeness: how much of the intended operator-visible workflow exists for the category. Use the default completeness process plus any surface-specific variation before changing this score.
  • LTS: derived from Quality, release-evidence Coverage, and human_lts_override; do not hand-edit generated Markdown to change LTS status.

Bands:

  • Clawesome: 95-100
  • Stable: 80-95
  • Beta: 70-80
  • Alpha: 50-70
  • Experimental: 0-50

Artifacts

Do not add the maintainer repo's docs/kevinslin/maturity-scorecard/inventory/ tree to openclaw. Evidence-enriched scorecard outputs belong in short-lived artifacts, not committed generated docs, unless this repo adds an explicit renderer/check workflow first.