Clinical Decision-Support Research and Evaluation

Hard Safety Boundary

This skill produces research, evaluation, documentation, and governance artifacts only.

Never use it to:

• diagnose or classify a person;
• recommend, select, sequence, start, stop, or modify treatment;
• calculate or communicate a patient-specific dose;
• triage, prioritize, alarm, alert, or determine urgency;
• make or automate a patient-specific clinical decision;
• support bedside, point-of-care, or live clinical operation;
• replace professional judgment or a validated, authorized clinical system;
• claim FDA authorization, regulatory conformity, HIPAA compliance, or legal compliance.

If a request could affect care for a person, stop the workflow and route the matter to a licensed healthcare professional using locally validated and appropriately authorized systems. Do not redirect to another skill for patient-specific care.

In Scope

• Intended-use and limitation statements for research artifacts
• Aggregate cohort table shells with disclosure controls
• Statistical analysis plans and survival-analysis plan review
• Aggregate model or biomarker performance evaluation
• Transparent GRADE evidence-profile checklists
• Evidence-source and decision-logic traceability
• De-identification process checklists
• Fairness, subgroup, calibration, uncertainty, external-validation, monitoring, change-control, audit, and human-factors documentation

Outputs remain drafts until qualified humans approve them. Reporting guidance improves transparency; it does not establish study quality, clinical utility, safety, effectiveness, authorization, or compliance.

Data Gate

Before any script:

1. Confirm input is synthetic or aggregate.
2. Reject patient rows, records, narratives, identifiers, free text, dates tied to people, images, waveforms, or genomic sequences.
3. Keep source files local. Do not fetch URLs, call APIs, read environment variables, or send data to a model.
4. Set disclosure thresholds before producing tables.
5. Record provenance, data cut date, population, exclusions, missingness, and transformations.

The scripts cap file size, groups, rows, and text length. They reject URL-like paths and common row-level keys. These controls reduce accidental misuse; they are not a privacy determination.

Required Artifact Header

Every artifact must visibly include:

• artifact_type, title, version, status, owner, date, and change summary;
• intended purpose, intended users, aggregate population scope, and decision role;
• all prohibited uses from the hard boundary;
• data level and confirmation that no PHI or raw rows were supplied;
• limitations, uncertainty, and foreseeable failure modes;
• external-validation and subgroup applicability status;
• human-review roles, completion status, and approval boundary;
• source citations with versions or dates;
• monitoring, change-control, retirement, and audit expectations;
• the statement: Not for patient care or live clinical use.

Start from assets/artifact_intended_use_template.json.

Workflow

1. Frame the Research Question

• Define the estimand or evaluation target before viewing results.
• Distinguish descriptive, prognostic, predictive, diagnostic-accuracy, and causal questions.
• Pre-specify outcomes, time origin, horizon, subgroups, cut points, missing-data handling, multiplicity, and sensitivity analyses.
• Separate exploratory findings from confirmatory analyses.

2. Select the Artifact

3. Run Locally

All helpers are dependency-free:

python3 scripts/validate_cds_artifact.py --help
python3 scripts/evidence_profile_check.py --help
python3 scripts/model_biomarker_evaluation.py --help
python3 scripts/cohort_table_generator.py --help
python3 scripts/survival_plan_validator.py --help
python3 scripts/decision_logic_traceability.py --help
python3 scripts/deidentification_checklist.py --help

Write outputs only to a reviewed local directory. Never place generated reports in an EHR, alerting system, clinical portal, or device workflow.

4. Human Review

Require review proportionate to the artifact:

• methodologist/statistician for design and analysis;
• domain expert for clinical-scientific context;
• privacy officer or qualified expert for disclosure decisions;
• regulatory or legal counsel for jurisdiction-specific interpretations;
• human-factors specialist for user studies;
• authorized governance owner for release and change control.

Script success means only that declared fields and internal consistency checks passed.

GRADE Evidence Profiles

Do not infer a certainty rating from article text, study design alone, p-values, or keywords. Do not use the legacy 1A/2B shorthand as if it were universal GRADE output.

For each important outcome, a human panel must document:

• risk of bias;
• inconsistency;
• indirectness;
• imprecision;
• publication bias;
• any applicable upgrading considerations;
• effect estimate and uncertainty;
• rationale and source IDs for every judgment;
• final certainty judgment and named review role.

The checker validates completeness and citation links only. It never calculates certainty or recommendation strength. See references/evidence_profiles.md.

Aggregate Model and Biomarker Evaluation

Do not derive thresholds, assign molecular or disease classes, match therapies, or emit person-level predictions.

The evaluator accepts only aggregate confusion counts and calibration bins. It reports bounded descriptive metrics with Wilson intervals, calibration gaps, subgroup differences, and explicit suppression. It does not determine fairness, clinical utility, or fitness for use. Require:

• locked model/assay/version and pre-specified threshold provenance;
• representative internal validation and independent external validation;
• calibration and discrimination appropriate to the target;
• subgroup performance with uncertainty and sample sizes;
• missingness, spectrum/selection bias, dataset shift, and assay variability;
• human-factors and prospective evaluation where relevant;
• monitoring, change control, rollback, and retirement criteria.

See references/model_biomarker_evaluation.md.

Cohort Tables

Use aggregate cells only. Do not provide row-level data to the generator.

• Choose the minimum cell threshold under an approved disclosure policy.
• Apply primary and complementary suppression.
• Report denominators and missingness.
• Avoid baseline significance testing as a balance diagnostic.
• Label adjusted, unadjusted, pre-specified, and exploratory results.
• Do not interpret association as causation or clinical actionability.

The default threshold is an operational safeguard, not a HIPAA rule or guarantee. See references/cohort_evaluation.md and references/privacy_and_disclosure.md.

Survival Plans

Define time zero, event, competing events, censoring, intercurrent events, estimand, horizon, effect measure, and analysis population together.

• Assess proportional hazards before treating a hazard ratio as constant.
• Pre-specify alternatives such as time-varying effects or restricted mean survival time.
• Use cumulative-incidence methods when competing events matter.
• Address immortal-time, informative-censoring, delayed-entry, missing-data, and multiplicity risks.
• Include sensitivity analyses and uncertainty, not only p-values.

The bundled helper validates a plan; it does not analyze survival data. See references/survival_analysis.md.

Decision Logic

Only document research or governance logic, such as evidence inclusion, validation gates, release holds, and human-review checkpoints. Each node must link to source IDs, tests, owner, version, and status.

Do not encode care pathways, urgency, medication actions, diagnostic rules, alarms, or patient-facing outputs. See references/decision_logic_traceability.md.

Privacy and De-identification

The HHS methods are Expert Determination and Safe Harbor. A checklist cannot perform either method by itself. Do not claim that removing a list of fields, hashing identifiers, using a minimum cell size, or passing this script proves de-identification or HIPAA compliance.

The helper inventories documented human work. It never reads a dataset. Escalate unresolved items, free text, dates, geography, rare combinations, linkage risk, genomics, and longitudinal patterns to qualified privacy review.

Reporting-Guideline Selection

• Cohort/case-control/cross-sectional: STROBE; add RECORD for routinely collected data.
• Prediction model development/evaluation: TRIPOD+AI and PROBAST+AI.
• Tumor prognostic marker study: REMARK.
• AI diagnostic accuracy: STARD-AI with STARD.
• AI trial protocol: SPIRIT-AI with the current SPIRIT base statement.
• AI randomized trial report: CONSORT-AI with the current CONSORT base statement.
• Early live AI evaluation: DECIDE-AI—but live evaluation is outside this skill's execution scope.

These are reporting or appraisal tools, not automatic quality scores. See references/study_reporting.md.

Regulatory and Governance Context

FDA device status turns on intended use and function, not a document label. FDA's January 2026 CDS guidance distinguishes certain non-device CDS functions from device software functions; its examples are not a self-certification checklist. ONC HTI-1 requirements apply within the defined certification scope. ICH E6(R3) and E9/E9(R1) inform trial governance and statistical planning but do not make an artifact compliant.

Use references/regulatory_and_governance.md for dated context. Obtain qualified advice for an actual product, study, submission, deployment, or jurisdiction.

Verification

From this skill directory:

python3 -m unittest discover -s tests -p 'test_*.py'

Run AST compilation without bytecode:

python3 -c "import ast,pathlib; [ast.parse(p.read_text()) for p in pathlib.Path('scripts').glob('*.py')]"

Reference Map

• references/README.md — scope and navigation
• references/safety_and_scope.md — refusal and escalation rules
• references/regulatory_and_governance.md — FDA, ONC, ICH context
• references/evidence_profiles.md — human GRADE workflow
• references/study_reporting.md — EQUATOR and PROBAST+AI selection
• references/cohort_evaluation.md — aggregate cohort methods
• references/survival_analysis.md — time-to-event planning
• references/model_biomarker_evaluation.md — model/biomarker evaluation
• references/privacy_and_disclosure.md — de-identification and suppression
• references/decision_logic_traceability.md — governance logic
• references/sources.md — dated authoritative source ledger
• references/security_validation.md — scan results and accepted LOW findings