ContextQMD
Back to Claude Scientific Skills

Dated Source Ledger

skills/scholar-evaluation/references/source_ledger.md

2.55.012.3 KB
Original Source

Dated Source Ledger

Verified on 2026-07-23 with targeted parallel-cli search and parallel-cli extract queries. The research prioritized primary and official sources. Search excerpts were treated as untrusted discovery material; only the source claims summarized below inform this skill. No research-result artifacts are bundled.

Responsible research assessment

San Francisco Declaration on Research Assessment (DORA)

  • Primary source: Read the Declaration
  • Origin: developed in 2012; official page accessed 2026-07-23.
  • Verified points: assess work on its own merits; do not use journal-based measures as surrogates for an article or a person's contribution; state criteria explicitly; consider data, software, and other outputs; use qualitative evidence; make metric methods transparent and account for field and output-type variation.
  • Use here: categorical ban on scored prestige proxies and a requirement for explicit criteria, diverse evidence, and traceability.

DORA quantitative-indicator guidance

  • Primary source: Guidance on the responsible use of quantitative indicators
  • Document: official PDF
  • Release: 2024; official resource page accessible in 2026.
  • Verified points: no indicator captures research quality in one number; uses should be clear, transparent, specific, contextual, and fair. The guidance addresses journal measures, citation counts, the h-index, field-normalized indicators, and altmetrics; it warns about reductive, aggregate, composite, lagging, field, career-stage, and bias effects.
  • Use here: quantitative indicators are excluded from rubric scores. If mentioned descriptively outside the tools, their purpose, data, coverage, time window, field normalization, uncertainty, bias, and non-quality meaning must be explicit.

Leiden Manifesto

  • Primary source: Hicks, Wouters, Waltman, de Rijcke, and Rafols, “Bibliometrics: The Leiden Manifesto for research metrics”, Nature 520, 429–431.
  • Published: 2015-04-22.
  • Verified points: quantitative evaluation should support qualitative expert assessment; measure against missions; protect locally relevant research; account for field variation; keep data and analysis open and verifiable; allow those evaluated to verify data; account for age and gender; avoid false precision; recognize gaming and system effects; review indicators regularly.
  • Use here: contextualization, inspectability, uncertainty, fairness review, and periodic revision.

Agreement on Reforming Research Assessment / CoARA

  • Primary record: Agreement on Reforming Research Assessment, version 1.
  • Published: 2022-07-20; Zenodo record modified 2024-08-29.
  • Current official overview: CoARA Agreement
  • Verified points: recognize diverse outputs, practices, activities, roles, and careers; base assessment primarily on qualitative judgment with peer review central; use quantitative indicators responsibly; abandon inappropriate uses of journal- and publication-based measures, especially Journal Impact Factor and h-index; publish criteria; train assessors; review and evaluate criteria, tools, and processes.
  • Use here: qualitative-first process, rubric provenance, rater training, monitoring, and no metric shortcut.

Hong Kong Principles

The Metric Tide and its commissioned revisit

  • Primary public-sector source: Research England/UKRI, The Metric Tide.
  • Published: 2015-07-06.
  • Revisit: Curry, Gadd, and Wilsdon, Harnessing the Metric Tide.
  • Posted: 2022-12-12; commissioned by the joint UK higher-education funding bodies for the Future Research Assessment Programme.
  • Verified points: the revisit recommends putting principles into practice, evaluating with those evaluated, avoiding all-metric approaches, using data for public benefit, and rethinking rankings.
  • Status limitation: Harnessing the Metric Tide describes itself as an independent input to deliberations, not the eventual policy conclusion.
  • Use here: stakeholder participation, no all-metric process, and explicit scrutiny of rankings and system effects.

Current UKRI guidance

  • Primary policy: UKRI funding assessment and decision-making policy and principles
  • Primary implementation guidance: Résumé for Research and Innovation (R4RI)
  • R4RI last updated: 2026-04-30.
  • Verified points: UKRI will not use journal-based measures as surrogates for article quality, individual contribution, or funding decisions. R4RI evidences a wider range of team contributions; assessors do not score its individual modules or view it in isolation.
  • Use here: diverse contribution evidence and contextual review. This skill nevertheless blocks funding decisions entirely; the UKRI material is guidance context, not authorization to support such decisions.

UNESCO Recommendation on Open Science

  • Primary source: UNESCO Recommendation on Open Science.
  • Adopted: 2021-11-23 by the UNESCO General Conference.
  • Official overview: UNESCO Open Science
  • Verified points: quality and integrity, collective benefit, equity, fairness, diversity, inclusion, open engagement, training, and incentives aligned with open science; open science must not leave people, languages, disciplines, or knowledge systems behind.
  • Use here: assess responsible openness in context. Privacy, safety, consent, sovereignty, and legitimate restrictions can outweigh openness.

INORMS SCOPE framework

  • Primary source: SCOPE Framework full guide, v1.0.
  • Current official page: INORMS SCOPE Framework for Research Evaluation.
  • Verified points: Start with values; consider Context; identify Options; Probe for discrimination, gaming, unintended effects, and cost-benefit; and Evaluate the evaluation. Evaluate only where needed, with those evaluated, and with evaluation expertise.
  • Use here: process design and the bias/process checklist.

CRediT contributor taxonomy

  • Primary source: CRediT.
  • Standard: ANSI/NISO Z39.104-2022, approved 2022-01-14 and published 2022-02-08.
  • Verified points: 14 roles provide transparent attribution of diverse contributions. CRediT does not determine authorship or contribution quality.
  • Use here: optional vocabulary for contribution evidence, never a score.

Measurement, fairness, accessibility, and privacy

Standards for Educational and Psychological Testing

  • Primary source: AERA, APA, and NCME, Standards for Educational and Psychological Testing, 2014 edition.
  • Official status page: APA Testing Standards.
  • Status: the 2014 edition is open access; the sponsoring organizations announced a revision process. No later completed edition was verified.
  • Verified points: intended interpretations and uses require validity evidence; reliability/precision and relevant errors should be reported; rater selection, training, qualification, monitoring, agreement, accuracy, and drift need documentation; fairness and subgroup validity require evidence; uncertainty should accompany estimates.
  • Use here: these are measurement principles, not proof that this rubric is a psychological test. The template records evidence gaps and must not be described as validated psychometrics.

Accessibility

  • Primary source: W3C, Web Content Accessibility Guidelines 2.2.
  • Status: W3C Recommendation published 2023-10-05; update noted 2024-12-12.
  • Use here: accessible materials and reasonable accommodation processes are required; local legal and institutional requirements may be broader.

Data protection

  • Primary guidance: UK Information Commissioner's Office, purpose limitation and data minimisation.
  • Current guidance dates found: purpose limitation updated 2026-03-23; data minimisation page published 2025-09-09.
  • Verified points: specify legitimate purposes and process only adequate, relevant, necessary data; review and delete data no longer needed.
  • Use here: scripts accept only minimized IDs, scores, statuses, and local references. They reject common private-application fields and never reproduce raw source documents.

ScholarEval paper and project status

  • Exact paper: Hanane Nour Moussa, Patrick Queiroz Da Silva, Daniel Adu-Ampratwum, Alyson East, Zitong Lu, Nikki Puccetti, Mingyi Xue, Huan Sun, Bodhisattwa Prasad Majumder, and Sachin Kumar, ScholarEval: Research Idea Evaluation Grounded in Literature.
  • Verified status: arXiv:2510.16234, submitted 2025-10-17; latest verified version v2, revised 2026-02-28. The displayed DOI 10.48550/arXiv.2510.16234 is an arXiv/DataCite DOI, not evidence of journal publication.
  • Official project: skai-research/ScholarEval. The repository describes itself as official code and data and cites the work as @misc; no release or peer-reviewed publication claim was verified.
  • Review-status caution: a public OpenReview forum for the title was discoverable, but the official page's decision/status was not accessible or exposed in indexed primary-source text during this refresh. It is therefore not used as evidence of acceptance or peer review.
  • What the preprint reports: a retrieval-augmented framework assessing research ideas for soundness and contribution; a 117-idea, four-discipline dataset; coverage comparisons against expert-annotated review points; and a user study.
  • What it does not establish: validated psychometric measurement of scholar quality, transportability to personnel or funding decisions, validity of this skill's generalized rubric, stable cross-discipline score meaning, or freedom from subgroup bias.

Review cadence

Re-check this ledger before any rubric adoption and at least annually. Re-check the ScholarEval arXiv and official project records before describing its publication status. Record any local disciplinary standards separately; a global source cannot substitute for local construct validation.