ContextQMD
Back to Claude Scientific Skills

Practical Workflows (matchms 0.33.1)

skills/matchms/references/workflows.md

2.55.012.7 KB
Original Source

Practical Workflows (matchms 0.33.1)

These patterns use the current 0.33.1 API. Adapt tolerances and quality filters to the instrument, acquisition method, and scientific question.

1. Standard Query/Reference Preprocessing

Apply the same peak processing to both collections. Metadata enrichment can differ when a reference library contains known structures and queries do not.

python
from matchms import SpectrumProcessor
from matchms.filtering import (
    default_filters,
    normalize_intensities,
    require_minimum_number_of_peaks,
    require_precursor_mz,
    select_by_mz,
    select_by_relative_intensity,
)
from matchms.importing import load_spectra


def preprocess(path):
    raw = [default_filters(spectrum) for spectrum in load_spectra(path)]
    processor = SpectrumProcessor(
        [
            (require_precursor_mz, {"minimum_accepted_mz": 50.0}),
            normalize_intensities,
            (select_by_mz, {"mz_from": 20.0, "mz_to": 1500.0}),
            (select_by_relative_intensity, {"intensity_from": 0.01}),
            (require_minimum_number_of_peaks, {"n_required": 5}),
        ]
    )
    cleaned, _ = processor.process_spectra(
        raw,
        progress_bar=False,
        create_report=False,
    )
    return cleaned, ["default_filters", *processor.processing_steps]


queries, query_steps = preprocess("queries.mgf")
references, reference_steps = preprocess("library.msp")
assert query_steps == reference_steps

SpectrumProcessor may reorder built-in filters. Persist the final processing_steps, not only the requested list. Run the aggregate default_filters before the processor; if passed as one callable, it is treated as a custom filter and placed after registered filters.

The bundled CLI is the fastest route to a reproducible CSV:

bash
uv run python scripts/library_search.py \
  queries.mgf library.msp hits.csv \
  --metric modified \
  --tolerance 0.02 \
  --top-k 10 \
  --min-score 0.6 \
  --min-matches 5

The script:

  • applies the same configurable peak filters to both inputs;
  • supports cosine, exact/greedy modified cosine, neutral-loss, BLINK, and Flash modes;
  • handles both scalar and structured matchms score records;
  • records identifiers, precursor m/z, rank, score, and matched peaks;
  • refuses pickle inputs;
  • refuses to overwrite output unless --force; and
  • stops before an unexpectedly large pairwise matrix unless --max-pairs is explicitly raised.

Use --help before batch use.

python
from matchms import calculate_scores
from matchms.similarity import ModifiedCosineGreedy

metric = ModifiedCosineGreedy(tolerance=0.02)
scores = calculate_scores(
    references=references,
    queries=queries,
    similarity_function=metric,
)

score_name = "ModifiedCosineGreedy_score"
matches_name = "ModifiedCosineGreedy_matches"
rows = []

for query_index, query in enumerate(queries):
    ranked = scores.scores_by_query(query, name=score_name, sort=True)
    for rank, (reference, value) in enumerate(ranked[:10], start=1):
        rows.append(
            {
                "query_index": query_index,
                "query_id": query.get("spectrum_id", query.get("id")),
                "reference_id": reference.get(
                    "spectrum_id",
                    reference.get("compound_name"),
                ),
                "rank": rank,
                "score": float(value[score_name]),
                "matches": int(value[matches_name]),
            }
        )

Filter on both score and matched peaks only after examining their distributions. Do not label arbitrary cutoffs as universal high/medium/low confidence.

For identity-oriented search, a precursor gate can reduce expensive spectral calculations:

python
from matchms import calculate_scores
from matchms.similarity import ModifiedCosineGreedy, PrecursorMzMatch

scores = calculate_scores(
    references,
    queries,
    PrecursorMzMatch(tolerance=10, tolerance_type="ppm"),
    array_type="sparse",
)
scores.filter_by_range(
    name="PrecursorMzMatch",
    low=0.5,
    above_operator=">=",
)
scores.calculate(
    ModifiedCosineGreedy(tolerance=0.02),
    array_type="sparse",
    join_type="left",
)

This pattern calculates the second metric on retained coordinates. It is inappropriate when the scientific goal is broad analog discovery across precursor shifts or adducts.

5. Reproducible Pipeline Workflow

Pipeline combines import, ordered filtering, sparse score gating, and score calculation. A workflow can also be written to YAML.

python
from matchms import Pipeline
from matchms.Pipeline import create_workflow

common_filters = [
    "make_charge_int",
    "add_compound_name",
    "derive_adduct_from_name",
    "derive_formula_from_name",
    "clean_compound_name",
    "interpret_pepmass",
    "add_precursor_mz",
    "derive_ionmode",
    "correct_charge",
    ["require_precursor_mz", {"minimum_accepted_mz": 50.0}],
    "normalize_intensities",
    ["select_by_relative_intensity", {"intensity_from": 0.01}],
    ["require_minimum_number_of_peaks", {"n_required": 5}],
]

workflow = create_workflow(
    yaml_file_name=None,
    query_filters=common_filters,
    reference_filters=common_filters,
    score_computations=[
        ["precursormzmatch", {"tolerance": 10, "tolerance_type": "ppm"}],
        ["filter_by_range", {"name": "PrecursorMzMatch", "low": 0.5}],
        ["modifiedcosinegreedy", {"tolerance": 0.02}],
        [
            "filter_by_range",
            {"name": "ModifiedCosineGreedy_score", "low": 0.6},
        ],
    ],
)

pipeline = Pipeline(
    workflow,
    progress_bar=False,
    logging_level="WARNING",
    logging_file="matchms-pipeline.log",
)
pipeline.run(
    query_files="queries.mgf",
    reference_files="library.msp",
    cleaned_query_file="queries-cleaned.mgf",
    cleaned_reference_file="library-cleaned.msp",
    create_report=False,
)
scores = pipeline.scores

Notes:

  • create_workflow() expects matchms filter callables/names and score class names lowercased in score_computations.
  • The first score should establish the coordinate set when later metrics should be candidate-gated.
  • filter_by_range mutates the retained coordinates.
  • Pipeline.run() stores final scores on pipeline.scores.
  • Output and YAML paths must not already exist.
  • For all-vs-all scoring, omit reference_files; Pipeline then sets symmetric mode.

To persist and reload a workflow:

python
from matchms import Pipeline
from matchms.Pipeline import create_workflow
from matchms.yaml_file_functions import load_workflow_from_yaml_file

create_workflow(
    yaml_file_name="workflow.yaml",
    query_filters=common_filters,
    reference_filters=common_filters,
    score_computations=[["cosinegreedy", {"tolerance": 0.02}]],
)

workflow = load_workflow_from_yaml_file("workflow.yaml")
Pipeline(workflow).run("queries.mgf", "library.msp")

Treat YAML as configuration, not as a substitute for recording software and input versions.

6. All-vs-All Comparison

python
from matchms import calculate_scores
from matchms.similarity import CosineGreedy

scores = calculate_scores(
    references=spectra,
    queries=spectra,
    similarity_function=CosineGreedy(tolerance=0.02),
    array_type="sparse",
    is_symmetric=True,
)

is_symmetric=True avoids redundant calculation for symmetric methods. The logical pair count still grows quadratically. Estimate size before launching:

python
n = len(spectra)
unique_pairs_with_diagonal = n * (n + 1) // 2

For large collections, consider:

  • precursor/metadata candidate masks;
  • BlinkCosine or matrix-oriented FlashSimilarity;
  • batched query subsets;
  • sparse score thresholds; and
  • approximate-neighbor indexing with validated recall.

Do not densify a large result with to_array() merely to find top hits.

7. Fast Matrix Scoring

Flash entropy

python
from matchms import calculate_scores
from matchms.similarity import FlashSimilarity

scores = calculate_scores(
    references,
    queries,
    FlashSimilarity(
        score_type="spectral_entropy",
        matching_mode="fragment",
        tolerance=0.02,
        noise_cutoff=0.01,
    ),
)

Fast modified-cosine-like mode

python
scores = calculate_scores(
    references,
    queries,
    FlashSimilarity(
        score_type="cosine",
        matching_mode="hybrid",
        tolerance=0.02,
    ),
)

Flash outputs a scalar FlashSimilarity field without a matched-peak count. Validate ranking agreement against a transparent baseline on representative data.

8. Mirror-Plot Validation

python
from pathlib import Path

output = Path("top-hit-mirror.png")
axis = query.plot_against(reference, figsize=(10, 6), dpi=200)
axis.figure.savefig(output, bbox_inches="tight")

Inspect:

  • unshifted and shifted matched peaks;
  • precursor and adduct agreement;
  • whether the score is driven by one dominant fragment;
  • unexplained high-intensity peaks; and
  • collision-energy/acquisition differences.

9. Spectral Similarity Network

Networks require an all-vs-all Scores object and a unique identifier on each spectrum:

python
from matchms import calculate_scores
from matchms.networking import SimilarityNetwork
from matchms.similarity import ModifiedCosineGreedy

scores = calculate_scores(
    spectra,
    spectra,
    ModifiedCosineGreedy(tolerance=0.02),
    array_type="sparse",
    is_symmetric=True,
)

network = SimilarityNetwork(
    identifier_key="spectrum_id",
    top_n=20,
    max_links=10,
    score_cutoff=0.7,
    link_method="mutual",
    keep_unconnected_nodes=True,
)
network.create_network(
    scores,
    score_name="ModifiedCosineGreedy_score",
)
network.export_to_file("spectral-network.graphml", graph_format="graphml")

Supported export formats include GraphML, GEXF, GML, Cytoscape JSON, and JSON. top_n must be at least max_links. Equal scores near the strict link limit can make tie selection order-dependent; use deterministic identifiers and document parameters.

Network components are hypotheses about spectral relatedness, not proof of shared structure or biosynthetic origin.

10. Structure Versus Spectrum Similarity

Use separate labels and outputs:

  • spectral similarity compares measured peak patterns;
  • fingerprint similarity compares known molecular structures;
  • correlation between the two can be analyzed only where reliable structures are available.

Do not include a query's true structure in candidate ranking when evaluating a spectral identification method; that leaks the answer.

11. USI-Based Reproducible Pair

python
from matchms.importing import load_from_usi
from matchms.similarity import CosineGreedy

reference = load_from_usi(
    "mzspec:GNPS:GNPS-LIBRARY:accession:CCMSLIB00000424840"
)
query = load_from_usi(
    "mzspec:MSV000086109:BD5_dil2x_BD5_01_57213:scan:760"
)
result = CosineGreedy(tolerance=0.02).pair(reference, query)

Record resolver, retrieval date, and returned metadata. A USI is stable provenance, but resolver availability and returned annotations can change.

12. Provenance Record

At minimum, save:

python
import json
import platform

import matchms

provenance = {
    "matchms_version": matchms.__version__,
    "python_version": platform.python_version(),
    "query_files": ["queries.mgf"],
    "reference_files": ["library.msp"],
    "metadata_harmonization": True,
    "processing_steps": query_steps,
    "similarity": {
        "class": "ModifiedCosineGreedy",
        "tolerance_da": 0.02,
    },
    "selection": {
        "top_k": 10,
        "minimum_score": 0.6,
        "minimum_matches": 5,
    },
}

with open("provenance.json", "w", encoding="utf-8") as handle:
    json.dump(provenance, handle, indent=2, default=str)

Also retain input checksums, library release/date, acquisition information, code/configuration, and any manual curation decisions.

Failure Modes

  • No modified-cosine results: check precursor m/z after harmonization and filtering.
  • Unexpected TypeError from SpectrumProcessor: call process_spectrum()/process_spectra(); the processor is not callable.
  • Cannot format a score as float: extract a named field from the structured score record.
  • Top-hit value treated as an index: scores_by_query() returns a Spectrum.
  • Huge memory use: avoid dense conversion; gate candidates and estimate pair count.
  • No network edges: verify identifier fields, score layer name, threshold, and that the scores are all-vs-all.
  • Fingerprint warnings: migrate from add_fingerprint() to Fingerprints and bridge to the current FingerprintSimilarity only when required.
  • Output already exists: matchms intentionally refuses overwrite in generic writers and pipeline outputs; choose a new path or remove only after explicit confirmation.