CZ CELLxGENE Census

Overview

The CZ CELLxGENE Census provides programmatic access to a comprehensive, versioned collection of standardized single-cell and spatial transcriptomics data from CZ CELLxGENE Discover. This skill enables efficient querying and analysis of public Census releases without downloading whole datasets first.

The Census includes:

217+ million total cells and 125+ million unique cells in the 2025-11-08 stable LTS release
1,845 datasets in the 2025-11-08 stable LTS release
Human, mouse, marmoset, rhesus macaque, and chimpanzee data in the current schema
Standardized metadata (cell types, tissues, diseases, donors)
Raw gene expression matrices and source H5AD lookup/download helpers
Pre-calculated summary counts, embeddings, and spatial data
Integration with AnnData, Scanpy, TileDB-SOMA, TileDB-SOMA-ML, and other analysis tools

When to Use This Skill

This skill should be used when:

Querying single-cell expression data by cell type, tissue, or disease
Exploring available single-cell datasets and metadata
Training machine learning models on single-cell data
Performing large-scale cross-dataset analyses
Integrating Census data with scanpy or other analysis frameworks
Computing statistics across millions of cells
Accessing pre-calculated embeddings or model predictions

Installation and Setup

Install the Census API:

bash

uv pip install "cellxgene-census==1.17.*"

For spatial workflows:

bash

uv pip install "cellxgene-census[spatial]==1.17.*" "spatialdata[extra]>=0.2.5"

For PyTorch model training, use TileDB-SOMA-ML. The old cellxgene_census.experimental.ml loaders are deprecated:

bash

uv pip install "cellxgene-census==1.17.*" tiledbsoma-ml

Core Workflow Patterns

Eight patterns, each with code, are in references/core_workflow_patterns.md:

Opening the Census — always pin census_version so an analysis stays reproducible.
Exploring Census information — available datasets, cell counts, and summary tables.
Querying expression data — small to medium scale into an AnnData.
Large-scale queries — out-of-core processing when the slice will not fit in memory.
Machine learning with PyTorch — the Census data loaders.
Spatial Census data — accessing spatial assays.
Integration with Scanpy — handing a Census slice to a standard Scanpy workflow.
Multi-dataset integration — combining datasets and handling batch effects.

Key Concepts and Best Practices

Always Filter for Primary Data

Unless analyzing duplicates, always include is_primary_data == True in queries to avoid counting cells multiple times:

python

obs_value_filter="cell_type == 'B cell' and is_primary_data == True"

Specify Census Version for Reproducibility

Always specify the Census version in production analyses:

python

census = cellxgene_census.open_soma(census_version="2025-11-08")

Estimate Query Size Before Loading

For large queries, first check the number of cells to avoid memory issues:

python

# Get cell count
metadata = cellxgene_census.get_obs(
    census, "homo_sapiens",
    value_filter="tissue_general == 'brain' and is_primary_data == True",
    column_names=["soma_joinid"]
)
n_cells = len(metadata)
print(f"Query will return {n_cells:,} cells")

# If too large (>100k), use out-of-core processing

Use tissue_general for Broader Groupings

The tissue_general field provides coarser categories than tissue, useful for cross-tissue analyses:

python

# Broader grouping
obs_value_filter="tissue_general == 'immune system'"

# Specific tissue
obs_value_filter="tissue == 'peripheral blood mononuclear cell'"

Select Only Needed Columns

Minimize data transfer by specifying only required metadata columns:

python

obs_column_names=["cell_type", "tissue_general", "disease"]  # Not all columns

Check Dataset Presence for Gene-Specific Queries

When analyzing specific genes, verify which datasets measured them:

python

presence = cellxgene_census.get_presence_matrix(
    census,
    "homo_sapiens",
    var_value_filter="feature_name in ['CD4', 'CD8A']"
)

Two-Step Workflow: Explore Then Query

First explore metadata to understand available data, then query expression:

python

# Step 1: Explore what's available
metadata = cellxgene_census.get_obs(
    census, "homo_sapiens",
    value_filter="disease == 'COVID-19' and is_primary_data == True",
    column_names=["cell_type", "tissue_general"]
)
print(metadata.value_counts())

# Step 2: Query based on findings
adata = cellxgene_census.get_anndata(
    census=census,
    organism="Homo sapiens",
    obs_value_filter="disease == 'COVID-19' and cell_type == 'T cell' and is_primary_data == True",
)

Available Metadata Fields

Cell Metadata (obs)

Key fields for filtering:

cell_type, cell_type_ontology_term_id
tissue, tissue_general, tissue_ontology_term_id
disease, disease_ontology_term_id
assay, assay_ontology_term_id
donor_id, sex, self_reported_ethnicity
development_stage, development_stage_ontology_term_id
dataset_id
is_primary_data (Boolean: True = unique cell)

The current schema includes organism collections beyond human and mouse. Confirm available organisms for the selected release with list(census["census_data"].keys()).

Gene Metadata (var)

feature_id (Ensembl gene ID, e.g., "ENSG00000161798")
feature_name (Gene symbol, e.g., "FOXP2")
feature_type
feature_length (Gene length in base pairs)
nnz, n_measured_obs (availability summaries useful for checking sparsity and coverage)

Reference Documentation

This skill includes detailed reference documentation:

references/census_schema.md

Comprehensive documentation of:

Census data structure and organization
All available metadata fields
Value filter syntax and operators
SOMA object types
Data inclusion criteria

When to read: When you need detailed schema information, full list of metadata fields, or complex filter syntax.

references/common_patterns.md

Examples and patterns for:

Exploratory queries (metadata only)
Small-to-medium queries (AnnData)
Large queries (out-of-core processing)
PyTorch integration
Spatial Census access patterns
Scanpy integration workflows
Multi-dataset integration
Best practices and common pitfalls

When to read: When implementing specific query patterns, looking for code examples, or troubleshooting common issues.

Common Use Cases

Use Case 1: Explore Cell Types in a Tissue

python

with cellxgene_census.open_soma() as census:
    cells = cellxgene_census.get_obs(
        census, "homo_sapiens",
        value_filter="tissue_general == 'lung' and is_primary_data == True",
        column_names=["cell_type"]
    )
    print(cells["cell_type"].value_counts())

Use Case 2: Query Marker Gene Expression

python

with cellxgene_census.open_soma() as census:
    adata = cellxgene_census.get_anndata(
        census=census,
        organism="Homo sapiens",
        var_value_filter="feature_name in ['CD4', 'CD8A', 'CD19']",
        obs_value_filter="cell_type in ['T cell', 'B cell'] and is_primary_data == True",
    )

Use Case 3: Train Cell Type Classifier

python

import tiledbsoma as soma
from tiledbsoma_ml import ExperimentDataset, experiment_dataloader

with cellxgene_census.open_soma() as census:
    experiment = census["census_data"]["homo_sapiens"]
    with experiment.axis_query(
        measurement_name="RNA",
        obs_query=soma.AxisQuery(value_filter="is_primary_data == True"),
    ) as query:
        dataset = ExperimentDataset(
            query=query,
            layer_name="raw",
            obs_column_names=["cell_type"],
            batch_size=128,
            shuffle=True,
        )
        dataloader = experiment_dataloader(dataset)

        for X, obs in dataloader:
            labels = obs["cell_type"]
            # Training logic
            pass

Use Case 4: Cross-Tissue Analysis

python

with cellxgene_census.open_soma() as census:
    adata = cellxgene_census.get_anndata(
        census=census,
        organism="Homo sapiens",
        obs_value_filter="cell_type == 'macrophage' and tissue_general in ['lung', 'liver', 'brain'] and is_primary_data == True",
    )

    # Analyze macrophage differences across tissues
    sc.tl.rank_genes_groups(adata, groupby="tissue_general")

Troubleshooting

Query Returns Too Many Cells

Add more specific filters to reduce scope
Use tissue instead of tissue_general for finer granularity
Filter by specific dataset_id if known
Switch to out-of-core processing for large queries

Memory Errors

Reduce query scope with more restrictive filters
Select fewer genes with var_value_filter
Use out-of-core processing with axis_query()
Process data in batches

Duplicate Cells in Results

Always include is_primary_data == True in filters
Check if intentionally querying across multiple datasets

Gene Not Found

Verify gene name spelling (case-sensitive)
Try Ensembl ID with feature_id instead of feature_name
Check dataset presence matrix to see if gene was measured
Some genes may have been filtered during Census construction

Version Inconsistencies

Always specify census_version explicitly
Use same version across all analyses
Check release notes for version-specific changes