Data Access, Hierarchies, and Transfers

Use this reference for bounded reads and explicit import/export scopes. Read connection.md first.

Object Hierarchies

Common container paths are:

Project -> Dataset -> Image
Screen -> Plate -> Well -> WellSample -> Image
Image -> Pixels -> Channel
Image -> Fileset -> OriginalFile(s)

Links are model objects and may be many-to-many. Do not assume an image has exactly one dataset or a dataset exactly one project. Traverse links returned by the server instead of synthesizing parent paths.

Common BlitzGateway object names documented by OME include:

• Project, Dataset, Image
• Screen, Plate, PlateAcquisition, Well
• Roi, Shape
• Experimenter, ExperimenterGroup
• OriginalFile, Fileset
• Annotation and specific annotation subtypes

Object-name support is not permission. A returned None may mean nonexistent or inaccessible.

One Object by ID

Use explicit IDs whenever possible:

image_id = 123
image = conn.getObject("Image", image_id)
if image is None:
    raise LookupError("Image was not found or is not accessible")

print(image.getId())
print(image.getSizeX(), image.getSizeY())

Do not print names, descriptions, owner names, or acquisition metadata unless the requested output includes them.

For multiple explicit IDs:

requested_ids = [101, 102, 103]
for image in conn.getObjects(
    "Image",
    requested_ids,
    respect_order=True,
):
    print(image.getId())

Keep the input list bounded. Check whether inaccessible IDs were omitted.

Bounded Pagination

getObjects() returns a generator. Use both an overall cap and page size:

def iter_bounded(conn, object_type, *, limit=100, page_size=25):
    if not 1 <= limit <= 1000:
        raise ValueError("limit must be between 1 and 1000")
    if not 1 <= page_size <= min(limit, 200):
        raise ValueError("page_size must be between 1 and min(limit, 200)")

    emitted = 0
    offset = 0
    while emitted < limit:
        size = min(page_size, limit - emitted)
        page = list(
            conn.getObjects(
                object_type,
                opts={
                    "limit": size,
                    "offset": offset,
                    "order_by": "obj.id",
                },
            )
        )
        if not page:
            return

        for obj in page:
            yield obj
            emitted += 1

        if len(page) < size:
            return
        offset += len(page)

Do not write list(conn.getObjects(...)) without server-side limits. If another process changes rows during offset paging, results may shift; record the extraction time and selected group.

The bundled inventory helper implements a cap of 1000 and page cap of 200:

python -B scripts/inventory.py \
  --object-type Image \
  --limit 50 \
  --page-size 25

# Review the dry-run JSON, then explicitly connect:
python -B scripts/inventory.py \
  --object-type Image \
  --limit 50 \
  --page-size 25 \
  --execute \
  --output ./image-inventory.json

Names are redacted unless --include-names is requested.

Group and Owner Filters

Prefer one selected group:

group_id = 42
conn.SERVICE_OPTS.setOmeroGroup(str(group_id))

for project in conn.getObjects(
    "Project",
    opts={"limit": 20, "offset": 0, "order_by": "obj.id"},
):
    print(project.getId())

Filters can further narrow a query:

owner_id = conn.getUser().getId()
projects = conn.getObjects(
    "Project",
    opts={
        "owner": owner_id,
        "group": group_id,
        "limit": 20,
        "offset": 0,
        "order_by": "obj.id",
    },
)

Cross-group context (-1) must be separately approved and paired with a hard limit. Never use it as a fallback when an object is not found.

Traversing Containers

Downward traversal lazily loads children:

project = conn.getObject("Project", project_id)
if project is None:
    raise LookupError("Project unavailable")

dataset_limit = 10
for dataset_index, dataset in enumerate(project.listChildren()):
    if dataset_index >= dataset_limit:
        break
    print(dataset.getId())

    image_limit = 25
    for image_index, image in enumerate(dataset.listChildren()):
        if image_index >= image_limit:
            break
        print(image.getId())

countChildren() can help plan a cap but does not replace one. A count may change before retrieval.

For a direct dataset image query, prefer a server filter:

images = conn.getObjects(
    "Image",
    opts={
        "dataset": dataset_id,
        "limit": 50,
        "offset": 0,
        "order_by": "obj.id",
    },
)

Screening Data

Bound each hierarchy level:

plate = conn.getObject("Plate", plate_id)
if plate is None:
    raise LookupError("Plate unavailable")

for well_index, well in enumerate(plate.listChildren()):
    if well_index >= 96:
        break
    print(well.getId())

    field_count = min(well.countWellSample(), 10)
    for field_index in range(field_count):
        image = well.getImage(field_index)
        if image is not None:
            print(image.getId())

Well rows/columns and field counts can reveal experiment design. Include them only when requested.

Image Metadata

Basic dimensions do not retrieve pixel planes:

summary = {
    "id": image.getId(),
    "size_x": image.getSizeX(),
    "size_y": image.getSizeY(),
    "size_z": image.getSizeZ(),
    "size_c": image.getSizeC(),
    "size_t": image.getSizeT(),
    "pixels_type": image.getPixelsType(),
}

Physical sizes may be absent:

size_x = image.getPixelSizeX(units=True)
if size_x is not None:
    print(size_x.getValue(), size_x.getSymbol())

Names, descriptions, acquisition dates, owner names, group names, and channel labels are potentially sensitive metadata. Redact by default in broad reports.

Filesets and Original Files

A fileset groups original imported files and may represent several images. Inspect metadata before downloading:

fileset = image.getFileset()
if fileset is not None:
    print(fileset.getId())

Downloading an Image or Fileset may retrieve several original files and their directory structure. Estimate scope first; never use a container-wide download merely because it is convenient.

The current CLI supports:

# One OriginalFile:
omero download OriginalFile:123 ./explicit-local-file

# Original files linked to one image:
omero download Image:123 ./explicit-empty-directory

# Original files in one fileset:
omero download Fileset:456 ./explicit-empty-directory

Authenticate through an already prompted CLI session. Do not add -w, --password, or -k to reusable command text. Reject symlinked destinations and collisions; never derive a local path directly from an untrusted remote filename.

Import Planning and Import

The OMERO importer can scan without a running server:

omero import -f ./explicit-input
omero import --depth 4 -f ./explicit-directory

-f lists files that would be imported, grouped into filesets, then exits. This is the correct first pass; it is not a remote import.

The bundled local planner is even more conservative and does not invoke OMERO:

python -B scripts/plan_transfer.py import \
  --target Dataset:id:42 \
  --max-files 100 \
  ./explicit-input

After review and a prompted omero login, an actual scoped import is:

omero import -T Dataset:id:42 ./explicit-input

Important:

• The target must be in the current session group.
• Import needs compatible importer Java libraries; set OMERODIR to the matching extracted server distribution.
• --parallel-fileset and --parallel-upload are documented as experimental; high values can crash the client or make the server unresponsive.
• --report --upload can send broken source files and logs to the OME team. Never use it without explicit authorization to disclose that data.
• In-place imports change repository assumptions and are administrator workflows, not a routine client optimization.

OME-TIFF and XML Export

The documented omero export command currently supports:

omero export --file ./image-123.ome.tiff Image:123
omero export --file ./image-123.ome.xml --type XML Image:123

This is not the same as downloading original files:

• export serializes an OMERO image as OME-TIFF or its metadata as XML;
• download retrieves original files associated with an OriginalFile, FileAnnotation, Image, or Fileset.

Dataset iteration exists only as an experimental export mode. Do not use it for broad exports by default. Plan explicit image IDs instead:

python -B scripts/plan_transfer.py export \
  --format ome-tiff \
  --output-dir ./reviewed-output \
  Image:123 Image:124

The planner does not connect or export. Review file collisions, image count, and available storage before running each proposed command.

Transfer Checklist

Before any import, export, or download:

1. Confirm current session group.
2. Confirm explicit source paths or object IDs.
3. Cap file/object count and directory scan depth.
4. Distinguish derived OME-TIFF/XML export from original-file download.
5. Estimate bytes and review data-sharing authorization.
6. Use a dedicated existing output directory with no symlinks/collisions.
7. Never use credential flags.
8. Do not upload diagnostics or broken files without separate consent.