DpCompression Subtopology — Software Design Document (SDD)

The DpCompression subtopology packages the F´ data product compression services — a compression processor, a ZLib compressor, and a dedicated buffer manager — into a single pre-wired subtopology. It plugs into a data product pipeline (typically the DataProducts subtopology's dpWriterProcOut processing hook) to compress data products before they are stored and downlinked.

1. Requirements

2. Design & Core Functions

2.1 Instance Summary

2.2 Internal Wiring

• dpCompressProc.compressChunk -> dpZLibCompressor.compressChunk — chunks flow to the compressor.
• dpZLibCompressor.bufferCompressionGet / bufferCompressionReturn -> dpZLibCompressorBufferManager — compression output buffer allocation and return.
• dpZLibCompressor.bufferZLibGet / bufferZLibReturn -> dpZLibCompressorBufferManager — ZLib working buffer allocation and return.

2.3 Configuration Hooks inside the Subtopology

• dpZLibCompressorBufferManager: Two buffer bins are set up at startup from DpCompressionConfig::ZLibBuffMgr — one for compression output buffers and one for ZLib allocation (working) buffers — and cleaned up at teardown.

2.4 Required Inputs for Operation

The DpCompression subtopology is not a stand-alone application. It requires connections from the including deployment topology:

• Processing requests: Connect a data product processing output (e.g., DataProducts.Subtopology.dpWriterProcOut) to dpCompressProcIn.
• Rate group: Connect a scheduler output to dpZLibBufferManagerSchedIn for buffer manager telemetry.

3. Usage

3.1 Exposed Topology Ports

3.2 Example Usage

topology Flight {
  instance DataProducts.Subtopology
  instance DpCompression.Subtopology

  connections Compression {
    DataProducts.Subtopology.dpWriterProcOut -> DpCompression.Subtopology.dpCompressProcIn
  }

  connections RateGroups {
    rg1Hz.RateGroupMemberOut[3] -> DpCompression.Subtopology.dpZLibBufferManagerSchedIn
  }
}

4. Configuration

Configure the buffer store properties in Svc/Subtopologies/DpCompression/DpCompressionConfig/DpCompressionConfig.fpp:

• Base ID — Base identifier for the subtopology; component IDs are offset from this base.
• ZLibBuffMgr — Compression buffer store size/count, ZLib allocation buffer store size/count, and the buffer manager ID.

The memory allocator used by the subtopology is provided by DpCompressionSubtopologyConfig.cpp (DpCompression::Allocation::memAllocator).

5. Traceability Matrix