Svc::DpCatalog Component

1 Introduction

DpCatalog is an active F´ component that manages the downlink of generate data products. The data products were previously generated by components into a project-specific set of directories. Upon command, DpCatalog will build a catalog of existing data products and start downloading them. They are downloaded based on a priority stored in the file and by generation time. DpCatalog also has commands to reprioritize and delete data products.

NOTE: This release has an early prototype that will do the following:

1. Reads the data product files from the specified directory.
2. Puts them in priority sorted list.
3. Sends requests to Svc/FileDownlink to downlink each file in the list.
4. Stop transmissions in progress
5. Allow insertion of new products after the catalog is built
6. Clear the catalog via command

It does not:

1. Send data product files in pieces. It only sends whole files, so if the downlink is interrupted, the whole file will have to be retransmitted.

Features and more extended unit testing will be added over time. Use at your own risk!

2 Requirements

3 Design

3.1 Assumptions

The design of DpCatalog assumes the following:

1. A file system exists to store the data product files.
2. The contents of the data product files match the data product specification.
3. The file downlink will acknowledge completion of each file

3.3 Ports

3.3.1 Role Ports

These ports will be automatically connected in the topology to F Prime services.

3.3.2 Component-Specific Ports

3.4 Constants

DpCatalog can be statically configured with the following constants:

These constants are located in DpCatalogCfg.hpp in the config directory.

3.5 Configuration

During initialization, the configuration function takes a set of parameters:

        void configure(
            Fw::FileNameString directories[DP_MAX_DIRECTORIES],
            FwSizeType numDirs,
            Fw::FileNameString& stateFile,
            FwEnumStoreType memId,
            Fw::MemAllocator& allocator
        );

3.6 Commands

Sequence of Commands

When the software is first started, the catalog data structure is empty. The catalog must be built before starting downlink. The BUILD_CATALOG command was implemented separately from the START_XMIT_CATALOG command to start downlinking so the software could build the catalog prior to a communication session and execute the downlink during communication. Downlink can be halted by issuing the STOP_XMIT_COMMAND in the middle of the downlink. If for some reason the state in the state file, the contents of the tree and the data products get out of sync, a CLEAR_CATALOG command can be issued. Then the BUILD_CATALOG command can be invoked to rebuild the state based on the existing set of data product files only. This will caused downlinked state to be lost, so data products not deleted after downlink would be readded to the pending list of downlinks.

3.7 Algorithms

3.7.1 Data Product Sorting

Overall priority is determined by comparing two records' priority, timestamp, and id in that order with lower numerical values holding higher priority (see DpCatalog::DpStateEntry::compareEntries). The following logic is implemented:

1. Data Product Priority - Data products are generated with a priority, where the lower the number the higher the priority.
2. Data Product Generation Time - If priorities are the same, the older data is prioritized over the newer.
3. Data Product ID - If priorities and time are the same (highly unlikely), then lower IDs are prioritized first.

3.7.2 Reading Files

The initialize() function is provided an array of directories where data product files are generated by Svc/DpWriter. When the BUILD_CATALOG command is executed, the headers of the data product files are read and the metadata in their headers is processed and stored as a data structure for sorting. The file name is not stored to conserve memory.

3.7.2 Sorting Algorithm

The data products are sorted using an unbalanced, non-recursive binary tree algorithm. The node to be inserted is placed into the tree according to its priority. Higher priority nodes are left children, while lower (and unlikely, equal) priority nodes are placed as right children. No rotations or rebalancing is performed, so all new nodes will be initially be leaves.

3.7.2 Tree Traversal for Downlink

When data products are downlinked, the tree is traversed in priority order. As each node is visited, the file is downlinked and the node is removed from the tree upon completion. The current node to be explored is always checked for left children to ensure the highest priority node is transmitted. Afterward, the right child and, ultimately, the parent are transmitted. If a node is added to the tree with a higher priority than the node to be explored, the added node becomes the node to be explored.

3.7.3 State File

When a data product is downlinked, it is marked in the node as completed, but the state is also written to a file so that downlinked state is preserved across restarts of the software. When the catalog is built, the state file is first read into a data structure in memory.

6 Unit Testing