docs/classtf_1_1DataPipeline.html
class to create a data-parallel pipeline scheduling framework
| Template parameters |
|---|
| Ps |
Similar to tf::Pipeline, a tf::DataPipeline is a composable graph object for users to create a data-parallel pipeline scheduling framework using a module task in a taskflow. The only difference is that tf::DataPipeline provides a data abstraction for users to quickly express dataflow in a pipeline. The following example creates a data-parallel pipeline of three stages that generate dataflow from void to int, std::string, and void.
#include \<taskflow/taskflow.hpp\>#include \<taskflow/algorithm/data\_pipeline.hpp\>int main() {// data flow =\> void -\> int -\> std::string -\> voidtf::Taskflow taskflow("pipeline");tf::Executor executor;const size\_t num\_lines = 4;tf::DataPipeline pl(num\_lines,tf::make\_data\_pipe\<void, int\>(tf::PipeType::SERIAL, [&](tf::Pipeflow& pf) -\> int{if(pf.token() == 5) {pf.stop();return 0;}else {return pf.token();}}),tf::make\_data\_pipe\<int, std::string\>(tf::PipeType::SERIAL, [](int& input) {return std::to\_string(input + 100);}),tf::make\_data\_pipe\<std::string, void\>(tf::PipeType::SERIAL, [](std::string& input) {std::cout \<\< input \<\< std::endl;}));// build the pipeline graph using compositiontaskflow.composed\_of(pl).name("pipeline");// dump the pipeline graph structure (with composition)taskflow.dump(std::cout);// run the pipelineexecutor.run(taskflow).wait();return 0;}
The pipeline schedules five tokens over four parallel lines in a circular fashion, as depicted below:
o -> o -> o|||v v v
o -> o -> o|||v v v
o -> o -> o|||v v v
o -> o -> o
using data_t = unique_variant_t<std::variant<std::conditional_t<std::is_void_v<typename Ps::output_t>, std::monostate, std::decay_t<typename Ps::output_t>>...>> internal storage type for each data token (default std::variant)
DataPipeline(size_t num_lines, Ps && ... ps)constructs a data-parallel pipeline objectDataPipeline(size_t num_lines, std::tuple<Ps...>&& ps)constructs a data-parallel pipeline object
auto num_lines() const -> size_t noexceptqueries the number of parallel linesauto num_pipes() const -> size_t constexpr noexceptqueries the number of pipesvoid reset()resets the pipelineauto num_tokens() const -> size_t noexceptqueries the number of generated tokens in the pipelineauto graph() -> Graph&obtains the graph object associated with the pipeline construct
constructs a data-parallel pipeline object
| Parameters |
|---|
| num_lines |
| ps |
Constructs a data-parallel pipeline of up to num_lines parallel lines to schedule tokens through the given linear chain of pipes. The first pipe must define a serial direction (tf::PipeType::SERIAL) or an exception will be thrown.
constructs a data-parallel pipeline object
| Parameters |
|---|
| num_lines |
| ps |
Constructs a data-parallel pipeline of up to num_lines parallel lines to schedule tokens through the given linear chain of pipes stored in a std::tuple. The first pipe must define a serial direction (tf::PipeType::SERIAL) or an exception will be thrown.
queries the number of parallel lines
The function returns the number of parallel lines given by the user upon the construction of the pipeline. The number of lines represents the maximum parallelism this pipeline can achieve.
queries the number of pipes
The Function returns the number of pipes given by the user upon the construction of the pipeline.
resets the pipeline
Resetting the pipeline to the initial state. After resetting a pipeline, its token identifier will start from zero as if the pipeline was just constructed.
queries the number of generated tokens in the pipeline
The number represents the total scheduling tokens that has been generated by the pipeline so far.
obtains the graph object associated with the pipeline construct
This method is primarily used as an opaque data structure for creating a module task of this pipeline.