docs/notes/framegraph.html
FrameGraph is a framework within Filament for computing resources needed to render a frame. The framework enables declaring dependencies between resources.
For example, when rendering shadows, we would need to first compute and store the shadow map into a texture resource, and then the later color pass would then sample that texture to attenuate the final output color. That creates a dependency on the shadow map from the color pass. Filament uses FrameGraph to declare that dependency.
The core of this framework is a class that defines a dependency graph — that is, the class defines nodes and connections between nodes. This class makes assumptions about the types of its nodes. Like many other classes within Filament, this class is without virtual function declaration to avoid paying the cost of virtual calls.
This class has additional functions to detect whether there is a cycle in the graph, and it is able to cull unreachable nodes.
A frame graph consists of two types of nodes
Resource
Pass
Edges can be created in the following three directions:
To better understand FrameGraph, we consider the following graphical representation of a real graph. In this graph, blue nodes denote "Resources" and orange nodes denote "Passes."
In this graph, we see that the "Color Pass" takes as input the "Shadowmap", which has edges going into it, meaning that it's a texture array. The output of the "Color Pass" are "viewRenderTarget" and "Depth Buffer."
Note that there is an outgoing edge from "viewRenderTarget", where the color buffer will be used as input in the post-processing passes. But since "Depth Buffer" is not relevant to the rest of the rendering, it does not have an outgoing edge.
Since the graph is guaranteed to be acyclic, we can produce a dependency-respecting ordering of the nodes by traversal of the graph (e.g. topological sort).
We take a snippet of in production code to look through the details of building a graph.
struct StructurePassData {
FrameGraphId<FrameGraphTexture> depth;
FrameGraphId<FrameGraphTexture> picking;
};
...
// generate depth pass at the requested resolution
auto& structurePass = fg.addPass<StructurePassData>("Structure Pass",
[&](FrameGraph::Builder& builder, auto& data) {
bool const isES2 = mEngine.getDriverApi().getFeatureLevel() == FeatureLevel::FEATURE_LEVEL_0;
data.depth = builder.createTexture("Structure Buffer", {
.width = width, .height = height,
.levels = uint8_t(levelCount),
.format = isES2 ? TextureFormat::DEPTH24 : TextureFormat::DEPTH32F });
// workaround: since we have levels, this implies SAMPLEABLE (because of the gl
// backend, which implements non-sampleables with renderbuffers, which don't have levels).
// (should the gl driver revert to textures, in that case?)
data.depth = builder.write(data.depth,
FrameGraphTexture::Usage::DEPTH_ATTACHMENT | FrameGraphTexture::Usage::SAMPLEABLE);
if (config.picking) {
data.picking = builder.createTexture("Picking Buffer", {
.width = width, .height = height,
.format = isES2 ? TextureFormat::RGBA8 : TextureFormat::RG32F });
data.picking = builder.write(data.picking,
FrameGraphTexture::Usage::COLOR_ATTACHMENT);
}
builder.declareRenderPass("Structure Target", {
.attachments = { .color = { data.picking }, .depth = data.depth },
.clearFlags = TargetBufferFlags::COLOR0 | TargetBufferFlags::DEPTH
});
},
[=, renderPass = pass](FrameGraphResources const& resources,
auto const&, DriverApi&) mutable {
Variant structureVariant(Variant::DEPTH_VARIANT);
structureVariant.setPicking(config.picking);
auto out = resources.getRenderPassInfo();
renderPass.setRenderFlags(structureRenderFlags);
renderPass.setVariant(structureVariant);
renderPass.appendCommands(mEngine, RenderPass::CommandTypeFlags::SSAO);
renderPass.sortCommands(mEngine);
renderPass.execute(mEngine, resources.getPassName(), out.target, out.params);
}
);
The addPass method creates a node and it take in two lambda functions as its parameter. The first lambda sets up the resources that will be used in the execution of the Pass. This lambda is executed immediately and synchronously when addPass is called. The second lambda is the actual execution of the pass; it is executed when the graph has been completed and is traversed.
In the above, we see through a graph and code what a frame graph looks like and how to build it. We provide here a more detailed description of what it does:
Manages the lifetime of the resources
Calculates the usage bit of the texture resource
Calculates the load/store bits of the rendertargets within a renderpass.
In a previous version of FrameGraph, there were only edges between Resource and Pass. For example, a Pass and Pass edge would not make logical sense. The following iteration, allowed for edges between two Resource nodes to indicate that one is a subresource of another (i.e. a layer in a mip-mapped texture).
There are two extra features of FrameGraph that are important but has a lot subtlety, and, incidentally, their inclusion added great complexity to the implementation
Future Work
matdbg."RenderGraph" might be a more fitting name for this framework.