third_party/blink/renderer/modules/webaudio/docs/tail-processing.md
The WebAudio API has a concept of a
tail-time in
which an AudioNode must continue to process even if the input(s) to
the node are disconnected. For example, if you have a DelayNode with
a delay of 1 sec, the node must continue to output data for at least 1
sec to flush out all the delayed data.
To implement this, we introduce the TailTime and LatencyTime
methods for each node. This isn't required in the spec, but the
implementation makes this distinction. (Not sure why). TailTime is
how long it takes before a node that has silent input will produce
silent output. LatencyTime is how long it takes non-silent input to
produce non-silent output.
The sum of these two terms tells us how long a node needs to process to flush out its internal state.
To support tail processing several routines are used:
PropagatesSilence: returns true if the node is producing silence
after knowing the inputs are silent
RequiresTailProcessing: returns true if the node needs tail processing. For example, GainNode`s have no memory so tail
processing is not needed.
The actual details are a bit complicated, but basically
AudioNode::ProcessIfNecessary manages this. It keeps track of when
the node was last non-silent. If the inputs are silent and
PropagatesSilence returns false, the nodes Process method is still
called.
For most nodes, PropagatesSilence checks to see if the last
non-silent time plus the TailTime plus the LatencyTime is greater
than the context currentTime. If so, then the node produces silence
now because the internal state has been flushed out.
There is another way to start tail processing, and this is the more
difficult case that we need to handle. Consider an OscillatorNode
connected to a DelayNode. When the OscillatorNode stops, it
disables its output, basically marking its output as silent. This
normally propagates down the graph disabling the output of each node.
However, the DelayNode needs to continue processing. The logic for
tail processing is in AudioNode::DisableOutputsIfNecessary. If
RequiresTailProcessing() returns true, this node is added to the
tail processing handler list (tail_processing_handlers_) via
DeferredTaskHandler::AddTailProcessingHandler(). If not, the output
of the node is disabled which propagates through the downstream
nodes.
Then during the beginning and end of each render quantum, we check the
tail processing list to see if the handler would be silent (via
PropagatesSilence()). If it would produce silence, it's removed from
the list. Otherwise, nothing is done.
Note also that if a connection is made to the node, it is removed from the tail processing list since it's not processing the tail anymore.
Because WebAudio has two threads: the main thread and the audio
rendering thread, things are a bit more complicated. Removing a
handler from tail processing cannot be done on the audio thread
because it requires changing the state of the output. Thus, when this
happens, the handler is removed from the tail processing list and
placed on the finished_tail_processing_handlers_ list. At Each
render quantum, a task is posted to the main thread to update the
output state.
In addition, we had to make some guesses on the tail time for
BiquadFilterNodes and IIRFilterNodes. In theory, these have an
infinite tail since the both of thse are infinite impulse response
filters. In practice, we don't really want the TailTime to be
infinite because then the nodes basically never go away.
For an IIRFilterNode, we actually compute the impulse response and
determine approximately where the impulse response is low enough to
say it is done. We also arbirarily limit the maximum value, just to
prevent huge tail times.
For a BiquadFilterNode, we don't compute the impulse response
because automations of the filter parameters can change the tail.
Instead, we determine the poles of the filter and from that determine
roughly the analytical impulse response. From the response, we
determine when the response is low enough to say the tail is done.
Like the IIRFIlterNode, this is limited to a max value.
For a DelayNode, we just set the tail time to be the max delay value
for the node instead of trying to determine a tail time from the
actual delayTime parameter.
DeferredTaskHandler::tail_processing_handlers_DeferredTaskHandler::finished_tail_processing_handlers_AudioNodeHandler::ProcessIfNecessaryAudioNodeHandler::EnableOutputsIfNecessaryAudioNodeHandler::TailTimeAudioNodeHandler::LatencyTimeAudioNodeHandler::PropagatesSilenceAudioNodeHandler::RequiresTailProcessingDeferredTaskHandler::AddTailProcessingHandlerDeferredTaskHandler::RemoveTailProcessingHandlerDeferredTaskHandler::UpdateTailProcessingHandlers