ADRs/0021 - Create View.md
Discussion
Proposed by: Adam Gibson (27th April 2022) Discussed with: Paul Dubs Finalized by: Adam Gibson (5th May 2022)
Samediff's op graph mainly focuses on immutability to prevent bugs making many variable inputs and outputs for various ops copy on write.
This prevents performance gains with in place ops such as +=.
Currently, the samediff strided_slice operation works to provide a view of an array but the view is a copy. This limitation prevents performance gains you typically see in views.
CreateView is an op that takes in a set of SDVariables that represents index information similar to nd4j's point, interval,all, and new axis. This op allows for dynamic generation of views of variables.
CreateView is a building block for other ops to execute in place operations. Usage of CreateView should be deliberate and only used in certain circumstances. CreateView simply (using the aforementioned index inputs) creates a view using the existing data buffer and returns an output that wraps the same exact buffer as is rendered as an alternative view in a similar way as nd4j's indexing mechanisms.
These inputs are represented as follows:
This describes the general pattern the above described buffers follow:
Of note here are a few constants representing types to be passed to the ops:
These are created as INT64 ndarrays passed in to the operation itself.
An omission of indexing here is SpecifiedIndex. Since SpecifiedIndex requires a copy most of the time, this op will mainly be focused on indexing that is guaranteed to use the same buffer.
Usually, arrays during training should not modify their outputs. Instead, new output arrays are allocated with calculated results being inserted into these pre-defined outputs.
However, CreateView is, by definition, special since it is a building block for enabling manipulation of a view of the same data buffer as the input. The gradient checks in the NonInplaceValidationListener make an exception to this rule to account for this particular behaviour.