Back to Delta

Protocol RFCs

protocol_rfcs/README.md

4.2.07.3 KB
Original Source

Protocol RFCs

This directory contains information about the process of making Delta protocol changes via RFCs and all the RFCs that have been proposed since this process was adopted.

Table of RFCs

Here is the history of all the RFCs propose/accepted/rejected since Feb 6, 2024, when this process was introduced.

Proposed RFCs

Date proposedRFC fileGithub issueRFC title
2023-02-26column-mapping-usage.tracking.mdhttps://github.com/delta-io/delta/issues/2682Column Mapping Usage Tracking
2023-04-24variant-type.mdhttps://github.com/delta-io/delta/issues/2864Variant Data Type
2024-04-30collated-string-type.mdhttps://github.com/delta-io/delta/issues/2894Collated String Type
2025-03-13checkpoint-protection.mdhttps://github.com/delta-io/delta/issues/4152Checkpoint Protection
2025-03-18iceberg-writer-compat-v1.mdhttps://github.com/delta-io/delta/issues/4284IcebergWriterCompatV1
2025-05-06variant-shredding.mdhttps://github.com/delta-io/delta/issues/4032Variant Shredding
2025-11-20materialize-partition-columns.mdhttps://github.com/delta-io/delta/issues/5555Materialize Partition Columns

Accepted RFCs

Date proposedDate acceptedRFC fileGithub issueRFC title
2025-04-072026-02-17catalog-managed.mdhttps://github.com/delta-io/delta/issues/4381Catalog-Managed Tables
2023-02-282023-03-26vacuum-protocol-check.mdhttps://github.com/delta-io/delta/issues/2630Enforce Vacuum Protocol Check
2023-02-022023-07-24in-commit-timestamps.mdhttps://github.com/delta-io/delta/issues/2532In-Commit Timestamps
2023-02-092025-01-28type-widening.mdhttps://github.com/delta-io/delta/issues/2623Type Widening

Rejected RFCs

Date proposedDate rejectedRFC fileGithub issueRFC title
2023-02-142025-04-07managed-commits.mdhttps://github.com/delta-io/delta/issues/2598Managed Commits

RFC process

1. Make initial proposal

Create a Github issue of type [Protocol Change Request].

  • The description of the issue may have links to design docs, etc.
  • This issue will serve as the central location for all discussions related to the protocol change.
  • If the proposal comes with a prototype or other pathfinding, the changes should be in an open PR.

2. Add the RFC doc

After creating the issue and discussing with the community, if a basic consensus is reached that this feature should be implemented, then create a PR to add the protocol RFC before merging code in master.

  • Clone the RFC template template.md and create a new RFC markdown doc.
  • Cross-link with the issue with "see #xxx". DONT USE "closes #xxx" or "fixes #xxx" or "resolves #xxx" because we don't want the issue to be closed when this RFC PR is merged.

Note:

  • For table features, it is strongly recommended that any experimental support for the feature uses a temporary feature name with a suffix like -dev. This will communicate to the users that are about to use experimental feature with no future compatibility guarantee.
  • Code related to a proposed feature should not be merged into the main branch until the RFC attains "proposed" status (that is, the RFC PR has been through public review and merged). Until the RFC has been accepted (that is, the proposed changes have been merged into the Delta specification), any code changes should be isolated from production code behind feature flags, etc. so that existing users are not affected in any way.

3. Finally, accept or reject the RFC

For a RFC to be accepted, it must satisfy the following criteria:

  • There is a production implementation (for example, in delta-spark) of the feature that has been thoroughly well tested.
  • There is at least some discussion and/or prototype (preferred) that ensure the feasibility of the feature in Delta Kernel.

When the success criteria are met, then the protocol can be finalized by making a PR to make the following changes:

  • Closely validate that the protocol spec changes are actually consistent with the production implementation.
  • Cross-link the PR with the original issue with "closes #xxx" as now we are ready to close the issue. In addition, update the title of the issue to say [ACCEPTED] to make it obvious how the proposal was resolved.
  • Update protocol.md.
  • Move the RFC doc to the accepted subdirectory, and update the state in index.md.
  • Remove the temporary/preview suffix like -dev in the table feature name from all the code.

However, if the RFC is to be rejected, then make a PR to do the following changes:

  • Cross-link the PR with the original issue with "closes #xxx" as now we are ready to close the issue. In addition, update the title of the issue to say [REJECTED] to make it obvious how the proposal was resolved.
  • Move the RFC doc to the rejected subdirectory.
  • Update the state in index.md.
  • Remove any experimental/preview code related to the feature.