rfc/20250211-s3-locking-with-conditional-writes.md
Issue: https://github.com/opentofu/opentofu/issues/599
Considering the request from the ticket above, and the newly AWS released S3 feature, we can now have state locking without relying on DynamoDB.
The main reasons for such a change could be summarized as follows:
The most important things that need to be handled during this implementation:
backend block configuration contains no attributes related to the new locking option.Until recently, most of the approaches that could have been taken for this implementation, could have been prone to data races. But AWS has released new functionality for S3, supporting conditional writes on objects in any S3 bucket.
For more details on the AWS S3 feature and the way it works, you can read more on the official docs.
By using conditional writes, you can add an additional header to your WRITE requests to specify preconditions for your Amazon S3 operation. To conditionally write objects, add the HTTP
If-None-MatchorIf-Matchheader.The
If-None-Matchheader prevents overwrites of existing data by validating that there's not an object with the same key name already in your bucket.Alternatively, you can add the
If-Matchheader to check an object's entity tag (ETag) before writing an object. With this header, Amazon S3 compares the providedETagvalue with theETagvalue of the object in S3. If theETagvalues don't match, the operation fails.
To allow this in OpenTofu, the backend "s3" will receive one new attribute:
use_lockfile bool (Default: false) - Flag to indicate if the locking should be performed by using strictly the S3 bucket.[!NOTE]
The name
use_lockfilewas selected this way to keep the feature parity with Terraform.
In order to make use of this new feature, the users will have to add the attribute in the backend block:
terraform {
backend "s3" {
bucket = "tofu-state-backend"
key = "statefile"
region = "us-east-1"
dynamodb_table = "tofu_locking"
use_lockfile = true
}
}
use_lockfile exists and dynamodb_table is missing, OpenTofu will try to acquire the lock inside the configured S3 bucket.use_lockfile will exist alongside dynamodb_table, OpenTofu will:
The usage of workspaces will not impact this new way of locking. The locking object will always be stored right next to its related state object.
[!NOTE]
OpenTofu recommends to have versioning enabled for the S3 buckets used to store state objects.
Acquiring and releasing locks will add a good amount of writes and reads to the bucket. Therefore, for a versioning-enabled S3 bucket, the number of versions for that object could grow significantly. Even though the cost should be negligible for the locking objects, any user using this feature could consider configuring the lifecycle of the S3 bucket to limit the number of versions of an object.
[!WARNING]
When OpenTofu S3 backend is used with an S3 compatible provider, it needs to be checked that the provider supports conditional writes in the same way AWS S3 is offering.
In this case, the user can just add the new use_lockfile=true and run tofu init -reconfigure.
In case the user has DynamoDB enabled, there are two paths forward:
use_lockfile=true and run tofu init -reconfigure
dynamodb_table attribute and run tofu init -reconfigure again.use_lockfile=true, remove the dynamodb_table one and run tofu init -reconfigure
OpenTofu recommends to have both locking mechanisms enabled for a limited amount of time and later remove the DynamoDB locking. This is to ensure that the changes pushed upstream will be propagated into all of your places that the configuration could be executed from.
In order to achieve and ensure a proper state locking via S3 bucket, we want to attempt to create the locking object only when it is missing.
In order to do so we need to call s3client.PutObject with the property IfNoneMatch: "*".
For more information, please check the official documentation.
But the simplified implementation would look like this:
input := &s3.PutObjectInput{
Bucket: aws.String(bucket),
Key: aws.String(key),
Body: bytes.NewReader([]byte(lockInfo)),
IfNoneMatch: aws.String("*"),
}
_, err := actor.S3Client.PutObject(ctx, input)
The err returned above should be handled accordingly with the behaviour defined in the official docs:
[!NOTE] Right now, when locking is enabled on DynamoDB, at the moment of updating the state object content, OpenTofu also writes an entry in DynamoDB with the MD5 sum of the state object. The reason is to be able to check the integrity of the state object from the S3 bucket in a future run. This is done by reading the digest from DynamoDB and comparing it with the ETag attribute of the state object from S3.
By moving to the S3 based locking, OpenTofu will store no other file for the digest of the state object. This was a mechanism to validate the state object integrity when the lock was stored in DynamoDB. More info about this topic can be found on the official documentation.
But if both locks are enabled (use_lockfile=true and dynamodb_table=<actual_table_name>), the state digest will still be stored in DynamoDB.
[!WARNING]
By enabling the S3 locking and disabling the DynamoDB one, the digest from DynamoDB will become stale. This means that if it is desired to go back to the DynamoDB locking, the digest needs to be cleaned up or updated in order to allow the content integrity check to work.
Later, DynamoDB based locking might be considered for deprecation once (at least) the following are true:
Since this new feature relies on the S3 conditional writes, there is hardly other reliable alternative to implement this.