docs/provider-design.md
The Terraform AWS Provider follows the guidelines established in the HashiCorp Provider Design Principles. That general documentation provides many high-level design points gleaned from years of experience with Terraform's design and implementation concepts. Sections below will expand on specific design details between that documentation and this provider, while others will capture other pertinent information that may not be covered there. Other pages of the contributing guide cover implementation details such as code, testing, and documentation specifics.
The AWS provider implements support for the AWS service APIs using the AWS Go SDK. The API and SDK limits extend to the provider. In general, SDK operations manage the lifecycle of AWS components, such as creating, describing, updating, and deleting a database. Operations do not usually handle functionality within those components, such as executing a query on a database. If you are interested in other APIs/SDKs, we invite you to view the many Terraform Providers available, as each has a community of domain expertise.
Some examples of functionality that is not expected in this provider:
The provider maintainers' design goal is to cover as much of the AWS API as pragmatically possible. However, not every aspect of the API is compatible with an infrastructure-as-code (IaC) conception. Specifically: IaC is best suited for immutable rather than mutable infrastructure -- i.e. for resources with a single desired state described in its entirety, as opposed to resources defined via a dynamic process.
If such limits affect you, we recommend that you open an AWS Support case and encourage others to do the same. Request that AWS components be made more self-contained and compatible with IaC. These AWS Support cases can also yield insights into the AWS service and API that are not well documented.
Terraform resources work best as the smallest infrastructure blocks on which practitioners can build more complex configurations and abstractions, such as Terraform Modules. The general heuristic guiding when to implement a new Terraform resource for an aspect of AWS is whether the AWS service API provides create, read, update, and delete (CRUD) operations. However, not all AWS service API functionality falls cleanly into CRUD lifecycle management. In these situations, there is extra consideration necessary for properly mapping API operations to Terraform resources.
This section highlights design patterns when considering an implementation within a singular Terraform resource or as separate Terraform resources.
Please note: the overall design and implementation across all AWS functionality is federated: individual services may implement concepts and use terminology differently. As such, this guide is not exhaustive. The aim is to provide general concepts and basic terminology that point contributors in the right direction, especially in understanding previous implementations.
Some AWS services use an authorization-acceptance model for cross-account associations or access. Examples include:
Depending on the API and components, AWS uses two basic ways of creating cross-region and cross-account associations. One way is to generate an invitation (or proposal) identifier from one AWS account to another. Then in the other AWS account, that identifier is used to accept the invitation. The second way is configuring a reference to another AWS account identifier. These may not require explicit acceptance on the receiving account to finish creating the association or begin working.
To model creating an association using an invitation or proposal, follow these guidelines.
To model the second type of association, implicit associations, create an "association" resource and, optionally, an "authorization" resource. Map create, read, and delete to the corresponding operations in the AWS service API.
Many AWS service APIs build on top of other AWS services. Some examples of these include:
Some cross-service API implementations lack the management or description capabilities of the other service. The lack can make the Terraform resource implementation seem incomplete or unsuccessful in end-to-end configurations. Given the overall “resources should represent a single API object” goal from the HashiCorp Provider Design Principles, a resource must only communicate with a single AWS service API. As such, maintainers will not approve cross-service resources.
The rationale behind this design decision includes the following:
A poignant real-world example of the last point involved a Lambda resource. The resource helped clean up extra resources (ENIs) due to a common misconfiguration. Practitioners found the functionality helpful since the issue was hard to diagnose. Years later, AWS updated the Lambda API. Immediately, practitioners reported that Terraform executions were failing. Downgrading the provider was not possible since many configurations depended on recent releases. For environments running many versions behind, forcing an upgrade with the fix would likely cause unrelated and unexpected changes. In the end, HashiCorp and AWS performed a large-scale outreach to help upgrade and fix the misconfigurations. Provider maintainers and practitioners lost considerable time.
A separate class of Terraform resource types are data sources. These are typically intended as a configuration method to lookup or fetch data in a read-only manner. Data sources should not have side effects on the remote system.
When discussing data sources, they are typically classified by the intended number of return objects or data. Singular data sources represent a one-to-one lookup or data operation. Plural data sources represent a one-to-many operation.
These data sources are intended to return zero, one, or many results, usually associated with a managed resource type. Typically results are a set unless ordering guarantees are provided by the remote system. These should be named with a plural suffix (e.g., s or es) and should not include any specific attribute in the naming (e.g., prefer aws_ec2_transit_gateways instead of aws_ec2_transit_gateway_ids).
These data sources are intended to return one result or an error. These should not include any specific attribute in the naming (e.g., prefer aws_ec2_transit_gateway instead of aws_ec2_transit_gateway_id).
For some AWS components, the AWS API allows specifying an IAM resource-based policy, the IAM policy to associate with a component. Some examples include:
Provider developers should implement this capability in a new resource rather than adding it to the associated resource. Reasons for this include:
One rare exception to this guideline is where the policy is required during resource creation.
The AWS API provides the ability to start, stop, enable, or disable some AWS components. Some examples include:
In this situation, provider developers should implement this ability within the resource instead of creating a separate resource. Since a practitioner cannot practically manage interaction with a resource's states in Terraform's declarative configuration, developers should implement the state management in the resource. This design provides consistency and future-proofing even where updating a resource in the current API is not problematic.
Some AWS operations are asynchronous. Terraform requests that AWS perform a task. Initially, AWS only notifies Terraform that it received the request. Terraform then requests the status while awaiting completion. Examples of this include:
In this situation, provider developers should create a separate resource representing the task, assuming that the AWS service API provides operations to start the task and read its status. Adding the task functionality to the parent resource muddies its infrastructure-management purpose. The maintainers prefer this approach even though there is some duplication of an existing resource. For example, the provider has a resource for copying an EC2 AMI in addition to the EC2 AMI resource itself. This modularity allows practitioners to manage the result of the task resource with another resource.
For a related consideration, see the Managing Resource Running State section.
AWS supports having multiple versions of some components. Examples of this include:
In general, provider developers should create a separate resource to represent a single version. For example, the provider has both the aws_secretsmanager_secret and aws_secretsmanager_secret_version resources. However, in some cases, developers should handle versioning in the main resource.
In deciding when to create a separate resource, follow these guidelines:
triggers attribute. If this changes in the future, then this guidance may be updated towards separate resources, following the Task Execution and Waiter Resources guidance.In the interest of security, the maintainers will not approve data sources that provide the ability to reference or export the AWS credentials of the running provider. There are valid use cases for this information, such as executing AWS CLI calls as part of the same Terraform configuration. However, this mechanism may allow credentials to be discovered and used outside of Terraform. Some specific concerns include: