docs/adr/0002-api-enhancement-drift-detection.md
Date: 2025-01-21
Proposed
The current Atlantis API (/api/plan, /api/apply, /api/locks) provides basic functionality for triggering Terraform operations but has several issues that need addressing:
error types serialize as {} or null in JSON responses instead of the actual error messageapproved and mergeable requirement checks, preventing drift detection use casesPR #6093 (Error Serialization):
{} instead of error messagesProjectResult from embedding ProjectCommandOutput to a named field"ProjectCommandOutput": {...}MarshalJSON on ProjectResult that handles the embedded struct while converting errors to stringsPR #6095 (API Non-PR Support):
approved/mergeable requirementsAPI flag to ProjectContext and skip PR-specific requirements when ctx.API && ctx.Pull.Num == 0| Endpoint | Method | Auth Required | Description |
|---|---|---|---|
/api/plan | POST | Yes | Trigger terraform plan |
/api/apply | POST | Yes | Trigger terraform apply |
/api/locks | GET | No | List active locks |
/status | GET | No | Server status |
/readyz | GET | No | Readiness check |
/healthz | GET | No | Health check |
{
"Error": null,
"Failure": "",
"ProjectResults": [
{
"Error": null, // BUG: Shows {} when error exists
"Failure": "",
"PlanSuccess": {...},
"PolicyCheckResults": null,
"ApplySuccess": "",
"VersionSuccess": "",
"ImportSuccess": null,
"StateRmSuccess": null,
"Command": 1,
"SubCommand": "",
"RepoRelDir": ".",
"Workspace": "default",
"ProjectName": "",
"SilencePRComments": null
}
],
"PlansDeleted": false
}
We will enhance the Atlantis API in five phases, implementing the fixes from PRs #6093 and #6095 properly as part of a comprehensive design.
Goal: Fix error serialization WITHOUT changing the API response structure.
Problem Analysis:
The current ProjectResult struct embeds ProjectCommandOutput:
type ProjectResult struct {
ProjectCommandOutput // embedded
Command Name
// ...
}
PR #6093's approach changes this to a named field, which changes JSON output from flat fields to nested ProjectCommandOutput: {...} - a breaking change.
Correct Implementation:
Add MarshalJSON to ProjectResult that:
// server/events/command/project_result.go
// MarshalJSON implements custom JSON marshaling to properly serialize the Error field
// while maintaining backwards compatibility with the existing API response structure.
func (p ProjectResult) MarshalJSON() ([]byte, error) {
// Convert error to string pointer for proper serialization
var errMsg *string
if p.Error != nil {
msg := p.Error.Error()
errMsg = &msg
}
// Use anonymous struct to control JSON output
// This preserves the flat structure (no ProjectCommandOutput wrapper)
return json.Marshal(&struct {
Error *string `json:"Error"`
Failure string `json:"Failure"`
PlanSuccess *models.PlanSuccess `json:"PlanSuccess"`
PolicyCheckResults *models.PolicyCheckResults `json:"PolicyCheckResults"`
ApplySuccess string `json:"ApplySuccess"`
VersionSuccess string `json:"VersionSuccess"`
ImportSuccess *models.ImportSuccess `json:"ImportSuccess"`
StateRmSuccess *models.StateRmSuccess `json:"StateRmSuccess"`
Command Name `json:"Command"`
SubCommand string `json:"SubCommand"`
RepoRelDir string `json:"RepoRelDir"`
Workspace string `json:"Workspace"`
ProjectName string `json:"ProjectName"`
SilencePRComments []string `json:"SilencePRComments"`
}{
Error: errMsg,
Failure: p.Failure,
PlanSuccess: p.PlanSuccess,
PolicyCheckResults: p.PolicyCheckResults,
ApplySuccess: p.ApplySuccess,
VersionSuccess: p.VersionSuccess,
ImportSuccess: p.ImportSuccess,
StateRmSuccess: p.StateRmSuccess,
Command: p.Command,
SubCommand: p.SubCommand,
RepoRelDir: p.RepoRelDir,
Workspace: p.Workspace,
ProjectName: p.ProjectName,
SilencePRComments: p.SilencePRComments,
})
}
// MarshalJSON for Result to properly serialize top-level Error
func (c Result) MarshalJSON() ([]byte, error) {
type Alias Result
var errMsg *string
if c.Error != nil {
msg := c.Error.Error()
errMsg = &msg
}
return json.Marshal(&struct {
Error *string `json:"Error"`
*Alias
}{
Error: errMsg,
Alias: (*Alias)(&c),
})
}
Why This Approach:
Files to Modify:
server/events/command/project_result.go - Add MarshalJSON methodsserver/events/command/result.go - Add MarshalJSON methodserver/events/command/result_test.go - Add comprehensive testsGoal: Allow API operations without PR context for drift detection use cases.
Implementation (based on PR #6095 with enhancements):
API flag to ProjectContext:// server/events/command/project_context.go
type ProjectContext struct {
// ... existing fields ...
// API is true if this command was triggered via API endpoint rather than PR comment.
// When true and Pull.Num == 0, PR-specific requirements (approved, mergeable) are skipped.
API bool
}
// server/events/project_command_context_builder.go
func newProjectCommandContext(...) command.ProjectContext {
return command.ProjectContext{
// ... existing fields ...
API: ctx.API,
}
}
// server/events/command_requirement_handler.go
func (a *DefaultCommandRequirementHandler) validateCommandRequirement(...) (string, error) {
for _, req := range requirements {
switch req {
case raw.ApprovedRequirement:
// Skip for API calls without PR - approval is PR-specific
if ctx.API && ctx.Pull.Num == 0 {
ctx.Log.Info("skipping approval requirement for API call without PR number")
continue
}
// ... existing approval check ...
case raw.MergeableRequirement:
// Skip for API calls without PR - mergeability is PR-specific
if ctx.API && ctx.Pull.Num == 0 {
ctx.Log.Info("skipping mergeable requirement for API call without PR number")
continue
}
// ... existing mergeable check ...
}
}
}
Design Rationale:
approved and mergeable requirements are PR-specific checks - they have no meaning outside a PR contextundivergedDependency Checks for Non-PR API Applies:
Non-PR API applies must also handle depends_on without relying on pull
request status from a VCS event. Before building apply project contexts for
ctx.API && ctx.Pull.Num == 0, seed ctx.PullStatus with a synthetic
models.PullStatus containing every selected project and its dependencies.
Each entry should use the project config's ProjectName, RepoRelDir, and
Workspace, and start as models.PlannedPlanStatus unless the current
operation already knows it has no changes.
APIController.apiApply currently runs projects in a simple loop instead of
using project_command_pool_executor, so the API apply path must update
ctx.PullStatus.Projects immediately after each project apply result. The
update should use the same Workspace, RepoRelDir, and ProjectName match
as the pool executor and assign result.PlanStatus() so later project contexts
see dependencies as AppliedPlanStatus or PlannedNoChangesPlanStatus after
earlier applies succeed. Alternatively, route non-PR API applies through the
pool executor so it owns the same status update behavior.
ValidateProjectDependencies must also return a clear failure when
ctx.PullStatus is nil for a project with dependencies, or when a declared
dependency is absent from the synthetic status. Non-PR API apply requests should
therefore include dependent projects in the same request or fail before apply
instead of panicking or treating the dependency as satisfied.
Files to Modify:
server/events/command/project_context.go - Add API fieldserver/events/project_command_context_builder.go - Propagate API flagserver/events/project_command_builder.go - Seed synthetic PullStatus for non-PR API appliesserver/controllers/api_controller.go - Update synthetic PullStatus after each API apply result or route applies through the pool executorserver/events/command_requirement_handler.go - Skip PR-specific requirementsserver/events/command_requirement_handler_test.go - Add test cases for PR-specific requirements and dependency failuresserver/events/project_command_builder_test.go - Add non-PR API apply dependency status coverageGoal: Enable infrastructure drift detection outside of PR workflows.
New Endpoints:
| Endpoint | Method | Description |
|---|---|---|
POST /api/plan | Enhanced | Support PR: 0 or omitted for drift detection |
GET /api/drift/status | New | Get drift status for repository/projects |
API Authentication:
All new /api/drift/* endpoints must enforce the same API authentication as
/api/plan and /api/apply before parsing request input or returning drift
data. Implementations should reuse the X-Atlantis-Token validation currently
performed by APIController.apiParseAndValidate, or extract that check into a
shared API-auth helper for handlers that do not accept the same request body.
The handler must reject requests when the API secret is unset or the
X-Atlantis-Token header does not match the configured secret. Do not rely on
web basic auth for these endpoints, because /api/* routes are API routes and
their handlers are responsible for API-token enforcement.
Enhanced Plan Endpoint for Drift Detection:
The existing /api/plan endpoint already supports omitting the PR field. With Phase 2's non-PR workflow support, it becomes usable for drift detection:
# Drift detection request (no PR)
curl -X POST 'https://atlantis/api/plan' \
-H 'X-Atlantis-Token: <secret>' \
-H 'Content-Type: application/json' \
-d '{
"Repository": "org/repo",
"Ref": "main",
"Type": "Github",
"Paths": [{"Directory": ".", "Workspace": "default"}]
}'
Response Enhancement for Drift Detection:
Add drift detection helpers to expose plan output as structured drift status.
These helpers must reuse models.NewPlanSuccessStats (or extend it if
Terraform/OpenTofu adds new counters) instead of adding a separate regex parser,
so drift API responses stay consistent with PR plan comments and existing
summary rendering.
// server/events/models/drift.go
// DriftSummary represents detected infrastructure drift
type DriftSummary struct {
HasDrift bool `json:"has_drift"`
ToImport int `json:"to_import"`
ToAdd int `json:"to_add"`
ToChange int `json:"to_change"`
ToDestroy int `json:"to_destroy"`
ToForget int `json:"to_forget"`
ChangesOutside bool `json:"changes_outside"`
Summary string `json:"summary"` // e.g., "2 to import, 2 to add, 1 to change, 0 to destroy, 1 to forget"
}
// DriftRepositoryKey identifies the VCS repository that owns drift status.
// VCSHostType should be Repo.VCSHost.Type.String().
type DriftRepositoryKey struct {
VCSHostType string `json:"type"`
Hostname string `json:"hostname"`
Repository string `json:"repository"`
}
// DriftProjectKey identifies a drift status record for an Atlantis project on
// a specific ref. ProjectName is optional in atlantis.yaml, so Path and
// Workspace must be part of the storage key to avoid collisions between
// unnamed projects in the same repository. Ref must also be part of the key so
// drift checks against different branches do not overwrite each other.
type DriftProjectKey struct {
Ref string `json:"ref"`
ProjectName string `json:"project_name,omitempty"`
Path string `json:"path"`
Workspace string `json:"workspace"`
}
// ParseDriftFromPlan extracts drift information from terraform plan output.
func ParseDriftFromPlan(planOutput string) DriftSummary {
stats := NewPlanSuccessStats(planOutput)
return DriftSummary{
HasDrift: stats.Changes || stats.ChangesOutside,
ToImport: stats.Import,
ToAdd: stats.Add,
ToChange: stats.Change,
ToDestroy: stats.Destroy,
ToForget: stats.Forget,
ChangesOutside: stats.ChangesOutside,
Summary: (&PlanSuccess{TerraformOutput: planOutput}).Summary(),
}
}
New Drift Status Endpoint:
// GET /api/drift/status?type=Github&hostname=github.com&repository=org/repo&ref=main
// &project=myproject&path=.&workspace=default
// type must use the canonical models.VCSHostType.String() value. Type,
// hostname, repository, and ref identify the VCS source. Project, path, and
// workspace are optional filters; path and workspace identify unnamed Atlantis
// projects. Ref is required so callers can distinguish drift runs for the same
// project on different branches.
type DriftStatusResponse struct {
Repository DriftRepositoryKey `json:"repository"`
Projects []ProjectDrift `json:"projects"`
CheckedAt time.Time `json:"checked_at"`
}
type ProjectDrift struct {
Repository DriftRepositoryKey `json:"repository"`
ProjectName string `json:"project_name,omitempty"`
Path string `json:"path"`
Workspace string `json:"workspace"`
Ref string `json:"ref"`
Drift DriftSummary `json:"drift"`
LastChecked time.Time `json:"last_checked"`
}
Drift Storage:
The status endpoint is a read endpoint, not a recomputation endpoint. Each
successful drift-detection /api/plan run must parse every project plan result
with ParseDriftFromPlan and persist one ProjectDrift record per project.
That record must include the VCS repository identity (VCS type, hostname, and
repository full name), the request source metadata (Ref), the project
identity (ProjectName, Path, Workspace), the parsed drift summary, and
LastChecked. GET /api/drift/status then filters and returns the persisted
records for the requested VCS repository/ref/project key.
The repository key must not include a separate derived RepoID; derive
models.Repo.ID() from the same hostname and repository fields only when a log
message or downstream API call needs that string.
If the implementation instead chooses to compute drift on demand, the status
endpoint request shape must change to include the required Ref, VCS type,
hostname, repository, and Paths inputs and must run a plan synchronously.
With the endpoint shape above, storage is required.
// server/core/drift/storage.go
type DriftStorage interface {
StoreDriftStatus(repo models.DriftRepositoryKey, status models.ProjectDrift) error
GetDriftStatus(repo models.DriftRepositoryKey, key models.DriftProjectKey) (*models.ProjectDrift, error)
ListDriftStatus(repo models.DriftRepositoryKey, ref string) ([]models.ProjectDrift, error)
}
// In-memory implementation for initial version
type InMemoryDriftStorage struct {
mu sync.RWMutex
status map[models.DriftRepositoryKey]map[models.DriftProjectKey]models.ProjectDrift
}
Files to Create/Modify:
server/events/models/drift.go - New drift response, parser, and key modelsserver/events/models/models.go - Reuse or extend NewPlanSuccessStats for any new drift countersserver/core/drift/storage.go - Required drift status storage for /api/drift/statusserver/controllers/api_controller.go - Add drift status endpointserver/controllers/api_controller_test.go - Add status endpoint API-token coverageGoal: Enable automated PR/MR creation to consolidate drift fixes.
Prerequisites:
VCS Client Extension:
// server/events/vcs/client.go
type Client interface {
// ... existing methods ...
// CreatePullRequest creates a new PR/MR for the repository.
// Returns the created pull request or an error.
CreatePullRequest(
logger logging.SimpleLogging,
repo models.Repo,
opts CreatePullRequestOptions,
) (models.PullRequest, error)
// CreateBranch creates a new branch from the given ref.
CreateBranch(
logger logging.SimpleLogging,
repo models.Repo,
branchName string,
fromRef string,
) error
}
type CreatePullRequestOptions struct {
Title string
Body string
HeadBranch string // Source branch (e.g., "drift-remediation-2025-01-21")
BaseBranch string // Target branch (e.g., "main")
Draft bool
Labels []string
Assignees []string
}
GitHub Implementation:
// server/events/vcs/github/client.go
func (g *Client) CreatePullRequest(
logger logging.SimpleLogging,
repo models.Repo,
opts vcs.CreatePullRequestOptions,
) (models.PullRequest, error) {
owner, repoName := repo.Owner, repo.Name
pr, _, err := g.client.PullRequests.Create(g.ctx, owner, repoName, &github.NewPullRequest{
Title: github.String(opts.Title),
Body: github.String(opts.Body),
Head: github.String(opts.HeadBranch),
Base: github.String(opts.BaseBranch),
Draft: github.Bool(opts.Draft),
})
if err != nil {
return models.PullRequest{}, err
}
return models.PullRequest{
Num: pr.GetNumber(),
HeadBranch: pr.GetHead().GetRef(),
BaseBranch: pr.GetBase().GetRef(),
URL: pr.GetHTMLURL(),
State: models.OpenPullState,
// ... other fields
}, nil
}
func (g *Client) CreateBranch(
logger logging.SimpleLogging,
repo models.Repo,
branchName string,
fromRef string,
) error {
owner, repoName := repo.Owner, repo.Name
// Get the SHA of the source ref
ref, _, err := g.client.Git.GetRef(g.ctx, owner, repoName, "refs/heads/"+fromRef)
if err != nil {
return fmt.Errorf("failed to get ref %s: %w", fromRef, err)
}
// Create new branch
_, _, err = g.client.Git.CreateRef(g.ctx, owner, repoName, github.CreateRef{
Ref: "refs/heads/" + branchName,
SHA: ref.Object.GetSHA(),
})
if err != nil {
return fmt.Errorf("failed to create branch %s: %w", branchName, err)
}
return nil
}
GitLab Implementation:
// server/events/vcs/gitlab/client.go
func (g *Client) CreatePullRequest(
logger logging.SimpleLogging,
repo models.Repo,
opts vcs.CreatePullRequestOptions,
) (models.PullRequest, error) {
labels := gitlab.LabelOptions(opts.Labels)
title := opts.Title
if opts.Draft {
// GitLab's create-MR API uses a title prefix to create draft MRs.
title = "Draft: " + title
}
mr, _, err := g.Client.MergeRequests.CreateMergeRequest(repo.FullName, &gitlab.CreateMergeRequestOptions{
Title: gitlab.String(title),
Description: gitlab.String(opts.Body),
SourceBranch: gitlab.String(opts.HeadBranch),
TargetBranch: gitlab.String(opts.BaseBranch),
Labels: &labels,
})
if err != nil {
return models.PullRequest{}, err
}
return models.PullRequest{
Num: mr.IID,
HeadBranch: mr.SourceBranch,
BaseBranch: mr.TargetBranch,
URL: mr.WebURL,
State: models.OpenPullState,
}, nil
}
Drift Remediation Endpoint:
// POST /api/drift/remediate
type DriftRemediatePath struct {
Directory string `json:"directory" validate:"required"`
Workspace string `json:"workspace,omitempty"`
}
type DriftRemediateRequest struct {
Repository string `json:"repository" validate:"required"`
Ref string `json:"ref" validate:"required"` // Base branch
Type string `json:"type" validate:"required"` // VCS type
Projects []string `json:"projects,omitempty"` // Projects to remediate
Paths []DriftRemediatePath `json:"paths,omitempty"`
Title string `json:"title,omitempty"` // PR title
Description string `json:"description,omitempty"` // PR body
Draft bool `json:"draft,omitempty"`
AutoMerge bool `json:"auto_merge,omitempty"` // Enable auto-merge after approval
}
type DriftRemediateResponse struct {
Success bool `json:"success"`
PullRequest *PullRequestInfo `json:"pull_request,omitempty"`
Error string `json:"error,omitempty"`
}
type PullRequestInfo struct {
Number int `json:"number"`
URL string `json:"url"`
Branch string `json:"branch"`
Title string `json:"title"`
}
The remediation handler must enforce the shared API-token authentication before creating branches, commits, or PR/MRs. It must reject unauthenticated requests before running a plan or exposing drift data.
Remediation Workflow:
atlantis-drift-remediation-{timestamp})CreatePullRequest.Files to Create/Modify:
server/events/vcs/client.go - Add CreatePullRequest, CreateBranch interface methodsserver/events/vcs/proxy.go - Proxy new methods to the concrete VCS clientserver/events/vcs/not_configured_vcs_client.go - Return explicit unsupported errors for unconfigured providersserver/events/vcs/github/client.go - Implement for GitHubserver/events/vcs/gitlab/client.go - Implement for GitLabserver/events/vcs/bitbucketcloud/client.go - Implement for Bitbucket Cloudserver/events/vcs/bitbucketserver/client.go - Implement for Bitbucket Server or return explicit unsupported errorsserver/events/vcs/azuredevops/client.go - Implement for Azure DevOpsserver/events/vcs/gitea/client.go - Implement for Gitea or return explicit unsupported errorsserver/events/vcs/mocks/mock_client.go - Regenerate after changing vcs.Clientserver/controllers/api_controller.go - Add remediation endpointserver/controllers/api_controller_test.go - Add remediation endpoint API-token coverageGoal: Add useful operational endpoints and improve API stability.
New Endpoints:
| Endpoint | Method | Auth | Description |
|---|---|---|---|
GET /api/projects | GET | Yes | List configured projects |
GET /api/runs | GET | Yes | List recent plan/apply runs |
DELETE /api/locks?id={lockID} | DELETE | Yes | Force unlock a specific lock |
Projects Endpoint:
// GET /api/projects?repository=org/repo
type ProjectsResponse struct {
Projects []ProjectInfo `json:"projects"`
}
type ProjectInfo struct {
Name string `json:"name"`
Repository string `json:"repository"`
Path string `json:"path"`
Workspace string `json:"workspace"`
Workflow string `json:"workflow,omitempty"`
AutoplanEnabled bool `json:"autoplan_enabled"`
}
Force Unlock Endpoint:
// DELETE /api/locks?id=<url-escaped lock key from GET /api/locks>
// Requires X-Atlantis-Token authentication
func (a *APIController) ForceUnlock(w http.ResponseWriter, r *http.Request) {
// Validate authentication
// Extract lock ID from the id query parameter. Atlantis lock keys contain
// slash separators, so they must not be modeled as a single path segment.
// Call a.Locker.Unlock(lockID)
// Return success/failure
}
The force-unlock endpoint must accept the exact lock key returned by
GET /api/locks as LockDetail.Name. Those keys are generated as
<repo>/<path>/<workspace>/<project> and contain /, so implementations must
use a query parameter, request body field, or explicit catch-all route that
preserves the full URL-escaped value before calling Locker.Unlock(lockID).
MarshalJSON to ProjectResult with explicit field listingMarshalJSON to ResultAPI field to ProjectContextAPI flag in command buildermodels.NewPlanSuccessStatsDriftSummary to plan response (optional enhancement)/api/plan project result/api/drift/status endpointCreatePullRequest and CreateBranch to VCS Client interfaceClientProxy, NotConfiguredVCSClient, and generated VCS mocks/api/drift/remediate endpoint/api/projects endpoint/api/runs endpoint (if run history available)DELETE /api/locks?id={lockID} endpoint| File | Phase | Changes |
|---|---|---|
server/events/command/project_result.go | 1 | Add MarshalJSON method |
server/events/command/result.go | 1 | Add MarshalJSON method |
server/events/command/result_test.go | 1 | Add serialization tests |
server/events/command/project_context.go | 2 | Add API field |
server/events/project_command_context_builder.go | 2 | Propagate API flag |
server/events/command_requirement_handler.go | 2 | Skip PR-specific requirements |
server/events/command_requirement_handler_test.go | 2 | Add test cases |
server/events/models/models.go | 3 | Reuse or extend NewPlanSuccessStats for drift counters |
server/events/vcs/client.go | 4 | Add CreatePullRequest, CreateBranch and shared option types |
server/events/vcs/proxy.go | 4 | Proxy branch and PR creation calls |
server/events/vcs/not_configured_vcs_client.go | 4 | Return unsupported errors for unconfigured clients |
server/events/vcs/github/client.go | 4 | Implement PR creation |
server/events/vcs/gitlab/client.go | 4 | Implement MR creation |
server/events/vcs/bitbucketcloud/client.go | 4 | Implement PR creation |
server/events/vcs/bitbucketserver/client.go | 4 | Implement PR creation or explicit unsupported errors |
server/events/vcs/azuredevops/client.go | 4 | Implement PR creation |
server/events/vcs/gitea/client.go | 4 | Implement PR creation or explicit unsupported errors |
server/events/vcs/mocks/mock_client.go | 4 | Regenerate after vcs.Client changes |
server/controllers/api_controller.go | 3,4,5 | Add new endpoints |
server/server.go | 3,4,5 | Register new routes |
| File | Phase | Purpose |
|---|---|---|
server/events/models/drift.go | 3 | Drift models |
server/core/drift/storage.go | 3 | Required drift status storage |
runatlantis.io/docs/api-endpoints.md | 5 | Documentation updates |
| Risk | Mitigation |
|---|---|
| Field synchronization in MarshalJSON | Add maintainer warning comment, add linter check |
| VCS provider differences | Abstract through interface, comprehensive testing |
| Security (non-PR applies) | Keep other requirements enforced, audit logging |
| API stability | Consider versioning in future, maintain compatibility |