doc/website/blog/2022-01-20-announcing-new-migration-engine.md
Dear community,
I'm very happy to announce the release of the next version of Ent: v0.10. It has been almost six months since v0.9.1, so naturally there's a ton of new stuff in this release. Still, I wanted to take the time to discuss one major improvement we have been working on for the past few months: a brand-new migration engine.
Ent's current migration engine is great, and it does some pretty neat stuff which our community has been using in production for years now, but as time went on issues which we could not resolve with the existing architecture started piling up. In addition, we feel that existing database migration frameworks leave much to be desired. We have learned so much as an industry about safely managing changes to production systems in the past decade with principles such as Infrastructure-as-Code and declarative configuration management, that simply did not exist when most of these projects were conceived.
Seeing that these problems were fairly generic and relevant to application regardless of the framework or programming language it was written in, we saw the opportunity to fix them as common infrastructure that any project could use. For this reason, instead of just rewriting Ent's migration engine, we decided to extract the solution to a new open-source project, Atlas (GitHub).
Atlas is distributed as a CLI tool that uses a new DDL based on HCL (similar to Terraform), but can also be used as a Go package. Just as Ent, Atlas is licensed under the Apache License 2.0.
Finally, after much work and testing, the Atlas integration for Ent is finally ready to use. This is great news to many of our users who opened issues (such as #1652, #1631, #1625, #1546 and #1845) that could not be well addressed using the existing migration system, but are now resolved using the Atlas engine.
As with any substantial change, using Atlas as the migration engine for your project is currently opt-in. In the near future, we will switch to an opt-out mode, and finally deprecate the existing engine. Naturally, this transition will be made slowly, and we will progress as we get positive indications from the community.
First, upgrade to the latest version of Ent:
go get entgo.io/[email protected]
Next, in order to execute a migration with the Atlas engine, use the WithAtlas(true) option.
package main
import (
"context"
"log"
"<project>/ent"
"<project>/ent/migrate"
"entgo.io/ent/dialect/sql/schema"
)
func main() {
client, err := ent.Open("mysql", "root:pass@tcp(localhost:3306)/test")
if err != nil {
log.Fatalf("failed connecting to mysql: %v", err)
}
defer client.Close()
ctx := context.Background()
// Run migration.
err = client.Schema.Create(ctx, schema.WithAtlas(true))
if err != nil {
log.Fatalf("failed creating schema resources: %v", err)
}
}
And that's it!
One of the great improvements of the Atlas engine over the existing Ent code, is it's layered structure, that cleanly separates between inspection (understanding the current state of a database), diffing (calculating the difference between the current and desired state), planning (calculating a concrete plan for remediating the diff), and applying. This diagram demonstrates the way Ent uses Atlas:
In addition to the standard options (e.g. WithDropColumn,
WithGlobalUniqueID), the Atlas integration provides additional options for
hooking into schema migration steps.
Here are two examples that show how to hook into the Atlas Diff and Apply steps.
package main
import (
"context"
"log"
"<project>/ent"
"<project>/ent/migrate"
"ariga.io/atlas/sql/migrate"
atlas "ariga.io/atlas/sql/schema"
"entgo.io/ent/dialect"
"entgo.io/ent/dialect/sql/schema"
)
func main() {
client, err := ent.Open("mysql", "root:pass@tcp(localhost:3306)/test")
if err != nil {
log.Fatalf("failed connecting to mysql: %v", err)
}
defer client.Close()
ctx := context.Background()
// Run migration.
err := client.Schema.Create(
ctx,
// Hook into Atlas Diff process.
schema.WithDiffHook(func(next schema.Differ) schema.Differ {
return schema.DiffFunc(func(current, desired *atlas.Schema) ([]atlas.Change, error) {
// Before calculating changes.
changes, err := next.Diff(current, desired)
if err != nil {
return nil, err
}
// After diff, you can filter
// changes or return new ones.
return changes, nil
})
}),
// Hook into Atlas Apply process.
schema.WithApplyHook(func(next schema.Applier) schema.Applier {
return schema.ApplyFunc(func(ctx context.Context, conn dialect.ExecQuerier, plan *migrate.Plan) error {
// Example to hook into the apply process, or implement
// a custom applier. For example, write to a file.
//
// for _, c := range plan.Changes {
// fmt.Printf("%s: %s", c.Comment, c.Cmd)
// if err := conn.Exec(ctx, c.Cmd, c.Args, nil); err != nil {
// return err
// }
// }
//
return next.Apply(ctx, conn, plan)
})
}),
)
if err != nil {
log.Fatalf("failed creating schema resources: %v", err)
}
}
I know we took a while to get this release out the door, but the next one is right around the corner. Here's what's in store for v0.11:
Aside from the exciting announcement about the new migration engine, this release is huge in size and contents, featuring 199 commits from 42 unique contributors. Ent is a community effort and keeps getting better every day thanks to all of you. So here's huge thanks and infinite kudos to everyone who took part in this release (alphabetically sorted):
attackordie, bbkane, bodokaiser, cjraa, dakimura, dependabot, EndlessIdea, ernado, evanlurvey, freb, genevieve, giautm, grevych, hedwigz, heliumbrain, hilakashai, HurSungYun, idc77, isoppp, JeremyV2014, Laconty, lenuse, masseelch, mattn, mookjp, msal4, naormatania, odeke-em, peanut-cc, posener, RiskyFeryansyahP, rotemtam, s-takehana, sadmansakib, sashamelentyev, seiichi1101, sivchari, storyicon, tarrencev, ThinkontrolSY, timoha, vecpeng, yonidavidson, and zeevmoney.
Best, Ariel
:::note For more Ent news and updates:
:::