docs/design/2019-11-18-tidb-with-wasm.md
This proposal proposes a way to build a bridge between TiDB ecosystem and WebAssembly ecosystem.
As WebAssembly technology matures, more and more applications can be running in browsers, so that users no longer have to endure cumbersome downloads and installation processes, which greatly reduces the cost of promoting these applications. Obviously users who are using TiDB for the first time are facing these problems. Using WebAssembly technology, we can reduce the barriers to user engagement with TiDB, and we can also expand the TiDB audience with the help of the WebAssembly community.
This proposal proposes to compile TiDB on various runtimes of WebAssembly and build TiDB applications in the WebAssembly ecosystem:
WebAssembly is an open standard that defines a portable binary code format for executable programs, a corresponding assembly language, and an interface to interact with the hosting environment. Its main goal is to enable applications with high performance requirements to run on web pages. Of course, its instruction format is also designed to be able to run on other platforms.
The WebAssembly virtual machine has better performance than JavaScript because WebAssembly's binary format file has a more compact instruction structure, similar to ELF files (the most commonly used binary instruction format on Unix systems, which is parsed by the loader and loaded into memory for execution). WebAssembly is compiled into memory for execution by a specific runtime. The most common runtimes include mainstream browsers, nodejs, and a generic implementation specifically designed for WebAssembly - Wasmer. Even someone gives the Linux kernel a feature patch to integrate the WebAssembly runtime into the kernel so that the user-written program can easily run in kernel mode.
Currently, all mainstream browsers support WebAssembly, so you don't have to worry about its compatibility with browsers. The support for WebAssembly on various browsers is as follows:
Running in a non-browser environment involves interacting with the host system, which can be resolved by WASI. We know that WebAssembly is an assembly language designed for abstract virtual machines, so it requires a system interface to manipulate the abstracted operating system, which is defined by WASI. With WASI, the application can be compiled once, run anywhere.
Go 1.11 experimentally added the support for WebAssembly and optimized it in Go 1.12. TiDB is written in pure Golang. In theory, it can be compiled directly into WebAssembly according to Golang's official document, allowing TiDB to run anywhere.
Our implementation will be divided into several stages:
In the case of the problem of stage 1, incompatibility with WASM can be divided into the following cases:
For case 1, we add the Wasm implementation for TiDB directly, according to Golang's platform-related file naming rules, the file name format is:
So the only thing we need do is to add *_wasm.go, and mock an implementation based on the implementation for other platforms.
For case 2, the solution is same as case 1, because we can modify the third-party code directly.
For case 3 and case 4, we can’t modify the code directly. There are 3 solutions:
In summary, when a third-party package does not support WebAssembly, we need to look at it in different situations. The priority can be set as follows:
We found that some specific system functions in Golang will crash if we try to run them in browser:
As shown in the figure above, when we execute the os.stat function, the runtime found that there
was no such a function in the environment. This situation occurs because Golang does not currently
support WASI, so it mocks the host interface using the wasm_exec.js file. However, only a few system
functions are mocked in this file, and os.stat is not among them. The solution is to mount the
functions we need to the fs object on the global object before we execute Wasm. For example:
function unimplemented(callback) {
const err = new Error("not implemented");
err.code = "ENOSYS";
callback(err);
}
function unimplemented1(_1, callback) { unimplemented(callback); }
function unimplemented2(_1, _2, callback) { unimplemented(callback); }
fs.stat = unimplemented1;
fs.lstat = unimplemented1;
fs.unlink = unimplemented1;
fs.rmdir = unimplemented1;
fs.mkdir = unimplemented2;
go.run(result.instance);
Another problem for browser is that TiDB requires to listen to a port to provide service normally, which is prohibited by the browser. Therefore, we need to change the way TiDB provides services — embedding a SQL terminal. SQL terminal in the front-end field is very common, and we can find a ready-made one. The only problem we need to deal with is the interface between the SQL terminal and the TiDB kernel. There is such a method in TiDB:
// Exec executes a sql statement.
func (tk *TestKit) Exec(sql string, args ...interface{}) (sqlexec.RecordSet, error) {
var err error
if tk.Se == nil {
tk.Se, err = session.CreateSession4Test(tk.store)
tk.c.Assert(err, check.IsNil)
id := atomic.AddUint64(&connectionID, 1)
tk.Se.SetConnectionID(id)
}
ctx := context.Background()
if len(args) == 0 {
var rss []sqlexec.RecordSet
rss, err = tk.Se.Execute(ctx, sql)
if err == nil && len(rss) > 0 {
return rss[0], nil
}
return nil, errors.Trace(err)
}
stmtID, _, _, err := tk.Se.PrepareStmt(sql)
if err != nil {
return nil, errors.Trace(err)
}
params := make([]types.Datum, len(args))
for i := 0; i < len(params); i++ {
params[i] = types.NewDatum(args[i])
}
rs, err := tk.Se.ExecutePreparedStmt(ctx, stmtID, params)
if err != nil {
return nil, errors.Trace(err)
}
err = tk.Se.DropPreparedStmt(stmtID)
if err != nil {
return nil, errors.Trace(err)
}
return rs, nil
}
This Exec function is exactly what we need. We can modify it slightly and use it in browsers.
The last two problems are that we need to access files and support SQL operations in batches. These two problems can be combined into one problem, because the user executes SQL in batches in the form of files. The only way for a browser to access a file is to upload and download, so our file access must be converted to file upload/download in the browser. The solution is: detecting the SQL statements executed by the user, and requesting the browser to pop the upload window when the user needs to access files. Then the user selects a file to upload, and the data of the file is submitted to TiDB for processing.
Since the webkit has restrictions on the executable size in mobile browsers (usually 64 MB), and our TiDB-WASM size is about 76 MB. An exception will occur when we run it in a mobile browser:
There are two options to reduce the size of the compiled file:
In addition to the browser's size limit on the Wasm binary file, we also need to solve the SQL terminal input problem on the mobile phone. At present, the SQL terminal on PC browsers has been implemented, but it cannot be used normally on mobile browsers, which process events differently as PC does. We need to adapt the front end for mobile phones.
To run TiDB across platforms, we need to compile TiDB to WASI. The current version of Golang does not support WASI, so running with wasmer after compilation will result in an error:
The above picture is the Hello world program compiled by Golang.
Golang does not plan to support WASI till v1.14, but this won’t stop us from compiling TiDB to WASI because we already have this feature in a Golang fork. We can use this version to get TiDB running in all WASI-enabled environments before Golang 1.14.