docs-src/docs/releases/9.0.0.md
So I was working hard the past month to prepare everything for the next major release of RxDB. The last major release was 1,5 years ago by the way.
When I started listing up the planned changes I had big ambitions about basically rewriting everything. But I found out this time has not come yet. There is some work to be done first. So version 9.0.0 was more about fixing all these small things that made improving the codebase difficult. Much has been refactored and moved. Some API-parts have been changed to have a more simple project with a cleaner codebase.
Notice that I use major releases to bundle stuff that breaks the RxDB usage in your project. By having only few major releases you can be sure that you can upgrade in one big block instead of changing stuff each few months. Big features are released in non-major releases because they mostly can be implemented without side effects.
You have to apply these changes to your codebase when upgrading RxDB.
Using default exports and imports can be helpful when you want to write code fast. But using them also disabled the tree-shaking of your bundler which means you added much code to your bundle that was not even used. To prevent this common behavior, I removed all default exports and renamed functions so that they are more unlikely to clash with other non-RxDB function names.
Instead of doing
import RxDB from 'rxdb';
RxDB.plugin(/* ... */);
await RxDB.create();
You now do
import { createRxDatabase, addRxPlugin } from 'rxdb';
addRxPlugin(/* ... */);
await createRxDatabase();
Also removeDatabase() is renamed to removeRxDatabase() and plugin() is now addRxPlugin().
Same goes for all previous default exports of the plugins.
In the past the indexes of a collection had to be specified at the field level of the schema like
{
"firstName": {
"type": "string",
"index": true
}
}
This made it complex to list up the index fields which had a bad performance on startup. To fix this the indexes are now specified at the top level of the schema like
{
"title": "my schema",
"version": 0,
"type": "object",
"properties": {},
"indexes": [
"firstName",
["compound", "index"]
]
}
Same as the indexes, encrypted fields are now also defined in the top level like
{
"title": "my schema",
"version": 0,
"type": "object",
"properties": {},
"encrypted": [
"password"
]
}
In the past we had stuff that is only wanted for development in the two plugins error-messages and schema-check.
Now we have a single plugin dev-mode that contains all these checks and development helper functions. I also moved many other checks out of the core-module into dev-mode.
import { RxDBDevModePlugin } from 'rxdb/plugins/dev-mode';
addRxPlugin(RxDBDevModePlugin);
The data migration was not used by all users of this project. Often it was easier to just wipe the local data store and let the client sync everything from the server. Because the migration has so much code, it is now in a separate plugin so that you do not have to ship this code to the clients if not necessary.
import { RxDBMigrationPlugin } from 'rxdb/plugins/migration';
addRxPlugin(RxDBMigrationPlugin);
The key-compression logic was fully coded only for RxDB. This was a problem because it was not usable for other stuff and also badly tested. We had a known problem with nested arrays that caused much confusion because some queries did not find the correct documents.
I now created a npm-package jsonschema-key-compression that has cleaner code, better tests and can also be used for non-RxDB stuff.
If you used the key-compression in the past and have clients out there with old data, you have to find a way to migrate that data by using the json-import or other solutions depending on your project.
One big benefit of having a realtime database is that big performance optimizations can be done when the database knows a query is observed and the updated results are needed continuously. In the past this optimization was done by the internal queryChangeDetection which was a big tree of if-else-statements that hopefully worked. This was also the reason why queryChangeDetection was in beta mode and not enabled by default.
After months of research and testing I was able to create Event-Reduce: An algorithm to optimize database queries that run multiple times. This JavaScript module contains an algorithm that is able to optimize realtime queries in a way that was not possible before. The algorithm is not RxDB specific and also heavily tested.
Instead of setting queryChangeDetection when creating a RxDatabase, you now set eventReduce which defaults to true.
In the past, only the selector of a query could be passed to find() and findOne() if you wanted to also do sort, skip or limit, you had to call additional functions like
const query = myRxCollection.find({
age: {
$gt: 10
}
}).sort('name').skip(5).limit(10);
Now you can pass the full query to the function call like
const query = myRxCollection.find({
selector: {
age: {
$gt: 10
}
},
sort: [{name: 'asc'}],
skip: 5,
limit: 10
});
The query builder that allowed to create queries like .where('foo').eq('bar') etc. was not really used by many people. Most of the time it is better to just pass the full query as simple json object. Also the code for the query builder was big and increased the build size much more than its value added. Only some edge-cases where recursive query modification was needed made the query builder useful. If you still want to use the query builder, you have to import the plugin.
import { RxDBQueryBuilderPlugin } from 'rxdb/plugins/query-builder';
addRxPlugin(RxDBQueryBuilderPlugin);
The whole data structure of RxChangeEvent was way more complicated than it had to be. I refactored the whole class to be more simple. If you directly use RxChangeEvent in your project you have to adapt to these changes. Also the stream of RxDatabase().$ will no longer emit the COLLECTION event when a new collection is created.
The internal hash function was used to store hashes of database passwords to compare them and directly throw errors when the wrong password was used with an existing data set. This was dangerous because you could use rainbow tables or even just google the hash to find out the plain password. So now the internal hashing is using a salt to prevent these attacks. If you have used the encryption in the past, you have to migrate your internal schema storage.
By default RxDocument.toJSON() always returned also the _rev field and the _attachments. This was confusing behavior which is why I changed the default to RxDocument().toJSON(withRevAndAttachments = false)
Because RxDB and some subdependencies extensively use export type ... you now need typescript 3.8.0 or newer.
In dev-mode, the GraphQL-replication will run a schema validation of each document that comes from the server before it is saved to the database.
I refactored much internal stuff and moved much code out of the core into the specific plugins.
RxSchema.jsonID to RxSchema.jsonSchemainternalStorerequire() CommonJS loadingRxCollection.docChanges$() because all events are from the docsRxDB is an open source project an heavily relies on the contribution of its users. There are some things that must be done, but I have no time for them.
The current implementation has some flaws and should be completely rewritten.
The react example has no end-to-end tests which is why the CI does not ensure that it works all the time. We should add some basic tests like we did for the other example projects.
There is a bug in pouchdb that prevents the upgrade of pouchdb-find. This is why RxDB relies on an old version of pouchdb-find that also requires different sub-dependencies. This increases the build size a lot because for example we ship multiple version of spark-md5 and others.
At the moment RxDB is a realtime database based on pouchdb. In the future I want RxDB to be a wrapper around pull-based databases that also works with other source-dbs like mongoDB or PostgreSQL. As a start I defined the RxStorage interface and created a RxStoragePouchdb class that implements it and contains all pouchdb-specific logic. I want to move every direct storage usage into that interface so that later we can create other implementations of it for other source databases.