docs-src/docs/releases/10.0.0.md
One year after version 9.0.0 we now have RxDB version 10.0.0.
The main goal of version 10 was to change things that make RxDB ready for the future.
Notice that I use major releases to bundle stuff that breaks the RxDB usage in your project, not to add new features.
In the past, RxDB was build around Pouchdb. Before I started making RxDB I tried to solve the problems of my current project with other existing databases out there. I evaluated all of them and then started using Pouchdb and added many features via plugin. Then I realised it will be easier to create a separate project that wraps around Pouchdb, that was RxDB. Back then pouchdb was the most major browser database out there and it was well maintained and had a big community. But in the last 5 years, things have changed. A big part of the RxDB users do not use couchdb in the backend and do not need the couchdb replication. Therefore they do not really need the overhead with revision handling that slows down the performance of pouchdb. Also there where many other problems with using pouchdb. It is not actively developed, many bugs are not fixed and no new features get added. Also there are many unsolved problems like how to finally delete document data or how to replicate more than 6 databases at the same time, how to use replication without attachments data, and so on...
So for this release, I abstracted all parts that we use from pouchdb into the RxStorage interface. RxDB works on top of any implementation of the RxStorage interface. This means it is now possible to use RxDB together with other underlying storages like SQLite, PostgreSQL, Minimongo, MongoDB, and so on, as long as someone writes the RxStorage class for it.
This means, to create a RxDatabase you have to pass the storage class instead of pouchdb specific settings:
// import pouchdb specific stuff and add pouchdb adapters
import {
addPouchPlugin,
getRxStoragePouch
} from 'rxdb/plugins/pouchdb';
// IMPORTANT: Do not use addRxPlugin to add
// pouchdb adapter, instead use addPouchPlugin
addPouchPlugin(require('pouchdb-adapter-memory'));
import {
addRxPlugin,
createRxDatabase,
randomCouchString,
} from 'rxdb/plugins/core';
// create the database with the storage creator.
const db = await createRxDatabase({
name: 'mydatabase',
storage: getRxStoragePouch('memory'),
});
To access the internal pouch instance of a collection, you have to go over the storageInstance:
const pouch = myRxCollection.storageInstance.internals.pouch;
In the past, using a primary key was optional. When no primary key was defined, RxDB filled up the _id field with an uuid-like string which was then used as primary. When I researched on github how people use RxDB, I found out that many use a secondary index for what should be the primary key.
Also having the primary key optional, caused much confusing when using RxDB with typescript.
So now the primary key MUST be set when creating a schema for RxDB.
Also the primary key is defined with the primaryKey property at the top level of the schema. This ensures that typescript will complain if no primaryKey is defined.
// when using the type `RxJsonSchema<DocType>` the `DocType` is now required
const mySchema: RxJsonSchema<MyDocumentData> = {
version: 0,
primaryKey: 'passportId',
type: 'object',
properties: {
passportId: {
type: 'string'
}
},
// primaryKey is always required
required: ['passportId']
}
In the past, an RxAttachment could be stored with Blob, Buffer and string data. If a string was passed, pouchdb internally transformed the data to a Blob or Buffer, depending on in which environment it is running.
This behavior caused much trouble and weird edge cases because of how the data is transformed from and to string.
So now you can only store Blob or Buffer as attachment data. string is no longer allowed. You can still transform a string to a Blob or Buffer by yourself and then store it.
import { blobBufferUtil } from 'rxdb';
const attachment = await myDocument.putAttachment(
{
id: 'cat.txt',
data: blobBufferUtil.createBlobBuffer('miau', 'text/plain')
type: 'text/plain'
}
);
Also putAttachment() now defaults to skipIfSame=true. This means when you write attachment data that already is exactly the same in the database, no write will be done.
RxDB often uses outgoing data also in the internals. For example the result of a query is not only send to the user, but also used inside of RxDB's query-change-detection. To ensure that mutation of the outgoing data is not changing internal stuff, which would cause strange bugs, outgoing data was always deep-cloned before handing it out to the user. This is a common practice on many javascript libraries.
The problem is that deep-cloning big objects can be very CPU/Memory expensive. So instead of doing a deep-clone, RxDB does now assume that outgoing data is immutable. If the users wants to modify that data, it has to be deep-cloned by the user.
To ensure immutability, RxDB runs a deep-freeze in the dev-mode (about same expensive as deep clone). Also typescript will throw a build-time error because we use ReadonlyArray and readonly to define outgoing data immutable. In production-mode, there will be nothing besides typescript that ensures immutability to have best performance.
const data = myRxDocument.toJSON();
data.foo = bar; // This does NOT work!
// instead clone the data before changing it
import { clone } from 'rxjs';
const clonedData = clone(data);
data.foo = bar; // This works!
The in-memory plugin was used to spawn in-memory collections on top of a normal RxCollection. The benefit is to have the data replicated into the memory of the javascript runtime, which allows for faster queries.
After doing many tests and observations, I found out that the in-memory plugin was slow. Really slow, even slower than just using the indexeddb adapter in the browser. You can reproduce my observations at the event-reduce testpage. Here you can see that random-writes+query are slower on the memory-adapter than on indexeddb. The reason for this are the big abstraction layers. Pouchdb uses the adapter system. The memory adapter uses the leveldown abstraction layer. Each write/read goes to the memdown module.
So the in-memory plugin is not working for now. In the future it will be reimplemented in a custom memory based RxStorage class.
:::note
You can of course still use the pouchdb memory adapter as usual. It is not affected by this change.
:::
atomicSet(), use atomicPatch() instead.RxDatabase.collection() use RxDatabase().addCollections() instead.preCreatePouchDb because it is no longer needed.watch-for-changes plugin. We now overwrite pouchdbs bulkDocs method to generate events. This is faster and more reliable.adapter-check plugin. (The function adapterCheck is move to the pouchdb plugin).RxDatabase.server() now returns a promise that resolves when the server is started up.PouchDBExpressServerOptions from the server() method, by default we now store logs in the tmp folder and the config is in memory.replication-plugin to replication-couchdb to be more consistent in naming like with replication-graphql
RxCollection().sync() to RxCollection().syncCouchDB()dump() is now exportJSON()importDump() is now importJSON()RxCollection uses a separate pouchdb instance for local documents, so that they can persist during migrations.required array at the top level and it must contain the primary key.You can now use a composite primary key for the schema where you can join different properties of the document data to create a primary key.
const mySchema = {
keyCompression: true, // set this to true, to enable the keyCompression
version: 0,
title: 'human schema with composite primary',
primaryKey: {
// where should the composed string be stored
key: 'id',
// fields that will be used to create the composed key
fields: [
'firstName',
'lastName'
],
// separator which is used to concat the fields values.
separator: '|'
}
type: 'object',
properties: {
id: {
type: 'string'
},
firstName: {
type: 'string'
},
lastName: {
type: 'string'
}
},
required: [
'id',
'firstName',
'lastName'
]
};
With these changes, RxDB is now ready for the future plans:
revision handling of documents with conflict resolution strategies that can always directly resolve conflicts instead of maintaining the revision tree.RxStorage. I will first work on a memory based version. I am in good hope that the community will create other implementations depending on their needs.There are many things that can be done by you to improve RxDB:
Please discuss here.