Replication

{% include anchor.html edit="true" title="Replicate a database" hash="replication" %} {% highlight "js" %} PouchDB.replicate(source, target, [options]) {% endhighlight %} Replicate data from source to target. Both the source and target can be a PouchDB instance or a string representing a CouchDB database URL or the name of a local PouchDB database. If options.live is true, then this will track future changes and also replicate them automatically. This method returns an object with the method cancel(), which you call if you want to cancel live replication. Replication is an [event emitter][] like changes() and emits the 'complete', 'active', 'paused', 'change', 'denied', 'error' and 'checkpoint' events. ### Options All options default to false unless otherwise specified. * options.live: If true, starts subscribing to future changes in the source database and continue replicating them. * options.retry: If true will attempt to retry replications in the case of failure (due to being offline), using a backoff algorithm that retries at longer and longer intervals until a connection is re-established, with a maximum delay of 10 minutes. Only applicable if options.live is also true. **Filtering Options:** * options.filter: Reference a filter function from a design document to selectively get updates. To use a view function, pass _view here and provide a reference to the view function in options.view. See filtered replication for details. * options.doc_ids: Only show changes for docs with these ids (array of strings). * options.query_params: Object containing properties that are passed to the filter function, e.g. {"foo":"bar"}, where "bar" will be available in the filter function as params.query.foo. To access the params, define your filter function like function (doc, params). * options.view: Specify a view function (e.g. 'design_doc_name/view_name' or 'view_name' as shorthand for 'view_name/view_name') to act as a filter. Documents counted as "passed" for a view filter if a map function emits at least one record for them. **Note**: options.filter must be set to '_view' for this option to work. * options.selector: Filter using a query/pouchdb-find selector. **Advanced Options:** * options.since: Replicate changes after the given sequence number. * options.heartbeat: Configure the heartbeat supported by CouchDB which keeps the change connection alive. * options.timeout: Request timeout (in milliseconds). * options.batch_size: Number of change feed items to process at a time. Defaults to 100. This affects the number of docs and attachments held in memory and the number sent at a time to the target server. You may need to adjust downward if targeting devices with low amounts of memory (e.g. phones) or if the documents and/or attachments are large in size or if there are many conflicted revisions. If your documents are small in size, then increasing this number will probably speed replication up. * options.batches_limit: Number of batches to process at a time. Defaults to 10. This (along with batch_size) controls how many docs are kept in memory at a time, so the maximum docs in memory at once would equal batch_size × batches_limit. * options.back_off_function: backoff function to be used in retry replication. This is a function that takes the current backoff as input (or 0 the first time) and returns a new backoff in milliseconds. You can use this to tweak when and how replication will try to reconnect to a remote database when the user goes offline. Defaults to a function that chooses a random backoff between 0 and 2 seconds and doubles every time it fails to connect. The default delay will never exceed 10 minutes. (See Customizing retry replication below.) * options.checkpoint: Can be used if you want to disable checkpoints on the source, target, or both. Setting this option to false will prevent writing checkpoints on both source and target. Setting it to source will only write checkpoints on the source. Setting it to target will only write checkpoints on the target. * options.style: Specifies whether all revisions of a document including conflicts and deleted former conflicts (all_docs) or only the winning revision (main_only) should be replicated. This option is passed to the changes endpoint of the replication source. Defaults to all_docs. #### Example Usage: {% highlight "js" %} const rep = PouchDB.replicate('mydb', 'http://localhost:5984/mydb', { live: true, retry: true }).on('change', function (info) { // handle change }).on('paused', function (err) { // replication paused (e.g. replication up to date, user went offline) }).on('active', function () { // replicate resumed (e.g. new changes replicating, user went back online) }).on('denied', function (err) { // a document failed to replicate (e.g. due to permissions) }).on('complete', function (info) { // handle complete }).on('error', function (err) { // handle error }); rep.cancel(); // whenever you want to cancel {% endhighlight %} There are also shorthands for replication given existing PouchDB objects. These behave the same as PouchDB.replicate(): {% highlight "js" %} db.replicate.to(remoteDB, [options]); // or db.replicate.from(remoteDB, [options]); {% endhighlight %} The remoteDB can either be a string or a PouchDB object. If you have a fetch override on a remote database, you will want to use PouchDB objects instead of strings, so that the options are used. #### Replication events * __change__ (info) - This event fires when the replication has written a new document. info will contain details about the change. info.docs will contain the docs involved in that change. See below for an example response. * __complete__ (info) - This event fires when replication is completed or cancelled. In a live replication, only cancelling the replication should trigger this event. info will contain details about the replication. See below for an example response. * __paused__ (err) - This event fires when the replication is paused, either because a live replication is waiting for changes, or replication has temporarily failed, with err, and is attempting to resume. * __active__ - This event fires when the replication starts actively processing changes; e.g. when it recovers from an error or new changes are available. * __denied__ (err) - This event fires if a document failed to replicate due to validation or authorization errors. * __error__ (err) - This event is fired when the replication is stopped due to an unrecoverable failure. If retry is false, this will also fire when the user goes offline or another network error occurs (so you can handle retries yourself, if you want). * __checkpoint__ - This event exposes information about the internal steps of the replication process. It includes one of the fields pending_batch, start_next_batch, revs_diff or checkpoint. #### Single-shot As with changes(), you can also omit live, in which case you can use replicate() in the callback/promise style and it will be treated as a single-shot operation. {% include code/start.html id="replication1" type="callback" %} {% highlight "js" %} db.replicate.to(remote, function (err, result) { if (err) { return console.log(err); } // handle 'completed' result }); {% endhighlight %} {% include code/end.html %} {% include code/start.html id="replication1" type="async" %} {% highlight "js" %} try { const result = await db.replicate.to(remote); } catch (err) { console.log(err); } {% endhighlight %} {% include code/end.html %} {% include code/start.html id="replication1" type="promise" %} {% highlight "js" %} db.replicate.to(remote).then(function (result) { // handle 'completed' result }).catch(function (err) { console.log(err); }); {% endhighlight %} {% include code/end.html %} For non-live replications, the returned object is also an event emitter as well as a promise, and you can use all the events described above (except for 'paused' and 'active', which only apply to retry replications). #### Example Response: Example response in the 'change' listener: {% highlight "js" %} { "doc_write_failures": 0, "docs_read": 1, "docs_written": 1, "errors": [], "last_seq": 1, "ok": true, "start_time": "Fri May 16 2014 18:23:12 GMT-0700 (PDT)", "docs": [{ _id: 'docId', _rev: '1-e798a1a7eb4247b399dbfec84ca699d4', and: 'data' }] } {% endhighlight %} Example response in the 'complete' listener: {% highlight "js" %} { "doc_write_failures": 0, "docs_read": 2, "docs_written": 2, "end_time": "Fri May 16 2014 18:26:00 GMT-0700 (PDT)", "errors": [], "last_seq": 2, "ok": true, "start_time": "Fri May 16 2014 18:26:00 GMT-0700 (PDT)", "status": "complete" } {% endhighlight %} Note that replication is supported for both local and remote databases. So you can replicate from local to local or from remote to remote. However, if you replicate from remote to remote, then the changes will flow through PouchDB. If you want to trigger a server-initiated replication, please use regular ajax to POST to the CouchDB _replicate endpoint, as described in the CouchDB docs. #### Filtered replication As with changes(), you can filter from the source database using: * an ad-hoc filter function * an array of doc_ids * a filter function inside of a design document * a filter function inside of a design document, with query_params * a view function inside of a design document If you are replicating from a remote CouchDB, then the first method will run client-side, whereas the last four will filter on the server side. Therefore the last four should be preferred, especially if the database is large, because you want to send as few documents over the wire as possible. You should also beware trying to use filtered replication to enforce security, e.g. to partition a database per user. A better strategy is the "one database per user" method. {% include alert/start.html variant="warning" %} {% markdown %} **Deleting filtered docs**: When you use filtered replication, you should avoid using remove() to delete documents, because that removes all their fields as well, which means they might not pass the filter function anymore, causing the deleted revision to not be replicated. Instead, set the doc._deleted flag to true and then use put() or bulkDocs(). {% endmarkdown %} {% include alert/end.html %} #### Filtering examples In these examples, we'll work with some mammals. Let's imagine our docs are: {% highlight "js" %} [{_id: 'a', name: 'Kangaroo', type: 'marsupial'}, {_id: 'b', name: 'Koala', type: 'marsupial'}, {_id: 'c', name: 'Platypus', type: 'monotreme'}] {% endhighlight %} Here are 5 examples using the 5 different systems. **Example 1: Ad-hoc filter function** *Warning*: this runs client-side, if you are replicating from a remote database. Filter by type === 'marsupial': {% highlight "js" %} remote.replicate.to(local, { filter: function (doc) { return doc.type === 'marsupial'; } }); {% endhighlight %} **Example 2: Array of doc_ids** Filter documents with _ids ['a', 'c']. {% highlight "js" %} remote.replicate.to(local, { doc_ids: ['a', 'c'] }); {% endhighlight %} **Example 3: filter function inside of a design document** First put() a design document in the remote database: {% highlight "js" %} { _id: '_design/mydesign', filters: { myfilter: function (doc) { return doc.type === 'marsupial'; }.toString() } } {% endhighlight %} Then filter by type === 'marsupial': {% highlight "js" %} remote.replicate.to(local, { filter: 'mydesign/myfilter' }); {% endhighlight %} **Example 4: filter function inside of a design document, with query_params** This is the most powerful way to filter, because it allows you to pass in arbitrary options to your filter function. First put() a design document in the remote database: {% highlight "js" %} { _id: '_design/mydesign', filters: { myfilter: function (doc, req) { return doc.type === req.query.type; }.toString() } } {% endhighlight %} Then filter by type === 'marsupial': {% highlight "js" %} remote.replicate.to(local, { filter: 'mydesign/myfilter', query_params: {type: 'marsupial'} }); {% endhighlight %} **Example 5: view function inside of a design document** This doesn't really offer any advantages compared to the previous two methods, unless you are already using a view for map/reduce queries, and you want to reuse it. Any documents that emit() anything will be considered to have passed this filter method. First put() a design document in the remote database: {% highlight "js" %} { _id: '_design/mydesign', views: { myview: { map: function(doc) { if (doc.type === 'marsupial') { emit(doc._id); } }.toString() } } } {% endhighlight %} Then filter by type === 'marsupial': {% highlight "js" %} remote.replicate.to(local, { filter: '_view', view: 'mydesign/myview' }); {% endhighlight %} #### Customizing retry replication During retry replication, you can customize the backoff function that determines how long to wait before reconnecting when the user goes offline. Here's a simple backoff function that starts at 1000 milliseconds and triples it every time a remote request fails: {% highlight "js" %} db.replicate.to(remote, { live: true, retry: true, back_off_function: function (delay) { if (delay === 0) { return 1000; } return delay * 3; } }); {% endhighlight %} The first time a request fails, this function will receive 0 as input. The next time it fails, 1000 will be passed in, then 3000, then 9000, etc. When the user comes back online, the delay goes back to 0. By default, PouchDB uses a backoff function that chooses a random starting number between 0 and 2000 milliseconds and will roughly double every time, with some randomness to prevent client requests from occurring simultaneously.