Back to Watermill

Firestore

docs/content/pubsubs/firestore.md

1.5.13.2 KB
Original Source

+++ title = "Firestore Pub/Sub" description = "A scalable document database from Google" date = 2021-07-29T15:30:00+02:00 bref = "A scalable document database from Google" weight = 30 +++

Cloud Firestore is a cloud-hosted, NoSQL database from Google.

This Pub/Sub comes with two publishers. To publish messages in a transaction use the TransactionalPublisher. If you do not want to publish messages in transaction use the normal Publisher.

Using Firestore as a Pub/Sub instead of using a dedicated Pub/Sub system can be useful to publish messages in transaction while at the same time saving other data in Firestore. Thanks to that the data and the messages can be consistently persisted. If the messages and the data weren't being published transactionally you could end up in situations where messages were emitted even though the data wasn't saved or messages weren't emitted even though the data was saved. After transactionally publishing messages in Firestore you can then subscribe to them and relay them to a different Pub/Sub system.

Godoc: https://pkg.go.dev/github.com/ThreeDotsLabs/watermill-firestore

Firestore documentation: https://firebase.google.com/docs/firestore/

Installation

bash
go get github.com/ThreeDotsLabs/watermill-firestore

Characteristics

FeatureImplementsNote
ConsumerGroupsyes
ExactlyOnceDeliveryno
GuaranteedOrderno
Persistentyes

Configuration

Publisher configuration

{{% load-snippet-partial file="src-link/watermill-firestore/pkg/firestore/publisher.go" first_line_contains="type PublisherConfig struct {" last_line_equals="}" %}}

Subscriber configuration

{{% load-snippet-partial file="src-link/watermill-firestore/pkg/firestore/subscriber.go" first_line_contains="type SubscriberConfig struct {" last_line_equals="}" %}}

Subscription name

To receive messages published to a topic, you must create a subscription to that topic. Only messages published to the topic after the subscription is created will be received by the subscribers.

A topic can have multiple subscriptions, but a given subscription belongs to a single topic.

In Watermill, the subscription is created automatically during calling Subscribe(). Subscription name is generated by function passed to SubscriberConfig.GenerateSubscriptionName. By default, it is just the topic name with a suffix _sub appended to it.

If you want to consume messages from a topic with multiple subscribers processing the incoming messages in a different way, you should use a custom function to generate unique subscription names for each subscriber.

Marshaler

Watermill's messages cannot be stored directly in Firestore. The marshaler is responsible for converting them to a type which can be stored by Firestore. The default implementation should be enough for most applications so it is unlikely that you need to implement your own marshaler.

{{% load-snippet-partial file="src-link/watermill-firestore/pkg/firestore/marshaler.go" first_line_contains="// Marshaler" last_line_equals="}" padding_after="0" %}}