docs/content/pubsubs/io.md
+++ title = "io.Writer/io.Reader" description = "Pub/Sub implemented as Go stdlib's most loved interfaces" date = 2019-07-06T22:30:00+02:00 bref = "Pub/Sub implemented as Go stdlib's most loved interfaces" weight = 70 +++
This is an experimental Pub/Sub implementation that leverages the standard library's io.Writer and io.Reader interfaces as sources of Publisher and Subscriber, respectively.
Note that these aren't full-fledged Pub/Subs like Kafka, RabbitMQ, or the likes, but given the ubiquity of implementations of Writer and Reader they may come in handy, for uses like:
io.Writer or io.Reader, like github.com/colinmarc/hdfs or github.com/mholt/archiver.go get github.com/ThreeDotsLabs/watermill-io
This is a very bare-bones implementation for now, so no extra features are supported. However, it is still sufficient for applications like a CLI producer/consumer.
| Feature | Implements | Note |
|---|---|---|
| ConsumerGroups | no | |
| ExactlyOnceDelivery | no | |
| GuaranteedOrder | no | |
| Persistent | no |
The publisher configuration is relatively simple.
{{% load-snippet-partial file="src-link/watermill-io/pkg/io/publisher.go" first_line_contains="type PublisherConfig struct" last_line_contains="// Publisher" %}}
The subscriber may work in two modes – either perform buffered reads of constant size from the io.Reader, or split the byte stream into messages using a delimiter byte.
The reading will continue even if the reads come up empty, but they will not be sent out as messages. The time to wait after an empty read is configured through the PollInterval parameter. As soon as a non-empty input is read, it will be packaged as a message and sent out.
{{% load-snippet-partial file="src-link/watermill-io/pkg/io/subscriber.go" first_line_contains="type SubscriberConfig struct" last_line_contains="// Subscriber" %}}
The continuous reading may be used, for example, to emulate the behaviour of a tail -f command, like in this snippet:
{{% load-snippet-partial file="docs/snippets/tail-log-file/main.go" first_line_contains="// this will" last_line_contains="return false" padding_after="1" %}}
The MarshalFunc is an important part of io.Publisher, because it fully controls the format in the underlying io.Writer will obtain the messages.
Correspondingly, the UnmarshalFunc regulates how the bytes read by the io.Reader will be interpreted as Watermill messages.
{{% load-snippet-partial file="src-link/watermill-io/pkg/io/marshal.go" first_line_contains="// MarshalMessageFunc" last_line_contains="// PayloadMarshalFunc" %}}
{{% load-snippet-partial file="src-link/watermill-io/pkg/io/marshal.go" first_line_contains="// UnmarshalMessageFunc" last_line_contains="// PayloadUnmarshalFunc" %}}
The package comes with some predefined marshal and unmarshal functions, but you might want to write your own marshaler/unmarshaler to work with the specific implementation of io.Writer/io.Reader that you are working with.
For the Publisher/Subscriber implementation itself, the topic has no meaning. It is difficult to interpret the meaning of topic in the general context of io.Writer and io.Reader interfaces.
However, the topic is passed as a parameter to the marshal/unmarshal functions, so the adaptations to particular Writer/Reader implementation may take it into account.