Back to Pulsar

PIP-120: Enable client memory limit by default

pip/pip-120.md

4.2.11008 B
Original Source

PIP-120: Enable client memory limit by default

In Pulsar 2.8, we have introduced a setting to control the amount of memory used by a client instance.

java
interface ClientBuilder {
    ClientBuilder memoryLimit(long memoryLimit, SizeUnit unit);
}

By default, in 2.8 and 2.9 this setting is set to 0, meaning no limit is being enforced.

I think it's a good time for 2.10 to enable this setting by default and, correspondingly, to disable by default the producer queue size limit.

This will simplify a lot the configuration that a producer application will have to come up with, when publishing with many topic/partitions or when messages are bigger than expected.

Proposed changes

In 2.10 release, for the ClientBuilder, change   * memoryLimit: 0 -> 64 MB

For the ProducerBuilder, changes   * maxPendingMessages: 1000 -> 0

64MB is picked because it's a small enough memory size that will guarantee a very high producer throughput, irrespective of the individual messages size.