Back to Materialize

Azure deployment guidelines

doc/user/content/self-managed-deployments/deployment-guidelines/azure-deployment-guidelines.md

1233.0 KB
Original Source

As a general guideline, we recommend:

  • ARM-based CPU.
  • A 1:8 ratio of vCPU to GiB memory.
  • An 8:1 ratio of GiB local instance storage to GiB memory when using swap.

When operating on Azure in production, we recommend Epdsv6 sizes Azure VM Types with Local NVMe Disk:

VM SizevCPUsMemoryEphemeral DiskDisk-to-RAM Ratio
Standard_E2pds_v6216 GiB75 GiB~4.7:1
Standard_E4pds_v6432 GiB150 GiB~4.7:1
Standard_E8pds_v6864 GiB300 GiB~4.7:1
Standard_E16pds_v616128 GiB600 GiB~4.7:1
Standard_E32pds_v632256 GiB1,200 GiB~4.7:1

{{< warning >}}

These VM types provide <red>ephemeral</red> local NVMe SSD disks. Data is <red>lost</red> when the VM is stopped or deleted.

{{</ warning >}}

Locally-attached NVMe storage

Configuring swap on nodes to use locally-attached NVMe storage allows Materialize to spill to disk when operating on datasets larger than main memory. This setup can provide significant cost savings and provides a more graceful degradation rather than OOMing. Network-attached storage (like EBS volumes) can significantly degrade performance and is not supported.

Swap support

{{< tabs >}} {{< tab "New Terraform" >}}

New Terraform

The new Materialize Terraform module supports configuring swap out of the box.

{{< /tab >}} {{< tab "Legacy Terraform" >}}

Legacy Terraform

The Legacy Terraform provider, adds preliminary swap support in v0.6.1, via the swap_enabled variable. With this change, the Terraform:

  • Creates a node group for Materialize.
  • Configures NVMe instance store volumes as swap using a daemonset.
  • Enables swap at the Kubelet.

See Upgrade Notes.

{{< note >}} If deploying v25.2, Materialize clusters will not automatically use swap unless they are configured with a memory_request less than their memory_limit. In v26, this will be handled automatically. {{< /note >}}

{{< /tab >}} {{< /tabs >}}

Materialize writes block blobs on Azure. As a general guideline, we recommend Premium block blob storage accounts.

TLS

When running with TLS in production, run with certificates from an official Certificate Authority (CA) rather than self-signed certificates.

Upgrading guideline

{{< include-md file="shared-content/self-managed/general-rules-for-upgrades.md"

}}