Partitioning tables - Supabase

Table partitioning is a technique that allows you to divide a large table into smaller, more manageable parts called “partitions”.

Each partition contains a subset of the data based on a specified criteria, such as a range of values or a specific condition. Partitioning can significantly improve query performance and simplify data management for large datasets.

Benefits of table partitioning

Improved query performance: allows queries to target specific partitions, reducing the amount of data scanned and improving query execution time.
Scalability: With partitioning, you can add or remove partitions as your data grows or changes, enabling better scalability and flexibility.
Efficient data management: simplifies tasks such as data loading, archiving, and deletion by operating on smaller partitions instead of the entire table.
Enhanced maintenance operations: can optimize vacuuming and indexing, leading to faster maintenance tasks.

Partitioning methods

Postgres supports various partitioning methods based on how you want to partition your data. The commonly used methods are:

Range Partitioning: Data is divided into partitions based on a specified range of values. For example, you can partition a sales table by date, where each partition represents a specific time range (e.g., one partition for each month).
List Partitioning: Data is divided into partitions based on a specified list of values. For instance, you can partition a customer table by region, where each partition contains customers from a specific region (e.g., one partition for customers in the US, another for customers in Europe).
Hash Partitioning: Data is distributed across partitions using a hash function. This method provides a way to evenly distribute data among partitions, which can be useful for load balancing. However, it doesn't allow direct querying based on specific values.

Creating partitioned tables

Let's consider an example of range partitioning for a sales table based on the order date. We'll create monthly partitions to store data for each month:

sql

create table sales (
    id bigint generated by default as identity,
    order_date date not null,
    customer_id bigint,
    amount bigint,

    -- We need to include all the
    -- partitioning columns in constraints:
    primary key (order_date, id)
)
partition by range (order_date);

create table sales_2000_01
	partition of sales
  for values from ('2000-01-01') to ('2000-02-01');

create table sales_2000_02
	partition of sales
	for values from ('2000-02-01') to ('2000-03-01');

To create a partitioned table you append partition by range (<column_name>) to the table creation statement. The column that you are partitioning with must be included in any unique index, which is the reason why we specify a composite primary key here (primary key (order_date, id)).

Querying partitioned tables

To query a partitioned table, you have two options:

Querying the parent table
Querying specific partitions

Querying the parent table

When you query the parent table, Postgres automatically routes the query to the relevant partitions based on the conditions specified in the query. This allows you to retrieve data from all partitions simultaneously.

Example:

sql

select *
from sales
where order_date >= '2000-01-01' and order_date < '2000-03-01';

This query will retrieve data from both the sales_2000_01 and sales_2000_02 partitions.

Querying specific partitions

If you only need to retrieve data from a specific partition, you can directly query that partition instead of the parent table. This approach is useful when you want to target a specific range or condition within a partition.

sql

select *
from sales_2000_02;

This query will retrieve data only from the sales_2000_02 partition.

When to partition your tables

There is no real threshold to determine when you should use partitions. Partitions introduce complexity, and complexity should be avoided until it's needed. A few guidelines:

If you are considering performance, avoid partitions until you see performance degradation on non-partitioned tables.
If you are using partitions as a management tool, it's fine to create the partitions any time.
If you don't know how you should partition your data, then it's probably too early.

Examples

Here are simple examples for each of the partitioning types in Postgres.

Range partitioning

Let's consider a range partitioning example for a table that stores sales data based on the order date. We'll create monthly partitions to store data for each month.

In this example, the sales table is partitioned into two partitions: sales_january and sales_february. The data in these partitions is based on the specified range of order dates:

sql

create table sales (
    id bigint generated by default as identity,
    order_date date not null,
    customer_id bigint,
    amount bigint,

    -- We need to include all the
    -- partitioning columns in constraints:
    primary key (order_date, id)
)
partition by range (order_date);

create table sales_2000_01
	partition of sales
  for values from ('2000-01-01') to ('2000-02-01');

create table sales_2000_02
	partition of sales
	for values from ('2000-02-01') to ('2000-03-01');

List partitioning

Let's consider a list partitioning example for a table that stores customer data based on their region. We'll create partitions to store customers from different regions.

In this example, the customers table is partitioned into two partitions: customers_americas and customers_asia. The data in these partitions is based on the specified list of regions:

sql

-- Create the partitioned table
create table customers (
    id bigint generated by default as identity,
    name text,
    country text,

    -- We need to include all the
    -- partitioning columns in constraints:
    primary key (country, id)
)
partition by list(country);

create table customers_americas
	partition of customers
	for values in ('US', 'CANADA');

create table customers_asia
	partition of customers
  for values in ('INDIA', 'CHINA', 'JAPAN');

Hash partitioning

You can use hash partitioning to evenly distribute data.

In this example, the products table is partitioned into two partitions: products_one and products_two. The data is distributed across these partitions using a hash function:

sql

create table products (
    id bigint generated by default as identity,
    name text,
    category text,
    price bigint
)
partition by hash (id);

create table products_one
	partition of products
  for values with (modulus 2, remainder 1);

create table products_two
	partition of products
  for values with (modulus 2, remainder 0);

Other tools

There are several other tools available for Postgres partitioning, most notably pg_partman. Native partitioning was introduced in Postgres 10 and is generally thought to have better performance.