Search Provider Interface

Status: In progress Target: Spree 5.4 (provider + DB provider + PgSearch removal + add_search_scope removal + Meilisearch provider), 6.0 (metafield faceting) Depends on: None for 5.4. ProductType (6.0-product-types.md) for metafield faceting in 6.0. Author: Damian + Claude Last updated: 2026-03-18

Summary

Add a pluggable search provider interface (Spree::SearchProvider::Base) that abstracts product search, filtering, sorting, and facet aggregation behind a clean contract. The default implementation (Spree::SearchProvider::Database) uses standard SQL/ILIKE scopes. Meilisearch is the recommended external provider, using meilisearch-ruby (raw client, no model overrides).

This plan also:

• Removes the pg_search dependency (legacy, replaced by the provider interface)
• Removes the add_search_scope macro (legacy, replaced by standard Rails scopes)
• Extends faceted filtering to include MetafieldDefinition-based facets

Problem

1. No search abstraction. Search logic is spread across four layers — model scopes (ProductScopes), Ransack integration (ResourceController), a finder service (Products::Find), and a filter aggregator (FiltersAggregator). Replacing any one part means understanding and patching all four.

2. SQL search is inadequate for enterprise. The search scope uses ILIKE (or PgSearch trigram when installed). No typo tolerance, synonyms, stemming, or relevance tuning. Every serious deployment replaces this, but there's no clean seam to do so.

3. PgSearch is a legacy dependency. The if defined?(PgSearch) conditional in Product creates two code paths for search — one with PgSearch, one without. PgSearch only works with PostgreSQL, adds complexity for marginal benefit over plain ILIKE, and doesn't solve the real problems (relevance, typo tolerance, faceting). A proper search engine (Meilisearch) solves all of these. PgSearch should be removed entirely, not maintained alongside a provider interface.

4. add_search_scope is a non-standard macro. ProductScopes defines ~25 scopes using add_search_scope which uses singleton_class.send(:define_method, ...) instead of standard Rails scope. This is legacy cruft — there's no benefit over standard scopes, and it creates confusion about which scopes are "search scopes" vs regular scopes. All should be converted to standard scope definitions.

5. Faceted filtering is OptionType-only. FiltersAggregator builds facets from OptionType.filterable + OptionValue counts. Metafield-based attributes (material, brand, weight class) can't participate in faceted navigation, despite MetafieldDefinitions being the structured attribute system.

6. No autocomplete/typeahead endpoint. Search-as-you-type is expected by enterprise buyers. No endpoint exists.

7. Filter counts are naive. FiltersAggregator counts products per option value against the full scope, not adjusted for already-applied filters. This means selecting "Size: M" doesn't update the count on "Color: Red" to reflect only products that are both M and Red.

Current State

Key Decisions (do not deviate without discussion)

Provider owns search + filtering + facets; AR owns security + visibility

The provider handles everything that a search engine does better than SQL:

1. Search — text query → ranked results
2. Filter — structured params (price, options, metafields, availability, categories)
3. Aggregate — facets with adjusted counts

AR scopes handle what must be enforced at the database level:

1. Store scoping — current_store.products
2. Visibility — .active(currency) for Store API (not applied for Admin)
3. Authorization — .accessible_by(current_ability, :show)

The controller builds a base AR scope for security/visibility, then hands everything else to the provider:

# Store API controller
base_scope = current_store.products.active(currency).accessible_by(current_ability, :show)

result = search_provider.search_and_filter(
  scope: base_scope,
  query: params.dig(:q, :search),
  filters: params[:q],       # price_gte, with_option_value_ids, etc.
  sort: params[:sort],
  page: params[:page],
  limit: params[:limit]
)

result.products    # → AR relation (filtered, sorted, paginated)
result.filters     # → facets with counts
result.total_count # → total matches

The Database provider implements this by chaining Ransack scopes + FiltersAggregator (same as today). The Meilisearch provider does search + filter + facets in one Meilisearch API call, then intersects with the base AR scope for security.

Store API Controller
  │
  ├─ base_scope = store.products.active.accessible_by(ability)  ← AR (security)
  │
  └─ search_provider.search_and_filter(scope: base_scope, ...)  ← Provider (search + filter + facets)
       │
       ├─ Database: scope.search(query).ransack(filters) + FiltersAggregator
       │
       └─ Meilisearch: meilisearch_client.search(query, filters, facets)
            → Product.where(id: result_ids).merge(base_scope)   ← intersect for security

Product.search stays as a plain ILIKE scope

• Product.search(query) is a simple SQL scope. ILIKE on name + SKU. It knows nothing about providers.
• Admin controllers, rake tasks, and console can call it directly — no dependency on Meilisearch or the provider.
• The Database provider delegates to it internally. The Meilisearch provider ignores it entirely and uses the meilisearch-ruby client.
• No model-level magic. We use meilisearch-ruby (raw HTTP client), not meilisearch-rails (ActiveRecord override). The model stays clean.

Provider is registered via Spree::Dependencies

Spree.search_provider = 'Spree::SearchProvider::Database'       # default
Spree.search_provider = 'SpreeMeilisearch::SearchProvider'       # with meilisearch

Consistent with existing products_finder, products_sorter pattern.

MetafieldDefinition gains filterable flag

• Add filterable boolean to MetafieldDefinition (default: false).
• Both providers include filterable metafield definitions as facets.
• Only choice-type metafields are facet-ready in v1: single_line_text and number types where values repeat across products. Rich text and JSON types are not filterable.
• Metafield facets use the same filter response shape as option type facets, with type: 'metafield'.

Remove PgSearch dependency

• Drop pg_search entirely. No conditional if defined?(PgSearch) branches. The search scope on Product uses plain ILIKE (the current fallback path).
• Remove PgSearch scopes from Product — search_by_name (trigram) removed. ILIKE on name is sufficient for the Database provider.
• Remove PgSearch scopes from Variant — search_by_sku, search_by_sku_or_options replaced by ILIKE equivalents.
• Remove pg_search from gemspec. Currently an optional dependency. No migration needed — it's a gem removal.
• Meilisearch is the recommended upgrade path. For merchants who want typo tolerance, stemming, and relevance tuning, install spree_search_meilisearch. This gives them far more than PgSearch ever did.

Remove add_search_scope macro

• Convert all add_search_scope calls to standard scope definitions. There are ~25 scopes in ProductScopes using this macro. All become standard Rails scopes.
• Remove the add_search_scope method, search_scopes class variable, and related infrastructure from ProductScopes.
• No behavior change. The scopes do the same thing — only the definition mechanism changes.
• Ransack whitelisted_ransackable_scopes stays as-is. Standard scopes work with Ransack identically.

Meilisearch is the recommended external provider

• Ship spree_search_meilisearch as an official Spree gem. Uses meilisearch-ruby (raw client), NOT meilisearch-rails (no model-level overrides).
• Index all products (all statuses). Store API applies .active scope to the base AR scope before passing to the provider. Admin passes all products. The provider searches the full index, AR scope intersects for visibility.
• Meilisearch handles search + filtering + faceting in one call — replaces both Ransack filtering and FiltersAggregator with Meilisearch's native capabilities. Adjusted facet counts come for free.
• Docs recommend Meilisearch as the production search solution. The Database provider is for development and small catalogs.

Design Details

Provider Base Class

module Spree
  module SearchProvider
    class Base
      attr_reader :store

      def initialize(store)
        @store = store
      end

      # Search, filter, and return facets in one call.
      #
      # @param scope [ActiveRecord::Relation] base scope (store-scoped, visibility-filtered, authorized)
      # @param query [String, nil] text search query
      # @param filters [Hash] structured filters (price_gte, with_option_value_ids, in_category, in_categories, etc.)
      # @param sort [String, nil] sort param (e.g. 'price', '-price', 'best_selling')
      # @param page [Integer] page number
      # @param limit [Integer] results per page
      # @return [SearchResult] { products: AR::Relation, filters: Array, total_count: Integer }
      def search_and_filter(scope:, query: nil, filters: {}, sort: nil, page: 1, limit: 25)
        raise NotImplementedError
      end

      # Index a product — called after product save. No-op for database provider.
      #
      # @param product [Spree::Product] the product to index
      def index(product)
        # no-op by default
      end

      # Remove a product from the index.
      #
      # @param product [Spree::Product] the product to remove
      def remove(product)
        # no-op by default
      end

      # Bulk reindex — full catalog sync. Called manually or via rake task.
      #
      # @param scope [ActiveRecord::Relation] products to reindex (default: all in store)
      def reindex(scope = nil)
        # no-op by default
      end
    end

    # Returned by search_and_filter
    SearchResult = Struct.new(:products, :filters, :sort_options, :total_count, keyword_init: true)
  end
end

Database Provider (default)

Wraps existing Ransack + SQL scopes + FiltersAggregator. Behavior identical to today.

module Spree
  module SearchProvider
    class Database < Base
      def search_and_filter(scope:, query: nil, filters: {}, sort: nil, page: 1, limit: 25)
        # 1. Text search via ILIKE scope
        scope = scope.search(query) if query.present?

        # 2. Structured filtering via Ransack
        scope = scope.ransack(sanitize_filters(filters)).result(distinct: true) if filters.present?

        # 3. Sorting
        scope = apply_sort(scope, sort) if sort.present?

        # 4. Facets via FiltersAggregator + metafield facets
        filter_facets = build_facets(scope)

        # 5. Pagination
        total = scope.count
        products = scope.offset((page - 1) * limit).limit(limit)

        SearchResult.new(
          products: products,
          filters: filter_facets,
          sort_options: available_sort_options,
          total_count: total
        )
      end

      private

      def build_facets(scope)
        aggregator = Spree::Api::V3::FiltersAggregator.new(
          scope: scope,
          store: store,
          currency: Spree::Current.currency
        )

        result = aggregator.call
        result[:filters].concat(build_metafield_facets(scope))
        result[:filters]
      end

      def build_metafield_facets(scope)
        definitions = Spree::MetafieldDefinition
          .where(resource_type: 'Spree::Product', filterable: true)
          .order(:sort_order)

        product_ids = scope.select(:id)

        definitions.filter_map do |definition|
          values_with_counts = Spree::Metafield
            .where(resource_type: 'Spree::Product', resource_id: product_ids)
            .where(namespace: definition.namespace, key: definition.key)
            .group(:value)
            .count

          next if values_with_counts.empty?

          {
            id: definition.prefixed_id,
            type: 'metafield',
            name: definition.key,
            namespace: definition.namespace,
            presentation: definition.name,
            options: values_with_counts.map do |value, count|
              { value: value, count: count }
            end.sort_by { |o| -o[:count] }
          }
        end
      end

      def sanitize_filters(filters)
        # Strip text search param — already handled by scope.search
        filters.except(:search)
      end

      def apply_sort(scope, sort)
        # Delegate to existing sort logic (Ransack s param or custom scopes)
        scope
      end

      def available_sort_options
        %w[price -price best_selling name -name -available_on available_on]
      end
    end
  end
end

Meilisearch Provider (spree_search_meilisearch gem)

Uses meilisearch-ruby (raw HTTP client). No model-level overrides.

module SpreeMeilisearch
  class SearchProvider < Spree::SearchProvider::Base
    def search_and_filter(scope:, query: nil, filters: {}, sort: nil, page: 1, limit: 25)
      # One Meilisearch call: search + filter + facets
      ms_result = client.index(index_name).search(query || '', {
        filter: build_meilisearch_filters(filters),
        facets: facet_attributes,
        sort: build_meilisearch_sort(sort),
        offset: (page - 1) * limit,
        limit: limit
      })

      ids = ms_result['hits'].map { |h| h['id'] }

      # Intersect with AR scope for security/visibility
      products = scope.where(id: ids).in_order_of(:id, ids)

      SearchResult.new(
        products: products,
        filters: build_filters_from_facet_distribution(ms_result['facetDistribution']),
        sort_options: available_sort_options,
        total_count: ms_result['estimatedTotalHits']
      )
    end

    def index(product)
      client.index(index_name).add_documents([serialize(product)])
    end

    def remove(product)
      client.index(index_name).delete_document(product.id)
    end

    def reindex(scope = nil)
      scope ||= store.products
      scope.find_in_batches(batch_size: 500) do |batch|
        client.index(index_name).add_documents(batch.map { |p| serialize(p) })
      end
    end

    private

    def client
      @client ||= MeiliSearch::Client.new(
        ENV['MEILISEARCH_URL'] || 'http://localhost:7700',
        ENV['MEILISEARCH_API_KEY']
      )
    end

    def index_name
      "#{store.code}_products"
    end

    def serialize(product)
      {
        id: product.id,
        prefixed_id: product.prefixed_id,
        name: product.name,
        description: product.description,
        slug: product.slug,
        status: product.status,
        price: product.price_in(store.default_currency)&.amount&.to_f,
        currency: store.default_currency,
        category_ids: product.taxon_ids,
        option_values: product.variants.flat_map { |v| v.option_values.map(&:presentation) },
        sku: product.sku,
        in_stock: product.in_stock?,
        primary_media_url: product.primary_media&.url(:large),
        created_at: product.created_at.to_i,
        available_on: product.available_on&.to_i,
        # Filterable metafields
        **filterable_metafields(product)
      }
    end

    def filterable_metafields(product)
      Spree::MetafieldDefinition.where(resource_type: 'Spree::Product', filterable: true).each_with_object({}) do |defn, hash|
        metafield = product.metafields.detect { |m| m.namespace == defn.namespace && m.key == defn.key }
        hash["mf_#{defn.namespace}_#{defn.key}"] = metafield&.value
      end
    end

    def facet_attributes
      # Option types + filterable metafields + price + stock
      ['price', 'in_stock', 'category_ids'] +
        Spree::OptionType.filterable.pluck(:name).map { |n| "option_#{n}" } +
        Spree::MetafieldDefinition.where(resource_type: 'Spree::Product', filterable: true)
          .pluck(:namespace, :key).map { |ns, k| "mf_#{ns}_#{k}" }
    end
  end
end

Integration Points

1. Products Controller

The Store API products controller delegates to the provider for all search + filtering:

module Spree::Api::V3::Store
  class ProductsController < ResourceController
    private

    def collection
      base_scope = scope.accessible_by(current_ability, :show)

      result = search_provider.search_and_filter(
        scope: base_scope,
        query: params.dig(:q, :search),
        filters: params[:q]&.except(:search),
        sort: params[:sort],
        page: params[:page],
        limit: params[:limit]
      )

      # Store filters for the response (if requested via expand=filters)
      @filters = result.filters
      @total_count = result.total_count

      result.products
    end

    def search_provider
      @search_provider ||= Spree.search_provider.constantize.new(current_store)
    end
  end
end

2. Filters Controller

module Spree::Api::V3::Store
  class Products::FiltersController < BaseController
    def index
      base_scope = current_store.products
        .active(current_currency)
        .accessible_by(current_ability, :show)

      result = search_provider.search_and_filter(
        scope: base_scope,
        query: params.dig(:q, :search),
        filters: params[:q]&.except(:search),
        page: 1,
        limit: 0  # no products needed, just facets
      )

      render json: {
        filters: result.filters,
        sort_options: result.sort_options,
        total_count: result.total_count
      }
    end

    private

    def search_provider
      @search_provider ||= Spree.search_provider.constantize.new(current_store)
    end
  end
end

3. Product Indexing Hooks

module Spree
  module SearchIndexable
    extend ActiveSupport::Concern

    included do
      after_commit :index_in_search, on: [:create, :update]
      after_commit :remove_from_search, on: :destroy
    end

    private

    def index_in_search
      return unless stores.any?

      stores.each do |store|
        provider = Spree.search_provider.constantize.new(store)
        provider.index(self)
      end
    end

    def remove_from_search
      stores.each do |store|
        provider = Spree.search_provider.constantize.new(store)
        provider.remove(self)
      end
    end
  end
end

# In Spree::Product
class Spree::Product < Spree.base_class
  include Spree::SearchIndexable
end

For the Database provider, index and remove are no-ops. For external providers, these trigger async indexing (via a job wrapper if needed).

4. Rake Task

# rake spree:search:reindex
namespace :spree do
  namespace :search do
    task reindex: :environment do
      Spree::Store.all.find_each do |store|
        provider = Spree.search_provider.constantize.new(store)
        puts "Reindexing #{store.name} (#{store.products.count} products)..."
        provider.reindex(store.products)
        puts "Done."
      end
    end
  end
end

MetafieldDefinition Changes

class Spree::MetafieldDefinition < Spree.base_class
  # New column
  attribute :filterable, :boolean, default: false

  scope :filterable, -> { where(filterable: true) }
end

Store API Filter Response (extended)

The filter response shape stays the same, with a new type: 'metafield' filter type:

{
  "filters": [
    { "id": "price", "type": "price_range", "min": 10, "max": 100, "currency": "USD" },
    { "id": "availability", "type": "availability", "options": [...] },
    { "id": "opt_abc", "type": "option", "name": "size", "presentation": "Size", "options": [...] },
    {
      "id": "mfdef_xyz",
      "type": "metafield",
      "name": "material",
      "namespace": "product",
      "presentation": "Material",
      "options": [
        { "value": "Cotton", "count": 42 },
        { "value": "Polyester", "count": 18 }
      ]
    }
  ],
  "sort_options": [...],
  "total_count": 60
}

Product Filtering by Metafield

New Ransack scope on Product:

# In Product model
scope :with_metafield, ->(namespace, key, value) {
  joins(:metafields)
    .where(spree_metafields: { namespace: namespace, key: key, value: value })
}

API usage: q[with_metafield][]=product|material|Cotton

SDK Types (generated)

interface MetafieldFilter {
  id: string          // prefixed MetafieldDefinition ID
  type: 'metafield'
  name: string        // key
  namespace: string
  presentation: string // MetafieldDefinition.name
  options: Array<{
    value: string
    count: number
  }>
}

// ProductFilter union gains MetafieldFilter
type ProductFilter = PriceRangeFilter | AvailabilityFilter | OptionFilter | CategoryFilter | MetafieldFilter

Migration Path

Phase 1: Remove add_search_scope macro

• Convert all ~25 add_search_scope calls in ProductScopes to standard scope definitions
• Remove add_search_scope method, search_scopes class variable, and self.search_scopes accessor
• Update whitelisted_ransackable_scopes if needed (should be identical)
• Run full test suite to verify no behavior change

Phase 2: Remove PgSearch

• Remove all if defined?(PgSearch) conditionals in Product and Variant
• Keep only the ILIKE/SQL branches (the current fallback paths)
• Remove pg_search_scope declarations: search_by_name, search_by_sku, search_by_sku_or_options
• Replace with simple ILIKE scopes where needed (Product name search, Variant SKU search)
• Remove pg_search from spree_core.gemspec optional dependencies
• Remove any PgSearch-related translation model scopes (Translation.search_by_name)
• Run full test suite — no test should depend on PgSearch

Phase 3: Provider interface + Database provider

• Create Spree::SearchProvider::Base and Spree::SearchProvider::Database
• Register search_provider in Spree::Dependencies
• Database provider wraps existing search scope, FiltersAggregator, and ILIKE autocomplete
• Zero behavior change — existing code paths are wrapped, not replaced

Phase 4: Controller integration

• Wire search_provider.search() into ProductsController for q[search] param
• Wire search_provider.aggregate() into FiltersController
• Add SearchIndexable concern to Product (no-op for database provider)
• Add rake spree:search:reindex task

Phase 5: Meilisearch provider (official gem)

• Create spree_search_meilisearch gem (official, maintained by Spree team)
• Uses meilisearch-ruby gem (raw client, no model-level overrides)
• Indexes all products (all statuses) — storefront scopes handle visibility
• Implements: search + filter + faceted aggregation with adjusted counts in one call
• SearchIndexable concern on Product triggers async indexing
• rake spree:search:reindex delegates to Meilisearch bulk indexing
• Documented as the recommended production search solution

Phase 6: MetafieldDefinition.filterable (6.0)

class AddFilterableToMetafieldDefinitions < ActiveRecord::Migration[7.2]
  def change
    add_column :spree_metafield_definitions, :filterable, :boolean, null: false, default: false
    add_index :spree_metafield_definitions, :filterable
  end
end

• Add filterable column and scope
• Extend both providers to include metafield facets
• Add with_metafield Ransack scope to Product (for Database provider)
• Meilisearch provider indexes filterable metafields as document attributes

Constraints on Current Work

• Don't add new add_search_scope calls. Use standard scope definitions. The macro is being removed.
• Don't add new PgSearch scopes. PgSearch is being removed. Use ILIKE for new search logic.
• Don't add new search logic to ProductScopes. New search/filter behavior should be designed for the provider interface.
• Don't hardcode FiltersAggregator usage in controllers. It will be wrapped by the provider.
• New filterable attributes should use MetafieldDefinition. Don't add one-off filterable columns on other models.

Resolved Questions

1. Admin search. Same provider powers both Store API and Admin API. Meilisearch indexes all products (all statuses). Store API applies .active scope to filter results. Admin has no such scope — gets full search across drafts, archived, paused. No separate admin provider needed.

2. PgSearch vs Meilisearch. Drop PgSearch entirely. It's PostgreSQL-only, adds conditional code paths, and provides marginal benefit over ILIKE. Meilisearch is the recommended upgrade for serious search needs — typo tolerance, stemming, synonyms, faceting, and relevance tuning. The Database provider uses plain ILIKE as the baseline. We use meilisearch-ruby (raw client), not meilisearch-rails, to avoid model-level method overrides and own the integration fully.

3. add_search_scope removal. Convert to standard Rails scopes. The macro adds no value — it was created before Rails scopes were as capable as they are today. Standard scopes work identically with Ransack.

Open Questions

1. Per-store providers. Should different stores in the same Spree instance use different search providers? (e.g., small store uses Database, large store uses Meilisearch). If yes, provider config moves to Store model instead of global Dependencies. Leaning toward global for 6.0, per-store for 6.1.

2. Cross-model search. Enterprise buyers want unified search across products, categories, pages, and blog posts. Should the provider interface support multiple resource types, or is product search the only contract? Leaning toward product-only for 6.0.

3. Filter count adjustment. Should the Database provider attempt cross-filter count adjustment (selecting "Size: M" updates "Color: Red" count)? This requires N+1 subqueries in SQL. Meilisearch handles this natively via facetDistribution. Leaning toward naive counts for Database provider, adjusted counts for Meilisearch.

References

• Current search scopes: spree/core/app/models/concerns/spree/product_scopes.rb (has add_search_scope macro — being removed)
• Current finder: spree/core/app/finders/spree/products/find.rb
• Current filter aggregator: spree/api/app/services/spree/api/v3/filters_aggregator.rb
• Current filters endpoint: spree/api/app/controllers/spree/api/v3/store/products/filters_controller.rb
• Current products controller: spree/api/app/controllers/spree/api/v3/store/products_controller.rb
• Dependencies: spree/core/lib/spree/core/dependencies.rb
• Product search scope: spree/core/app/models/spree/product.rb (has PgSearch conditional — being removed)
• Variant search scopes: spree/core/app/models/spree/variant.rb (has PgSearch scopes — being removed)
• MetafieldDefinition: spree/core/app/models/spree/metafield_definition.rb
• meilisearch-ruby client: https://github.com/meilisearch/meilisearch-ruby
• Rename: multi_search → search shipped in 5.4 (backward compat alias kept until 6.0)
• Related plan: 6.0-product-types.md (MetafieldDefinition schema enforcement)
• Related plan: 6.0-channels-catalogs-b2b.md (Catalog-scoped product visibility)