DataHub Search Command

The datahub search command provides a powerful interface to search across all DataHub entities directly from the command line. It supports both keyword search (default) and semantic search with flexible filtering, multiple output formats, and comprehensive discovery features.

The search command is a command group with multiple subcommands:

query (default) - Execute search queries across DataHub entities
diagnose - Diagnose search configuration and availability

Quick Start

shell

# Basic keyword search (query is the default subcommand)
datahub search "users"
datahub search query "users"  # Explicit subcommand

# Search with filters
datahub search "*" --filter platform=snowflake --filter entity_type=dataset

# Semantic search
datahub search --semantic "financial reports"

# Get results in table format
datahub search "dashboard" --table

# Get just URNs for piping
datahub search "dataset" --urns-only

# Preview query without executing
datahub search "users" --filter platform=snowflake --dry-run

# Custom field projection (reduce response size)
datahub search "users" --projection "urn type platform { name }"

# Diagnose search configuration
datahub search diagnose
datahub search diagnose --format json

Core Features

1. Dual Search Modes

Keyword Search (Default): Traditional text-based search using DataHub's search index.

shell

datahub search "users"
datahub search "customer_data"

Semantic Search: AI-powered search that understands meaning and context.

shell

datahub search --semantic "financial reports"
datahub search --semantic "user analytics dashboards"

2. Flexible Filtering

Simple Filters: Easy key=value syntax with implicit AND logic.

shell

# Single filter
datahub search "*" --filter platform=snowflake

# Multiple filters (AND)
datahub search "*" --filter platform=snowflake --filter entity_type=dataset

# OR logic with comma-separated values
datahub search "*" --filter platform=snowflake,bigquery

Complex Filters: JSON-based filters with AND/OR/NOT logic.

shell

# AND logic
datahub search "*" --filters '{"and": [{"platform": ["snowflake"]}, {"env": ["PROD"]}]}'

# OR logic
datahub search "*" --filters '{"or": [{"entity_type": ["chart"]}, {"entity_type": ["dashboard"]}]}'

# NOT logic
datahub search "*" --filters '{"not": {"platform": ["mysql"]}}'

# Complex nested logic
datahub search "*" --filters '{
  "and": [
    {"platform": ["snowflake", "bigquery"]},
    {"env": ["PROD"]},
    {"not": {"tag": ["urn:li:tag:Deprecated"]}}
  ]
}'

3. Multiple Output Formats

JSON (Default): Structured output with full metadata.

shell

datahub search "users" --format json
# or simply
datahub search "users"

Table: Human-readable table format.

shell

datahub search "users" --table
# or
datahub search "users" --format table

URNs: List of URNs only, perfect for piping.

shell

datahub search "dataset" --urns-only
# or
datahub search "dataset" --format urns

4. Filter Discovery

List Available Filters: See all filter types and their descriptions.

shell

datahub search --list-filters

Output includes:

entity_type, entity_subtype
platform, domain, container
tag, glossary_term, owner
env, status
custom, and, or, not

Describe Specific Filter: Get detailed information about a filter.

shell

datahub search --describe-filter platform
datahub search --describe-filter entity_type
datahub search --describe-filter custom

5. Faceted Search

Facets-Only Mode: Get aggregation counts without search results.

shell

datahub search "*" --facets-only

# With filters
datahub search "*" --filter entity_type=dataset --facets-only --format table

Useful for:

Understanding data distribution
Building filter UIs
Data exploration

6. Pagination

shell

# Get first 20 results
datahub search "*" --limit 20

# Get next page (results 20-40)
datahub search "*" --limit 20 --offset 20

# Maximum limit is 50
datahub search "*" --limit 50

7. Sorting

shell

# Sort by field
datahub search "*" --sort-by lastModified

# Sort ascending
datahub search "*" --sort-by name --sort-order asc

# Sort descending (default)
datahub search "*" --sort-by name --sort-order desc

8. DataHub Views

Apply saved DataHub Views to searches:

shell

datahub search "*" --view "urn:li:dataHubView:12345678"

9. Dry Run

Preview the compiled GraphQL query and variables without connecting to DataHub:

shell

# See what query would be sent
datahub search "users" --dry-run

# Dry run with filters and sorting
datahub search "*" --filter platform=snowflake --sort-by lastModified --dry-run

# Dry run with semantic search
datahub search --semantic "financial data" --dry-run

# Dry run with projection
datahub search "users" --projection "urn type" --dry-run

Output is a JSON object containing operation_name, graphql_field, variables, and optionally projection and query (when using --projection).

10. Field Projection

Control which GraphQL fields are returned per entity to reduce response size:

shell

# Return only URN and type
datahub search "users" --projection "urn type"

# Return URN with platform info
datahub search "users" --projection "urn platform { ...PlatformFields }"

# Load projection from a file
datahub search "users" --projection @my_fields.gql

# Braces are optional — these are equivalent
datahub search "users" --projection "{ urn type }"
datahub search "users" --projection "urn type"

When --projection is omitted, the full default selection set from search_queries.gql is used. Projections have a 5000-character limit and must not contain mutations, introspection queries, variables, or directives.

11. Agent Context

Print best-practice guidance for AI agents consuming the search CLI:

shell

# Print agent context and exit
datahub search --agent-context

When stdout is not a TTY (e.g., piped to an AI agent), --help automatically appends the agent context.

Command Reference

Main Command

shell

datahub search [OPTIONS] COMMAND [ARGS]

The search command is a command group. The default subcommand is query, so datahub search "text" is equivalent to datahub search query "text".

Subcommands

`query` (default)

Execute search queries across DataHub entities.

shell

datahub search [QUERY] [OPTIONS]
datahub search query [QUERY] [OPTIONS]

Options:

Option	Type	Description
`QUERY`	string	Search query string (default: "*" for all entities)
`--semantic`	flag	Use semantic search instead of keyword search
`-f, --filter`	string	Simple filter: key=value (repeatable, comma for OR)
`--filters`	string	Complex filters as JSON string (AND/OR/NOT logic)
`-n, --limit`	int	Number of results to return (default: 10, max: 50)
`--offset`	int	Starting position for pagination (default: 0)
`--sort-by`	string	Field name to sort by
`--sort-order`	choice	Sort order: `asc` or `desc` (default: desc)
`--format`	choice	Output format: `json`, `table`, or `urns` (default: json)
`--table`	flag	Shortcut for --format table
`--urns-only`	flag	Shortcut for --format urns
`--list-filters`	flag	List all available filter fields
`--describe-filter`	string	Describe a specific filter field
`--facets-only`	flag	Return only facets, no search results
`--view`	string	DataHub View URN to apply
`--dry-run`	flag	Show compiled GraphQL query and variables without executing
`--projection`	string	Custom GraphQL selection set for entity (inline or @file)
`--agent-context`	flag	Print agent skill context (best practices for AI agents)

`diagnose`

Diagnose search configuration and availability. Checks semantic search configuration, backend connectivity, embedding provider, model details, and enabled entity types.

shell

datahub search diagnose [OPTIONS]

Options:

Option	Type	Description
`--format`	choice	Output format: `text` or `json` (default: text)

Examples:

shell

# Text output (default)
datahub search diagnose

# JSON output
datahub search diagnose --format json

Available Filters

Entity Filters

entity_type: Filter by entity type (dataset, dashboard, chart, corpuser, etc.)
entity_subtype: Filter by entity subtype (Table, View, Model, etc.)

Platform & Location

platform: Filter by data platform (snowflake, bigquery, looker, etc.)
container: Filter by container URN (database, schema, etc.)
domain: Filter by domain URN

Metadata

tag: Filter by tag URN
glossary_term: Filter by glossary term URN
owner: Filter by owner URN (user or group)

Environment

env: Filter by environment (PROD, DEV, STAGING, QA)
status: Filter by deletion status (NOT_SOFT_DELETED, SOFT_DELETED)

Advanced

custom: Custom field filter with flexible conditions
and: Logical AND of filters
or: Logical OR of filters
not: Logical NOT of a filter

Usage Examples

Basic Searches

shell

# Search for everything
datahub search "*"

# Search for specific term
datahub search "users"

# Search with limit
datahub search "dashboard" --limit 20

Semantic Search

shell

# Find financial datasets
datahub search --semantic "financial reports and revenue data"

# Find user analytics
datahub search --semantic "user behavior analysis dashboards"

# Find ML models
datahub search --semantic "machine learning models for prediction"

Simple Filtering

shell

# Platform filter
datahub search "*" --filter platform=snowflake

# Entity type filter
datahub search "*" --filter entity_type=dataset

# Multiple filters (AND)
datahub search "*" --filter platform=snowflake --filter env=PROD

# OR with comma separation
datahub search "*" --filter platform=snowflake,bigquery

# Combine different filters
datahub search "customer" --filter platform=mysql --filter entity_type=dataset,dashboard

Complex Filtering

shell

# Find Snowflake PROD datasets
datahub search "*" --filters '{
  "and": [
    {"platform": ["snowflake"]},
    {"entity_type": ["dataset"]},
    {"env": ["PROD"]}
  ]
}'

# Find charts OR dashboards in Looker
datahub search "*" --filters '{
  "and": [
    {"platform": ["looker"]},
    {"or": [
      {"entity_type": ["chart"]},
      {"entity_type": ["dashboard"]}
    ]}
  ]
}'

# Exclude deprecated datasets
datahub search "*" --filters '{
  "and": [
    {"entity_type": ["dataset"]},
    {"not": {"tag": ["urn:li:tag:Deprecated"]}}
  ]
}'

# Custom field filter
datahub search "*" --filters '{
  "custom": {
    "field": "name",
    "condition": "CONTAIN",
    "values": ["customer"]
  }
}'

Output Formats

shell

# JSON (default) - full metadata
datahub search "users" --format json

# Table - human readable
datahub search "users" --table

# URNs only - for piping to other commands
datahub search "*" --filter entity_type=dataset --urns-only | while read urn; do
  datahub get --urn "$urn" --aspect ownership
done

Discovery & Exploration

shell

# List all available filters
datahub search --list-filters

# Learn about a specific filter
datahub search --describe-filter platform
datahub search --describe-filter entity_type
datahub search --describe-filter custom

# Explore data distribution with facets
datahub search "*" --facets-only --format table
datahub search "*" --filter entity_type=dataset --facets-only

# Diagnose search configuration
datahub search diagnose
datahub search diagnose --format json

Pagination

shell

# First page (10 results)
datahub search "dataset"

# Second page
datahub search "dataset" --limit 10 --offset 10

# Large page
datahub search "*" --limit 50 --offset 100

Sorting

shell

# Sort by last modified (newest first)
datahub search "*" --sort-by lastModified --sort-order desc

# Sort by name (alphabetically)
datahub search "*" --sort-by name --sort-order asc

Advanced Use Cases

Finding Datasets in Multiple Platforms

shell

# Quick version with simple filters
datahub search "*" --filter platform=snowflake,bigquery,postgres --filter entity_type=dataset --table

# Complex version with AND/OR
datahub search "*" --filters '{
  "and": [
    {"entity_type": ["dataset"]},
    {"or": [
      {"platform": ["snowflake"]},
      {"platform": ["bigquery"]},
      {"platform": ["postgres"]}
    ]}
  ]
}' --table

Finding Production Data with Specific Tags

shell

datahub search "*" --filters '{
  "and": [
    {"env": ["PROD"]},
    {"tag": ["urn:li:tag:PII", "urn:li:tag:Sensitive"]}
  ]
}' --table

Finding Datasets Owned by Specific Team

shell

datahub search "*" --filters '{
  "and": [
    {"entity_type": ["dataset"]},
    {"owner": ["urn:li:corpGroup:data-eng"]}
  ]
}' --table

Pipeline: Search and Process

shell

# Find all PROD datasets and check their ownership
datahub search "*" \
  --filter entity_type=dataset \
  --filter env=PROD \
  --urns-only | \
while read urn; do
  echo "Checking $urn"
  datahub get --urn "$urn" --aspect ownership
done

# Find datasets and export to file
datahub search "*" \
  --filter platform=snowflake \
  --format json > snowflake_datasets.json

Semantic Search for Data Discovery

shell

# Find datasets about customers
datahub search --semantic "customer information and profiles"

# Find ML models for forecasting
datahub search --semantic "predictive models and forecasting algorithms"

# Find compliance-related documentation
datahub search --semantic "GDPR compliance and privacy documentation"

Output Examples

JSON Output (Default)

json

{
  "count": 2,
  "facets": [
    {
      "aggregations": [
        { "count": 10, "value": "DATASET" },
        { "count": 5, "value": "DASHBOARD" }
      ],
      "displayName": "Type",
      "field": "_entityType"
    }
  ],
  "searchResults": [
    {
      "entity": {
        "platform": { "name": "snowflake" },
        "properties": {
          "description": "User data table",
          "name": "users"
        },
        "urn": "urn:li:dataset:(urn:li:dataPlatform:snowflake,db.users,PROD)"
      },
      "matchedFields": [{ "name": "name", "value": "users" }]
    }
  ],
  "start": 0,
  "total": 15
}

Table Output

+--------------------------------------------------------------+------------+-----------+-------------------------+
| URN                                                          | Name       | Platform  | Description             |
+==============================================================+============+===========+=========================+
| urn:li:dataset:(urn:li:dataPlatform:snowflake,db.users,PROD)| users      | snowflake | User data table         |
+--------------------------------------------------------------+------------+-----------+-------------------------+
| urn:li:dashboard:(looker,dashboards.19)                     | User Stats | looker    | User analytics dash...  |
+--------------------------------------------------------------+------------+-----------+-------------------------+

Showing 1-2 of 15 results

URNs Output

urn:li:dataset:(urn:li:dataPlatform:snowflake,db.users,PROD)
urn:li:dashboard:(looker,dashboards.19)
urn:li:chart:(looker,chart.123)

Integration Examples

Shell Scripts

bash

#!/bin/bash
# Find and tag all PROD datasets in Snowflake

DATASETS=$(datahub search "*" \
  --filter platform=snowflake \
  --filter entity_type=dataset \
  --filter env=PROD \
  --urns-only)

for urn in $DATASETS; do
  echo "Tagging $urn"
  datahub put --urn "$urn" --aspect globalTags -d '{
    "tags": [{"tag": "urn:li:tag:Production"}]
  }'
done

Data Quality Checks

bash

#!/bin/bash
# Find datasets without ownership

DATASETS=$(datahub search "*" \
  --filter entity_type=dataset \
  --limit 50 \
  --format json)

echo "$DATASETS" | jq -r '.searchResults[] |
  select(.entity.ownership == null) |
  .entity.urn'

Monitoring

bash

#!/bin/bash
# Monitor new datasets added today

COUNT=$(datahub search "*" \
  --filter entity_type=dataset \
  --format json | jq '.total')

echo "Total datasets: $COUNT"

Error Handling

The CLI provides clear error messages for common issues:

shell

# Semantic search not enabled
datahub search --semantic "test"
# Error: Semantic search is not enabled on this DataHub instance.
# Use keyword search (default) or contact your DataHub administrator.

# Invalid filter format
datahub search "*" --filter invalid_format
# Error: Invalid filter: invalid_format. Expected key=value format.

# Conflicting filter options
datahub search "*" --filter platform=snowflake --filters '{"env": ["PROD"]}'
# Error: Cannot use both --filter and --filters.
# Use --filter for simple filters or --filters for complex logic.

# Invalid limit
datahub search "*" --limit 0
# Error: --limit must be at least 1

# Limit too high
datahub search "*" --limit 100
# Warning: limit capped at 50

Diagnostics

The diagnose subcommand helps troubleshoot search configuration issues, particularly for semantic search:

shell

# Check semantic search configuration
datahub search diagnose

# Get detailed JSON output
datahub search diagnose --format json

The diagnostic output includes:

Semantic search availability and status
Embedding provider configuration
Model ID and embedding key
AWS region (if applicable)
Enabled entity types for semantic search
Recommendations for resolving issues

Tips and Best Practices

Start Broad: Begin with datahub search "*" and add filters progressively
Use Discovery: Run --list-filters and --describe-filter to learn available options
Test with --table: Use table format for quick exploration before JSON processing
Semantic Search: Use semantic search for natural language queries and concept-based discovery
Facets First: Use --facets-only to understand data distribution before filtering
Pipe URNs: Combine --urns-only with other datahub commands for bulk operations
Limit Wisely: Start with small limits during development, increase for production
Save Complex Filters: Store complex JSON filters in files for reusability
Check Pagination: Use total field in JSON output to determine if pagination is needed
Environment Variables: Configure DATAHUB_GMS_URL and DATAHUB_GMS_TOKEN for easier access

Troubleshooting

Common Issues

"Semantic search is not enabled": Your DataHub instance doesn't have semantic search configured. Use keyword search instead or contact your administrator.

"No results found": Try broadening your search query or removing filters. Use datahub search "*" to verify connectivity.

"Invalid filter": Check filter syntax. Use --list-filters to see available filters and --describe-filter <name> for details.

Slow searches: Use more specific filters to narrow results. Check network latency to DataHub server.

Permission denied: Ensure you've run datahub init and have valid credentials configured.

Getting Help

shell

# Command help
datahub search --help

# Subcommand help
datahub search query --help
datahub search diagnose --help

# List available filters
datahub search --list-filters

# Learn about specific filter
datahub search --describe-filter platform

# Diagnose search configuration
datahub search diagnose

# Check DataHub connection
datahub check plugins

DataHub Search Command

DataHub Search Command

Quick Start

Core Features

1. Dual Search Modes

2. Flexible Filtering

3. Multiple Output Formats

4. Filter Discovery

5. Faceted Search

6. Pagination

7. Sorting

8. DataHub Views

9. Dry Run

10. Field Projection

11. Agent Context

Command Reference

Main Command

Subcommands

query (default)

diagnose

Available Filters

Entity Filters

Platform & Location

Metadata

Environment

Advanced

Usage Examples

Basic Searches

Semantic Search

Simple Filtering

Complex Filtering

Output Formats

Discovery & Exploration

Pagination

Sorting

Advanced Use Cases

Finding Datasets in Multiple Platforms

Finding Production Data with Specific Tags

Finding Datasets Owned by Specific Team

Pipeline: Search and Process

Semantic Search for Data Discovery

Output Examples

JSON Output (Default)

Table Output

URNs Output

Integration Examples

Shell Scripts

Data Quality Checks

Monitoring

Error Handling

Diagnostics

Tips and Best Practices

Troubleshooting

Common Issues

Getting Help

See Also

`query` (default)

`diagnose`