Back to Kibana

Osquery Scripts

x-pack/platform/plugins/shared/osquery/scripts/readme.md

9.4.05.1 KB
Original Source

Osquery Scripts

Synthetic Data Generator (create_actions/)

Creates synthetic osquery action, response, result, and scheduled response documents for performance testing and development.

The script populates three Elasticsearch data streams:

  • Actions (.logs-osquery_manager.actions-default) -- on-demand and pack queries with agents, users, cases
  • Responses (logs-osquery_manager.action.responses-default) -- per-agent query responses (on-demand + scheduled)
  • Results (logs-osquery_manager.result-default) -- individual result rows with realistic osquery column data
bash
node x-pack/platform/plugins/shared/osquery/scripts/create_actions [options]

Options

Actions & Responses

OptionDefaultDescription
--count, -c500Number of action documents to create
--packRatio, --pr0.2Fraction of actions that are pack queries (0.0-1.0)
--queriesPerPack5Number of sub-queries per pack action
--minAgents1Minimum agents per action
--maxAgents40Maximum agents per action (randomized between min and max)
--errorRate, --er0.25Fraction of responses that are errors (0.0-1.0)
--users, -u5Number of unique synthetic user profiles
--ruleRatio, --rr0.1Fraction of actions that are rule-triggered with no user_id (0.0-1.0)
--cases15Number of real cases to create via Cases API
--caseRatio0.3Fraction of actions with case_ids attached (0.0-1.0)

Results

OptionDefaultDescription
--results, -rtrueGenerate mock result documents in the results index
--maxResultRows50Maximum result rows per agent per query (randomized 1 to N)

Scheduled Responses

Scheduled responses simulate periodic pack execution over the last 30 days. Each document includes space_id, pack_id, pack_name, and schedule_id to mirror real scheduled query data.

OptionDefaultDescription
--scheduledtrueGenerate scheduled response documents
--scheduledPacks5Number of simulated scheduled packs
--scheduledQueriesPerPack5Queries per scheduled pack
--scheduledExecutions200Total execution cycles to generate per query
--scheduledAgents10Number of agents reporting per scheduled execution
--scheduledErrorRate0.05Fraction of scheduled responses that are errors (0.0-1.0)

Total scheduled docs = scheduledPacks × scheduledQueriesPerPack × scheduledExecutions × scheduledAgents.

Infrastructure

OptionDefaultDescription
--es, -ehttp://elastic:[email protected]:9200Elasticsearch URL
--kibana, -khttp://elastic:[email protected]:5601Kibana URL (for Cases API)
--batchSize, --bs500Documents per bulk request
--delete, -dfalseDelete previously generated synthetic data first
--deleteOnlyfalseOnly delete existing synthetic data, do not create new

Examples

bash
# Default run - 500 actions + responses + results + scheduled responses
node x-pack/platform/plugins/shared/osquery/scripts/create_actions

# Clean slate with 1000 actions
node x-pack/platform/plugins/shared/osquery/scripts/create_actions --count 1000 --delete

# Only delete all previously generated synthetic data (actions, responses, results, cases)
node x-pack/platform/plugins/shared/osquery/scripts/create_actions --deleteOnly

# Stress test - 10k pack actions, 100 agents each, all with cases
node x-pack/platform/plugins/shared/osquery/scripts/create_actions \
  --count 10000 \
  --packRatio 1.0 \
  --queriesPerPack 10 \
  --minAgents 100 --maxAgents 100 \
  --caseRatio 1.0 \
  --delete

# Actions only - skip results and scheduled responses
node x-pack/platform/plugins/shared/osquery/scripts/create_actions \
  --no-results \
  --no-scheduled

# Heavy scheduled data - 10 packs, 10 queries each, 500 executions, 50 agents (2.5M docs)
node x-pack/platform/plugins/shared/osquery/scripts/create_actions \
  --count 100 \
  --scheduledPacks 10 \
  --scheduledQueriesPerPack 10 \
  --scheduledExecutions 500 \
  --scheduledAgents 50 \
  --delete

# High error-rate scenario for testing error handling UI
node x-pack/platform/plugins/shared/osquery/scripts/create_actions \
  --count 200 \
  --errorRate 0.8 \
  --scheduledErrorRate 0.5

# Custom Elasticsearch/Kibana endpoints
node x-pack/platform/plugins/shared/osquery/scripts/create_actions \
  --es http://elastic:changeme@localhost:9220 \
  --kibana http://elastic:changeme@localhost:5620

Schema Formatter (schema_formatter/)

Extracts only the currently used fields from osquery schema files (manually curated selection). Output goes to public/editor/osquery_schema.

bash
node x-pack/platform/plugins/shared/osquery/scripts/schema_formatter/ecs.js --schema_version=4.6.0
node x-pack/platform/plugins/shared/osquery/scripts/schema_formatter/osquery.js --schema_version=4.6.0

Note: CSV exports may have capitalized field names that need to be transformed to lowercase.