ContextQMD
Back to N8n Mcp

P0-R3 Feature Test Coverage Plan

P0-R3-TEST-PLAN.md

2.50.412.8 KB
Original Source

P0-R3 Feature Test Coverage Plan

Executive Summary

This document outlines comprehensive test coverage for the P0-R3 feature (Template-based Configuration Examples). The feature adds real-world configuration examples from popular templates to node search and essentials tools.

Feature Overview:

  • New database table: template_node_configs (197 pre-extracted configurations)
  • Enhanced tools: search_nodes({includeExamples: true}) and get_node_essentials({includeExamples: true})
  • Breaking changes: Removed get_node_for_task tool

Test Files Created

Unit Tests

1. /tests/unit/scripts/fetch-templates-extraction.test.ts ✅

Purpose: Test template extraction logic from fetch-templates.ts

Coverage:

  • extractNodeConfigs() - 90%+ coverage

    • Valid workflows with multiple nodes
    • Empty workflows
    • Malformed compressed data
    • Invalid JSON
    • Nodes without parameters
    • Sticky note filtering
    • Credential handling
    • Expression detection
    • Special characters
    • Large workflows (100 nodes)

  • detectExpressions() - 100% coverage

    • ={{...}} syntax detection
    • $json references
    • $node references
    • Nested objects
    • Arrays
    • Null/undefined handling
    • Multiple expression types

Test Count: 27 tests Expected Coverage: 92%+

2. /tests/unit/mcp/search-nodes-examples.test.ts ✅

Purpose: Test search_nodes tool with includeExamples parameter

Coverage:

  • includeExamples parameter behavior
    • false: no examples returned
    • undefined: no examples returned (default)
    • true: examples returned
  • Example data structure validation
  • Top 2 limit enforcement
  • Backward compatibility
  • Performance (<100ms)
  • Error handling (malformed JSON, database errors)
  • searchNodesLIKE integration
  • searchNodesFTS integration

Test Count: 12 tests Expected Coverage: 85%+

3. /tests/unit/mcp/get-node-essentials-examples.test.ts ✅

Purpose: Test get_node_essentials tool with includeExamples parameter

Coverage:

  • includeExamples parameter behavior
  • Full metadata structure
    • configuration object
    • source (template, views, complexity)
    • useCases (limited to 2)
    • metadata (hasCredentials, hasExpressions)
  • Cache key differentiation
  • Backward compatibility
  • Performance (<100ms)
  • Error handling
  • Top 3 limit enforcement

Test Count: 13 tests Expected Coverage: 88%+

Integration Tests

4. /tests/integration/database/template-node-configs.test.ts ✅

Purpose: Test database schema, migrations, and operations

Coverage:

  • Schema validation
    • Table creation
    • All columns present
    • Correct types and constraints
    • CHECK constraint on complexity
  • Indexes
    • idx_config_node_type_rank
    • idx_config_complexity
    • idx_config_auth
  • View: ranked_node_configs
    • Top 5 per node_type
    • Correct ordering
  • Foreign key constraints
    • CASCADE delete
    • Referential integrity
  • Data operations
    • INSERT with all fields
    • Nullable fields
    • Rank updates
    • Delete rank > 10
  • Performance
    • 1000 records < 10ms queries
  • Migration idempotency

Test Count: 19 tests Expected Coverage: 95%+

5. /tests/integration/mcp/template-examples-e2e.test.ts ✅

Purpose: End-to-end integration testing

Coverage:

  • Direct SQL queries
    • Top 2 examples for search_nodes
    • Top 3 examples with metadata for get_node_essentials
  • Data structure validation
    • Valid JSON in all fields
    • Credentials when has_credentials=1
  • Ranked view functionality
  • Performance with 100+ configs
    • Query performance < 5ms
    • Complexity filtering
  • Edge cases
    • Non-existent node types
    • Long parameters_json (100 params)
    • Special characters (Unicode, emojis, symbols)
  • Data integrity
    • Foreign key constraints
    • Cascade deletes

Test Count: 14 tests Expected Coverage: 90%+

Test Fixtures

6. /tests/fixtures/template-configs.ts ✅

Purpose: Reusable test data

Provides:

  • sampleConfigs: 7 realistic node configurations

    • simpleWebhook
    • webhookWithAuth
    • httpRequestBasic
    • httpRequestWithExpressions
    • slackMessage
    • codeNodeTransform
    • codeNodeWithExpressions

  • sampleWorkflows: 3 complete workflows

    • webhookToSlack
    • apiWorkflow
    • complexWorkflow

  • Helper Functions:

    • compressWorkflow() - Compress to base64
    • createTemplateMetadata() - Generate metadata
    • createConfigBatch() - Batch create configs
    • getConfigByComplexity() - Filter by complexity
    • getConfigsWithExpressions() - Filter with expressions
    • getConfigsWithCredentials() - Filter with credentials
    • createInsertStatement() - SQL insert helper

Existing Tests Requiring Updates

High Priority

1. tests/unit/mcp/parameter-validation.test.ts

Line 480: Remove get_node_for_task from legacyValidationTools array

typescript
// REMOVE THIS:
{ name: 'get_node_for_task', args: {}, expected: 'Missing required parameters for get_node_for_task: task' },

Status: ⚠️ BREAKING CHANGE - Tool removed

2. tests/unit/mcp/tools.test.ts

Update: Remove get_node_for_task from templates category

typescript
// BEFORE:
templates: ['list_tasks', 'get_node_for_task', 'search_templates', ...]

// AFTER:
templates: ['list_tasks', 'search_templates', ...]

Add: Tests for new includeExamples parameter in tool definitions

typescript
it('should have includeExamples parameter in search_nodes', () => {
  const searchNodesTool = tools.find(t => t.name === 'search_nodes');
  expect(searchNodesTool.inputSchema.properties.includeExamples).toBeDefined();
  expect(searchNodesTool.inputSchema.properties.includeExamples.type).toBe('boolean');
  expect(searchNodesTool.inputSchema.properties.includeExamples.default).toBe(false);
});

it('should have includeExamples parameter in get_node_essentials', () => {
  const essentialsTool = tools.find(t => t.name === 'get_node_essentials');
  expect(essentialsTool.inputSchema.properties.includeExamples).toBeDefined();
});

Status: ⚠️ REQUIRED UPDATE

3. tests/integration/mcp-protocol/session-management.test.ts

Remove: Test case calling get_node_for_task with invalid task

typescript
// REMOVE THIS TEST:
client.callTool({ name: 'get_node_for_task', arguments: { task: 'invalid_task' } }).catch(e => e)

Status: ⚠️ BREAKING CHANGE

4. tests/integration/mcp-protocol/tool-invocation.test.ts

Remove: Entire get_node_for_task describe block

Add: Tests for new includeExamples functionality

typescript
describe('search_nodes with includeExamples', () => {
  it('should return examples when includeExamples is true', async () => {
    const response = await client.callTool({
      name: 'search_nodes',
      arguments: { query: 'webhook', includeExamples: true }
    });

    expect(response.results).toBeDefined();
    // Examples may or may not be present depending on database
  });

  it('should not return examples when includeExamples is false', async () => {
    const response = await client.callTool({
      name: 'search_nodes',
      arguments: { query: 'webhook', includeExamples: false }
    });

    expect(response.results).toBeDefined();
    response.results.forEach(node => {
      expect(node.examples).toBeUndefined();
    });
  });
});

describe('get_node_essentials with includeExamples', () => {
  it('should return examples with metadata when includeExamples is true', async () => {
    const response = await client.callTool({
      name: 'get_node_essentials',
      arguments: { nodeType: 'nodes-base.webhook', includeExamples: true }
    });

    expect(response.nodeType).toBeDefined();
    // Examples may or may not be present depending on database
  });
});

Status: ⚠️ REQUIRED UPDATE

Medium Priority

5. tests/unit/services/task-templates.test.ts

Status: ✅ No changes needed (TaskTemplates marked as deprecated but not removed)

Note: TaskTemplates remains for backward compatibility. Tests should continue to pass.

Test Execution Plan

Phase 1: Unit Tests

bash
# Run new unit tests
npm test tests/unit/scripts/fetch-templates-extraction.test.ts
npm test tests/unit/mcp/search-nodes-examples.test.ts
npm test tests/unit/mcp/get-node-essentials-examples.test.ts

# Expected: All pass, 52 tests

Phase 2: Integration Tests

bash
# Run new integration tests
npm test tests/integration/database/template-node-configs.test.ts
npm test tests/integration/mcp/template-examples-e2e.test.ts

# Expected: All pass, 33 tests

Phase 3: Update Existing Tests

bash
# Update files as outlined above, then run:
npm test tests/unit/mcp/parameter-validation.test.ts
npm test tests/unit/mcp/tools.test.ts
npm test tests/integration/mcp-protocol/session-management.test.ts
npm test tests/integration/mcp-protocol/tool-invocation.test.ts

# Expected: All pass after updates

Phase 4: Full Test Suite

bash
# Run all tests
npm test

# Run with coverage
npm run test:coverage

# Expected coverage improvements:
# - src/scripts/fetch-templates.ts: +20% (60% → 80%)
# - src/mcp/server.ts: +5% (75% → 80%)
# - Overall project: +2% (current → current+2%)

Coverage Expectations

New Code Coverage

FileFunctionTargetTests
fetch-templates.tsextractNodeConfigs95%15 tests
fetch-templates.tsdetectExpressions100%12 tests
server.tssearchNodes (with examples)90%8 tests
server.tsgetNodeEssentials (with examples)90%10 tests
Database migrationtemplate_node_configs100%19 tests

Overall Coverage Goals

  • Unit Tests: 90%+ coverage for new code
  • Integration Tests: All happy paths + critical error paths
  • E2E Tests: Complete feature workflows
  • Performance: All queries <10ms (database), <100ms (MCP)

Test Infrastructure

Dependencies Required

All dependencies already present in package.json:

  • vitest (test runner)
  • better-sqlite3 (database)
  • @vitest/coverage-v8 (coverage)

Test Utilities Used

  • TestDatabase helper (from existing test utils)
  • createTestDatabaseAdapter (from existing test utils)
  • Standard vitest matchers

No New Dependencies Required ✅

Regression Prevention

Critical Paths Protected

  1. Backward Compatibility

    • Tools work without includeExamples parameter
    • Existing workflows unchanged
    • Cache keys differentiated

  2. Performance

    • No degradation when includeExamples=false
    • Indexed queries <10ms
    • Example fetch errors don't break responses

  3. Data Integrity

    • Foreign key constraints enforced
    • JSON validation in all fields
    • Rank calculations correct

CI/CD Integration

GitHub Actions Updates

No changes required. Existing test commands will run new tests:

yaml
- run: npm test
- run: npm run test:coverage

Coverage Thresholds

Current thresholds maintained. Expected improvements:

  • Lines: +2%
  • Functions: +3%
  • Branches: +2%

Manual Testing Checklist

Pre-Deployment Verification

  • Run npm run rebuild - Verify migration applies cleanly
  • Run npm run fetch:templates --extract-only - Verify extraction works
  • Check database: SELECT COUNT(*) FROM template_node_configs - Should be ~197
  • Test MCP tool: search_nodes({query: "webhook", includeExamples: true})
  • Test MCP tool: get_node_essentials({nodeType: "nodes-base.webhook", includeExamples: true})
  • Verify backward compatibility: Tools work without includeExamples parameter
  • Performance test: Query 100 nodes with examples < 200ms

Rollback Plan

If issues are detected:

  1. Database Rollback:

    sql
    DROP TABLE IF EXISTS template_node_configs;
DROP VIEW IF EXISTS ranked_node_configs;

  2. Code Rollback:

    • Revert server.ts changes
    • Revert tools.ts changes
    • Restore get_node_for_task tool (if critical)

  3. Test Rollback:

    • Revert parameter-validation.test.ts
    • Revert tools.test.ts
    • Revert tool-invocation.test.ts

Success Metrics

Test Metrics

  • ✅ 85+ new tests added
  • ✅ 0 tests failing after updates
  • ✅ Coverage increase 2%+
  • ✅ All performance tests pass

Feature Metrics

  • ✅ 197 template configs extracted
  • ✅ Top 2/3 examples returned correctly
  • ✅ Query performance <10ms
  • ✅ No backward compatibility breaks

Conclusion

This test plan provides comprehensive coverage for the P0-R3 feature with:

  • 85+ new tests across unit, integration, and E2E levels
  • Complete coverage of extraction, storage, and retrieval
  • Backward compatibility protection
  • Performance validation (<10ms queries)
  • Clear migration path for existing tests

All test files are ready for execution. Update the 4 existing test files as outlined, then run the full test suite.

Estimated Total Implementation Time: 2-3 hours for updating existing tests + validation