ContextQMD
Back to Ruflo

SPARC BatchTool Orchestration Guide

v2/examples/06-tutorials/sparc-batchtool-orchestration.md

3.6.308.6 KB
Original Source

SPARC BatchTool Orchestration Guide

Overview

BatchTool enables powerful parallel and concurrent execution of SPARC modes, allowing for sophisticated orchestration patterns that dramatically accelerate development workflows.

Key Concepts

1. Parallel Execution

Run multiple independent SPARC modes simultaneously:

bash
batchtool run --parallel \
  "npx claude-flow sparc run code 'user authentication' --non-interactive" \
  "npx claude-flow sparc run code 'database schema' --non-interactive" \
  "npx claude-flow sparc run code 'API endpoints' --non-interactive"

2. Boomerang Pattern

Iterative development where each phase informs the next:

bash
batchtool orchestrate --boomerang \
  --phase1 "Research and gather requirements" \
  --phase2 "Design based on research" \
  --phase3 "Implement the design" \
  --phase4 "Test the implementation" \
  --phase5 "Optimize based on test results"

3. Dependency-Aware Execution

Handle task dependencies intelligently:

bash
batchtool run --dependency-aware \
  --task "db:npx claude-flow sparc run code 'database layer' --non-interactive" \
  --task "auth:npx claude-flow sparc run code 'auth service' --non-interactive:depends=db" \
  --task "api:npx claude-flow sparc run code 'API layer' --non-interactive:depends=auth,db"

Complete Workflow Examples

Full Application Development

bash
# Complete application development with SPARC + BatchTool
batchtool orchestrate --name "full-app-development" --boomerang \
  --phase1-parallel "Research Phase" \
    "npx claude-flow sparc run ask 'research best practices for modern web apps' --non-interactive" \
    "npx claude-flow sparc run ask 'analyze competitor features' --non-interactive" \
    "npx claude-flow sparc run security-review 'identify security requirements' --non-interactive" \
  --phase2-sequential "Design Phase" \
    "npx claude-flow sparc run spec-pseudocode 'create detailed specifications' --non-interactive" \
    "npx claude-flow sparc run architect 'design system architecture' --non-interactive" \
  --phase3-parallel "Implementation Phase" \
    "npx claude-flow sparc run code 'implement frontend' --non-interactive" \
    "npx claude-flow sparc run code 'implement backend' --non-interactive" \
    "npx claude-flow sparc run code 'implement database' --non-interactive" \
  --phase4-sequential "Integration Phase" \
    "npx claude-flow sparc run integration 'integrate all components' --non-interactive" \
    "npx claude-flow sparc run tdd 'comprehensive testing suite' --non-interactive" \
  --phase5-parallel "Optimization Phase" \
    "npx claude-flow sparc run optimization 'optimize performance' --non-interactive" \
    "npx claude-flow sparc run security-review 'security hardening' --non-interactive" \
    "npx claude-flow sparc run docs-writer 'complete documentation' --non-interactive"

Microservices Development

bash
# Develop multiple microservices concurrently
batchtool run --parallel --tag "microservices" \
  "npx claude-flow sparc run code 'user-service' --non-interactive" \
  "npx claude-flow sparc run code 'auth-service' --non-interactive" \
  "npx claude-flow sparc run code 'notification-service' --non-interactive" \
  "npx claude-flow sparc run code 'payment-service' --non-interactive" \
  "npx claude-flow sparc run code 'api-gateway' --non-interactive"

# Then integrate and test
batchtool run --sequential \
  "npx claude-flow sparc run integration 'integrate all microservices' --non-interactive" \
  "npx claude-flow sparc run tdd 'end-to-end testing' --non-interactive" \
  "npx claude-flow sparc run devops 'kubernetes deployment' --non-interactive"

Feature Development Sprint

bash
# Sprint planning and execution
batchtool orchestrate --sprint "feature-sprint-1" \
  --planning \
    "npx claude-flow sparc run sparc 'break down epic into features' --non-interactive" \
  --development-parallel \
    "npx claude-flow sparc run code 'feature: user profile' --non-interactive" \
    "npx claude-flow sparc run code 'feature: notifications' --non-interactive" \
    "npx claude-flow sparc run code 'feature: settings page' --non-interactive" \
  --testing-parallel \
    "npx claude-flow sparc run tdd 'test user profile' --non-interactive" \
    "npx claude-flow sparc run tdd 'test notifications' --non-interactive" \
    "npx claude-flow sparc run tdd 'test settings' --non-interactive" \
  --review \
    "npx claude-flow sparc run security-review 'audit all features' --non-interactive" \
    "npx claude-flow sparc run optimization 'performance review' --non-interactive"

Advanced Patterns

1. Continuous Refinement Loop

bash
# Continuous improvement cycle
while true; do
  batchtool orchestrate --boomerang \
    --monitor "npx claude-flow sparc run monitoring 'analyze metrics' --non-interactive" \
    --identify "npx claude-flow sparc run debug 'identify bottlenecks' --non-interactive" \
    --optimize "npx claude-flow sparc run optimization 'improve performance' --non-interactive" \
    --validate "npx claude-flow sparc run tdd 'verify improvements' --non-interactive"
  sleep 3600  # Run every hour
done

2. A/B Implementation Testing

bash
# Implement two approaches in parallel and compare
batchtool run --ab-test \
  --variant-a "npx claude-flow sparc run code 'implement with approach A' --non-interactive" \
  --variant-b "npx claude-flow sparc run code 'implement with approach B' --non-interactive" \
  --compare "npx claude-flow sparc run optimization 'compare performance' --non-interactive"

3. Progressive Enhancement

bash
# Build features progressively
batchtool orchestrate --progressive \
  --mvp "npx claude-flow sparc run code 'minimal viable product' --non-interactive" \
  --enhance-1 "npx claude-flow sparc run code 'add user authentication' --non-interactive" \
  --enhance-2 "npx claude-flow sparc run code 'add real-time updates' --non-interactive" \
  --enhance-3 "npx claude-flow sparc run code 'add analytics' --non-interactive" \
  --polish "npx claude-flow sparc run optimization 'final optimizations' --non-interactive"

Monitoring and Control

Real-time Monitoring

bash
# Monitor all running SPARC tasks
batchtool monitor --dashboard --refresh 5s

# Get status of specific orchestration
batchtool status --orchestration "full-app-development"

# View logs from all parallel executions
batchtool logs --follow --filter "error|warning"

Resource Management

bash
# Limit concurrent executions
batchtool config --max-concurrent 5

# Set memory limits per task
batchtool run --memory-limit 2GB \
  "npx claude-flow sparc run code 'heavy processing task' --non-interactive"

# Priority-based execution
batchtool run --priority high \
  "npx claude-flow sparc run security-review 'urgent security audit' --non-interactive"

Best Practices

  1. Use --non-interactive for all BatchTool orchestrations

    • Ensures smooth automated execution
    • Enables proper log capture and monitoring

  2. Tag related tasks for easier management

    bash
    batchtool run --tag "auth-system" --parallel ...

  3. Implement proper error handling

    bash
    batchtool run --on-error retry --max-retries 3 ...

  4. Use memory namespaces to share context

    bash
    --namespace "project-x-auth"

  5. Monitor resource usage

    bash
    batchtool stats --orchestration "my-workflow"

Integration with CI/CD

yaml
# Example GitHub Actions workflow
name: SPARC Development Pipeline
on: [push]

jobs:
  develop:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      
      - name: Install dependencies
        run: |
          npm install -g claude-flow
          npm install -g batchtool
      
      - name: Run SPARC orchestration
        run: |
          batchtool orchestrate --boomerang \
            --analyze "npx claude-flow sparc run architect 'analyze changes' --non-interactive" \
            --implement "npx claude-flow sparc run code 'implement features' --non-interactive" \
            --test "npx claude-flow sparc run tdd 'run tests' --non-interactive" \
            --deploy "npx claude-flow sparc run devops 'deploy to staging' --non-interactive"

Troubleshooting

Common Issues

  1. Tasks not running in parallel: Ensure BatchTool is installed and --non-interactive flag is used
  2. Memory namespace conflicts: Use unique namespaces for different workflows
  3. Resource exhaustion: Limit concurrent executions with --max-concurrent

Debug Commands

bash
# Enable verbose logging
batchtool run --verbose --debug ...

# Dry run to see execution plan
batchtool orchestrate --dry-run ...

# Export orchestration plan
batchtool export --format json --output plan.json