Workflow Output Structure Documentation

This document describes the expected output structure for ComputerUse workflows to properly integrate with the CLI and backend systems.

Table of Contents

• Overview
• Core Response Structure
• Output Parser Structure
• Workflow States
• Examples
• Validation

Overview

ComputerUse workflows return structured data that allows the CLI and backend to:

1. Determine if the workflow succeeded from a business logic perspective
2. Extract structured data from UI automation or API calls
3. Display results in a user-friendly format
4. Provide detailed debugging information when failures occur

Core Response Structure

When a workflow is executed via execute_sequence, it returns the following structure:

{
  "status": "success | partial_success | error",
  "results": [...],
  "executed_tools": 5,
  "total_duration_ms": 1234,
  "output": { ... }  // Optional, if output parser is defined
}

Fields

• status (string): The execution-level status

  • "success": All steps executed successfully
  • "partial_success": Some steps failed but execution continued
  • "error": Critical failure that stopped execution

• results (array): Array of individual step results

• executed_tools (number): Count of steps that were executed

• total_duration_ms (number): Total execution time in milliseconds

• output (object): Optional structured data from the output parser

Output Parser Structure

The output field is generated by the workflow's output parser (JavaScript code) and should follow this structure for optimal CLI rendering:

{
  "success": true,
  "state": "success | failure | skipped",
  "message": "Human-readable message about the result",
  "data": { ... } | [...],
  "error": "Error details if failed",
  "validation": { ... },
  "skipped": false
}

Required Fields

• success (boolean): Whether the workflow achieved its business goal

  • true: The workflow successfully completed its intended task
  • false: The workflow failed to achieve its goal (even if technically executed)

• message (string): Human-readable description of the result

  • Should be concise and actionable
  • Examples:
    • "Successfully extracted 5 invoices"
    • "No data found matching the criteria"
    • "Login failed - invalid credentials"

Optional Fields

• state (string): Explicit workflow state

  • "success": Business goal achieved
  • "failure": Business goal not achieved
  • "skipped": Workflow skipped due to conditions

• data (any): The extracted/processed data

  • Can be an array, object, or primitive value
  • For scrapers: Array of extracted items
  • For validators: Boolean or validation object
  • For API calls: Response data

• error (string): Detailed error information

  • Only present when success: false
  • Should provide actionable error details

• validation (object): Additional validation information

  • Custom validation results
  • Field-level validation details

• skipped (boolean): Whether the workflow was skipped

  • Used when conditions weren't met to run the workflow

Workflow States

The CLI interprets workflow states as follows:

Examples

Successful Data Extraction

// Output parser code
const items = tree.findAll('[data-testid="invoice"]');
return {
  success: items.length > 0,
  message: `Found ${items.length} invoices`,
  data: items.map(item => ({
    id: item.getAttribute('data-id'),
    amount: item.querySelector('.amount')?.text,
    date: item.querySelector('.date')?.text
  }))
};

Failed Validation

// Output parser code
const loginForm = tree.find('#login-form');
const errorMsg = tree.find('.error-message');

return {
  success: false,
  message: errorMsg?.text || "Login failed",
  error: "Invalid credentials provided",
  validation: {
    usernameField: !!tree.find('#username'),
    passwordField: !!tree.find('#password'),
    errorDisplayed: !!errorMsg
  }
};

Skipped Workflow

// Output parser code
const prerequisite = tree.find('.required-element');

if (!prerequisite) {
  return {
    success: false,
    skipped: true,
    message: "Workflow skipped - required element not found",
    data: null
  };
}

// Continue with normal processing...

API Response Processing

// Output parser code (no UI tree, processing API response)
const response = sequenceResult.results?.[0]?.result;

if (response?.status === 200) {
  return {
    success: true,
    message: `Retrieved ${response.data.length} records`,
    data: response.data
  };
} else {
  return {
    success: false,
    message: "API request failed",
    error: response?.error || "Unknown error",
    data: null
  };
}

Validation

To ensure your workflow output is properly structured:

1. Required Fields Check

• Always include success and message
• Ensure success accurately reflects business logic success

2. Data Consistency

• If success: true, include meaningful data
• If success: false, include helpful error details

3. Message Quality

• Messages should be actionable and specific
• Avoid generic messages like "Success" or "Failed"

4. Type Safety

// Good practice: Ensure data types
return {
  success: Boolean(items.length > 0),  // Ensure boolean
  message: String(statusText),          // Ensure string
  data: items || []                     // Provide default
};

CLI Integration

The ComputerUse CLI (computeruse command) uses this structure to:

1. Display Results: Shows success/failure with appropriate colors and icons
2. Extract Data: Pretty-prints the data field for user review
3. Show Errors: Displays error details when workflows fail
4. Export Data: Can export structured data to JSON/CSV formats

Backend Integration

Backend systems can use this structure to:

1. Store Results: Save structured data to databases
2. Trigger Actions: Use success/failure to trigger downstream processes
3. Generate Reports: Aggregate data across multiple workflow runs
4. Monitor Health: Track success rates and performance metrics

Best Practices

1. Always Return Structured Data: Even on failure, return a properly structured response
2. Be Specific: Use descriptive messages that explain what happened
3. Include Context: Add relevant data even in failure cases for debugging
4. Validate Early: Check prerequisites and skip gracefully if not met
5. Handle Edge Cases: Account for empty results, missing elements, etc.

Future Enhancements

Planned improvements to the output structure:

• Schema validation for output parsers
• Type definitions for common data patterns
• Built-in validators for common scenarios
• Standardized error codes
• Performance metrics in output

Related Documentation

• Output Parser Guide
• Workflow Creation Guide
• CLI Usage