COBOL Code Indexing

GitNexus indexes COBOL codebases using a regex-only extraction strategy, bypassing tree-sitter entirely. This document explains why, how the pipeline works, and links to detailed sub-documents.

Why Regex-Only?

The tree-sitter-cobol grammar (v0.0.1) has three critical limitations that make it unusable for production indexing:

Because the external scanner hang cannot be interrupted (there is no setTimeoutMicros equivalent for tree-sitter), using tree-sitter-cobol would hang the indexing pipeline on a non-trivial fraction of real-world files.

The regex-only approach provides:

• Speed: ~1ms per file average extraction time
• Reliability: zero hangs, zero crashes across 13,000+ files
• Coverage: captures all critical symbols -- program name, paragraphs, sections, CALL, PERFORM, COPY, data items (01-77, 88-level), file declarations, FD entries, EXEC SQL/CICS blocks, ENTRY points, and MOVE statements

Architecture

flowchart TD
    A[Repository Scan] --> B{File Detection}
    B -->|Extension match| C[COBOL file]
    B -->|GITNEXUS_COBOL_DIRS match| C
    B -->|No match| Z[Skip]

    C --> D{Copybook?}
    D -->|Yes| E[Add to Copybook Map]
    D -->|No| F[Source Program]

    E --> G[COPY Expansion Engine]
    F --> G

    G -->|Inline copybook content| H[Expanded Source]
    H --> I[Patch Marker Cleanup]
    I --> J[Regex State Machine]

    J --> K[Extracted Symbols]
    K --> L[Graph Model Builder]
    L --> M[Knowledge Graph]

    subgraph "Per-Chunk Processing"
        G
        H
        I
        J
        K
        L
    end

    subgraph "Post-Processing"
        M --> N[Community Detection]
        M --> O[Process Detection]
        M --> P[Contract Detection]
    end

    style J fill:#e8f5e9,stroke:#2e7d32
    style G fill:#e3f2fd,stroke:#1565c0

COBOL vs Tree-Sitter Languages

Sub-Documents

Key Source Files