docs/guides/dlt-mcp-continue-cookbook.mdx
import { OSAutoDetect } from '/snippets/OSAutoDetect.jsx' import CLIInstall from '/snippets/cli-install.mdx'
<OSAutoDetect /> <Card title="What You'll Build" icon="database"> An AI-powered data pipeline development system that uses Continue's AI agent with dlt MCP to inspect pipeline execution, retrieve schemas, analyze datasets, and debug load errors - all through simple natural language prompts </Card>Before starting, ensure you have:
For all options, first: <Steps> <Step title="Install Continue CLI"> <CLIInstall /> </Step>
<Step title="Install dlt"> ```bash pip install dlt ``` </Step> </Steps> <Warning> To use agents in headless mode, you need a [Continue API key](https://continue.dev/settings/api-keys). </Warning>After ensuring you meet the Prerequisites above, you have two paths to get started:
<Tabs> <Tab title="⚡ Quick Start (Recommended)"> <Steps> <Step title="Load the Pre-Built Agent"> Navigate to your pipeline project directory and run: ```bash cn --agent continuedev/dlt-agent ``` This agent includes:
- **dlt MCP** pre-configured and ready to use
- **Pipeline-focused rules** for data engineering best practices
</Step>
<Step title="Run Your First Pipeline Inspection">
Start with a comprehensive pipeline check:
```bash
# TUI mode
Inspect the execution of my dlt pipeline and summarize the load info, including timing and file sizes.
```
That's it! The agent handles everything automatically.
</Step>
</Steps>
<Info>
**Why Use the Agent?** The pre-built [dlt Agent](https://continue.dev/continuedev/dlt-agent) provides consistent pipeline development workflows and handles MCP configuration automatically, making it easier to get started with AI-powered data engineering. You can [remix and customize this agent](/guides/understanding-configs#how-to-get-started-with-hub-configs) later to fit your team's specific workflow.
</Info>
This will add dlt MCP to your agent's available tools. The Mission Control listing automatically configures the MCP command.
<Tip>
**Alternative installation methods:**
1. **Quick CLI install**: `cn --mcp dlthub/dlt-mcp`
2. **Manual configuration**: Add the MCP to your `~/.continue/config.json` under the `mcpServers` section
Once installed, dlt MCP tools become available to your Continue agent for all prompts.
</Tip>
<Info>
The MCP will work with your existing dlt pipelines in your current directory.
</Info>
The agent will automatically detect and use your configuration along with the pre-configured dlt MCP for pipeline operations.
dlt+ MCP extends these capabilities with cloud-based features for production deployments:
For local development and getting started, dlt MCP is the right choice. Consider dlt+ MCP when you need production deployment features and team collaboration. </Card>
Now you can use natural language prompts to develop and debug your dlt pipelines. The Continue agent automatically calls the appropriate dlt MCP tools.
<Info> You can add prompts to your agent's configuration for easy access in future sessions. Go to your agent in the [Continue Mission Control](https://continue.dev), click **Edit**, and add prompts under the **Prompts** section. </Info> <Info> **Where to run these workflows:** - **IDE Extensions**: Use Continue in VS Code, JetBrains, or other supported IDEs - **Terminal (TUI mode)**: Run `cn` to enter interactive mode, then type your prompts - **CLI (headless mode)**: Use `cn -p "your prompt"` for headless commandsTest in Plan Mode First: Before running pipeline operations that might make changes, test your prompts in plan mode (see the Plan Mode Guide; press Shift+Tab to switch modes in TUI/IDE). This shows you what the agent will do without executing it.
To run any of the example prompts below in headless mode, use cn -p "prompt"
</Info>
About the --auto flag: The --auto flag enables tools to run continuously without manual confirmation. This is essential for headless mode where the agent needs to execute multiple tools automatically to complete tasks like pipeline inspection, schema retrieval, and error analysis.
Prompt:
Inspect my dlt pipeline execution and provide a summary of the load info.
Show me the timing breakdown and file sizes for each table.
Prompt:
Show me the schema for my users table including all columns,
data types, and any constraints.
Prompt:
Get the last 10 records from my orders table and show me
the distribution of order statuses.
Prompt:
Check for any load errors in my last pipeline run. If there are errors,
explain what went wrong and suggest fixes.
Prompt:
Help me create a new dlt pipeline that loads data from the
JSONPlaceholder API users endpoint into DuckDB.
Prompt:
Check if my pipeline schema has evolved since the last run.
Show me what columns were added or modified.
This example demonstrates a Continuous AI workflow where data pipeline validation runs automatically in your CI/CD pipeline in headless mode using the dlt Assistant agent. Consider remixing this agent to add your organization's specific validation rules.
Navigate to Repository Settings → Secrets and variables → Actions and add:
CONTINUE_API_KEY: Your Continue API key from continue.dev/settings/api-keysThis workflow automatically validates your dlt data pipelines on pull requests using the Continue CLI in headless mode. It inspects pipeline schemas, checks for errors, and posts a summary report as a PR comment. The workflow can also be triggered manually via workflow_dispatch.
Create .github/workflows/dlt-pipeline-validation.yml in your repository:
name: Data Pipeline Validation with dlt MCP
on:
pull_request:
branches: [main]
workflow_dispatch:
jobs:
validate-pipeline:
runs-on: ubuntu-latest
env:
CONTINUE_API_KEY: ${{ secrets.CONTINUE_API_KEY }}
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: "18"
- name: Install dlt
run: |
pip install dlt
echo "✅ dlt installed"
- name: Install Continue CLI
run: |
npm install -g @continuedev/cli
echo "✅ Continue CLI installed"
- name: Validate Pipeline Schema
run: |
echo "🔍 Validating pipeline schema..."
cn --agent continuedev/dlt-agent \
-p "Inspect the pipeline schema and verify all required tables
and columns are present. Flag any missing or unexpected changes." \
--auto
- name: Check Pipeline Health
run: |
echo "📊 Checking pipeline health..."
cn --agent continuedev/dlt-agent \
-p "Analyze the last pipeline run for errors or warnings.
Report any issues that need attention." \
--auto
- name: Comment Pipeline Report on PR
if: always() && github.event_name == 'pull_request'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
REPORT=$(cn --agent continuedev/dlt-agent \
-p "Generate a concise summary (200 words or less) of:
- Pipeline schemas and row counts
- Any load errors or warnings
- Performance metrics (timing, file sizes)
- Recommended improvements" \
--auto)
gh pr comment ${{ github.event.pull_request.number }} --body "$REPORT"
Implement automated pipeline quality checks using Continue's rule system. See the Rules deep dive for authoring tips.
<Card title="Schema Validation" icon="check-circle"> ```bash "Before committing pipeline changes, verify the schema matches expectations and flag any unexpected modifications." ``` </Card> <Card title="Error Handling" icon="shield-exclamation"> ```bash "When load errors occur, analyze the error details and suggest specific code fixes to handle the data issues." ``` </Card> <Card title="Performance Monitoring" icon="gauge-high"> ```bash "Track pipeline execution times and file sizes. Alert if performance degrades significantly from baseline." ``` </Card> <Card title="Data Quality" icon="check-double"> ```bash "After each pipeline run, validate row counts and check for null values in critical columns." ``` </Card>"Check if there's a dlt pipeline in the current directory.
If not, help me initialize a new pipeline."
"Verify the destination connection and credentials for my pipeline.
Test the connection and report any issues."
After completing this guide, you have a complete AI-powered data pipeline development system that:
✅ Uses natural language — Simple prompts instead of complex pipeline commands ✅ Debugs automatically — AI analyzes errors and suggests fixes ✅ Runs continuously — Automated validation in CI/CD pipelines ✅ Ensures quality — Pipeline checks prevent bad data from shipping
<Card title="Continuous AI" icon="rocket"> Your data pipeline workflow now operates at **[Level 2 Continuous AI](https://blog.continue.dev/what-is-continuous-ai-a-developers-guide/)** - AI handles routine pipeline inspection and debugging with human oversight through review and approval of changes. </Card>