Crawl4AI v0.6.0 Release Notes

We're excited to announce the release of Crawl4AI v0.6.0, our biggest and most feature-rich update yet. This version introduces major architectural upgrades, brand-new capabilities for geo-aware crawling, high-efficiency scraping, and real-time streaming support for scalable deployments.

Highlights

1. World-Aware Crawlers

Crawl as if you’re anywhere in the world. With v0.6.0, each crawl can simulate:

Specific GPS coordinates
Browser locale
Timezone

Example:

python

CrawlerRunConfig(
    url="https://browserleaks.com/geo",
    locale="en-US",
    timezone_id="America/Los_Angeles",
    geolocation=GeolocationConfig(
        latitude=34.0522,
        longitude=-118.2437,
        accuracy=10.0
    )
)

Great for accessing region-specific content or testing global behavior.

2. Native Table Extraction

Extract HTML tables directly into usable formats like Pandas DataFrames or CSV with zero parsing hassle. All table data is available under result.media["tables"].

Example:

python

raw_df = pd.DataFrame(
    result.media["tables"][0]["rows"],
    columns=result.media["tables"][0]["headers"]
)

This makes it ideal for scraping financial data, pricing pages, or anything tabular.

3. Browser Pooling & Pre-Warming

We've overhauled browser management. Now, multiple browser instances can be pooled and pages pre-warmed for ultra-fast launches:

Reduces cold-start latency
Lowers memory spikes
Enhances parallel crawling stability

This powers the new Docker Playground experience and streamlines heavy-load crawling.

4. Traffic & Snapshot Capture

Need full visibility? You can now capture:

Full network traffic logs
Console output
MHTML page snapshots for post-crawl audits and debugging

No more guesswork on what happened during your crawl.

5. MCP API and Streaming Support

We’re exposing MCP socket and SSE endpoints, allowing:

Live streaming of crawl results
Real-time integration with agents or frontends
A new Playground UI for interactive crawling

This is a major step towards making Crawl4AI real-time ready.

6. Stress-Test Framework

Want to test performance under heavy load? v0.6.0 includes a new memory stress-test suite that supports 1,000+ URL workloads. Ideal for:

Load testing
Performance benchmarking
Validating memory efficiency

Core Improvements

Robots.txt compliance
Proxy rotation support
Improved URL normalization and session reuse
Shared data across crawler hooks
New page routing logic

Breaking Changes & Deprecations

Legacy crawl4ai/browser/* modules are removed. Update imports accordingly.
AsyncPlaywrightCrawlerStrategy.get_page now uses a new function signature.
Deprecated markdown generator aliases now point to DefaultMarkdownGenerator with warning.

Miscellaneous Updates

FastAPI validators replaced custom validation logic
Docker build now based on a Chromium layer
Repo-wide cleanup: ~36,000 insertions, ~5,000 deletions

New Examples Included

Geo-location crawling
Network + console log capture
Docker MCP API usage
Markdown selector usage
Crypto project data extraction

Watch the Release Video

Want a visual walkthrough of all these updates? Watch the video: 🔗 https://youtu.be/9x7nVcjOZks

If you're new to Crawl4AI, start here: 🔗 https://www.youtube.com/watch?v=xo3qK6Hg9AA&t=15s

Join the Community

We’ve just opened up our Discord for the public. Join us to:

Ask questions
Share your projects
Get help or contribute

💬 https://discord.gg/wpYFACrHR4

Install or Upgrade

bash

pip install -U crawl4ai

Live long and import crawl4ai. 🖖