Native MagicString

Overview

experimental.nativeMagicString is an optimization feature that replaces the JavaScript-based MagicString implementation with a native Rust version, enabling source map generation in background threads for improving performance.

What is MagicString?

MagicString is a JavaScript library developed by Rich Harris (the creator of Rollup and Svelte) that provides efficient string manipulation with automatic source map generation. It's commonly used by bundlers and build tools for:

• Code transformation in plugins
• Source map generation
• Precise line/column tracking
• Efficient string operations (replace, prepend, append, etc.)

The JavaScript Implementation vs Native Rust

Traditional JavaScript MagicString

The original MagicString implementation is written in JavaScript and runs in the Node.js environment. When bundlers perform code transformations, they typically:

1. Load source code as JavaScript strings
2. Apply transformations using MagicString API
3. Generate source maps for the transformed code
4. Process everything in the main JavaScript thread

Native Rust Implementation

Rolldown's native MagicString implementation rewrites the core functionality in Rust, providing several advantages:

• Performance: Rust's memory safety and zero-cost abstractions make string operations faster
• Parallel Processing: Source map generation can happen in background threads
• Memory Efficiency: Better memory management for large codebases
• Integration: Seamless integration with Rolldown's Rust-based architecture

How It Works

When experimental.nativeMagicString is enabled, Rolldown modifies the transformation pipeline. The diagrams below show the architectural differences:

:::info Some technical details are simplified for better illustration. The native MagicString implementation provides a magicString object in the meta parameter of transform hooks, which plugins can use just like the JavaScript version. :::

Without Native MagicString

(Correction in the image: "rolldown without js magic-string" should be "rolldown without native magic-string")

With Native MagicString

Key Difference: The native implementation is written in Rust, providing both Rust's performance advantages and background thread source map generation. Offloading to background threads improves overall CPU usage and enables significant performance improvements.

API Compatibility

The native implementation maintains API compatibility with the JavaScript version. The most commonly used APIs are already implemented, with the remaining APIs planned for completion in future releases.

Implemented Methods

The following MagicString methods are currently available in the native implementation:

String Manipulation:

• append(content) - Appends content to the end of the string
• prepend(content) - Prepends content to the beginning of the string
• appendLeft(index, content) - Appends content to the left of a specific index
• appendRight(index, content) - Appends content to the right of a specific index
• prependLeft(index, content) - Prepends content to the left of a specific index
• prependRight(index, content) - Prepends content to the right of a specific index
• overwrite(start, end, content) - Replaces content in a range
• update(start, end, content) - Updates content in a range
• remove(start, end) - Removes content in a range
• replace(from, to) - Replaces the first occurrence of a pattern
• replaceAll(from, to) - Replaces all occurrences of a pattern

Transformations:

• indent(indentor?) - Indents the content with optional custom indentation string
• relocate(start, end, to) - Moves content from one position to another

Utilities:

• toString() - Returns the transformed string
• hasChanged() - Checks if the string has been modified
• length() - Returns the length of the transformed string
• isEmpty() - Checks if the string is empty
• clone() - Returns a clone of the MagicString instance
• trim(charType?) - Trims whitespace or specified characters from both ends
• trimStart(charType?) - Trims whitespace or specified characters from the start
• trimEnd(charType?) - Trims whitespace or specified characters from the end
• trimLines() - Trims newlines from both ends
• snip(start, end) - Returns a clone with content outside the range removed
• slice(start?, end?) - Returns content between positions
• reset(start, end) - Resets a range to its original content
• lastChar() - Returns the last character
• lastLine() - Returns the content after the last newline

Source Map Generation:

• generateMap(options?) - Generates a source map as a JSON string
  • options.source - Source file name
  • options.includeContent - Include original source in the map
  • options.hires - High-resolution mode: true, false, or "boundary"

Not Yet Implemented

The following features are planned for future releases:

• generateDecodedMap() - Generate source map with decoded mappings

Real-World Performance

use rolldown/benchmarks as benchmark cases

Build time

Plugin transform time (build time - noop plugin build time)

For detailed benchmark results, see the benchmark pull request.

Usage Examples

Basic Plugin with Native MagicString

import { defineConfig } from 'rolldown';

export default defineConfig({
  experimental: {
    nativeMagicString: true,
  },
  output: {
    sourcemap: true,
  },
  plugins: [
    {
      name: 'transform-example',
      transform(code, id, meta) {
        if (!meta?.magicString) {
          // Fallback when nativeMagicString is not available
          return null;
        }

        const { magicString } = meta;

        // Example transformation: Add debug comments
        if (code.includes('console.log')) {
          magicString.replace(/console\.log\(/g, 'console.log("[DEBUG]", ');
        }

        // Example: Add file header
        magicString.prepend(`// Transformed from: ${id}\n`);

        return {
          code: magicString,
        };
      },
    },
  ],
});

Compatibility and Fallbacks

Checking for Native MagicString Availability

transform(code, id, meta) {
  if (meta?.magicString) {
    // Native MagicString is available
    const { magicString } = meta;

    // Use the native implementation
    // Note: Return the magicString object directly, not a string
    return {
      code: magicString
    };
  } else {
    // Fallback to regular string manipulation
    // or use the JavaScript MagicString library
    const MagicString = require('magic-string');
    const ms = new MagicString(code);

    // Your transformations here...

    return {
      code: ms.toString(),
      map: ms.generateMap()
    };
  }
}

Rollup Compatibility

This feature is Rolldown-specific and not available in Rollup. For plugins that need to work with both bundlers:

function createTransform() {
  return function (code, id, meta) {
    if (meta?.magicString) {
      // Rolldown with native MagicString
      return transformWithNativeMagicString(code, id, meta);
    } else {
      // Rollup or Rolldown without native MagicString
      return transformWithJsMagicString(code, id);
    }
  };
}

::: tip

You can use rolldown-string, which provides a unified interface that works with both bundlers.

:::

When to Use Native MagicString

Recommended Scenarios

1. Large Codebases: Projects with hundreds or thousands of files
2. Complex Transformations: Plugins that perform extensive code manipulation
3. Source Map Intensive: Projects requiring detailed source maps
4. Performance-Critical: Build processes where speed is crucial
5. Development Mode: Faster rebuild times during development

When to Be Cautious

1. Experimental Feature: As an experimental feature, API may change
2. Plugin Compatibility: Some plugins may expect specific JavaScript MagicString behavior
3. Debugging: Native implementation may have different error messages

Migration Guide

Enabling Native MagicString

1. Update Configuration:

export default {
  experimental: {
    nativeMagicString: true,
  },
  output: {
    sourcemap: true, // Required for source map generation
  },
};

2. Update Plugins:

// Before
transform(code, id) {
  const ms = new MagicString(code);
  // ... transformations
  return { code: ms.toString(), map: ms.generateMap() };
}

// After
transform(code, id, meta) {
  if (meta?.magicString) {
    const { magicString } = meta;
    // ... transformations (same API)
    return { code: magicString };
  }
  // Fallback logic
}

Limitations and Considerations

Current Limitations

1. Experimental Status: API may change in future versions
2. Edge Cases: Some edge cases may behave differently from JavaScript version
3. Debugging: Error messages may be less familiar

Best Practices

1. Always Check Availability: Verify meta?.magicString exists before using
2. Provide Fallbacks: Include fallback logic for compatibility
3. Test Thoroughly: Test transformations with both implementations
4. Report Issues: Report any behavior differences to the Rolldown team

Conclusion

experimental.nativeMagicString represents a significant performance optimization for Rolldown by leveraging Rust's efficiency for code transformation tasks. While it requires some considerations for compatibility, the performance benefits make it an attractive option for large-scale projects and performance-critical build processes.

As an experimental feature, it's recommended to test thoroughly in development environments before adopting in production workflows. The Rolldown team is actively working on this feature, and feedback from the community is valuable for its continued development.