ContextQMD
Back to Powertoys

Fuzzing Testing in PowerToys

doc/devdocs/tools/fuzzingtesting.md

0.98.19.6 KB
Original Source

Fuzzing Testing in PowerToys

Overview

Fuzzing is an automated testing technique that helps identify vulnerabilities and bugs by feeding random, invalid, or unexpected data into the application. This is especially important for PowerToys modules that handle file input/output or user input, such as Hosts File Editor, Registry Preview, and others.

PowerToys integrates Microsoft's OneFuzz service to systematically discover edge cases and unexpected behaviors that could lead to crashes or security vulnerabilities. Fuzzing testing is a requirement from the security team to ensure robust and secure modules.

Why Fuzzing Matters

  • Security Enhancement: Identifies potential security vulnerabilities before they reach production
  • Stability Improvement: Discovers edge cases that might cause crashes
  • Automated Bug Discovery: Finds bugs that traditional testing might miss
  • Reduced Manual Testing: Automates the process of testing with unusual inputs

Types of Fuzzing in PowerToys

PowerToys supports two types of fuzzing depending on the module's implementation language:

  1. .NET Fuzzing - For C# modules (using OneFuzz)
  2. C++ Fuzzing - For native C++ modules using libFuzzer

Setting Up .NET Fuzzing Tests

Step 1: Add a Fuzzing Test Project

Create a new test project within your module folder. Ensure the project name follows the format *.FuzzTests.

Step 2: Configure the Project

  1. Set up a .NET 8 (Windows) project

    • Note: OneFuzz currently supports only .NET 8 projects. The Fuzz team is working on .NET 9 support.

  2. Add the required files to your fuzzing test project:

    • Create fuzzing test code
    • Add OneFuzzConfig.json configuration file

Step 3: Configure OneFuzzConfig.json

The OneFuzzConfig.json file provides critical information for deploying fuzzing jobs. For detailed guidance, see the OneFuzzConfig V3 Documentation.

json
{
  "fuzzers": [
    {
      "name": "YourModuleFuzzer",
      "fuzzerLibrary": "libfuzzer-dotnet",
      "targetAssembly": "YourModule.FuzzTests.dll",
      "targetClass": "YourModule.FuzzTests.FuzzTestClass",
      "targetMethod": "FuzzTest",
      "FuzzingTargetBinaries": [
        "YourModule.FuzzTests.dll"
      ]
    }
  ],
  "adoTemplate": [
    {
      "AssignedTo": "[email protected]",
      "jobNotificationEmail": "[email protected]"
    }
  ],
  "oneFuzzJobs": [
    {
      "projectName": "PowerToys",
      "targetName": "YourModule",
      "jobDependencies": {
        "binaries": [
          "PowerToys\\x64\\Debug\\tests\\YourModule.FuzzTests\\net8.0-windows10.0.19041.0\\**"
        ]
      }
    }
  ],
  "configVersion": "3.0.0"
}

Key fields to update:

  1. Update the targetAssembly, targetClass, targetMethod, and FuzzingTargetBinaries fields
  2. Set the AssignedTo and jobNotificationEmail to your Microsoft email
  3. Update the projectName and targetName fields
  4. Define job dependencies pointing to your compiled fuzzing tests

Step 4: Configure the OneFuzz Pipeline

Modify the patterns in the job steps within job-fuzz.yml to match your fuzzing project name:

yaml
- download: current
  displayName: Download artifacts
  artifact: $(ArtifactName)
  patterns: |-
    **/tests/*.FuzzTests/**

Setting Up C++ Fuzzing Tests

Step 1: Create a New C++ Project

  • Use the Empty Project template
  • Name it <ModuleName>.FuzzingTest

Step 2: Update Build Configuration

  • In Configuration Manager, uncheck Build for both Release|ARM64, Debug|ARM64 and Debug|x64 configurations
  • ARM64 is not supported for fuzzing tests

Step 3: Enable ASan and libFuzzer in .vcxproj

Edit the project file to enable fuzzing:

xml
<PropertyGroup>
  <EnableASAN>true</EnableASAN>
  <EnableFuzzer>true</EnableFuzzer>
</PropertyGroup>

Step 4: Add Fuzzing Compiler Flags

Add these to AdditionalOptions under the Fuzzing configuration:

xml
/fsanitize=address
/fsanitize-coverage=inline-8bit-counters
/fsanitize-coverage=edge
/fsanitize-coverage=trace-cmp
/fsanitize-coverage=trace-div
%(AdditionalOptions)

In Linker → Input → Additional Dependencies, add:

text
$(VCToolsInstallDir)lib\$(Platform)\libsancov.lib

Step 6: Copy Required Runtime DLL

Add a PostBuildEvent to copy the ASAN DLL:

xml
<Command>
  xcopy /y "$(VCToolsInstallDir)bin\Hostx64\x64\clang_rt.asan_dynamic-x86_64.dll" "$(OutDir)"
</Command>

Step 7: Add Preprocessor Definitions

To avoid annotation issues, add these to the Preprocessor Definitions:

text
_DISABLE_VECTOR_ANNOTATION;_DISABLE_STRING_ANNOTATION

Step 8: Implement the Entry Point

Every C++ fuzzing project must expose this function:

cpp
extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
{
    std::string input(reinterpret_cast<const char*>(data), size);

    try
    {
        // Call your module with the input here
    }
    catch (...) {}

    return 0;
}

Running Fuzzing Tests

Running Locally (.NET)

To run .NET fuzzing tests locally, follow the Running a .NET Fuzz Target Locally guide:

powershell
# Instrument the assembly
.\dotnet-fuzzing-windows\sharpfuzz\SharpFuzz.CommandLine.exe path\to\YourModule.FuzzTests.dll

# Set environment variables
$env:LIBFUZZER_DOTNET_TARGET_ASSEMBLY="path\to\YourModule.FuzzTests.dll"
$env:LIBFUZZER_DOTNET_TARGET_CLASS="YourModule.FuzzTests.FuzzTestClass"
$env:LIBFUZZER_DOTNET_TARGET_METHOD="FuzzTest"

# Run the fuzzer
.\dotnet-fuzzing-windows\libfuzzer-dotnet\libfuzzer-dotnet.exe --target_path=dotnet-fuzzing-windows\LibFuzzerDotnetLoader\LibFuzzerDotnetLoader.exe

Running in the Cloud

To submit a job to the OneFuzz cloud service, follow the OneFuzz Cloud Testing Walkthrough:

  1. Run the pipeline:

    • Navigate to the fuzzing pipeline
    • Click "Run pipeline"
    • Choose your branch and start the run

  2. Alternative: Use OIP (OneFuzz Ingestion Preparation) tool:

    oip submit --config .\OneFuzzConfig.json --drop-path <your_submission_directory> --platform windows --do-not-file-bugs --duration 1
    • Use --do-not-file-bugs to prevent automatic bug creation during initial testing
    • --duration specifies the number of hours (default is 48 if not specified)

  3. OneFuzz will send you an email when the job has started with a link to view results

Reviewing Results

  1. You'll receive an email notification when your fuzzing job starts
  2. Click the link in the email to view the job status on the OneFuzz Web UI
  3. The OneFuzz platform will show statistics like inputs processed, coverage, and any crashes found
  4. If the final status is "success," your fuzzing test is working correctly

Current Status

PowerToys has implemented fuzzing for several modules:

  • Hosts File Editor
  • Registry Preview
  • Fancy Zones

Modules that still need fuzzing implementation:

  • Environmental Variables
  • Keyboard Manager

Requesting Access to OneFuzz

To log into the production instance of OneFuzz with the CLI, you must request access. Visit the OneFuzz Access Request Page.

Resources