OpenVINO Intel GPU Plugin

GPU plugin in OpenVINO toolkit supports inference on Intel® GPUs starting from Gen8 architecture.

Key Contacts

For assistance regarding GPU, contact a member of openvino-ie-gpu-maintainers group.

Components

GPU Plugin contains the following components:

• docs - developer documentation pages for the component.
• include - public API.
• src - sources of the component.
• tests - tests for OpenVINO Plugin component.
• thirdparty - third-party modules.

Tutorials

• Simplified workflow
  • Graph Optimization Passes
  • Execution of Inference
• Source code structure
  • Basic data structures of gpu graph and overall flow
  • Memory allocation in GPU plugin
• Memory formats
• Kernels and kernel selectors
• GPU plugin operations enabling flow
• GPU plugin unit test
• Run benchmark from device_mem

Documentation on dynamic-shape

This contents explain the internal implementation of dynamic shape support in the GPU Plugin. For general usage of dynamic shape and limitations of the GPU plugin, please refer to this link: GPU Device — OpenVINO™ documentation - Version(2026).

• Overall flow for dynamic shape execution
• Implementation details
  • Preprocessing: Shape inference / update weight / realloc memory
  • dynamic impl of kernels
  • in-memory kernel cache
  • async kernel compilation
  <!-- * weight compression (TBD)) -->
• Optimization features
  • Memory preallocation
  • Runtime operation skip
  • KV cache optimization

<!-- * Fake alignment of shape (TBD) * Shape-of subgraph on CPU (TBD) * Runtime buffer fusing (TBD) * Runtime reorder skip (TBD) * KV cache (TBD) * Performance analysis and debugging features (TBD) * Model caching for dynamic shape (TBD) -->

Troubleshooting / Debugging

• Debug utils
• Debugging guide
• OpenCL Runtime issues troubleshooting

Attached licenses

GPU plugin uses 3^rd-party components licensed under following licenses:

• googletest under Google License
• *OpenCL™ ICD and C++ Wrapper under Khronos™ License
• RapidJSON under Tencent License

Support

To report issues and make suggestions, see GitHub issues.

How to Contribute

Community contributions to GPU plugin are highly welcome. If you have a suggestion on how to improve the library:

• Share your proposal via GitHub issues
• Ensure you can build the product and run all the tests with your patch
• In case of a larger feature, create a test
• Submit a pull request

We will review your contribution and, if any additional fixes or modifications are necessary, we may provide feedback to guide you. Once your pull request has been approved, it will be merged into our GitHub repository.

System Requirements

GPU plugin supports Intel® HD Graphics, Intel® Iris® Graphics and Intel® Arc™ Graphics and is optimized for Gen9-Gen12LP, Gen12HP architectures

GPU plugin currently uses OpenCL™ with multiple Intel OpenCL™ extensions and requires Intel® Graphics Driver to run.

GPU plugin requires CPU with Intel® SSE/Intel® AVX support.

---

The software dependencies are:

• CMake 3.5 or later
• C++ compiler with C++11 standard support compatible with:
  • GNU Compiler Collection 4.8 or later
  • clang 3.5 or later
  • Intel® C++ Compiler 17.0 or later
  • Visual C++ 2015 (MSVC++ 19.0) or later
• python™ 3.8 or later.

Trademark Information

Intel, the Intel logo, Intel Atom, Intel Core, Intel Xeon Phi, Iris, OpenVINO, the OpenVINO logo, Pentium, VTune, and Xeon are trademarks of Intel Corporation or its subsidiaries.

* Other names and brands may be claimed as the property of others.

Microsoft, Windows, and the Windows logo are trademarks, or registered trademarks of Microsoft Corporation in the United States and/or other countries.

OpenCL and the OpenCL logo are trademarks of Apple Inc. used by permission by Khronos.

Copyright © 2023, Intel Corporation

See also

• OpenVINO™ README
• OpenVINO Core Components
• OpenVINO Plugins
• Developer documentation