quickstart/IntroNotebooks/0. Running This Guide.ipynb
This guide is presented as a series of Jupyter notebooks covering both Tensorflow and PyTorch using a Python runtime.
If you would like to run this code yourself, you can do so using the following steps:
Setup:
The basic prerequisites you will need to use TensorRT are:
You can make sure your GPU environment is properly configured and check which GPU and CUDA version you are using with nvidia-smi:
!nvidia-smi
For some of the examples you will also need cuda-python, skimage, matplotlib, and onnx:
%pip install cuda-python onnx scikit-image matplotlib
PyTorch:
We will be using PyTorch to walk through the basic steps of deploying a TensorRT model by exporting the model in ONNX format.
You can find PyTorch installation instructions here, or use one of NVIDIA's NGC containers here.
You will also need torchvision:
%pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
Using Colab:
You can also test these notebooks out using Google Colab, which includes PyTorch, as well as supported NVIDIA drivers. Make sure to select a GPU hardware accelerator in the runtime options. Just note that TensorRT performance is best on newer gpus, Colab often has trouble with reduced precision inference, and you will have to use an older version of TensorRT.
TensorRT Support:
TensorRT support for NVIDIA GPUs is determined by their compute capability. You can check the compute cabapility of your card on the NVIDIA website.
TensorRT supports different feautures depending on your compute capability. Higher compute capabilities allow additional TensorRT optimizations, like reduced precision inference. You can check which TensorRT features are supported by your compute capability in the TensorRT documentation.
Next Steps:
Now, start by opening 1. Introduction.ipynb and proceed through the notebook!
The notebooks included with this guide are: