[shoumikhin]

This is a draft because some links on the page go to Torch-TensorRT guide sections that only exist after pytorch/TensorRT#4927 merges and the Torch-TensorRT docs site is rebuilt. It should also wait until the install command in the Torch-TensorRT guide installs Torch-TensorRT 2.15 with its delegate, because this page describes 2.15 and newer.

The TensorRT backend page had drifted away from Torch-TensorRT, which builds and ships the delegate. It pointed readers at a nightly build from early October in a "Feature availability" section, and it said that CUDA graph replay never runs for engines on the shared activation scratch, which nightly builds now do. It carried its own install command, one of several that disagreed with each other. It also repeated most of the Torch-TensorRT deployment guide, so every change to the delegate had to be made in two places, and the backends table listed TensorRT as Linux only.

The page keeps the usual backend sections but says less in each: features, target and development requirements, export, runtime integration, and a short list of performance options. For installing, it links to the install section of the Torch-TensorRT guide, and each performance option links to its section of that guide. The page says it describes Torch-TensorRT 2.15 and newer, and marks what still needs a nightly build: engines in a separate data file, optimization profiles, CUDA graph replay on the shared scratch, setting runtime options from Python, and Windows, because the 2.15 release candidate publishes no Windows delegate. It also states the limits of the CUDA backend on Windows.

Tested on Linux x86-64 with an H100: ran every Python block of the page in order with nightly Torch-TensorRT, which exports and runs the plain, split and GPU-resident programs, and built and ran the C++ example against the installed packages with the CMake recipe from the page. Checked these 2.15 differences against the Torch-TensorRT 2.15.0 release candidate: the re-export default, the missing data file option, the Python runtime options, and a CPU input to a GPU-resident program. Windows was not tested. The page builds with Sphinx and MyST with no new warnings.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24136

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 950de92 with merge base 00f0654 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.