Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the ONNX TensorRT execution provider and respective device. #1133

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

nemphys
Copy link
Contributor

@nemphys nemphys commented Jan 2, 2025

ONNX has a TensorRT EP that is available for NodeJS along with the CUDA EP (under linux x64) when installing onnxruntime-node.
This commit makes it possible to use it by specifying 'tensorrt' as the device.

@xenova
Copy link
Collaborator

xenova commented Jan 28, 2025

Hi there! Thanks for the PR. Is there a way to detect when this execution provider is available on a device? Is it only available on linux x64? It doesn't seem to be listed in the README:
image

@nemphys
Copy link
Contributor Author

nemphys commented Jan 28, 2025

Hello, as mentioned above, the tensorrt EP is available only under linux and seems to be included with the cuda EP in the default package distribution.
Indeed, they do not mention it anywhere, but it is there from the first onnxruntime distribution that introduced cuda support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants