Skip to content

Commit

Permalink
Refactors trace operations to be more self-contained, separates front…
Browse files Browse the repository at this point in the history
…end/trace tensors more cleanly

- Refactors Trace operation so that it reports how many outputs it generates instead
     of requiring the caller to know.

     The trace op is now also responsible for creating its own output trace
     tensors. Additionally, `build`/`build_internal` have been removed, meaning
     the trace does *not* create frontend tensors anymore.

     Frontend tensors no longer create trace tensors directly but instead only
     interface with ops and wrap their outputs as needed.

- Consolidates and renames some frontend Tensor constructors to better reflect
     their purpose. For example, `create_directly` -> `fast_init`.

- Temporarily removes the "how to add ops" guide. A new version of this will be
     written once we have switched to the TRT dialect, which will signficantly affect
     how ops are added.
  • Loading branch information
pranavm-nvidia committed Feb 3, 2025
1 parent 6ece9ed commit ecb8238
Show file tree
Hide file tree
Showing 50 changed files with 262 additions and 638 deletions.
3 changes: 0 additions & 3 deletions tripy/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,9 +74,6 @@ We've written developer guides to help you understand the codebase:
[architecture](https://nvidia.github.io/TensorRT-Incubator/post0_developer_guides/architecture.html)
documentation.

- If you need to add a new operation, refer to
[this guide](https://nvidia.github.io/TensorRT-Incubator/post0_developer_guides/how-to-add-new-ops.html).


### Tests

Expand Down
351 changes: 0 additions & 351 deletions tripy/docs/post0_developer_guides/how-to-add-new-ops.md

This file was deleted.

4 changes: 2 additions & 2 deletions tripy/notebooks/resnet50.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
"metadata": {},
"outputs": [],
"source": [
"!python3 -m pip install nvtripy -f https://nvidia.github.io/TensorRT-Incubator/packages.html"
"%pip install nvtripy -f https://nvidia.github.io/TensorRT-Incubator/packages.html"
]
},
{
Expand All @@ -39,7 +39,7 @@
"metadata": {},
"outputs": [],
"source": [
"!pip install \"datasets==2.18.0\" \"matplotlib>=3.9.1\" \"pillow>=9.4.0\" \"transformers==4.46.2\" \"torch>=2.3.1\""
"%pip install \"datasets==2.18.0\" \"matplotlib>=3.9.1\" \"pillow>=9.4.0\" \"transformers==4.46.2\" \"torch>=2.3.1\""
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion tripy/nvtripy/backend/api/executable.py
Original file line number Diff line number Diff line change
Expand Up @@ -229,7 +229,7 @@ def add(a, b):

raise

output_tensors = [Tensor.create_directly(output, fetch_stack_info=False) for output in executor_outputs]
output_tensors = [Tensor.fast_init(output) for output in executor_outputs]
if len(output_tensors) == 1:
output_tensors = output_tensors[0]
return output_tensors
Expand Down
2 changes: 1 addition & 1 deletion tripy/nvtripy/backend/mlir/executor.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ def _get_output_tensor_info(self, outputs_runtime_shape, output_devices):

output_device = output_devices[index]
if not output_device:
output_device = device.create_directly(
output_device = device.fast_init(
"gpu" if memref.address_space == runtime.PointerType.device else "cpu", 0
)

Expand Down
Loading

0 comments on commit ecb8238

Please sign in to comment.