Diskann Benchmarking Wrapper #260
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tarang-jain
added
feature request
…diskann-wrapper
…diskann-wrapper
…diskann-wrapper
…into diskann-wrapper
…into diskann-wrapper
…diskann-wrapper
…diskann-wrapper
…diskann-wrapper
Authors: - rhdong (https://github.com/rhdong) Approvers: - James Lamb (https://github.com/jameslamb) - Corey J. Nolet (https://github.com/cjnolet) - Ishan Chattopadhyaya (https://github.com/chatman) URL: rapidsai#618
This PR adds docs for cuvs nn_descent Authors: - Severin Dicks (https://github.com/Intron7) Approvers: - Corey J. Nolet (https://github.com/cjnolet) URL: rapidsai#668
|
/ok to test |
…diskann-wrapper
…into diskann-wrapper
|
/ok to test |
|
/ok to test |
|
/ok to test |
… copy beyond 4B elems (rapidsai#671) ann-bench keeps data dimensions as `uint32_t`. We use `std::fread` to copy the data from a file to the host memory and pass `n_rows * n_cols` there, which gets casted to size_t only after the multiplication. This leads to integer overflow for the datasets larger than 4B elements and a partial data copy. This PR fixes the bug by casting the dimensions before the multiplication. The bug only affects the benchmark cases where the data is requested in the host memory not backed by a file. Authors: - Artem M. Chirkin (https://github.com/achirkin) Approvers: - Tamas Bela Feher (https://github.com/tfeher) URL: rapidsai#671
Authors: - Tarang Jain (https://github.com/tarang-jain) Approvers: - Corey J. Nolet (https://github.com/cjnolet) URL: rapidsai#670
Authors: - Ben Frederickson (https://github.com/benfred) Approvers: - Corey J. Nolet (https://github.com/cjnolet) URL: rapidsai#664
|
These lines should be added to the My phrasing above is bad, to be clearer, those lines should be removed. |
PR does the following:
- [x] Modifies CI to run pytest and e2e test of cuvs-bench
- [x] We need to test the additional time needed to run the tests. They should be fast, but if they are not, then we can add an additional job to run them in parallel.
- [x] Adds synthetic test-data generation so the CI jobs don't depend on downloading datasets, and users can have easy testing locally.
- [ ] Few improvements to be done to docs, yaml and other things to make it easy for users.
- [x] Check in some additional pytests that hadn't been checked in before.
Authors:
- Dante Gama Dessavre (https://github.com/dantegd)
- Corey J. Nolet (https://github.com/cjnolet)
- Micka (https://github.com/lowener)
Approvers:
- James Lamb (https://github.com/jameslamb)
- Corey J. Nolet (https://github.com/cjnolet)
URL: rapidsai#574
|
/ok to test |
|
/ok to test |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Brings DiskANN into cuvs-bench