Babak/upgrade triton to v2.44.0 #5
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Add test for Python BLS model loading API * Fix up
* Adding nested spans to OTel tracing + support of ensemble models
* Move multi-GPU dlpack test to a separate L0 test * Fix copyright * Fix up
* Upgrade OV to 2023.0.0 * Upgrade OV model gen script to 2023.0.0
* Add test to check the output memory type for onnx models * Remove unused import * Address comment
* Add testing for implicit state for PyTorch backend * Add testing for libtorch string implicit models * Fix CodeQL * Mention that libtorch backend supports implicit state * Fix CodeQL * Review edits * Fix output tests for PyTorch backend
Add test for uncompressed conda execution enviroments
* Fix expected instance name * Copyright year
* Fix name of client wheel to be looked for * Fix up
* Add pre-commit * Fix typos, exec/shebang, formatting * Remove clang-format * Update contributing md to include pre-commit * Update spacing in CONTRIBUTING * Fix contributing pre-commit link * Link to pre-commit install directions * Wording * Restore clang-format * Fix yaml spacing * Exclude templates folder for check-yaml * Remove unused vars * Normalize spacing * Remove unused variable * Normalize config indentation * Update .clang-format to enforce max line length of 80 * Update copyrights * Update copyrights * Run workflows on every PR * Fix copyright year * Fix grammar * Entrypoint.d files are not executable * Run pre-commit hooks * Mark not executable * Run pre-commit hooks * Remove unused variable * Run pre-commit hooks after rebase * Update copyrights * Fix README.md typo (decoupled) Co-authored-by: Ryan McCormick <[email protected]> * Run pre-commit hooks * Grammar fix Co-authored-by: Ryan McCormick <[email protected]> * Redundant word Co-authored-by: Ryan McCormick <[email protected]> * Revert docker file changes * Executable shebang revert * Make model.py files non-executable * Passin is proper flag * Run pre-commit hooks on init_args/model.py * Fix typo in init_args/model.py * Make copyrights one line --------- Co-authored-by: Ryan McCormick <[email protected]>
* Add test for sequence model instance update * Add gap for file timestamp update * Update test for non-blocking sequence update * Update documentation * Remove mentioning increase instance count case * Add more documentaion for scheduler update test * Update test for non-blocking batcher removal * Add polling due to async scheduler destruction * Use _ as private * Fix typo * Add docs on instance count decrease * Fix typo * Separate direct and oldest to different test cases * Separate nested tests in a loop into multiple test cases * Refactor scheduler update test * Improve doc on handling future test failures * Address pre-commit * Add best effort to reset model state after a single test case failure * Remove reset model method to make harder for chaining multiple test cases as one * Remove description on model state clean up
* Update README and versions for 2.36.0 / 23.07 * Update Dockerfile.win10.min * Fix formating issue * fix formating issue * Fix whitespaces * Fix whitespaces * Fix whitespaces
* Reduce instance count to 1 for python bls model loading test * Add comment when calling unload
* Update README and versions for 2.43.0 / 24.02 * Update Dockefile to reduce image size. * Update path in patch file for model generation Update README.md post-24.02
* patching git repository parameterization from production branch 1 * Fix go package directory name * pre-commit fixes * pre-commit fixes --------- Co-authored-by: kyle <[email protected]>
* Enhance bound check for shm offset * Add test for enhance bound check for shm offset * Fix off by 1 on max offset * Improve comments * Improve comment and offset * Separate logic between computation and validation
…6017) * Allow non-decoupled model to send response and FINAL flag separately * Update copyright * Defer sending error until FINAL flag is seen to avoid invalid reference * Move timestamp capture location * Delay time-point of response complete timestamp in GPRC and SageMaker endpoint * Move location of RESPONSE_COMPLETE timestamp capture to better align with the meaning.
Added a test case to check for optional/required input params in a request and appropriate response from server. Includes addition of 3 simple models with a combination of required/optional input params
Add flag to enable compile of OpenAI support in PA
* Test Correlation Id string support for BLS
* Add AsyncIO HTTP compression test * Improve command line option handling
* Update Docerkfile to install genai * Change the installation script * install both build and hatch * Update name --------- Co-authored-by: Elias Bermudez <[email protected]>
* Added TRITONSERVER_InferenceTraceSetContext logic
…odes (#6992) * Add documentation for mapping between Triton Errors and HTTP status codes * formatting * Update README.md
* Update README and versions for 2.44.0 / 24.03 (#6971) * Update README and versions for 2.44.0 / 24.03 * Mchornyi 24.03 (#6972) * Current location is dropped in 12.4 * Update Dockerfile.win10.min * Change to triton_sample_folder (#6973) --------- Co-authored-by: kyle <[email protected]> Co-authored-by: Misha Chornyi <[email protected]> * Specify path for PyTorch model extension library (#7025) * Update README.md 2.44.0 / 24.03 (#7032) * Update README.md post-24.03 --------- Co-authored-by: Kyle McGill <[email protected]> Co-authored-by: kyle <[email protected]>
* Fix Otel version * Fix version in CPU metrics * Update metrics.md * Update trace.md
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| # }, | ||
| "use_edit_page_button": False, | ||
| "use_issues_button": True, | ||
| "use_repository_button": True, |
Check warning
Code scanning / CodeQL
Duplicate key in dict literal Warning documentation
|
|
||
| class Config(dict): |
Check warning
Code scanning / CodeQL
`__eq__` not overridden when adding attributes Warning
babakbehzad
merged commit 4018497
into
verkada:babak/upgrade-triton-to-v2.44.0
5 checks passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.