Babak/upgrade triton to v2.44.0#5

Merged

babakbehzad merged 654 commits intoverkada:babak/upgrade-triton-to-v2.44.0from triton-inference-server:main

Apr 5, 2024

+82,907-33,584

This pull request is big! We're only showing the most recent 250 commits

Commits on Jun 29, 2023

Add test for Python BLS model loading API (#5980 )
krishung5
authored

Commits on Jun 30, 2023

Commits on Jul 5, 2023

Add dependency for L0_python_client_unit_tests (#6010 )
GuanLuo
authored

Commits on Jul 6, 2023

Improve performance tuning guide (#6026 )
Tabrizian
authored

Commits on Jul 10, 2023

Fix implicit state test (#6039 )
Tabrizian
authored

Commits on Jul 11, 2023

Adding target_compile_features cxx_std_17 to tracing lib (#6040 )
oandreeva-nv
authored

Commits on Jul 12, 2023

Commits on Jul 17, 2023

Fix broken links pointing to the grpc_server.cc file (#6068 )
matemijolovic
authored

Commits on Jul 18, 2023

Fix L0_backend_python expected instance name (#6073 )
kthui
authored

Commits on Jul 19, 2023

Fix L0_sdk: update the search name for the client wheel (#6074 )
GuanLuo
authored

Commits on Jul 20, 2023

Add GitHub action to format and lint code (#6022 )

dyastremsky
and
rmccorm4
authored

Commits on Jul 21, 2023

Fix default instance name change when count is 1 (#6088 )
kthui
authored

Commits on Jul 24, 2023

Commits on Jul 25, 2023

Removing unused tests (#6085 )
oandreeva-nv
authored

Commits on Jul 28, 2023

Commits on Aug 1, 2023

Reduce instance count to 1 for python bls model loading test (#6130 )
krishung5
authored

Commits on Aug 4, 2023

Commits on Aug 7, 2023

Adding the support tracing of child models invoked from a BLS model (#6063 )

oandreeva-nv
and
rmccorm4
authored

Commits on Aug 9, 2023

Commits on Aug 14, 2023

Commits on Aug 16, 2023

Commits on Aug 17, 2023

Commits on Aug 18, 2023

Check for log error on failing to find explicit load model (#6204 )
rmccorm4
authored

Commits on Aug 22, 2023

Set default shm size to 1MB for Python backend (#6209 )
Tabrizian
authored

Commits on Aug 28, 2023

Add watts units to GPU power metric descriptions (#6242 )
rmccorm4
authored

Commits on Aug 30, 2023

Commits on Sep 2, 2023

Commits on Sep 5, 2023

Add test for TensorRT version-compatible model support (#6255 )
dyastremsky
authored

Commits on Sep 6, 2023

Commits on Sep 8, 2023

Update 'main' to track development of 2.39.0 / 23.10 (#6277 )
mc-nv
authored

Commits on Sep 12, 2023

Adding installation of docker and docker-buildx (#6299 )
mc-nv
authored

Commits on Sep 13, 2023

Use targetmodel from header as model name in SageMaker (#6147 )
nikhil-sk
authored

Commits on Sep 14, 2023

Add more error messages, return codes, and refactor HTTP server (#6297 )
rmccorm4
authored

Commits on Sep 18, 2023

Commits on Sep 20, 2023

Tests and docs for ability to specify subdirectory to download for LocalizePath (#6308 )
oandreeva-nv
authored

Commits on Sep 21, 2023

Refactor HandleInfer into more readable chunks (#6332 )
rmccorm4
authored

Commits on Sep 25, 2023

Refactor model generation scripts (#6336 )
Tabrizian
authored

Commits on Sep 26, 2023

Add testing for scalar I/O in ORT backend (#6343 )
Tabrizian
authored

Commits on Sep 29, 2023

Update post-23.09 release (#6367 )
mc-nv
authored

Commits on Oct 3, 2023

Add security policy (#6376 )
dyastremsky
authored

Commits on Oct 4, 2023

Commits on Oct 6, 2023

Commits on Oct 7, 2023

Add TRT-LLM backend build to Triton (#6365 ) (#6392 )
krishung5
authored

Commits on Oct 8, 2023

Commits on Oct 9, 2023

Commits on Oct 11, 2023

Commits on Oct 16, 2023

Commits on Oct 17, 2023

Switch Jetson model TensorRT models generation to container (#6378 )
mc-nv
authored

Commits on Oct 18, 2023

Commits on Oct 19, 2023

Commits on Oct 20, 2023

Commits on Oct 24, 2023

Commits on Oct 25, 2023

Test Python BLS with different sizes of CUDA memory pool (#6276 )
krishung5
authored

Commits on Oct 26, 2023

Add documentation for K8s-onprem StartupProbe (#5257 )

authored

Commits on Oct 27, 2023

Update main post-23.10 release (#6484 )

authored

Commits on Oct 28, 2023

Adding structure reference to the new document (#6493 )
mc-nv
authored

Commits on Oct 30, 2023

Improve L0_backend_python test stability (ensemble / gpu_tensor_lifecycle) (#6490 )
rmccorm4
authored

Commits on Nov 1, 2023

Commits on Nov 3, 2023

Commits on Nov 6, 2023

Upgrade DCGM from 2.4.7 to 3.2.6 (#6515 )
kthui
authored

Commits on Nov 9, 2023

Commits on Nov 10, 2023

Commits on Nov 13, 2023

Add error during rescheduling test to L0_generative_sequence (#6550 )
GuanLuo
authored

Commits on Nov 15, 2023

Commits on Nov 19, 2023

Change name generative -> iterative (#6601 )
nnshah1
authored

Commits on Nov 20, 2023

Commits on Nov 29, 2023

Update protobuf after python update for testing (#6638 )
jbkyang-nvi
authored

Commits on Dec 4, 2023

Commits on Dec 5, 2023

Add note on lack of ensemble support (#6648 )
rmccorm4
authored

Commits on Dec 6, 2023

Added request id to span attributes (#6667 )
oandreeva-nv
authored

Commits on Dec 8, 2023

Add test for optional internal tensor within an ensemble (#6663 )
GuanLuo
authored

Commits on Dec 10, 2023

Set CMake version to 3.27.7 (#6675 )
mc-nv
authored

Commits on Dec 11, 2023

restore typo (#6680 )
mc-nv
authored

Commits on Dec 12, 2023

Commits on Dec 13, 2023

Commits on Dec 14, 2023

adding default value for TRITON_IGPU_BUILD=OFF (#6705 )

nv-kmcgill53
and
kyle
authored

Commits on Dec 19, 2023

Modify HTTP frontend to return error code reflecting Triton error. Add test (#6713 )
GuanLuo
authored

Commits on Dec 26, 2023

Update main branch post-23.12 release (#6726 )

authored

Commits on Dec 28, 2023

trtllm CI fixes (#6687 )
mc-nv
authored

Commits on Jan 2, 2024

Remove double unit test (#6714 )
kthui
authored

Commits on Jan 3, 2024

Commits on Jan 4, 2024

Fix L0_lifecycle on insufficient hardware concurrency (#6762 )
kthui
authored

Commits on Jan 5, 2024

Commits on Jan 8, 2024

Commits on Jan 11, 2024

Commits on Jan 12, 2024

Commits on Jan 17, 2024

Commits on Jan 18, 2024

Commits on Jan 19, 2024

Commits on Jan 22, 2024

Remove boost::filesystem (#6810 )
rmccorm4
authored

Commits on Jan 23, 2024

Generate unittest xml reports from L0_python_api (#6822 )
rmccorm4
authored

Commits on Jan 27, 2024

Add test for shutdown while unloading in background (#6835 )
kthui
authored

Commits on Jan 29, 2024

Commits on Jan 30, 2024

Commits on Jan 31, 2024

Add test for shutdown while loading model (#6837 )
kthui
authored

Commits on Feb 1, 2024

Commits on Feb 7, 2024

Updating vllm version to 0.3.0 (#6858 )
oandreeva-nv
authored

Commits on Feb 8, 2024

Python Backend Windows Support (#6830 )
fpetrini15
authored

Commits on Feb 9, 2024

Add support for Oracle Cloud in deploy (#6850 )
bruno-garbaccio
authored

Commits on Feb 13, 2024

Add link to TRTLLM metrics docs (#6874 )
rmccorm4
authored

Commits on Feb 16, 2024

Update 'main' to track development of 2.44.0 / 24.03 (#6892 )
mc-nv
authored

Commits on Feb 17, 2024

Add response statistics (#6869 )

kthui
and
rmccorm4
authored

Commits on Feb 22, 2024

Fix busyop test for L0_memory_growth (#6900 )
krishung5
authored

Commits on Feb 23, 2024

Add cancellation into response statistics (#6904 )
kthui
authored

Commits on Feb 24, 2024

Install required pip pkgs (#6906 )
krishung5
authored

Commits on Feb 27, 2024

Match forward headers case insensitively. (#6889 )
yinggeh
authored

Commits on Mar 1, 2024

Add note on --cache-config spacing and fix typos (#6929 )
rmccorm4
authored

Commits on Mar 2, 2024

Commits on Mar 4, 2024

Expose tritonserver args in values.yaml (#5582 )
okyspace
authored

Commits on Mar 6, 2024

Parameterize git repository (#6934 )

nv-kmcgill53
and
kyle
authored

Commits on Mar 9, 2024

Test improved input validation errors (#6933 )
indrajit96
authored

Commits on Mar 13, 2024

Commits on Mar 15, 2024

Commits on Mar 19, 2024

Add documentation for mapping between Triton Errors and HTTP status codes (#6992 )
Tabrizian
authored

Commits on Mar 21, 2024

Commits on Mar 25, 2024

Commits on Mar 27, 2024

Add meetup invite banner (#7049 )
rmccorm4
authored

Commits on Apr 1, 2024

Update 'main' post-24.03 (#7051 )

authored

Commits on Apr 4, 2024