Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add trt support for BF16 #195

Draft
wants to merge 232 commits into
base: main
Choose a base branch
from
Draft

Add trt support for BF16 #195

wants to merge 232 commits into from
+2,504 −1

Commits on Nov 14, 2024

  1. add question logger

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    f8747c2 View commit details
    Browse the repository at this point in the history

  2. add cli input for TRT support

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    bf71e81 View commit details
    Browse the repository at this point in the history

  3. initial support for TRT engine builder

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    223017d View commit details
    Browse the repository at this point in the history

  4. add onnx export functions taken from

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    fd7057e View commit details
    Browse the repository at this point in the history

  5. base class for convert to onnx

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    0713049 View commit details
    Browse the repository at this point in the history

  6. add missing dependencies

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    d0ba09a View commit details
    Browse the repository at this point in the history

  7. fix imports

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    959eaa7 View commit details
    Browse the repository at this point in the history

  8. moved to wrappers package

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    50149bd View commit details
    Browse the repository at this point in the history

  9. moved to wrapper package and renamed into base wrapper

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    71c1b0d View commit details
    Browse the repository at this point in the history

  10. implement load engines function

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    407a3ac View commit details
    Browse the repository at this point in the history

  11. remove old wrapper

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    ca691f1 View commit details
    Browse the repository at this point in the history

  12. add additional parameters to base class constructor

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    78fbb25 View commit details
    Browse the repository at this point in the history

  13. implemented CLIP wrapper

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    3548085 View commit details
    Browse the repository at this point in the history

  14. remove model as a property

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    6b01977 View commit details
    Browse the repository at this point in the history

  15. enable float16 optimization

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    2477223 View commit details
    Browse the repository at this point in the history

  16. reorder arguments

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    969e06d View commit details
    Browse the repository at this point in the history

  17. first wrapper fir onnx build

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    f8a183b View commit details
    Browse the repository at this point in the history

  18. add load_engines with minimal parameters

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    5572ae7 View commit details
    Browse the repository at this point in the history

  19. fix get_sample_input interface format and add get_model_to_trace

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    713a66e View commit details
    Browse the repository at this point in the history

  20. fix stage name

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    b4bebd2 View commit details
    Browse the repository at this point in the history

  21. ad imports for wrappers

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    e56078c View commit details
    Browse the repository at this point in the history

  22. set assert error message for missing stage

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    f1fa53b View commit details
    Browse the repository at this point in the history

  23. add assert to validate only 1 dtype is active

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    0afcc8e View commit details
    Browse the repository at this point in the history

  24. add set_model_to_dtype function to set the correct dtype

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    9b600b3 View commit details
    Browse the repository at this point in the history

  25. call set_model_to_dtype instead of doing it manually

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    da78bef View commit details
    Browse the repository at this point in the history

  26. T5wrapper as a copy of CLIPwrapper

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    2c7201f View commit details
    Browse the repository at this point in the history

  27. rename get_model_to_trace to get_model

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    d2b2567 View commit details
    Browse the repository at this point in the history

  28. do_constant_folding should not be configurable

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    d6c18c8 View commit details
    Browse the repository at this point in the history

  29. removed custom configurations

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    89f616e View commit details
    Browse the repository at this point in the history

  30. fix import

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    44a3280 View commit details
    Browse the repository at this point in the history

  31. add t5wrapper to possible imports

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    27c4268 View commit details
    Browse the repository at this point in the history

  32. use T5Wrapper

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    c35d4d7 View commit details
    Browse the repository at this point in the history

  33. fluxwrapper as a copy of t5wrapper

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    4e273e1 View commit details
    Browse the repository at this point in the history

  34. implement flux exporter

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    e111c72 View commit details
    Browse the repository at this point in the history

  35. use flux exporter

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    ec75372 View commit details
    Browse the repository at this point in the history

  36. vae as copy of clip wrapper

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    6acb1af View commit details
    Browse the repository at this point in the history

  37. rename and fix imports

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    6c941e6 View commit details
    Browse the repository at this point in the history

  38. removed embedding_dim as not used

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    1e11bc1 View commit details
    Browse the repository at this point in the history

  39. remove embedding_dim and forward function point to decode

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    37ddc7e View commit details
    Browse the repository at this point in the history

  40. get_output_names = latent

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    fa22eac View commit details
    Browse the repository at this point in the history

  41. fix interface of get_sample_input

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    eb4e7df View commit details
    Browse the repository at this point in the history

  42. save configuration parameters

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    bf1cca6 View commit details
    Browse the repository at this point in the history

  43. ae wrapper implemented

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    358c8a5 View commit details
    Browse the repository at this point in the history

  44. fix import

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    381267d View commit details
    Browse the repository at this point in the history

  45. add AEWrapper step

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    a8af1d8 View commit details
    Browse the repository at this point in the history

  46. from set_model_to_dtype to prepare_model

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    a47608c View commit details
    Browse the repository at this point in the history

  47. fix eval mode during inference

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    ea420c5 View commit details
    Browse the repository at this point in the history

  48. fix clip onnx export. Now it trace ony the needed outputs

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    af2f48b View commit details
    Browse the repository at this point in the history

  49. fix t5 wrapper

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    e6b66bb View commit details
    Browse the repository at this point in the history

  50. reorder input name flux

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    cb188d8 View commit details
    Browse the repository at this point in the history

  51. fix flux input format for text_ids and guidance

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    54002de View commit details
    Browse the repository at this point in the history

  52. fix Flux imports and scale of inputs to prevent nan 

    added `"latent": {0: "B", 1: "latent_dim"}` as additional dynamic axes
    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    1cdc0a8 View commit details
    Browse the repository at this point in the history

  53. add torch inference while tracing

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    c1c3a8d View commit details
    Browse the repository at this point in the history

  54. fix casting problem in onnx trace

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    bb0cc66 View commit details
    Browse the repository at this point in the history

  55. solve optimization problem by removing cleanup steps

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    21ec7d9 View commit details
    Browse the repository at this point in the history

  56. rename to notes

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    d6f5e2f View commit details
    Browse the repository at this point in the history

  57. prevent nan due to large inputs

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    577ba49 View commit details
    Browse the repository at this point in the history

  58. provide base implementation of get_model

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    dfd06fc View commit details
    Browse the repository at this point in the history

  59. format

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    54b2ceb View commit details
    Browse the repository at this point in the history

  60. add trt export step

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    d7ccef4 View commit details
    Browse the repository at this point in the history

  61. add engine class for trt build

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    505411b View commit details
    Browse the repository at this point in the history

  62. add get_input_profile and get_minmax_dims abstract methods

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    0154232 View commit details
    Browse the repository at this point in the history

  63. add build_strongly_typed attributed

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    cc2d921 View commit details
    Browse the repository at this point in the history

  64. implement get_minmax_dims and get_input_profile

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    a2fb731 View commit details
    Browse the repository at this point in the history

  65. remove static_shape from get_sample_input

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    0096f7a View commit details
    Browse the repository at this point in the history

  66. remove static sharpe and batch flags

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    dfb6ded View commit details
    Browse the repository at this point in the history

  67. add typing

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    50100b5 View commit details
    Browse the repository at this point in the history

  68. remove static shape and batch flags

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    ea240be View commit details
    Browse the repository at this point in the history

  69. offload to cpu

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    0c3720c View commit details
    Browse the repository at this point in the history

  70. enable device offloading while tracing

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    30f0140 View commit details
    Browse the repository at this point in the history

  71. check cuda is avaiable while building engines

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    f2b357a View commit details
    Browse the repository at this point in the history

  72. clip trt engine build

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    a30ec20 View commit details
    Browse the repository at this point in the history

  73. add pinned transformer dependency

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    dbeeed9 View commit details
    Browse the repository at this point in the history

  74. fix nan with onnx and trt when executed on CUDA

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    0682915 View commit details
    Browse the repository at this point in the history

  75. AE need to be traced in TF32 not FP16

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    bef25e0 View commit details
    Browse the repository at this point in the history

  76. add get_shape_dict abstract method and device as a property

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    c028d8d View commit details
    Browse the repository at this point in the history

  77. AE should be traced in TF32

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    8208e4c View commit details
    Browse the repository at this point in the history

  78. AE explicitly on TF32 and reactivate full pipeline

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    816ff12 View commit details
    Browse the repository at this point in the history

  79. add input provile to flux to enable trt engine build

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    3a341f8 View commit details
    Browse the repository at this point in the history

  80. format and add input_profile to t5 for TRT build

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    7aa6956 View commit details
    Browse the repository at this point in the history

  81. add TransformersModelWrapper

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    e68a993 View commit details
    Browse the repository at this point in the history

  82. add TransformersModelWrapper support

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    ea581b7 View commit details
    Browse the repository at this point in the history

  83. add get_shape_dict interface

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    7e883d5 View commit details
    Browse the repository at this point in the history

  84. add TransformersModelWrapper support

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    5080d86 View commit details
    Browse the repository at this point in the history

  85. add shape_dict interface

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    e2b65c4 View commit details
    Browse the repository at this point in the history

  86. t5 in TF32 for numerical reasons

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    87413e2 View commit details
    Browse the repository at this point in the history

  87. remove unused options

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    8629e50 View commit details
    Browse the repository at this point in the history

  88. remove unused code

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    5e711c7 View commit details
    Browse the repository at this point in the history

  89. add get_shape_dict

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    02235dc View commit details
    Browse the repository at this point in the history

  90. remove custom optimization

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    6c3c4db View commit details
    Browse the repository at this point in the history

  91. add garbage collector

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    4b8a973 View commit details
    Browse the repository at this point in the history

  92. return error

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    8e4b103 View commit details
    Browse the repository at this point in the history

  93. create wrapper specific to Onnx export operatio

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    8f45f81 View commit details
    Browse the repository at this point in the history

  94. user OnnxWrapper

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    3af1a33 View commit details
    Browse the repository at this point in the history

  95. create base wrapper for trt engines

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    fe024b8 View commit details
    Browse the repository at this point in the history

  96. moved to engine package

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    68060bd View commit details
    Browse the repository at this point in the history

  97. moved to engine package

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    0f8d8b3 View commit details
    Browse the repository at this point in the history

  98. forbit relative import of trt-builder

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    49dc6d1 View commit details
    Browse the repository at this point in the history

  99. remove wrapper and create BaseExporter or BaseEngine

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    098391b View commit details
    Browse the repository at this point in the history

  100. models not stored in builder class

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    bf9c4cb View commit details
    Browse the repository at this point in the history

  101. _prepare_model_configs as pure function

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    0ee9104 View commit details
    Browse the repository at this point in the history

  102. _get_onnx_exporters as a private method to get onnx exporters

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    c7136f8 View commit details
    Browse the repository at this point in the history

  103. remove unused dependencies

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    ee72695 View commit details
    Browse the repository at this point in the history

  104. from onnxwrapper to onnxengine

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    ecf6c4f View commit details
    Browse the repository at this point in the history

  105. trt engine class

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    2a14000 View commit details
    Browse the repository at this point in the history

  106. add calculate_max_device_memory to TRTBuilder

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    c791c53 View commit details
    Browse the repository at this point in the history

  107. get_shape_dict moved to trt-engine interface

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    ce343dc View commit details
    Browse the repository at this point in the history

  108. add common inference code

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    66ca1ce View commit details
    Browse the repository at this point in the history

  109. autoencder inference wrapper

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    7400072 View commit details
    Browse the repository at this point in the history

  110. add requirements.txt

    @andompesta
    scavallari authored and andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    aa0d474 View commit details
    Browse the repository at this point in the history

  111. support guidance for ev model

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    d676a18 View commit details
    Browse the repository at this point in the history

  112. ad support for trt based on evn variables

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    550f660 View commit details
    Browse the repository at this point in the history

  113. format flux

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    fa5993b View commit details
    Browse the repository at this point in the history

  114. remove stream from constructor

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    bdbbb19 View commit details
    Browse the repository at this point in the history

  115. fix iterate over onnx-exporters

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    f1d86f6 View commit details
    Browse the repository at this point in the history

  116. flux is not strongly type

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    f065b09 View commit details
    Browse the repository at this point in the history

  117. move back for numerical stability

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    c57410a View commit details
    Browse the repository at this point in the history

  118. add logging

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    69f4dca View commit details
    Browse the repository at this point in the history

  119. fix dtype casting for bfloat16

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    cc12a14 View commit details
    Browse the repository at this point in the history

  120. fix default value

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    961259e View commit details
    Browse the repository at this point in the history

  121. add version before merge

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    6e1ca02 View commit details
    Browse the repository at this point in the history

  122. hacky get it building the engines

    @ducktrA @andompesta
    ducktrA authored and andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    7217a7b View commit details
    Browse the repository at this point in the history

  123. requirements.txt

    @ducktrA @andompesta
    ducktrA authored and andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    c5481a1 View commit details
    Browse the repository at this point in the history

  124. adding a seperate _engine.py file for all the flux, t5 and clip engine

    @ducktrA @andompesta
    ducktrA authored and andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    54674c3 View commit details
    Browse the repository at this point in the history

  125. boilerroom and plating. getting parameters handle into setting up the… 

    … trt engines
    @ducktrA @andompesta
    ducktrA authored and andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    37003c7 View commit details
    Browse the repository at this point in the history

  126. remove _version.py from git

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    fd33eb5 View commit details
    Browse the repository at this point in the history

  127. create base mixin class to share parameters

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    99e72e9 View commit details
    Browse the repository at this point in the history

  128. clipmixin parameters

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    6678a3b View commit details
    Browse the repository at this point in the history

  129. remove parameters as are part of mixin class

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    395541d View commit details
    Browse the repository at this point in the history

  130. clip engine and exporter use common mixin for managing parameters

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    315dd9d View commit details
    Browse the repository at this point in the history

  131. use mixin cass to build engine from exporter

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    7cdbb03 View commit details
    Browse the repository at this point in the history

  132. ae-mixin for shared parameters

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    55497eb View commit details
    Browse the repository at this point in the history

  133. flux exporter and engine unified by mixin class

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    5917f38 View commit details
    Browse the repository at this point in the history

  134. formatting

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    7c156cd View commit details
    Browse the repository at this point in the history

  135. add common get_latent_dims method

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    92f13f8 View commit details
    Browse the repository at this point in the history

  136. add get_latent_dims common method

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    f5acd54 View commit details
    Browse the repository at this point in the history

  137. T5 based on mixin class

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    8b182cc View commit details
    Browse the repository at this point in the history

  138. build strongly typed flux

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    11570dc View commit details
    Browse the repository at this point in the history

  139. enable load with shared device memory

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    a9acfa0 View commit details
    Browse the repository at this point in the history

  140. remove boilderpart code to create engines

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    c6e94a6 View commit details
    Browse the repository at this point in the history

  141. add tokenizer to trt engine

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    7b07602 View commit details
    Browse the repository at this point in the history

  142. use static shape for reduce memory consumption

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    2dc2460 View commit details
    Browse the repository at this point in the history

  143. implemnet tokenizer into t5 engine

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    40de55c View commit details
    Browse the repository at this point in the history

  144. mix max_batch size to 8

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    c8273c7 View commit details
    Browse the repository at this point in the history

  145. add licence

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    b96fd96 View commit details
    Browse the repository at this point in the history

  146. add licence

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    6743bb7 View commit details
    Browse the repository at this point in the history

  147. enable trt runtime tracking

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    852b444 View commit details
    Browse the repository at this point in the history

  148. add static-batch and static-shape options

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    95f7822 View commit details
    Browse the repository at this point in the history

  149. add cuda steam to load method

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    8ac3f84 View commit details
    Browse the repository at this point in the history

  150. add inference code

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    f93fc87 View commit details
    Browse the repository at this point in the history

  151. add inference code

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    528621a View commit details
    Browse the repository at this point in the history

  152. enable static shape

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    23e1236 View commit details
    Browse the repository at this point in the history

  153. add static_shape option to reduce memory and _build_engine as sta… 

    …ticmethod
    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    dc326df View commit details
    Browse the repository at this point in the history

  154. add should_be_dtype filed to handle output type conversion

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    7e3fe14 View commit details
    Browse the repository at this point in the history

  155. from trtbuilder to trt_manager

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    41f18e7 View commit details
    Browse the repository at this point in the history

  156. from TRTBuilder to TRTManager

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    12dee48 View commit details
    Browse the repository at this point in the history

  157. AE engine interface

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    45997a9 View commit details
    Browse the repository at this point in the history

  158. trt_to_torch_dtype_dict as property

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    bb9f468 View commit details
    Browse the repository at this point in the history

  159. clip engine inference

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    2bde369 View commit details
    Browse the repository at this point in the history

  160. implement flux trt engine inference process

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    359572e View commit details
    Browse the repository at this point in the history

  161. add scale_factor and shift_factor

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    e3f0fd9 View commit details
    Browse the repository at this point in the history

  162. removed should_be_dtype

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    d91bbde View commit details
    Browse the repository at this point in the history

  163. removed should_be_dtype

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    df245db View commit details
    Browse the repository at this point in the history

  164. remove should_be_dtype from t5

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    33bc095 View commit details
    Browse the repository at this point in the history

  165. add scale and shift factor

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    c330491 View commit details
    Browse the repository at this point in the history

  166. max_batch to 8

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    90b4f11 View commit details
    Browse the repository at this point in the history

  167. implement TRTManager

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    17c1f7d View commit details
    Browse the repository at this point in the history

  168. from ae to vae to match DD

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    811f2ff View commit details
    Browse the repository at this point in the history

  169. remove autocast

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    f4ae3ca View commit details
    Browse the repository at this point in the history

  170. pooled_embeddings to match DD naming for clip

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    0fe7c84 View commit details
    Browse the repository at this point in the history

  171. rename flux to transformer engine

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    f71091a View commit details
    Browse the repository at this point in the history

  172. from flux to transformer mixin

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    4055a3e View commit details
    Browse the repository at this point in the history

  173. from flux to transforemer exporter

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    2b2bb5b View commit details
    Browse the repository at this point in the history

  174. fix trtmanger with naming

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    b088430 View commit details
    Browse the repository at this point in the history

  175. fix inputs names and dimentions. Nota that img_ids and txt_ids ar… 

    …e without batch dim
    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    82d658d View commit details
    Browse the repository at this point in the history

  176. fix shape of inputs according to text_maxlen and batch_size

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    3708773 View commit details
    Browse the repository at this point in the history

  177. reduce max_batch

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    7737426 View commit details
    Browse the repository at this point in the history

  178. fix stage naming

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    917d8ff View commit details
    Browse the repository at this point in the history

  179. add support for DD model

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    6473ca1 View commit details
    Browse the repository at this point in the history

  180. add support for DD models

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    6d39ad5 View commit details
    Browse the repository at this point in the history

  181. fix dtype configuration

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    753129b View commit details
    Browse the repository at this point in the history

  182. fix enginge dtype

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    149c27c View commit details
    Browse the repository at this point in the history

  183. trensformers inference interface to match DD

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    55568bf View commit details
    Browse the repository at this point in the history

  184. vae inference script dtype mapping

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    4872169 View commit details
    Browse the repository at this point in the history

  185. remove dtype checks as multiples can be actives

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    41ee44c View commit details
    Browse the repository at this point in the history

  186. by default tf32 always active

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    a31161d View commit details
    Browse the repository at this point in the history

  187. fix trt enginges names

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    3b91c51 View commit details
    Browse the repository at this point in the history

  188. add wrapper for fluxmodel to match DD onnx configuration

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    4ebca7d View commit details
    Browse the repository at this point in the history

  189. add autocast back in to match DD setup

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    3e9f64f View commit details
    Browse the repository at this point in the history

  190. fix dependencies for trt support

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    bb82e4b View commit details
    Browse the repository at this point in the history

  191. support trt

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    830358e View commit details
    Browse the repository at this point in the history

  192. add explicit kwargs

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    cdce3a3 View commit details
    Browse the repository at this point in the history

  193. vscode setup

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    b789e05 View commit details
    Browse the repository at this point in the history

  194. add setup instructions for trt

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    8b07e6e View commit details
    Browse the repository at this point in the history

  195. trt dependencies not part of all

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    5ffd6d6 View commit details
    Browse the repository at this point in the history

  196. from onnx_exporter to exporter

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    766d878 View commit details
    Browse the repository at this point in the history

  197. hide onnx parameters

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    6d83690 View commit details
    Browse the repository at this point in the history

  198. from onnx-exporter to exporter

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    2458486 View commit details
    Browse the repository at this point in the history

  199. exporter responsible to build trt engine and onnx exportr

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    80a52d7 View commit details
    Browse the repository at this point in the history

  200. hide onnx parameter

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    adf2d46 View commit details
    Browse the repository at this point in the history

  201. remove build function from engine class

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    e82311f View commit details
    Browse the repository at this point in the history

  202. remove unused import

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    17f6562 View commit details
    Browse the repository at this point in the history

  203. remove space

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    2512bb2 View commit details
    Browse the repository at this point in the history

  204. manage t5 and vae separately

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    86614a3 View commit details
    Browse the repository at this point in the history

  205. disable autocast

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    f14de69 View commit details
    Browse the repository at this point in the history

  206. stronglytyped t5

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    3410d34 View commit details
    Browse the repository at this point in the history

  207. fix input type and max image size

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    2422538 View commit details
    Browse the repository at this point in the history

  208. max image size

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    9bef65b View commit details
    Browse the repository at this point in the history

  209. T5 not strongly typed

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    a3bd8fc View commit details
    Browse the repository at this point in the history

  210. testing

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    e615fa0 View commit details
    Browse the repository at this point in the history

  211. fix torch sycronize problem

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    611efed View commit details
    Browse the repository at this point in the history

  212. don't build already present engines

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    13b1016 View commit details
    Browse the repository at this point in the history

  213. remove torch save

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    01b508c View commit details
    Browse the repository at this point in the history

  214. removed onnx dependencies

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    f57b5a5 View commit details
    Browse the repository at this point in the history

  215. add trt dependencies

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    9cffa24 View commit details
    Browse the repository at this point in the history

  216. remove trt dependencies from toml

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    63e29cc View commit details
    Browse the repository at this point in the history

  217. rename requirements and fix readme

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    c978cc3 View commit details
    Browse the repository at this point in the history

  218. remove unused files

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    3087c60 View commit details
    Browse the repository at this point in the history

  219. fix import format

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    5c2cba1 View commit details
    Browse the repository at this point in the history

  220. remove comments

    @andompesta
    andompesta committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    08fbb60 View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2024

  1. add gitignore

    @andompesta
    andompesta committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    1b4a41a View commit details
    Browse the repository at this point in the history

  2. reset dependencies

    @andompesta
    andompesta committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    a404144 View commit details
    Browse the repository at this point in the history

  3. add hidden setup files

    @andompesta
    andompesta committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    a8b8478 View commit details
    Browse the repository at this point in the history

  4. solve ruff check

    @andompesta
    andompesta committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    8fa1d22 View commit details
    Browse the repository at this point in the history

  5. fix imports with rufs

    @andompesta
    andompesta committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    3f20508 View commit details
    Browse the repository at this point in the history

  6. run ruff formatter

    @andompesta
    andompesta committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    7662313 View commit details
    Browse the repository at this point in the history

  7. update gitignore

    @andompesta
    andompesta committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    4691502 View commit details
    Browse the repository at this point in the history

Commits on Nov 18, 2024

  1. simplify dependencies

    @andompesta
    andompesta committed Nov 18, 2024
    Configuration menu
    Copy the full SHA
    deb5633 View commit details
    Browse the repository at this point in the history

  2. remove gitignore

    @andompesta
    andompesta committed Nov 18, 2024
    Configuration menu
    Copy the full SHA
    1de2799 View commit details
    Browse the repository at this point in the history

  3. add cli formatting

    @andompesta
    andompesta committed Nov 18, 2024
    Configuration menu
    Copy the full SHA
    64cbb8f View commit details
    Browse the repository at this point in the history

  4. fix import orders

    @andompesta
    andompesta committed Nov 18, 2024
    Configuration menu
    Copy the full SHA
    fd1455e View commit details
    Browse the repository at this point in the history

  5. Merge pull request #1 from andompesta/add-trt-support-push 

    Add trt support push
    @andompesta
    andompesta authored Nov 18, 2024
    Configuration menu
    Copy the full SHA
    095ee89 View commit details
    Browse the repository at this point in the history