Add boilerplate code

[jainapurva]

Stack from ghstack (oldest at bottom):

• Create separate float8 tensor subclass #1636
• -> Add boilerplate code #1635

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/1635

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 10 New Failures

As of commit d42c725 with merge base abd41e5 ():

NEW FAILURES - The following jobs have failed:

• Code Analysis with Ruff / build (3.9) (gh)
  Process completed with exit code 1.
• Run Float8 Tests / test (H100, linux.aws.h100, --pre torch torchvision torchaudio --index-url https://download.pytor... / linux-job (gh)
  RuntimeError: Command
• Run Regression Tests / test (CPU 2.3, linux.4xlarge, torch==2.3.0 --index-url https://download.pytorch.org/whl/cpu, cpu) / linux-job (gh)
  test/test_utils.py::TestTorchAOBaseTensor::test_print_arg_types
• Run Regression Tests / test (CPU 2.4, linux.4xlarge, torch==2.4.0 --index-url https://download.pytorch.org/whl/cpu, cpu) / linux-job (gh)
  test/test_utils.py::TestTorchAOBaseTensor::test_print_arg_types
• Run Regression Tests / test (CPU 2.5.1, linux.4xlarge, torch==2.5.1 --index-url https://download.pytorch.org/whl/cpu, cpu) / linux-job (gh)
  test/test_utils.py::TestTorchAOBaseTensor::test_print_arg_types
• Run Regression Tests / test (CUDA 2.3, linux.g5.12xlarge.nvidia.gpu, torch==2.3.0, cuda, 12.1) / linux-job (gh)
  test/test_utils.py::TestTorchAOBaseTensor::test_print_arg_types
• Run Regression Tests / test (CUDA 2.4, linux.g5.12xlarge.nvidia.gpu, torch==2.4.0, cuda, 12.1) / linux-job (gh)
  test/test_utils.py::TestTorchAOBaseTensor::test_print_arg_types
• Run Regression Tests / test (CUDA 2.5.1, linux.g5.12xlarge.nvidia.gpu, torch==2.5.1 --index-url https://download.pytorch... / linux-job (gh)
  test/test_utils.py::TestTorchAOBaseTensor::test_print_arg_types
• Run Regression Tests / test-nightly (CPU Nightly, linux.4xlarge, --pre torch==2.7.0.dev20250122 --index-url https://down... / linux-job (gh)
  test/test_utils.py::TestTorchAOBaseTensor::test_print_arg_types
• Run Regression Tests / test-nightly (CUDA Nightly, linux.g5.12xlarge.nvidia.gpu, --pre torch==2.7.0.dev20250122 --index-... / linux-job (gh)
  test/test_utils.py::TestTorchAOBaseTensor::test_print_arg_types

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[jerryzh168]

one thing here is what would happen when a tensor subclasses TorchAOBaseTensor and tried to overwrite these functions, to allow child tensor classes to overwrite these functions I think we'd have to copy the table

ao/torchao/utils.py

Line 402 in 7b0d2ce

cls._ATEN_OP_OR_TORCH_FN_TABLE = {}

when we detect a new child class is created, e.g.

def _implements(cls, aten_ops_or_torch_fns):
    # making sure we are querying the attribute from current class, not parent
    # please check if this works
    if "_ATEN_OP_OR_TORCH_FN_TABLE" not in dir(cls):
         # copy the table from parent

def _dispatch__torch_function__(cls, func, types, args=(), kwargs=None):
    # making sure we are querying the attribute from current class, not parent
    # please check if this works
    if "_ATEN_OP_OR_TORCH_FN_TABLE" not in dir(cls):
         # copy the table from parent if it exists

[vkuzo]

what are the pros and cons of using inheritance here versus just having utility functions and tensors using what they need from utility functions, without inheritance? It might be hard to come up with a TorchAOBaseTensor which is generic enough to truly handle all the important use cases in torchao.

[jerryzh168]

makes sense, we could have both I think, e.g. _get_to_kwargs can be a standalone util function

[jainapurva]

As per my understanding:
Option 1: We can move out all the util functions, and make an independent tensor-subclass (inheriting from torch.Tensor), then developer will have to define/inherit util functions.
Option 2: We can add the minimum needed util functions in TorchAOBaseTensor, so that it gives a base start to developer and build on top of it.

[jerryzh168]

depends on how much can be reused by other tensor subclasses I think, if the TorchAOBaseTensor is very small then it may not make sense to have it any more and it will not be useful for existing inference tensor subclasses. I feel it might be better to have both the current TorchAOBaseTensor and some utils that can be reused by other tensor subclasses as a starting point and then adapt based on use cases