#8962: modify output_mem_config to be optional in binary ops

[KalaivaniMCW]

modify output_mem_config to be std::optional in binary ops

Why this change ?

• Eltwise Binary ops now have output_tensor support so when output_tensor is passed output_mem_config is to be ignored

CI - https://github.com/tenstorrent/tt-metal/actions/runs/9450252404

[tt-aho]

We probably should have gotten rid of in_place before this mem_config change since this simplifies further changes

[KalaivaniMCW]

okay @tt-aho
I hope everyone is okay with removing in_place flag from binary ops and replacing tt_lib.operations.primary.add / ttnn.experimental.operations.primary.add ?
I'll remove the inline Tensor add and use binary add() with output_tensor i.e tt_lib.tensor.add / ttnn.experimental.tensor.add . This will involve changes in below files, shall I go ahead with it (should someone be informed) ?

1. models/demos/t3000/falcon40b/tt/falcon_decoder.py
2. models/demos/resnet/tt/metalResnetBlock50.py
3. models/demos/t3000/llama2_70b/tt/llama_decoder_galaxy.py
4. models/demos/t3000/llama2_70b/tt/llama_decoder_optimized.py
5. models/experimental/llama2_70b/tt/llama_decoder_galaxy.py
6. models/experimental/llama2_70b/tt/llama_decoder_optimized.py
7. models/demos/falcon7b/tests/unit_tests/test_falcon_matmuls_and_bmms_with_mixed_precision.py

[tt-aho]

Don't think we should be doing .value() here.

Is the plan to change the default value for output_mem_config to nullopt instead of operation::DEFAULT_OUTPUT_MEMORY_CONFIG?

I think here we should either:

• pass output_mem_config as is or
• pass output_mem_config.value_or(whatever we decide the default should be)

@eyonland do you have thoughts on this? Should we determine the default memconfig to use when none is provided here, or within the op struct fns?

[KalaivaniMCW]

Yes, the default value of output_mem_config will be nullopt. I was thinking of assigning it operation::DEFAULT_OUTPUT_MEMORY_CONFIG if both output_tensor and output_mem_config is nullopt.

I have an earlier comment from @eyonland regarding this, are you ok with this too ?

"If the optional output tensor is provided, then it should be used instead of the dtype or output memconfig which should not be required but made optional going forward.
The flow should be:
Does an optional output tensor actually exist that is NOT a promise? If so, ignore whatever the user provided for dtype and memconfig.
If dtype and memconfig are provided use this, if not use the dtype and memconfig of the first input tensor as the default(s)"

so I'm thinking of adding an if block inside the run_eltwise_binary and change output_mem_config as per above conditions and then pass it to the EltwiseBinary struct

[KalaivaniMCW]

Issue closed #8962