[Optimizer] Fix ttnn.ToLayout mem config optimizer override.

[nobradovictt]

Temp fix for discrepancy between MemoryConfigAttr and layout attribute of output tensor of generic version of ttnn.ToLayoutOp.

Remove op input overrides not needed anymore from Mnist sharding POC.

[jnie-TT]

Just out of curiosity what's the current discrepancy of layout and memconfig in ttnn.ToLayoutOp prior to decomposition?

[nobradovictt]

Just out of curiosity what's the current discrepancy of layout and memconfig in ttnn.ToLayoutOp prior to decomposition?

In Optimizer we use op output tensor layout to specify memory related attributes of the op. Now after moving to TTNN dialect it seems we need to set same in MemoryConfigAttr as well so situations like this don't happen:

#layout5 = #tt.layout<(d0, d1) -> (d0, d1), undef, <1x1>, memref<1x1x!tt.tile<32x32, f32>, #l1_>, width_sharded>
%2 = "ttnn.to_device"(%1, %0) <{memory_config = #ttnn.memory_config<<interleaved>, <dram>, <<1x1>>>}> : (tensor<32x32xf32, #layout5>, !tt.device<#device>) -> tensor<32x32xf32, #layout5>

Notice diff between memory config and output layout. It will compile without errors and just ignore layout5 attr in runtime.

[jnie-TT]

Just out of curiosity what's the current discrepancy of layout and memconfig in ttnn.ToLayoutOp prior to decomposition?

In Optimizer we use op output tensor layout to specify memory related attributes of the op. Now after moving to TTNN dialect it seems we need to set same in MemoryConfigAttr as well so situations like this don't happen:

#layout5 = #tt.layout<(d0, d1) -> (d0, d1), undef, <1x1>, memref<1x1x!tt.tile<32x32, f32>, #l1_>, width_sharded>
%2 = "ttnn.to_device"(%1, %0) <{memory_config = #ttnn.memory_config<<interleaved>, <dram>, <<1x1>>>}> : (tensor<32x32xf32, #layout5>, !tt.device<#device>) -> tensor<32x32xf32, #layout5>

Notice diff between memory config and output layout. It will compile without errors and just ignore layout5 attr in runtime.

I see that makes sense. Yeah in TTNN backend I think there was a discussion that while we want to maintain all the information in the tensor, we also want to pass relevant parameters into the op as well to make it more explicit, so there will be some work needed to maintain consistency there. This also popped up when I was looking at the forcing row_major/tiled issue, where the layout in the op parameter is tiled but it was not reflected in the tensor (the tensor layout was still row major). Thanks for the explanation!