Inconsistency of TTNN ops interface

[azecevicTT]

This issue is the remainder of some inconsistencies spotted in interfaces of TTNN ops. Some of those things can only be fixed on the Metal side, some can be mitigated on our side until they are (if ever) fixed. Fill free to add other things that you have spotted in comments.

• bit width (32 vs 64): This looks really random when you see it mixed from op to op. I believe there isn't any good reason for this.
• signed vs unsigned: This is particularly important with arrays. MLIR doesn't have a good way to represent array of unsigned integers. One option is using DenseIXArrayAttr when TTNN uses std::vector<unsigned X> and then verify that all of the elements are non-negative. As we discussed offline, probably the best course of action would be some kind of attribute constraint that would automatically do it for us. This solution comes with a downside that we lose half of the range (1 bit), but we don't expect it to have an impact in practice.
• vector vs SmallVector vs span: There should be a single way to represent a (dynamic) array of elements. The span is an ideal option as it represents an interface rather than a class, but even std::vector or SmallVector are okay if they are used consistently.
• Some ops have memory config while others don't.
• Ops that belong to the same broader class should all have a similar interface. @mtopalovicTT had some examples where this was violated.

FYI @sdjordjevicTT @svuckovicTT @mtopalovicTT @jserbedzijaTT

[mtopalovicTT]

Can we subtask this. We can start checking if I64 is possible and change it throughout the compiler.

vector vs SmallVector vs span: There should be a single way to represent a (dynamic) array of elements. The span is an ideal option as it represents an interface rather than a class, but even std::vector or SmallVector are okay if they are used consistently.

This is more feedback for metal right?

[svuckovicTT]

Can we subtask this. We can start checking if I64 is possible and change it throughout the compiler.

vector vs SmallVector vs span: There should be a single way to represent a (dynamic) array of elements. The span is an ideal option as it represents an interface rather than a class, but even std::vector or SmallVector are okay if they are used consistently.

This is more feedback for metal right?

This is all feedback for metal.

[azecevicTT]

Can we subtask this. We can start checking if I64 is possible and change it throughout the compiler.

vector vs SmallVector vs span: There should be a single way to represent a (dynamic) array of elements. The span is an ideal option as it represents an interface rather than a class, but even std::vector or SmallVector are okay if they are used consistently.

This is more feedback for metal right?

This is all feedback for metal.

Yeah, more or less, it doesn't make much sense to work on these things if they are going to stay in Metal as-is. That point is the particular pain point that repeats a lot during an uplift.

Btw, if anybody remembers anything else that's important you can edit the opening post so that we have these points in one place.