Converting TensorMaps to arrays to evaluate elements

[ErikLW]

Hey,
Is there a way to access the specific elements in a TensorMap (Without violating the AD compatability) ? E.g. findung the maximum element in a TensorMap or simply taking an element with []?
Best regards,
Erik

[maartenvd]

One possible problem with this is that tensors are generically not indexable. I can probably define the rule to convert them to and from dictionaries which can in turn be indexed, but do you have a particular use-case in mind?

[Jutho]

@ErikLW , am I correct in assuming that the elements that you want to access are diagonal elements of say a tensor with singular values or eigenvalues? At least, those are the only ones for which I think the elements are meaningful and not dependent on details such as the fusion trees etc (in the case of non-abelian symmetries).

I have been planning to implement a DiagonalTensorMap{S} <: AbstracTensorMap{S,1,1} that offers more possibilities to access its elements, and iterate over them, sort them, truncate them, ...

There can also be other cases to access specific elements, such as when initialing tensors to create specific operators. There should also be more functionality for this, but that's work in progress. I think MPSKit has some additional tools for that; is that correct @maartenvd ?

[ErikLW]

Hey,
For my usual code accessing specific elements is usually not nessessary exept for diagonal elements. However I am trying to test a method to avoid taking the inverse square root during the creation of the projectors in the CTMRG routine. Taking this inverse square root of a small singular value spectrum otherwise oftentimes leads to problems with AD code. This requires me to take element wise operations (exp. , log. ,maximum etc.) of matrix elements.
So the ability to convert to and from dictionaries would be really useful for this !

[Jutho]

Simply d=convert(Dict, t) and t = convert(TensorMap, d) will do what you need. The dict d has three fields :domain, :codomain and :data. d[:data] is another Dict that contains the blocks per sector, which are matrices. In the case of a TensorMap{S,1,1}, these blocks correspond exactly to the diagonal blocks of the tensor.

[ErikLW]

Simply d=convert(Dict, t) and t = convert(TensorMap, d) will do what you need. The dict d has three fields :domain, :codomain and :data. d[:data] is another Dict that contains the blocks per sector, which are matrices. In the case of a TensorMap{S,1,1}, these blocks correspond exactly to the diagonal blocks of the tensor.

I am not sure I understand you correctly. Do you mean to say that using convert(Dict, t) etc. I can access the elements of a TensorMap without violating the AD compatability? Because this does not seem to work for me.

E.g.:

A= TensorMap(randn, ComplexF64, ℂ^5 ← ℂ^2)
B = TensorMap(randn, ComplexF64,  ℂ^2 ← ℂ^5)

function test(A,B)
    
    @tensor C[(i,);(k,)] := B[i,j]*A[j,k]
    
    d = convert(Dict, C)
    test = maximum(real(d[:data]["Trivial()"]))
    
return test
end

gradient(A -> scalartest(A,B), A)

[ErikLW]

One possible problem with this is that tensors are generically not indexable. I can probably define the rule to convert them to and from dictionaries which can in turn be indexed, but do you have a particular use-case in mind?

That would be very useful. In case you decide to do it, let me know :)

[maartenvd]

can you try the latest master? It's quick'n dirty but your example code works

[ErikLW]

can you try the latest master? It's quick'n dirty but your example code works

Great! It works for me now.
However as a last caveat, the back-conversion from Dict to TensorMap is not AD compatable yet right?
Because the sample

A = TensorMap(randn, ComplexF64, ℂ^5 ← ℂ^2)
B = TensorMap(randn, ComplexF64,  ℂ^2 ← ℂ^5)

function test1(A,B)
    
    @tensor C[(i,);(k,)] := B[i,j]*A[j,k]
    
    d = convert(Dict, C)
    C1 = convert(TensorMap,d)
    test = norm(C1)
return test
end

gradient(x -> test1(x,B), A)

does not yet work.

[maartenvd]

good catch, there was a typo! (master should work)

[ErikLW]

good catch, there was a typo! (master should work)

Does the example work for you? For me, it sadly still fails

[maartenvd]

are you sure you have the latest master (the one since the typo commit)?

julia> using Zygote,TensorKitAD,TensorKit

julia> A = TensorMap(randn, ComplexF64, ℂ^5 ← ℂ^2)
TensorMap(ProductSpace(ℂ^5) ← ProductSpace(ℂ^2)):
  0.35301722003196173 + 1.3313942244558592im   -0.23583593997637312 + 0.9283102039974742im
 0.028944288006906713 + 0.06185915297781779im   -0.7152113209825572 - 0.6153898897841442im
 0.001354105986239224 + 0.16982397095739718im     1.530445443067156 + 0.03682138649046981im
 -0.03154775651286625 - 0.5304242429706616im   -0.11345186968891859 - 0.08395226868367779im
   0.9134419243718206 - 0.944315393937413im      0.8787710188525171 + 0.4570462352566594im


julia> B = TensorMap(randn, ComplexF64,  ℂ^2 ← ℂ^5)
TensorMap(ProductSpace(ℂ^2) ← ProductSpace(ℂ^5)):
  0.0883086971978418 + 0.7316810967227223im  0.5305760475466069 + 0.18530039233248144im  …  0.9040913780913438 + 1.1759071997833277im    1.3527336931556821 - 0.2907022451477966im
 -0.8488113736257397 + 0.9821307245069488im  0.6451068551494642 + 0.27252126720661646im     0.3616753595327775 - 0.14156121872771496im  -0.7493102236187205 - 0.48794223057502295im


julia> function test1(A,B)
           
           @tensor C[(i,);(k,)] := B[i,j]*A[j,k]
           
           d = convert(Dict, C)
           C1 = convert(TensorMap,d)
           test = norm(C1)
       return test
       end
test1 (generic function with 1 method)

julia> gradient(x -> test1(x,B), A)
(TensorMap(ProductSpace(ℂ^5) ← ProductSpace(ℂ^2)):
 0.09391535413926677 + 0.5915739744293476im   -0.12657783273117224 + 0.972697644031776im
 -0.4279228924977377 - 0.1562096748214261im     -0.521179569412039 - 0.1797466309690924im
 0.13518147355170376 + 0.36335213690435864im   0.29080470348052306 + 0.3151243369055586im
   -0.51406675931298 - 0.6187374489552967im   -0.29551391468605986 - 0.09895487549594564im
  0.7953438869848626 - 0.6362569305307908im     0.3849778995663963 + 0.02506654855451612im
,)

julia> 

[ErikLW]

Works now, thanks :)