volume not balance using zstar+cavity

[PengyangSong]

Dear developers (@patrickscholz ),

I found that simulating with cavity has a volume drift (by checking global ssh integration). The drift is ~0.5 cm/yr.
This happens for both ocean-only and coupled model, see Levante:

/home/a/a270146/post_fesom2_ice/OCE_run/ipynb_code/plot_timeseries/ssh_global_timeseries.ipynb
/home/a/a270146/post_fesom2_ice/ESM_run/ipynb_code/plot_timeseries/ssh_global_timeseries.ipynb

In both refactoring and refactoring_bugfix_cavity, you can add below lines after Line 1919 of oce_ale.F90, where the ssh_rhs_old is freshly calculated and not decorated by water_flux . And you will find that the integration of ssh_rhs_old over open ocean is not zero in a zstar+cavity setting, which further delivers to hbar and finally arrives at the ssh integration drift.

    rhs_sum0 = 0.0
    rhs_sum1 = 0.0
    rhs_sum2 = 0.0
    rhs_sum_ocecav = 0.0
    rhs_sum_oce = 0.0
    rhs_sum_cav = 0.0

    do n=1, myDim_nod2D
      ! ocean + cavity
      rhs_sum0 = rhs_sum0 + ssh_rhs_old(n)

      ! rhs_sum1: open ocean; rhs_sum2: cavity
      if (ulevels_nod2D(n) > 1) then
        rhs_sum1 = rhs_sum1 + 0.0
        rhs_sum2 = rhs_sum2 + ssh_rhs_old(n)
      else
        rhs_sum1 = rhs_sum1 + ssh_rhs_old(n)
        rhs_sum2 = rhs_sum2 + 0.0
      end if
    end do

    call MPI_AllREDUCE(rhs_sum0, rhs_sum_ocecav, 1, MPI_DOUBLE_PRECISION, MPI_SUM, &
        MPI_COMM_FESOM, MPIerr)
    call MPI_AllREDUCE(rhs_sum1, rhs_sum_oce, 1, MPI_DOUBLE_PRECISION, MPI_SUM, &
        MPI_COMM_FESOM, MPIerr)
    call MPI_AllREDUCE(rhs_sum2, rhs_sum_cav, 1, MPI_DOUBLE_PRECISION, MPI_SUM, &
        MPI_COMM_FESOM, MPIerr)

    if (mype==0)  write(*,*), 'ssh_rhs_old integration ocean + cavity: ', rhs_sum_ocecav
    if (mype==0)  write(*,*), 'ssh_rhs_old integration ocean: ',          rhs_sum_oce
    if (mype==0)  write(*,*), 'ssh_rhs_old integration cavity: ',         rhs_sum_cav

With CORE2 mesh, the runlog /home/a/a270146/work/FESOM2_ICE/ICE_testssh/work/fesom2.1.out.3509038 says at the first step:

  0:  ssh_rhs_old integration ocean + cavity:   2.384185791015625E-007
  0:  ssh_rhs_old integration ocean:   2.384185791015625E-007
  0:  ssh_rhs_old integration cavity:   0.000000000000000E+000

With cavity mesh, the runlog /home/a/a270146/work/FESOM2_ICE/ICE_testssh/work/fesom2.1.out.3509067 says at the first step:

  0:  ssh_rhs_old integration ocean + cavity:  -1.490116119384766E-007
  0:  ssh_rhs_old integration ocean:    5760.43414956331
  0:  ssh_rhs_old integration cavity:   -5760.43414971719

Thus, to use zstar in open ocean + linfs in cavity, we need to balance the net volume transport between open ocean and cavity.