Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compute local rank grid with lazy_grid #85

Merged
merged 46 commits into from
Jan 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
320b64e
docstrings
albert-de-montserrat Dec 28, 2023
befe7e9
mpi utils
albert-de-montserrat Dec 28, 2023
3ce4404
generalize mpi grids
albert-de-montserrat Dec 28, 2023
1342948
mpi lazy grid fix
albert-de-montserrat Dec 28, 2023
3c3b84a
grid tests
albert-de-montserrat Dec 28, 2023
3b2b3bd
update miniapps syntax
albert-de-montserrat Dec 28, 2023
26eebc9
fix 2d grid test
albert-de-montserrat Dec 28, 2023
1e38e7e
update miniapps syntax
albert-de-montserrat Dec 28, 2023
8bb2cd4
format
albert-de-montserrat Dec 28, 2023
74924e1
bugfix
albert-de-montserrat Dec 29, 2023
a28d4ac
2D diffusion MPI miniapp
albert-de-montserrat Dec 29, 2023
203a961
fixes
albert-de-montserrat Dec 29, 2023
a7c7aa9
add plotting
albert-de-montserrat Dec 29, 2023
1f44991
add 3D miniapp; update device for `PS_setup`
aelligp Jan 2, 2024
3c5d71b
MPI ShearBands
aelligp Jan 4, 2024
5ff63a5
add rank plots
aelligp Jan 5, 2024
cd35e6d
typo
aelligp Jan 5, 2024
5154548
update Stokes - `flow_bcs!()`
aelligp Jan 5, 2024
2241812
update flow_bcs everywhere
aelligp Jan 5, 2024
a6fd62a
fix rest of miniapps
aelligp Jan 5, 2024
a9cbe92
format
aelligp Jan 5, 2024
de2b017
Merge branch 'main' into adm/mpi; bump format_check
aelligp Jan 5, 2024
0449783
add Spell Check
aelligp Jan 5, 2024
3ed4565
fix typos and add typos.toml to ignore words
aelligp Jan 5, 2024
ec76d6c
test typo.toml
aelligp Jan 5, 2024
dfd6393
fix plotting of 3D benchmark
aelligp Jan 5, 2024
27bf872
update thermal halo in the right place
albert-de-montserrat Jan 5, 2024
81145b6
Merge branch 'main' into adm/mpi
aelligp Jan 6, 2024
b61c3be
3D shearheating example (not - working SH)
aelligp Jan 6, 2024
7498bac
update diff2D MPI miniapp
albert-de-montserrat Jan 8, 2024
dd3b250
fix device
albert-de-montserrat Jan 8, 2024
56beb77
remove commented line
albert-de-montserrat Jan 8, 2024
0f875fc
fix Diff3D MPI miniapp
albert-de-montserrat Jan 8, 2024
a49909f
rename some miniapp folders
albert-de-montserrat Jan 8, 2024
3f6051e
fix reference
aelligp Jan 8, 2024
4d372a3
fix MPI bug in Stokes2D
albert-de-montserrat Jan 9, 2024
749a297
update 2D mpi shear band miniapp
albert-de-montserrat Jan 9, 2024
211f04b
update ShearBands3D_MPI
albert-de-montserrat Jan 9, 2024
87fe804
fix 3D MPI stokes
albert-de-montserrat Jan 10, 2024
4f08860
update 3D MPI shear band miniapp
albert-de-montserrat Jan 10, 2024
e1f0904
minor formatting
albert-de-montserrat Jan 10, 2024
6b252dd
fix typos
albert-de-montserrat Jan 10, 2024
69861e1
suggested changes
aelligp Jan 10, 2024
05a4c91
more fixes
aelligp Jan 10, 2024
b8d6be6
add save(fig) again to SH2D
aelligp Jan 10, 2024
8a32d27
fix figure titles
aelligp Jan 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .github/workflows/SpellCheck.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
name: Spell Check

on: [pull_request, workflow_dispatch]

jobs:
typos-check:
name: Spell Check with Typos
runs-on: ubuntu-latest
steps:
- name: Checkout Actions Repository
uses: actions/checkout@v4
- name: Check spelling
uses: crate-ci/[email protected]
2 changes: 1 addition & 1 deletion .github/workflows/format_check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
strategy:
matrix:
version:
- '1.9'
- '1.10'
os:
- ubuntu-latest
arch:
Expand Down
3 changes: 3 additions & 0 deletions _typos.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[default.extend-words]
Ths = "Ths"
oly = "oly"
8 changes: 4 additions & 4 deletions miniapps/benchmarks/stokes2D/RunStokesBench2D.jl
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ using MPI: MPI
using GLMakie

# setup ParallelStencil.jl environment
model = PS_Setup(:cpu, Float64, 2)
model = PS_Setup(:cpu, Float64, 2) # :cpu | :CUDA | :AMDGPU
environment!(model)

# choose benchmark
Expand All @@ -16,13 +16,13 @@ nx, ny = 64, 64
finalize_MPI = false

# :single for a single run model with nx, ny resolution
# :multiple for grid sensitivy error plot
# :multiple for grid sensitivity error plot
runtype = :single

if benchmark == :solcx
# benchmark reference:
# Duretz, Thibault, et al. "Discretization errors and free surface stabilization
# in the finite difference and marker‐in‐cell method for applied geodynamics:
# in the finite difference and marker‐in‐cell method for applied geodynamics:
# A numerical study." Geochemistry, Geophysics, Geosystems 12.7 (2011).
# DOI: 10.1029/2011GC003567

Expand Down Expand Up @@ -51,7 +51,7 @@ if benchmark == :solcx
elseif benchmark == :solkz
# benchmark reference:
# Duretz, Thibault, et al. "Discretization errors and free surface stabilization
# in the finite difference and marker‐in‐cell method for applied geodynamics:
# in the finite difference and marker‐in‐cell method for applied geodynamics:
# A numerical study." Geochemistry, Geophysics, Geosystems 12.7 (2011).
# DOI: 10.1029/2011GC003567

Expand Down
27 changes: 15 additions & 12 deletions miniapps/benchmarks/stokes2D/VanKeken.jl/VanKeken.jl
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ backend = "CUDA_Float64_2D" # options: "CUDA_Float64_2D" "Threads_Float64_2D"
# set_backend(backend) # run this on the REPL to switch backend

# setup ParallelStencil.jl environment
@static if occursin("CUDA", JustPIC.backend)
@static if occursin("CUDA", JustPIC.backend)
model = PS_Setup(:CUDA, Float64, 2)
environment!(model)
else
aelligp marked this conversation as resolved.
Show resolved Hide resolved
model = PS_Setup(:Threads, Float64, 2)
model = PS_Setup(:cpu, Float64, 2)
environment!(model)
end

Expand Down Expand Up @@ -107,14 +107,15 @@ end
function main2D(igg; ny=16, nx=ny*8, figdir="model_figs")

# Physical domain ------------------------------------
ly = 1 # domain length in y
lx = ly # domain length in x
ni = nx, ny # number of cells
li = lx, ly # domain length in x- and y-
di = @. li / ni # grid step in x- and -y
origin = 0.0, 0.0 # origin coordinates
xci, xvi = lazy_grid(di, li, ni; origin=origin) # nodes at the center and vertices of the cells
dt = Inf
ly = 1 # domain length in y
lx = ly # domain length in x
ni = nx, ny # number of cells
li = lx, ly # domain length in x- and y-
di = @. li / ni # grid step in x- and -y
origin = 0.0, 0.0 # origin coordinates
grid = Geometry(ni, li; origin = origin)
(; xci, xvi) = grid # nodes at the center and vertices of the cells
dt = Inf

# Physical properties using GeoParams ----------------
rheology = (
Expand Down Expand Up @@ -167,10 +168,12 @@ function main2D(igg; ny=16, nx=ny*8, figdir="model_figs")
)

# Boundary conditions
flow_bcs = FlowBoundaryConditions(;
flow_bcs = FlowBoundaryConditions(;
free_slip = (left = true, right = true, top = false, bot = false),
no_slip = (left = false, right = false, top = true, bot = true),
)
)
flow_bcs!(stokes, flow_bcs)
update_halo!(stokes.V.Vx, stokes.V.Vy)

# IO ----- -------------------------------------------
# if it does not exist, make folder where figures are stored
Expand Down
16 changes: 9 additions & 7 deletions miniapps/benchmarks/stokes2D/elastic_buildup/Elastic_BuildUp.jl
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,13 @@ function elastic_buildup(;
# Here, we only explicitly store local sizes, but for some applications
# concerned with strong scaling, it might make more sense to define global sizes,
# independent of (MPI) parallelization
ni = nx, ny # number of nodes in x- and y-
li = lx, ly # domain length in x- and y-
di = @. li / ni # grid step in x- and -y
igg = IGG(init_global_grid(nx, ny, 1; init_MPI=init_MPI)...) # init MPI
origin = 0.0, 0.0
xci, xvi = lazy_grid(di, li, ni; origin=origin) # nodes at the center and vertices of the cells
ni = nx, ny # number of nodes in x- and y-
li = lx, ly # domain length in x- and y-
di = @. li / ni # grid step in x- and -y
igg = IGG(init_global_grid(nx, ny, 1; init_MPI=init_MPI)...) # init MPI
origin = 0.0, 0.0
grid = Geometry(ni, li; origin = origin)
(; xci, xvi) = grid # nodes at the center and vertices of the cells

## (Physical) Time domain and discretization
yr = 365.25 * 3600 * 24
Expand All @@ -59,12 +60,13 @@ function elastic_buildup(;
free_slip = (left = true, right = true, top = true, bot = true)
)
flow_bcs!(stokes, flow_bcs)
update_halo!(stokes.V.Vx, stokes.V.Vy)

# Physical time loop
t = 0.0
it = 0
ρg = @zeros(ni...), @ones(size(stokes.P)) .* g
av_τyy = Float64[]
av_τyy = Float64[]
sol_τyy = Float64[]
tt = Float64[]
local iters
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,22 +29,23 @@ end
function main(igg; nx=64, ny=64, figdir="model_figs")

# Physical domain ------------------------------------
ly = 1e0 # domain length in y
lx = ly # domain length in x
ni = nx, ny # number of cells
li = lx, ly # domain length in x- and y-
di = @. li / ni # grid step in x- and -y
origin = 0.0, 0.0 # origin coordinates
xci, xvi = lazy_grid(di, li, ni; origin=origin) # nodes at the center and vertices of the cells
dt = Inf
ly = 1e0 # domain length in y
lx = ly # domain length in x
ni = nx, ny # number of cells
li = lx, ly # domain length in x- and y-
di = @. li / ni # grid step in x- and -y
origin = 0.0, 0.0 # origin coordinates
grid = Geometry(ni, li; origin = origin)
(; xci, xvi) = grid # nodes at the center and vertices of the cells
dt = Inf

# Physical properties using GeoParams ----------------
η0 = 1e22 # viscosity
G0 = 10^10 # elastic shear modulus
εbg = 1e-14 # background strain-rate
dt = 1e11
el_bg = SetConstantElasticity(; G=G0, ν=0.5)
visc = LinearViscous(; η=η0)
η0 = 1e22 # viscosity
G0 = 10^10 # elastic shear modulus
εbg = 1e-14 # background strain-rate
dt = 1e11
el_bg = SetConstantElasticity(; G=G0, ν=0.5)
visc = LinearViscous(; η=η0)
rheology = (
# Low density phase
SetMaterialParams(;
Expand All @@ -57,22 +58,22 @@ function main(igg; nx=64, ny=64, figdir="model_figs")
)

# Initialize phase ratios -------------------------------
radius = 0.1
radius = 0.1
phase_ratios = PhaseRatio(ni, length(rheology))
init_phases!(phase_ratios, xci, radius)

# STOKES ---------------------------------------------
# Allocate arrays needed for every Stokes problem
stokes = StokesArrays(ni, ViscoElastic)
stokes = StokesArrays(ni, ViscoElastic)
pt_stokes = PTStokesCoeffs(li, di; ϵ=1e-6, CFL=0.75 / √2.1)

# Buoyancy forces
ρg = @zeros(ni...), @zeros(ni...)
ρg = @zeros(ni...), @zeros(ni...)
ρg[2] .= rheology[1].Density[1].ρ.val .* rheology[1].Gravity[1].g.val
args = (; T=@zeros(ni...), P=stokes.P, dt=dt)
args = (; T=@zeros(ni...), P=stokes.P, dt=dt)

# Rheology
η = @ones(ni...)
η = @ones(ni...)
η_vep = similar(η) # effective visco-elasto-plastic viscosity
@parallel (@idx ni) compute_viscosity!(
η,
Expand All @@ -94,18 +95,19 @@ function main(igg; nx=64, ny=64, figdir="model_figs")
stokes.V.Vx .= PTArray([x * εbg for x in xvi[1], _ in 1:(ny + 2)])
stokes.V.Vy .= PTArray([-y * εbg for _ in 1:(nx + 2), y in xvi[2]])
flow_bcs!(stokes, flow_bcs) # apply boundary conditions
update_halo!(stokes.V.Vx, stokes.V.Vy)
# IO ------------------------------------------------
# if it does not exist, make folder where figures are stored
!isdir(figdir) && mkpath(figdir)
# ----------------------------------------------------

# Time loop
t, it = 0.0, 0
tmax = 1.0e13
τII = Float64[0.0]
sol = Float64[0.0]
ttot = Float64[0.0]
P = Float64[0.0]
tmax = 1.0e13
τII = Float64[0.0]
sol = Float64[0.0]
ttot = Float64[0.0]
P = Float64[0.0]

while t < tmax

Expand Down Expand Up @@ -163,8 +165,8 @@ function main(igg; nx=64, ny=64, figdir="model_figs")
return nothing
end

N = 32
n = N + 2
N = 32
n = N + 2
nx = n - 2
ny = n - 2
figdir = "ElasticBuildUp_incompressible"
Expand Down
18 changes: 10 additions & 8 deletions miniapps/benchmarks/stokes2D/shear_band/ShearBand2D.jl
Original file line number Diff line number Diff line change
Expand Up @@ -34,14 +34,15 @@ end
function main(igg; nx=64, ny=64, figdir="model_figs")

# Physical domain ------------------------------------
ly = 1e0 # domain length in y
lx = ly # domain length in x
ni = nx, ny # number of cells
li = lx, ly # domain length in x- and y-
di = @. li / ni # grid step in x- and -y
origin = 0.0, 0.0 # origin coordinates
xci, xvi = lazy_grid(di, li, ni; origin=origin) # nodes at the center and vertices of the cells
dt = Inf
ly = 1e0 # domain length in y
lx = ly # domain length in x
ni = nx, ny # number of cells
li = lx, ly # domain length in x- and y-
di = @. li / ni # grid step in x- and -y
origin = 0.0, 0.0 # origin coordinates
grid = Geometry(ni, li; origin = origin)
(; xci, xvi) = grid # nodes at the center and vertices of the cells
dt = Inf

# Physical properties using GeoParams ----------------
τ_y = 1.6 # yield stress. If do_DP=true, τ_y stand for the cohesion: c*cos(ϕ)
Expand Down Expand Up @@ -110,6 +111,7 @@ function main(igg; nx=64, ny=64, figdir="model_figs")
stokes.V.Vx .= PTArray([ x*εbg for x in xvi[1], _ in 1:ny+2])
stokes.V.Vy .= PTArray([-y*εbg for _ in 1:nx+2, y in xvi[2]])
flow_bcs!(stokes, flow_bcs) # apply boundary conditions
update_halo!(stokes.V.Vx, stokes.V.Vy)

# IO ------------------------------------------------
# if it does not exist, make folder where figures are stored
Expand Down
Loading
Loading