Guide to parameters

The parameters in the params.txt file are mostly self explanatory and extensively commented, but this page aims to provide a brief guide to put the comments in context.

Note that a hash # denotes a comment and the rest of the line will not be read, so to activate a parameter this should be removed at the start of the line. Parameters that are unlikely to change are set to defaults, and the corresponding values are commented out in the params.txt file so they are easy to identify. If you are not sure, it is probably best to leave it on the default setting. In cases that are not commented, you will usually need to think about the value you want to use (or copy values from a similar example).

Filesystem parameters

• verbosity can be set to 0, 1, 2,... . Generally, the higher it is set, the more information is output to the pout file from each MPI rank (or just stdout if running in serial). GRTresna currently does not do anything different above 2 but the underlying Chombo library may output more up to 5.
• Each checkpoint filename prefix will be appended with a 6 digit number corresponding to the time step on the coarsest level, the number of spatial dimensions (which should be 3 for these examples) and the hdf5 file extension. For example the checkpoint file for the 10th iteration will be called SourceData_chk000010.3d.hdf5.
• If a directory is not explicitly specified, then files will be output to the directory in which the simulation has been executed. The can be overwritten but providing a base directory in output_path, which needs to exist at runtime. Generally, when running on a cluster, you'll want to change it to some local scratch storage or a space with fast I/O. Refer to your cluster documentation/administrator for help with this.
• Note that Unix style relative paths, for example ../filename-prefix, will probably not work and you should use absolute paths.
• During the simulation each MPI process outputs data on what it is doing in 'pout/pout.*' files by default.
• The error_filename file is used to output information about the convergence of the solver over non linear iterations.
• input_filename is the filename for the hdf5 data from which grids and the matter functions can be (optionally) read. If commented out analytic data is used instead.
• output_path = Outputs/ is the directory where the solver hdf5 files will be written
• output_filename = InitialDataFinal.3d.hdf5 is the filename for the final solution, which can be used as the restart checkpoint file in the GRChombo evolution code.
• One can control the frequency of the diagnostic outputs with respect to the non linear iteration steps by changing the diagnostic_interval, which is defaulted to 10. We recommend only writing these files during the testing stage, as they tend to slow down the performance of the code, and are memory intensive.

Grid parameters

• The three-valued parameter N gives the number of cells on the coarsest level along each dimension. Note that this needs to be a multiple of the block_factor (minimum box size). For general 3-dimensional problems, you'll probably want to set them to the same value but for problems that are essentially 2 or 1-dimensional dynamically, you can set different values appropriately.
• L is the coordinate length of the longest side of the grid. The spatial step size dx on the coarsest level is L/Nmax where Nmax is the maximum of N. Note that dx is the same along each dimension, even if the N values differ.
• max_level is the maximum number of refinement levels above the coarsest level, so there are max_level + 1 levels in total.
• refine_threshold is the overall threshold which is used to decide whether or not to tag the cell. It is defaulted to 0.5 - making it lower should refine more of the domain.
• regrid_radius can be set such that fixed-grid refinement always occurs at the chosen distance from the origin
• max_grid_size is the maximum number of cells along each dimension of a box (the boxes are the subsets of the grid which are shared between processors). This should definitely be greater than or equal to 8 and probably around 16 or 32 but no larger.
• block_factor is the minimum number of cells along each dimension of a box. This should ideally be an integer multiple of the vector width for doubles e.g. 8 for AVX-512 (assuming you've compiled with the appropriate flags) and probably shouldn't be smaller as then the high number of ghost cells can cause memory issues. It must be a power of 2 (for a ref_ratio of 2).
• You may want to set max_grid_size and block_factor to the same value (e.g. 16) so that you know exactly how large each box is going to be. This will make it easier to optimise your load balancing.
• buffer_size is the number of additional cells that are added to tagged regions to smooth them out, and defaults to 0.
• fill_ratio is the minimum proportion of cells in a box that need to be tagged for regridding to occur and defaults to 0.75.

Boundary Conditions parameters

• is_periodic is a vector of boolean values corresponding to whether the grid boundaries along each dimension are periodic or not. If one wants to have periodic boundary conditions it should be set to 1 1 1. However, CTTK also has extrapolating boundary and reflective conditions in which case is_periodic should be be 0 0 0 and the precise condition must be chosen by using the additional parameters below.
• The chosen ansatz for the decomposition of $W_i$ into $V_i$ and $U$ is chosen via use_compact_Vi_ansatz. See Appendix A.2 in the CTTK paper for more details. In general the compact ansatz is needed for asymptotically flat spacetimes, but this parameter should be set to 0 for periodic ones.
• hi_boundary and lo_boundary are two 3d vectors that each component carries information about the boundary conditions of the coordinate at the high and low part of the simulation box. They should be set to 0 for extrapolating and 1 for reflective. For example, if one wanted to have reflective boundary condition for the lower boundaries and extrapolating for the upper, the vectors should be set to lo_boundary = 1 1 1 and hi_boundary = 0 0 0.
• extrapolation_order sets the order of the extrapolation used to fill ghost cells of the relevant variables (e.g. K and Aij). Can be set to 0 or 1, with the default being the latter (i.e. linear extrapolation)

Initial data parameters

• G_Newton is the Gravitational Constant, $G$, in geometric units. The default is 1.0.
• sign_of_K sets whether the mean curvature K should be taken as positive or negative. Positive corresponds to locally converging normal observers (collapsing spatial volume) whereas negative is locally diverging normal observers (expanding spatial volume). In general the former is used for BHs and the latter for expanding cosmological spacetimes, like in inflation.
• regularised_part_psi sets the initial homogeneous value for the regularised part of the conformal factor (ignoring any BHs etc). The default is 1.0. In cosmology this is related to the scale factor as $a = \psi^2$.

The other parameters are specific to the examples, and set the matter profiles and mass etc of the scalar potential. They should be mostly self explanatory if one refers to their use in MatterFunctions.hpp.

Similarly the Bowen York data for the BHs should be self explanatory. Note that all parameters need to be set to zero to remove them from the example (ie, also the spins and boosts, not just the masses).

Solver parameters

• max_NL_iterations sets the maximum number of non linear loop iterations.
• NL_iter_tolerance sets the tolerance of the solver at which it will exit the non linear iterations loop and write the solution in an hdf5 file. It sets a bound on the percentage error of the constraints, relative to their initial values.
• There is an option to deactivate_zero_mode for periodic data, as explained in the CTTK paper.

Some other parameters related to the linear multigrid solver are defaulted as we expect the user will not need to change them in standard scenarios. Please refer to Chombo's documentation for further details about these settings.

• iter_tolerance sets the tolerance for the linear solver steps

• max_iter sets the maximum number of iterations of the linear solver

• numMGIter sets the number of MG v-cycles in the preconditioner

• numMGsmooth sets the number of GSRB smooths before and after MG coarsening

• preCondSolverDepth is defaulted to -1

• coefficient_average_type specifies how to average face-centered coefficients to coarser multigrid levels. It can be arithmetic or harmonic. It is currently defaulted to harmonic.

• The spatially-varying aCoef and bCoef are set in the method classes, and they expect alpha = 1 and beta = -1 (for pure laplacian, alpha = 0, beta=-1) alpha = 1.0 beta = -1.0

It may also be useful to refer to the GRChombo wiki, which describes the params.txt in the current BinaryBH example.