Skip to content

Commit

Permalink
Add docs and scope code in if envs
Browse files Browse the repository at this point in the history
- added first documentation on time averaged diags
- put more operations on summation multifabs into if-environments
  • Loading branch information
n01r committed Sep 18, 2024
1 parent 3ff90c9 commit d6c3bdf
Show file tree
Hide file tree
Showing 3 changed files with 90 additions and 39 deletions.
44 changes: 43 additions & 1 deletion Docs/source/usage/parameters.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2618,8 +2618,9 @@ Diagnostics and output
In-situ visualization
^^^^^^^^^^^^^^^^^^^^^

WarpX has four types of diagnostics:
WarpX has five types of diagnostics:
``FullDiagnostics`` consist in dumps of fields and particles at given iterations,
``TimeAveragedDiagnostics`` only allow field data which they output after averaging over a period of time,
``BackTransformedDiagnostics`` are used when running a simulation in a boosted frame, to reconstruct output data to the lab frame,
``BoundaryScrapingDiagnostics`` are used to collect the particles that are absorbed at the boundary, throughout the simulation, and
``ReducedDiags`` allow the user to compute some reduced quantity (particle temperature, max of a field) and write a small amount of data to text files.
Expand Down Expand Up @@ -2867,6 +2868,47 @@ In-situ capabilities can be used by turning on Sensei or Ascent (provided they a
* ``warpx.mffile_nstreams`` (`int`) optional (default `4`)
Limit the number of concurrent readers per file.


.. _running-cpp-parameters-diagnostics-timeavg:

Time-Averaged Diagnostics
^^^^^^^^^^^^^^^^^^^^^^^^^

``TimeAveraged`` diagnostics are a special type of ``FullDiagnostics`` that allows for the output of time-averaged field data.
This type of diagnostics can be created using ``<diag_name>.diag_type = TimeAveraged``.
We support only field data and related options from the list at `Full Diagnostics`_.

In addition, `TimeAveraged` diagnostic options include:

* ``<diag_name>.time_average_mode`` (`string`, default `none`)
Describes the operating mode for time averaged field output.

* ``none`` for no averaging (instantaneous fields)

* ``fixed_start`` for a diagnostic that averages all fields between the current output step and a fixed point in time

* ``dynamic_start`` for a constant averaging period and output at different points in time (non-overlapping)

.. note::

To enable time-averaged field output with intervals tightly spaced enough for overlapping averaging periods,
please create additional instances of ``TimeAveraged`` diagnostics.

* ``<diag_name>.average_period_steps`` (`int`)
Configures the number of time steps in an averaging period.
Set this only in the ``dynamic_start`` mode and only if ``average_period_time`` has not already been set.
Will be ignored in the ``fixed_start`` mode (with warning).

* ``<diag_name>.average_period_time`` (`float`, in seconds)
Configures the time (SI units) in an averaging period.
Set this only in the ``dynamic_start`` mode and only if ``average_period_steps`` has not already been set.
Will be ignored in the ``fixed_start`` mode (with warning).

* ``<diag_name>.average_start_step`` (`int`)
Configures the time step at which time-averaging begins.
Set this only in the ``fixed_start`` mode.
Will be ignored in the ``dynamic_start`` mode (with warning).

.. _running-cpp-parameters-diagnostics-btd:

BackTransformed Diagnostics
Expand Down
2 changes: 1 addition & 1 deletion Source/Diagnostics/FullDiagnostics.H
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ private:
amrex::Real m_average_period_time = 0;
/** Time step to start averaging */
int m_average_start_step = 0;
/** Flush m_mf_output and particles to file for the i^th buffer */
/** Flush m_mf_output or m_sum_mf_output and particles to file for the i^th buffer */
void Flush (int i_buffer, bool /* force_flush */) override;
/** Flush raw data */
void FlushRaw ();
Expand Down
83 changes: 46 additions & 37 deletions Source/Diagnostics/FullDiagnostics.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -192,24 +192,26 @@ FullDiagnostics::Flush ( int i_buffer, bool /* force_flush */ )
// is supported for BackTransformed Diagnostics, in BTDiagnostics class.
auto & warpx = WarpX::GetInstance();

m_flush_format->WriteToFile(
m_varnames, m_mf_output.at(i_buffer), m_geom_output.at(i_buffer), warpx.getistep(),
warpx.gett_new(0),
m_output_species.at(i_buffer), nlev_output, m_file_prefix,
m_file_min_digits, m_plot_raw_fields, m_plot_raw_fields_guards);
if (m_diag_type == DiagTypes::TimeAveraged) {
if (m_time_average_type == TimeAverageType::Static || m_time_average_type == TimeAverageType::Dynamic) {
// Loop over the output levels and divide by the number of steps in the averaging period
for (int lev = 0; lev < nlev_output; ++lev) {
m_sum_mf_output.at(i_buffer).at(lev).mult(1./m_average_period_steps);
}

if (m_time_average_type == TimeAverageType::Static || m_time_average_type == TimeAverageType::Dynamic) {
// Loop over the output levels and divide by the number of steps in the averaging period
for (int lev = 0; lev < nlev_output; ++lev) {
m_sum_mf_output.at(i_buffer).at(lev).mult(1./m_average_period_steps);
}
m_flush_format->WriteToFile(
m_varnames, m_sum_mf_output.at(i_buffer), m_geom_output.at(i_buffer), warpx.getistep(),
warpx.gett_new(0),
m_output_species.at(i_buffer), nlev_output, m_file_prefix,
m_file_min_digits, m_plot_raw_fields, m_plot_raw_fields_guards);

m_flush_format->WriteToFile(
m_varnames, m_sum_mf_output.at(i_buffer), m_geom_output.at(i_buffer), warpx.getistep(),
}
} else {
m_flush_format->WriteToFile(
m_varnames, m_mf_output.at(i_buffer), m_geom_output.at(i_buffer), warpx.getistep(),
warpx.gett_new(0),
m_output_species.at(i_buffer), nlev_output, m_file_prefix,
m_file_min_digits, m_plot_raw_fields, m_plot_raw_fields_guards);

}

FlushRaw();
Expand All @@ -233,30 +235,33 @@ FullDiagnostics::DoDump (int step, int /*i_buffer*/, bool force_flush)
bool
FullDiagnostics::DoComputeAndPack (int step, bool force_flush)
{
if (m_time_average_type == TimeAverageType::Dynamic) {
m_average_start_step = m_intervals.nextContains(step) - m_average_period_steps;
// check that the periods do not overlap and that the start step is not negative
if (m_average_start_step > 0) {
if (m_average_start_step < m_intervals.previousContains(step)) {
WARPX_ABORT_WITH_MESSAGE(
"Averaging periods may not overlap within a single diagnostic. "
"Please create a second diagnostic for overlapping time averaging periods "
"and account for the increased memory consumption."
);
} else {
WARPX_ABORT_WITH_MESSAGE("The step to begin time averaging may not be a negative number.");
}
}
}

// Start averaging at output step (from diag.intervals) - period + 1
bool in_averaging_period = false;
if (step > m_intervals.nextContains(step) - m_average_start_step && step <= m_intervals.nextContains(step)) {
in_averaging_period = true;
if (m_diag_type == DiagTypes::TimeAveraged) {
if (m_time_average_type == TimeAverageType::Dynamic) {
m_average_start_step = m_intervals.nextContains(step) - m_average_period_steps;
// check that the periods do not overlap and that the start step is not negative
if (m_average_start_step > 0) {
if (m_average_start_step < m_intervals.previousContains(step)) {
WARPX_ABORT_WITH_MESSAGE(
"Averaging periods may not overlap within a single diagnostic. "
"Please create a second diagnostic for overlapping time averaging periods "
"and account for the increased memory consumption."
);
} else {
WARPX_ABORT_WITH_MESSAGE("The step to begin time averaging may not be a negative number.");
}
}
}

if (m_time_average_type == TimeAverageType::Static) {
// Update time averaging period to current step
m_average_period_steps = step - m_average_start_step;
if (step > m_intervals.nextContains(step) - m_average_start_step && step <= m_intervals.nextContains(step)) {
in_averaging_period = true;

if (m_time_average_type == TimeAverageType::Static) {
// Update time averaging period to current step
m_average_period_steps = step - m_average_start_step;
}
}
}

Expand Down Expand Up @@ -719,10 +724,14 @@ FullDiagnostics::InitializeBufferData (int i_buffer, int lev, bool restart ) {
const int ngrow = (m_format == "sensei" || m_format == "ascent") ? 1 : 0;
int const ncomp = static_cast<int>(m_varnames.size());
m_mf_output[i_buffer][lev] = amrex::MultiFab(ba, dmap, ncomp, ngrow);
// Allocate MultiFab for cell-centered field output accumulation. The data will be averaged before flushing.
m_sum_mf_output[i_buffer][lev] = amrex::MultiFab(ba, dmap, ncomp, ngrow);
// Initialize to zero because we add data.
m_sum_mf_output[i_buffer][lev].setVal(0.);

if (m_diag_type == DiagTypes::TimeAveraged) {
// Allocate MultiFab for cell-centered field output accumulation. The data will be averaged before flushing.
m_sum_mf_output[i_buffer][lev] = amrex::MultiFab(ba, dmap, ncomp, ngrow);
// Initialize to zero because we add data.
m_sum_mf_output[i_buffer][lev].setVal(0.);
}

if (lev == 0) {
// The extent of the domain covered by the diag multifab, m_mf_output
//default non-periodic geometry for diags
Expand Down

0 comments on commit d6c3bdf

Please sign in to comment.