Summing Information Over the Grid When Using MPI

[ChrisO56]

I am trying to implement a source term that needs to factor in changes in the grid before modifying the energy in the grid. So far I have implemented a simple case of what I want.

I have two questions regarding this:

1. When moving to MPI, does Athena handle the book keeping when trying to gather information from the total grid or is there something I need to handle myself?
2. In my current source term implementation, I have a print statement I added to monitor the statistic I am interested in, but I noticed that it gets written into the log file out of time order. I assume that's due to how MPI sends the information to be written out. Is there something you recommend that solves that issue?

[c-white]

1. This depends on what information you want. Simple things like how many MeshBlocks there are can be seen by each MPI process. On the other hand, the different processes do not have access to each other's physical values like density.

2. Syncing I/O between different processes is tricky. You can try inserting a std::flush in Mesh::OutputCycleDiagnostics to encourage all messages to be written before the new timestep is started. More details about what is being tried and what the results are would help.