-
Notifications
You must be signed in to change notification settings - Fork 12
ARCHEReCSEDecemberMinutes
WP1, done
WP2, done
WP3, done. Mechanical model of the cell. Given a shape, what are the forces acting on each node (bending, volume/surface conservation). HemeLB to read cell template from file. Added readers/writers.
WP4, done. Implement IBM.
- Spreading of forces and integration of velocities. Add force terms to BGK kernel.
- Cell-cell and cell-wall interactions. Implemented algorithm (Divide&Conquer approach) for identifying pairs of interacting nodes efficiently.
WP5, done.
- Created special regions at inlets and outlets, where cells are added/removed.
- In those regions the cells feel the full effect of the fluid however the forces of the cells on the fluid are ramped in [0,1]. This is done so that the original LB iolet algorithm can still be used (effectively ignoring the cell-fluid interactions at x=0). This is region is not physical. Actual simulation starts when region is left behind by cells.
- This iolet takes (x,y,z) where cells have to be placed. It’s therefore decoupled from the algorithm that chooses where/when cells appear.
WP6&7
Done:
- A process knows which cells are relevant to its flow subdomain (whether own by itself or not)
- Velocity interpolation is done with no-blocking neighbouring collectives. Cell owner sums all contributions from local fluid sites and those coming from neighbouring subdomains.
- Node positions are updated.
- Concurrently in any order:
- FE solution:
- FE model solved by owner.
- Spread forces locally and send those that are not to neighbouring subdomains.
- Spread forces for cells that have local nodes but are owned by someone else.
- This leads to changes in the set of processes affected by a given node and cell ownership.
- Update list of cell nodes relevant to each subdomain.
- Update cell ownership.
- Communicate these changes across processes.
Missing (but required for report):
- Create graph communicator.
- Integrate everything into cell army.
WP8 partially met:
- Implemented half an infinite cylinder (conveyor belt analogy) where any distribution of cells can be initialised and advanced towards the finite end of the cylinder where cells will be migrated to the simulation. Currently only implemented with columns of cells. Fully developed and unit tested, needs integration into SimulationMaster. Parallelisation needs thinking through (who owns the conveyor belt? a separate process?).
- Not implemented: in order to achieve tighter packing of cells they need to be seeded into the cylinder and grown.
WP9 partially met:
- Only C interface to HDF5 is suitable for us (it’s the only one MPI parallelised). Any code using C++ interface has to be rewritten with the C interface.
- Design of output file format based on H5MD. Class that creates H5MD metadata.
- To do: using the cell listener interface call append operation on writer with a multidimensional array that describes the state of the simulation (i.e. barycentre, all nodes). Issue 559.
WP10:
- We had informal talks with ARCHER re running CI in ARCHER and at the moment this is not yet supported, although there’s an increasing awareness that the community wants to move in this direction.
- Instead we did the following Infrastructure improvements:
- Ported development to Github. Facilitates multisite development. Facilitates code review (through pull request features).
- Migrated CI to UCL Jenkins service: we now test with multiple compilers and OS (OS X, desktop linux and cluster linux). Ability to test branches.
- Create a Slack channel for developer communication and design discussions.
- Added support for parallel unit testing. To date all unit tests where run in sequential, therefore communication bugs, deadlocks, etc. could not be tested against. A user would only discover them once the code is run in production in a cluster, which makes difficult debugging and wastes CPU hours.
- Enabled the use of C+11 features. Some of its features had been manually added to HemeLB and had to be maintained by the team, now it uses the equivalent standard implementation (STATIC_ASSERT and boost:shared_ptr() -> std::shared_ptr()).
WP11: to be done for the final report.
WP12: code documentation and tests done. High level description will be written as part of the report. Draft of the paper later on.
Net::Graph() creates the MPI data structure for graph communicator (I need to figure out the pattern, vector of vectors, of the graphs)
MPINodeParallelizationTests has some examples of graph creations
Find shortest distance between two flow domains, if it is shorter than the effective size of a cell they need to be in communication
Look at GatherSpecialPositions (parallel/Fixtures.h) for example of how to get the indices of edge fluid sites.
Design solution with Mayeul once the previous is done. Some notes:
In practice, you want to replace spreading and integration in CellArmy (two for loops over cells) with calls to redblood::parallel::SpreadForce and redblood::parallel::IntegrateVelocities (see the unit-tests). Then you want to make sure that cells owned by other processes are taken into account when doing cell-cell cell-wall interactions (by letting CellArmy::cell2cell and CellArmy::cell2wall know about lent cells). And finally, there is managing cells introduces and removed from the simulation.
WP7:
- cell-cell interaction range. For two-point stencil there’s a chance that a process doesn’t know about cells in neighbouring subdomains that should be taken into account for cell-cell interaction. This is not a problem with three-point stencil as long as cell-cell interaction is 1 LB.
- cell-wall interaction: cell-wall interaction distance must be smaller than stencil size.
Suitable calls:
- BBSRC: the 2016 round of http://www.bbsrc.ac.uk/funding/opportunities/2015/tools-and-resources-2015-call2/