HPX Source Code Structure and Coding Standards

Contents

• Coding Standards
• Directory Structure
• Commit Message Standards
• General Structure
• Thread Stuff

---

Coding Standards

Coding standards used are: http://www.boost.org/development/requirements.html#Guidelines

The short version of the guidelines:

• 80-character lines.
• Absolutely no tabs (use spaces instead of tabs).
• No whitespace at line endings
• Because we use git, UNIX line endings.
• Identifiers are C++ STL style: no CamelCase. E.g. my_class instead of MyClass.
• Use expressive identifiers.
• Exceptions for error handling instead of C-style error codes.

There is a .editorconfig file in the HPX root directory which can be used for almost any widely available editor. Please see http://editorconfig.org to download plugins for your favorite editor.

There is also a .clang-format file in the HPX root directory which you can use to manually format the code you contribute. This configuration file can be used with clang-format, a tool created by the Clang project.

All of HPX (and we mean all of it) is released under the Boost Software License V1.0 (see: http://www.boost.org/LICENSE_1_0.txt). We insist in all contributions being released under the same license to simplify the life of our users. No sane lawyer will ever agree to use an external piece of software which does not follow a clear licensing policy. Since HPX relies on Boost we decided to keep HPX itself under the Boost license as well. Learn more about the Boost Software License at: http://www.boost.org/users/license.html .

A few additional ones:

• Use doxygen style comments to document API functions.
• Before writing a piece of utility code, see if there is something in hpx::util, Boost or the C++ standard library that can be used to save time.

We use a tool call inspect (derived from a tool in Boost with the same name) to check for a list of guidelines for every commit. This tool is run as part of our CircleCI continuous integration tests. The test will fail if any of the guidelines are violated. Here is a list of things checked by inspect:

• Every file has to contain the Boost license info
• Every file has to contain a copyright notice
• Files should contain only 'proper' file endings (i.e. '\n' on Linux, '\r\n' on Windows, etc.)
• Files should end with a newline
• File and directory name should not be longer than 32 characters, overall file names shouldn't be longer than 255 characters
• No tabs!
• No non-ASCII chars
• Certain Apple macros are used
• C-style assert macro is used
• The Boost min/max guidelines have been violated
• Usages of unnamed namespaces in headers (including .ipp files)
• Endline whitespace
• Lines exceeding the character limit (soft limit: 80 characters, hard limit: 90 characters)

---

Directory Structure

The directory structure follows the structure of the C++ namespaces, except for the runtime directory portion. Headers are in the hpx directory, and source files in the src directory (both relative to the top directory of the HPX source tree). E.g. hpx/lcos/mutex.hpp is hpx::lcos::mutex, and hpx/runtime/threads/threadmanager.hpp is hpx::threads::threadmanager.

Anything that ends in _impl is some technical details that are not part of the interface. In some places, things are called _impl_impl :p.

Likewise, anything in a namespace called detail is an implementation detail. Things in the very_detail namespace are really hacky details.

The hpx/util (hpx::util) code is utilities and implementation details.

We tell users, whenever possible, to avoid looking at implementation details. These implementation details tend to be highly technical. This is why we try to abstract them away.

For those who are doing work on the HPX core, you may inevitably have to delve into these sections of the code, though.

---

Commit Message Standards

Whenever commits are made to the HPX repository we ask that they adhere to the following guidelines: (These guidelines are derived from http://chris.beams.io/posts/git-commit/)

1. Separate subject from body with a blank line
2. Limit the subject line to 50 characters
3. Capitalize the subject line
4. Do not end the subject line with a period
5. Use the imperative mood in the subject line
6. Wrap the body at 72 characters
7. Use the body to explain what and why vs. how

An example of a properly formatted commit message is below:

Summarize changes in around 50 characters or less

More detailed explanatory text, if necessary. Wrap it to about 72
characters or so. In some contexts, the first line is treated as the
subject of the commit and the rest of the text as the body. The
blank line separating the summary from the body is critical (unless
you omit the body entirely); various tools like `log`, `shortlog`
and `rebase` can get confused if you run the two together.

Explain the problem that this commit is solving. Focus on why you
are making this change as opposed to how (the code explains that).
Are there side effects or other unintuitive consequenses of this
change? Here's the place to explain them.

Further paragraphs come after blank lines.

 - Bullet points are okay, too

 - Typically a hyphen or asterisk is used for the bullet, preceded
   by a single space, with blank lines in between, but conventions
   vary here

If you use an issue tracker, put references to them at the bottom,
like this:

Resolves: #123
See also: #456, #789

---

General Structure

HPX has a couple major subsystems. Most of them are listed below, by header path (replace hpx/ with src/ to get the path to the .cpp files):

• hpx/lcos - LCOs
• hpx/lcos/local - LCOs optimized for shared memory usage
• hpx/runtime - The guts
• hpx/runtime/parcelset - The parcel transport layer. Consists of two elements: the parcel port (an object that receives parcels) and parcel handlers (objects that process parcels). There are currently two parcelhandlers in use: the early parcel handler, which handles bootstrap, and the primary parcel handler, which passes parcels on to the action manager, where they are decoded, and then scheduled with the thread manager.
• hpx/runtime/applier - The code which implements hpx::apply (its location is due to historical reasons). Previously, contained interfaces to the runtime. This directory is liable to get moved at some point.
• hpx/runtime/actions - The implementation of HPX actions (e.g. the active message; the contents of a parcel). Also contains the action_manager, which is invoked by the primary parcelhandler. Highly technical.
• hpx/runtime/components - Implementation of HPX components (e.g. globally named objects; anything that has a GID and can be acted upon remotely). Contains a good bit of allocation related code. Also contains implementations of special-case components: remote logging, remote error handling, the remotable interface to the runtime system and memory blocks. Highly technical.
• hpx/runtime/agas and hpx/runtime/naming - Implementation of AGAS, including the software cache and large chunks of the bootstrap code. Extraordinarily technical in some places. This subsystem is largely transparent to the end-user, aside from hpx::id_type.
• hpx/runtime/threads and hpx/util/coroutine - Contains the threadmanager, NUMA/CPU affinity code, scheduling code and related data structures and classes.
• hpx/config - Preprocessor configuration code (boring).
• hpx/components - Standard system components (PAPI, memory counters, factories, iostreams).
• hpx/traits - Metaprogramming code (run away).
• hpx/util - Implementation details and utilities. This contains a collection of code too diverse for me to enumerate here.
• hpx/include - Forwarding headers (boring).
• hpx/runtime.hpp, src/runtime.cpp, hpx/runtime_impl.hpp, hpx/runtime_impl.cpp - The runtime data structure. Contains a portion of startup and initialization related code. An instance of the hpx::runtime class represents an instance of the HPX runtime system.
• hpx/hpx_init.hpp, hpx_init_impl.hpp, hpx_start.hpp, hpx_start_impl.hpp, hpx/hpx_finalize.hpp - Command line handling and pre-runtime initialization code.
• src/pre_main.cpp - Post-bootstrap, pre-hpx_main() initialization code.

---

Thread Stuff

The thread schedulers are in (relative to the top directory of the tarball):

• hpx/runtime/threads/policies - location of the pluggable schedulers.

These are other places that may be of interest to you, listed from least scary to most scary.

• hpx/runtime/threads - all threadmanager related code.
• src/runtime/threads - all threadmanager related code.
• src/runtime/threads/threadmanager.cpp - main thread scheduling loop (calls into the schedulers), tfunc and tfunc_impl.
• hpx/runtime/threads/topology.hpp - NUMA/CPU affinity code.
• hpx/runtime/threads/topology.cpp - NUMA/CPU affinity code.
• hpx/hpx_fwd.hpp - new thread schedulers need to be forward declared in this file.
• src/runtime_impl.cpp - new thread schedulers currently need to be manually instantiated in this file (see the last few lines, it's self explanatory).
• src/hpx_init.cpp - new thread schedulers need to be added to the command line options here.
• CMakeLists.txt - new thread schedulers need to be added to the CMakeLists.txt file.
• hpx/util/coroutine - low-level threading implementation.
• src/util/coroutine - low-level threading implementation.

The thread scheduler subsystem isn't as pluggable as it could be - there are a few places where you will need to copy/paste boilerplate code.