HPX Source Code Structure and Coding Standards

Contents

• Coding Standards
• Directory Structure
• Commit Message Standards
• File Structure
• Thread Stuff
• Documentation Style Guide

---

Coding Standards

Coding standards used can be found at: https://www.boost.org/development/requirements.html#Guidelines

The short version of the guidelines:

• 80-character lines.
• Tabs are forbidden—use spaces instead of tabs.
• No whitespace at line endings
• Because we use Git, use UNIX line endings. Identifiers are C++ STL style. (No CamelCase. e.g., my_class instead of MyClass.)
• Use expressive identifiers.
• Use exceptions for error handling instead of C-style error codes.

There is a .editorconfig file in the HPX root directory, which almost any widely available editor can use. Visit https://editorconfig.org to download plugins for your favorite editor.

There is also a .clang-format file in the HPX root directory, which you can use to format the code you contribute manually. clang-format, a tool created by the Clang project, uses this configuration file.

All of HPX (and we mean all of it) is released under the Boost Software License V1.0 (see: https://www.boost.org/LICENSE_1_0.txt). We insist on all contributions being released under the same license to simplify the life of our users. No sane lawyer will ever agree to use an external piece of software that does not follow a clear licensing policy. Since HPX relies on Boost, we decided to keep HPX itself under the Boost license as well. Learn more about the Boost Software License at: https://www.boost.org/users/license.html.

A few additional ones:

• Use Doxygen-style comments to document API functions.
• Before writing a piece of utility code, see if there is something in hpx::util, Boost, or the C++ standard library that can help save you time.

We use a tool called inspect (derived from a tool in Boost with the same name) to check for a list of guidelines for every commit. This tool is run as part of our CircleCI continuous integration tests. The test will fail if any of the guidelines are violated. Here is a list of things checked by inspect:

• Every file has to contain the Boost license info.
• Every file has to contain a copyright notice.
• Files should contain only 'proper' file endings. (i.e., '\n' on Linux, '\r\n' on Windows, etc.)
• Files should end with a newline.
• File and directory name should not be longer than 32 characters; overall file names shouldn't be longer than 255 characters.
• No tabs!
• No non-ASCII chars
• Certain Apple macros are used.
• C-style assert macro is used.
• The Boost min/max guidelines have been violated.
• Usages of unnamed namespaces in headers (including .ipp files)
• Endline whitespace
• Lines exceeding the character limit (soft limit: 80 characters, hard limit: 90 characters)

---

Directory Structure

The directory structure follows the structure of the C++ namespaces, except for the runtime directory portion. Headers are in the hpx directory, and source files in the src directory (both relative to the top directory of the HPX source tree). E.g. hpx/lcos/mutex.hpp is hpx::lcos::mutex, and hpx/runtime/threads/threadmanager.hpp is hpx::threads::threadmanager.

Anything that ends in _impl is some technical details that are not part of the interface. In some places, things are called _impl_impl :P.

Likewise, anything in a namespace called detail is an implementation detail. Things in the very_detail namespace are really hacky details.

The hpx/util (hpx::util) code is utilities and implementation details.

We tell users, whenever possible, to avoid looking at implementation details. These implementation details tend to be highly technical. This is why we try to abstract them away. However, if you work on the HPX core, you may inevitably have to delve into these sections of the code.

---

Commit Message Standards

Whenever commits are made to the HPX repository, we ask that they adhere to the following guidelines (derived from http://chris.beams.io/posts/git-commit/):

1. Separate subject from body with a blank line
2. Limit the subject line to 50 characters
3. Capitalize the subject line
4. Do not end the subject line with a period
5. Use the imperative mood in the subject line
6. Wrap the body at 72 characters
7. Use the body to explain what and why vs. how

An example of a properly formatted commit message is below:

Summarize changes in around 50 characters or less

More detailed explanatory text, if necessary. Wrap it to about 72
characters or so. In some contexts, the first line is treated as the
subject of the commit and the rest of the text as the body. The
blank line separating the summary from the body is critical (unless
you omit the body entirely); various tools like `log`, `shortlog`
and `rebase` can get confused if you run the two together.

Explain the problem that this commit is solving. Focus on why you
are making this change as opposed to how (the code explains that).
Are there side effects or other unintuitive consequences of this
change? Here's the place to explain them.

Further paragraphs come after blank lines.

 - Bullet points are okay, too

 - Typically, a hyphen or asterisk is used for the bullet, preceded
   by a single space, with blank lines in between, but conventions
   vary here

If you use an issue tracker, put references to them at the bottom,
like this:

Resolves: #123
See also: #456, #789

---

File Structure

HPX has a couple of major subsystems. Most of them are listed below, by header path (replace hpx/ with src/ to get the path to the .cpp files):

• hpx/lcos - LCOs
• hpx/lcos/local - LCOs optimized for shared memory usage
• hpx/runtime - The guts
• hpx/runtime/parcelset - The parcel transport layer. Consists of two elements: the parcel port (an object that receives parcels) and parcel handlers (objects that process parcels). There are currently two parcelhandlers in use: the early parcel handler, which handles bootstrap, and the primary parcel handler, which passes parcels on to the action manager, where they are decoded and then scheduled with the thread manager.
• hpx/runtime/applier - The code which implements hpx::apply() (its location is due to historical reasons). Previously, contained interfaces to the runtime. This directory is liable to get moved at some point.
• hpx/runtime/actions - The implementation of HPX actions (e.g. the active message; the contents of a parcel). It also contains the action_manager, which is invoked by the primary parcelhandler. (Warning: Highly technical).
• hpx/runtime/components - Implementation of HPX components (e.g. globally named objects; anything that has a GID and can be acted upon remotely). Contains a good bit of allocation-related code. Also contains implementations of special-case components: remote logging, remote error handling, the remotable interface to the runtime system, and memory blocks. Highly technical.
• hpx/runtime/agas and hpx/runtime/naming - Implementation of AGAS, including the software cache and large chunks of the bootstrap code. Extraordinarily technical in some places. This subsystem is largely transparent to the end-user, aside from hpx::id_type.
• hpx/runtime/threads and hpx/util/coroutine - Contains the threadmanager, NUMA/CPU affinity code, scheduling code and related data structures and classes.
• hpx/config - Preprocessor configuration code (boring).
• hpx/components - Standard system components (PAPI, memory counters, factories, iostreams).
• hpx/traits - Metaprogramming code (run away).
• hpx/util - Implementation details and utilities. This contains a collection of code too diverse for me to enumerate here.
• hpx/include - Forwarding headers (boring).
• hpx/runtime.hpp, src/runtime.cpp, hpx/runtime_impl.hpp, src/runtime_impl.cpp - The runtime data structure. Contains a portion of startup and initialization related code. An instance of the hpx::runtime class represents an instance of the HPX runtime system.
• hpx/hpx_init.hpp, hpx_init_impl.hpp, hpx_start.hpp, hpx_start_impl.hpp, hpx/hpx_finalize.hpp - Command line handling and pre-runtime initialization code.
• src/pre_main.cpp - Post-bootstrap, pre-hpx_main() initialization code.

---

Thread Stuff

The thread schedulers are in (relative to the top directory of the tarball):

• hpx/runtime/threads/policies - location of the pluggable schedulers.

These are other places that may be of interest to you, listed from least scary to most scary.

• hpx/runtime/threads and src/runtime/threads - all threadmanager related code.
• src/runtime/threads/threadmanager.cpp - main thread scheduling loop (calls into the schedulers), tfunc and tfunc_impl.
• hpx/runtime/threads/topology.hpp and src/runtime/threads/topology.cpp - NUMA/CPU affinity code.
• hpx/hpx_fwd.hpp - new thread schedulers need to be forward declared in this file.
• src/runtime_impl.cpp - new thread schedulers currently need to be manually instantiated in this file (see the last few lines, it's self-explanatory).
• src/hpx_init.cpp - new thread schedulers need to be added to the command line options here.
• CMakeLists.txt - new thread schedulers need to be added to the CMakeLists.txt file.
• hpx/runtime/threads/coroutines and src/runtime/threads/coroutines - low-level threading implementation.

The thread scheduler subsystem isn't as pluggable as it could be - there are a few places where you will need to copy/paste boilerplate code.

---

Documentation Style Guide

The following guidelines have been created to help standardize the look of the documentation. These guidelines are still being updated and may expand or change, so please revisit them before creating new content.

Capitalization

• HPX should always be rendered in ALL CAPS and italics.
• All other terms should be capitalized according to their standard spelling (try googling a term if you are uncertain of its standard spelling). For example, CMake should always be written with both a capital "C" and "M".

Headings

• All headings should be written in bold.
• When writing for the manual, only capitalize the first letter of the first word in the heading and any normally capitalized names (like CMake). For example: Welcome to the HPX documentation!
• When writing for the wiki, capitalize the first letter of all important words. For example: Thread Stuff

Hyperlinks

• Internal hyperlinks (links to other areas within the documentation) should be used every time a relevant term is mentioned. For example, you will use a hyperlink every time you mention locality in the documentation.
• External hyperlinks (links to areas outside the documentation) should only be used the first time a relevant term is mentioned on a page. For example, you will only need to include a hyperlink for CMake the first time the term is used on a page.