Local Environment Setup for BFD Development
In order to develop and run BFD locally, there are a number of required pieces of software that will need to be installed. This guide intends to walk through the installation steps for that software in order to quickly get you up to speed on contributing to BFD.
Most developers who contribute to BFD are using Mac, so this setup guide will be focused on Mac-centric setup steps. If you need windows-specific help contributing to BFD, please contact the BFD team for support.
Homebrew is a command line tool for easily downloading and installing scripts, software, and artifacts.
Install it by running the following in your terminal:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
Homebrew allows very easy installation of things with simple commands like brew install X where X is the name of the software/package/installation you want to download. Homebrew will be used for many of the installation instructions in this guide, since it simplifies the installation process.
BFD is currently using Java 17. There are many ways to get the SDK for java installed on your machine. BFD currently recommends the amazon corretto SDK, to align with our deployed SDK versions. https://docs.aws.amazon.com/corretto/latest/corretto-17-ug/macos-install.html
Another option is to use openJdk with brew:
brew install openjdk@17
Either SDK should be functionally equivalent, but corretto may have minor bugfixes and performance improvements over openJdk.
After installing Java, set your JAVA_HOME environment variable in the terminal:
export JAVA_HOME="$(/usr/libexec/java_home)"
Maven is used to manage building the application and artifacts/dependencies. Installing it is easy:
brew install maven
Once installed you can test it with mvn -version which should look something like this:
You'll need to configure your maven toolchain to point to your recently installed Java SDK. Configure your ~/.m2/toolchains.xml file to look like the following (change the jdkHome and version as needed to match your setup):
xml
>
<toolchains> <!-- JDK toolchains --> <toolchain> <type>jdk</type> <provides> <version>17</version> <vendor>sun</vendor> </provides> <configuration> <jdkHome>/Library/Java/JavaVirtualMachines/amazon-corretto-17.jdk</jdkHome> </configuration> </toolchain> </toolchains>
Python is a programming language used in ancillary BFD scripting and tool installations.
Your mac may come installed with python 3, but if not we will need the ability to run with python 3. Another easy brew install:
brew install python
You’ll need to grab git locally as well to start downloading/working on any code.
brew install git
After installing this, you’ll need to locate your github account (or create one if you don’t have one.) It would be a good idea at this point to adjust your github account to have your real name in your profile if it’s not set up.
You’ll need to set up a few things to connect your environment to the CMS repository in github:
- SSH - This key is used to encrypt the data between your computer and github. Adding a new SSH key to your GitHub account
- GPG - This is a key that tells github you are who you say you are when you make a commit. See GPG below for installing GPG and generating a key. Adding a new GPG key to your GitHub account
- PAT (Personal Access Token) - This must be used in place of a password when authenticating your push actions in the CMS repo. Creating a personal access token
- Set up your 2-factor authentication on github as it is required by the CMS organization for access. You can/should use Google Authenticator (phone app) for this and any other 2FA you can use it for at Ad Hoc. Configuring two-factor authentication
You’ll need GPG to generate a key to sign commits with.
brew install gpg
gpg --full-generate-key
Hit return/enter to accept default values. Make sure that the email address you use for the key matches the email address you have in GitHub.
Amazon Web Services (AWS) are heavily used in the project. The command line tool (CLI) lets you communicate with AWS with easy terminal commands and is occasionally useful for transferring files to and from your local machine during development.
Follow the instructions here to install the CLI: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
You should just need to download/run the PKG, but you can also install via the command line if you’d like.
Initially, you won’t have an AWS account to connect to and use AWS. Get an account by following the instructions here under “VPN and AWS”: https://confluence.cms.gov/pages/viewpage.action?pageId=302580951
Once you have an AWS account, you can log into AWS console site, which is used to manage the databases, load files, and other aspects of the deployed application environments.
You’ll want to properly setup your access key and 2FA in the AWS console as well:
- Log into AWS (and proceed to security credentials) via https://console.aws.amazon.com/iam/home?#/security_credentials
- Set up your 2 factor authentication by following the steps here: Enabling a virtual multi-factor authentication (MFA) device (console) - AWS Identity and Access Management
- It is recommended to use Google Authenticator for this.
- Once set up, your device should appear on the security credentials page at the bottom like this, with a unique device ID for your device:
- Click “Create Access Key”
4. Once clicked, it will create an access key and allow you to get the secret value once. Leave this window open (dont complete the setup of the key yet)
AWS uses a temporary credentialing system to restrict access via a token with a short expiry. The long-term credential is connected to your account details, and the other profile (your “short-term” profile) has its credentials generated from a call using the long-term credentials, which only have access to generate a short-term access token. Let's set up the files that will control these credentials locally.
- Generate an initial dummy aws configuration file by doing
aws configureon the command line and filling in any value when prompted (we'll replace them shortly) - Open the file in your text editor of choice. The file should have been created at
~/.aws/config
- In the brackets are the profile names, and below are the parameter key/values for that profile.
- Delete the dummy data and replace it with two cms profiles "cms" and "cms-long-term" and the region we'll need in both (us-east-1), it should end up like this:
[cms] region = us-east-1
[cms-long-term] region = us-east-1
- Save the file with the new profiles
- Open/Create a file named credentials in this directory (
~/.aws/) in your text editor of choice - We'll set up our two profiles in this file as well, which should end up looking like this (if you opened this file, replace whatever was in it):
[cms] aws_access_key_id = XXXXXXXXXXEXAMPLE aws_secret_access_key = XxXxXxXxEXAMPLE aws_mfa_device = arn:aws:iam::EXAMPLE:mfa/
[cms-long-term] aws_arm_mfa = arn:aws:iam::EXAMPLE:mfa/
Let's go over how to fill the credential file values out:
aws_access_key_id - the value of this should be the Access Key ID in the window you left open in step 4
aws_secret_access_key - the value of this should be the one-time-viewable secret in the window you left open in step 4. Reveal this secret, copy it, and paste it as the value for this field. You can now hit the button to complete the setup of the Access Key. It should show up in your AWS console like this:
aws_mfa_device - the value of this should be the id for the 2 factor authentication device you set up earlier in step 2. Use the id under Assigned MFA device which starts with arn:aws:iam for this value.
Fill out both profiles as in the example using your values.
Once that setup is done, you’ll need to generate the token for accessing AWS out of the credentials you supplied. The aws-mfa tool can help do that. Install it with:
pip3 install aws-mfa
This will install the tool on your local machine.
You’ll need to set the profile name to use before we do the final setup stage. Simply do:
export AWS_PROFILE=cms
This will tell the script which profile to use, which will align to the profiles we set up earlier.
Now that everything is set up, you can run the script for a temporary token by entering:
python3 aws-mfa
This will generate the short-term token under your cms profile in the credentials file. Note that this is (by design) a short-term token. After the token expires (after about a day) you'll need to run the aws-mfa command again to generate a new token.
Once you run this you will be able to use the AWS cli to perform commands; you can test it with
aws s3 ls
Which will display a list of files stored in AWS.
Postgres is a database tool used for creating/managing the database flavor we use in BFD. We can easily install this with homebrew:
brew install postgresql
Docker is used to create containers that run applications in sandboxes. This can be used initially to run your code locally by emulating some of the AWS pieces, and is used by the end-to-end tests to spin up mock servers.
Easy to install here: https://docs.docker.com/desktop/mac/install/
Download/run the DMG, and docker will be installed! (Restart your terminal or open a new tab to be able to use docker from the command line after this.)
Let's setup the docker database container which will be helpful in local development.
Postgres is the database variety used in BFD; let's set up a Postgres database with the following command. Data will be persisted between starts and stops in the bfd_pgdata volume.
> sh
docker run \ -d \ --name 'bfd-db' \ -e 'POSTGRES_DB=fhirdb' \ -e 'POSTGRES_USER=bfd' \ -e 'POSTGRES_PASSWORD=InsecureLocalDev' \ -p '5432:5432' \ -v 'bfd_pgdata:/var/lib/postgresql/data' \ postgres:14 -c max_connections=200
You can also use the create-bfd-db script located in the bfd repo under apps/utils/scripts to create the db. Apply the latest schema to the db with the run-db-migrator script afterwards.
This tool is used for encrypting/decrypting sensitive files. First let’s install it.
brew install ansible
Once installed, you can use ansible to decrypt files as such:
ansible-vault decrypt --ask-vault-pass ~/path/to/file/to/decrypt.ex
And encrypt files similarly:
ansible-vault encrypt --ask-vault-pass ~/path/to/file/to/encrypt.ex
Running these commands will ask for a password to use. This password can be found in Box and is rotated on a schedule. This password should be used when encrypting and decrypting files.
You'll need to download the bfd repository once you have Git installed.
First make a directory on your machine where you will story your Git projects; something like this command will create one.
mkdir -p ~/git/
Once you have this directory created, clone the bfd repository into this location.
git clone [email protected]:CMSgov/beneficiary-fhir-data.git ~/git/beneficiary-fhir-data.git
Once you've cloned the repo, you'll need to do a few important setup steps:
- Install pre-commit and pre-push hooks by building the repo from the apps folder (we can skip tests for now):
cd beneficiary-fhir-data/apps
mvn clean install -DskipITs -Dmaven.test.skip=true -Dmaven.build.cache.enabled=false
This will locally install the appropriate hooks to check your commits are clean and valid before pushing a branch. Note if you ever delete and re-clone the repo, this step will need to be run again, as these hooks are dropped in the repo as a local file.
- Change to the
apps/bfd-data-fdadirectory and runmvn clean install- This builds a dependent artifact that BFD needs to build/run
- Change to the
apps/bfd-data-npidirectory and runmvn clean install- This builds a dependent artifact that BFD needs to build/run
- Change to the
apps/directory andmvn clean install -DskipITs. The flag to skip the integration tests is important here. You may need to have AWS access for the integration tests to work correctly.
At this point, you may wish to test everything is working; see Testing Your Local Environment Setup for instructions on how to do so.
Terraform is used to automate the creation of cloud resources like provisioning infrastructure. This is used in many of our deployment flows and automates the creation of many permissions and AWS resources so they are easily recreated if needed.
While we can install terraform directly, there is a helpful tool tfenv to help with using various versions of terraform, which may be needed.
brew install tfenv
Once this is installed, you can install the latest terraform with
tfenv install latest
or if you need a specific version:
tfenv install <version>
Further information can be found at the tfenv docs: https://spacelift.io/blog/tfenv
You will need to generate an SSH key to SSH into running server instances. These keys are stored in AWS
First, generate a new RSA key locally (we'll need this later):
ssh-keygen -t rsa -b 4096 -C <emailaddress> -f ~/.ssh/bfd_aws_id_rsa
Follow the steps in How To Setup User SSH Key to add your SSH key to the appropriate location. (If you don't have permissions/access, ask for help on this step.)
SonarQube is our static analysis tool for the bfd project. See SonarQube Setup wiki page for setup instructions.
You only really need to initially worry about getting an ITOPs ticket created for you to be added as an admin to the project so you can see the project at the hosted sonarQube instance at https://sonarqube.cloud.cms.gov/.
Here are some other links/resources you can check out that will help understand the system.
FHIR Specification - FHIR is an industry standard format for medical data that is used as the return format of the data from BFD. The return is in JSON and the spec of which can be found here:
- V4 (newer revision which the newer BFD API uses): https://www.hl7.org/fhir/explanationofbenefit.html
- STU3 (older revision which the order BFD API uses): http://hl7.org/fhir/stu3/explanationofbenefit.html
Note that only some fields in the spec are used in BFD, as many are optional and/or conditional.
Oh-my-zsh - Adds functionality/customization to the default zsh terminal. Includes current directory and git branch (if the current path is a git repo) in its default plugins:
Iterm2 - A great functional terminal replacement that adds some nice features like split-screen and better tabs.
IntelliJ - Modern IDE for Java development; most of the team uses this IDE.
Run Scripts
The apps/utils/scripts directory contains bash scripts that simplify running the various components of BFD on a UNIX-like system.
Refer to the README files in that directory for detailed information about the scripts.
Project Setup - If you try to load the BFD project with IntelliJ, you may notice some issues when trying to build the project within the IDE. Adjusting your build settings to build with maven is one way to get around this:
Click the checkbox for “Delegate IDE build/run actions to Maven” and add the two properties there to avoid having the build take a long time.
It's also recommended to import the bfd project as a maven project, and import from the "apps" directory to make sure all the projects link their classes correctly.
Checkstyle Plugin - BFD uses checkstyle to keep documentation consistent and does automatic checks on build that documentation is valid. To see checkstyle violations in IDE instead of having to wait to build, you can install the checkstyle plugin.
Once installed, point it at the BFD checkstyle file which can be found at the root level of the apps folder. The checkstyle plugin settings can be found in Settings > Tools > Checkstyle. Add the file with the + under configuration file.
Java Formatting - The main intelliJ styling matches the codebase pretty well, but a couple adjustments will help avoid some common pitfalls:
Ensure tabs and indents are set to NOT use tabs, with a tab size/indent of 4:
Set imports such that the IDE does not automatically use * imports; this can be basically disabled by setting "Class count to use import with '*'" to 99.
The rest of the standard Code Style default in IntelliJ should be ok.
To get access to push to the CMS github repo a ticket will need to be opened to add you to the CMSGov github organization. You will also need to be added to the https://github.com/orgs/CMSgov/teams/beneficiary-fhir-data-staff/members team. Here’s an example:
https://jira.cms.gov/browse/WHSD-39857
Once added to the organization, navigate to the team link above and click the “Request Team Access” button located at the upper right corner of the roster.
Once you have the organization added to your account, are included in the above group, and have github set up with the appropriate keys, you should be able to push to the main repo.