Skip to content

Commit

Permalink
Codespaces native (#15935)
Browse files Browse the repository at this point in the history
* Add hybrid setup config

* Various fixes

* Improve startup

* Don't try to bring app up in start

* Add mockdata config

* Improve startup

* More QoL improvements

* Improve startup

* Add ruby lsp ext after start

* Create log dir on start

* Improvements

* Change setup to facilitate ruby lsp extension installation

* Remove manual extension installation

* Fix startup script

* More work on deps

* Set user id when starting deps

* More adjustments to startup scripts

* simplify docker deps setup

* Wait for the api to be ready before exiting the startup script

* More QoL improvements

* Change setup order

* Add output for permissions set

* Adjust permissions approach

* One more adjustment

* Belt & suspenders

* More perms adjustment

* Simplify startup

* Add native setup

* Change ruby setup to enable

* Add hardware requirements

* Add test DB config

* Add docs

* Update postgres version

* Update redis version

* Fix foreman output

* Use different redis package

* Minor debug output

* Don't bother with starting the app

* Improve test database config

* Update doc

* Remove host requirements

* Only notify for port forwarding on 3000

* Allow access from localhost and shared github URLs.

* Change ownership

* Remove host header check

This was obviated by the addition of the HostAuthorization middleware in Rails 6

* Add test DB setup to main local settings file

* Update tests
  • Loading branch information
acrollet authored Mar 26, 2024
1 parent de101d0 commit df914a8
Show file tree
Hide file tree
Showing 12 changed files with 171 additions and 72 deletions.
64 changes: 64 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
{
"name": "vets-api native setup",
"image": "mcr.microsoft.com/devcontainers/base:bullseye",

"features": {
"ghcr.io/devcontainers-contrib/features/ruby-asdf:0": {
"version": "3.2.3"
},
"ghcr.io/robbert229/devcontainer-features/postgresql-client:1": {
"version": "15"
},
"ghcr.io/devcontainers-contrib/features/redis-homebrew:1": {
"version": "6.2"
}
},

"forwardPorts": [
3000,
9293,
5432,
6379
],
"portsAttributes": {
"3000": {
"label": "vets-api",
"onAutoForward": "notify",
"requireLocalPort": true
},
"9293": {
"label": "vets-api-healthcheck",
"onAutoForward": "silent",
"requireLocalPort": true
},
"5432": {
"label": "postgis",
"onAutoForward": "silent",
"requireLocalPort": true
},
"6379": {
"label": "redis",
"onAutoForward": "silent",
"requireLocalPort": true
}
},

"postCreateCommand": "sh .devcontainer/post-create.sh",
"postStartCommand": "sh .devcontainer/post-start.sh",

"customizations": {
"codespaces": {
"repositories": {
"department-of-veterans-affairs/vets-api-mockdata": {
"permissions": {
"contents": "read",
"pull_requests": "write"
}
}
}
},
"vscode": {
"extensions": ["ms-azuretools.vscode-docker", "Shopify.ruby-lsp"]
}
}
}
51 changes: 51 additions & 0 deletions .devcontainer/post-create.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
#!/bin/sh

# Add welcome message
sudo cp .devcontainer/welcome.txt /usr/local/etc/vscode-dev-containers/first-run-notice.txt

# Switch to vets-api ruby version
export PATH="${HOME}/.asdf/shims:${HOME}/.asdf/bin:${PATH}"
asdf install ruby $( cat .ruby-version )
asdf global ruby $( cat .ruby-version )

git clone https://github.com/department-of-veterans-affairs/vets-api-mockdata.git ../vets-api-mockdata

sudo apt update
sudo apt install -y libpq-dev pdftk shared-mime-info postgresql-15-postgis-3

gem install bundler
NUM_CORES=$( cat /proc/cpuinfo | grep '^processor'|wc -l )
bundle config --global jobs `expr $NUM_CORES - 1`

# Update test DB config
echo 'test_database_url: postgis://postgres:password@localhost:5432/vets_api_test?pool=4' > config/settings/test.local.yml

# Add service config
if [ ! -f config/settings.local.yml ]; then
cp config/settings.local.yml.example config/settings.local.yml
cat <<EOT >> config/settings.local.yml
database_url: postgis://postgres:password@localhost:5432/vets_api_development?pool=4
test_database_url: postgis://postgres:password@localhost:5432/vets_api_test?pool=4
redis:
host: localhost
port: 6379
app_data:
url: redis://localhost:6379
sidekiq:
url: redis://localhost:6379
betamocks:
cache_dir: ../vets-api-mockdata
# Allow access from localhost and shared github URLs.
virtual_hosts: ["127.0.0.1", "localhost", !ruby/regexp /.*\.app\.github\.dev/]
EOT
fi

mkdir -p log
nohup bash -c '/home/linuxbrew/.linuxbrew/opt/[email protected]/bin/redis-server /home/linuxbrew/.linuxbrew/etc/redis.conf' >> log/redis.log 2>&1 &
sudo /etc/init.d/postgresql restart
pg_isready -t 60
sudo -u root sudo -u postgres psql -c "ALTER USER postgres PASSWORD 'password';"
./bin/setup
9 changes: 9 additions & 0 deletions .devcontainer/post-start.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/bin/sh

echo "Starting redis..."
nohup /home/linuxbrew/.linuxbrew/opt/[email protected]/bin/redis-server /home/linuxbrew/.linuxbrew/etc/redis.conf >> log/redis.log 2>&1 &

echo "Starting postgres..."
sudo /etc/init.d/postgresql restart
echo "Waiting for postgres to be ready..."
pg_isready -t 60
9 changes: 9 additions & 0 deletions .devcontainer/welcome.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
~~~~~~ Welcome to vets-api on codespaces! ~~~~~~

For more information, see the codespaces README in docs/setup.

~~~~~~ Quickstart ~~~~~

To start vets-api, run this command:

foreman start -m all=1,clamd=0,freshclam=0
2 changes: 2 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Gemfile @department-of-veterans-affairs/backend-review-group
Gemfile.lock @department-of-veterans-affairs/backend-review-group
Jenkinsfile @department-of-veterans-affairs/backend-review-group
Makefile @department-of-veterans-affairs/backend-review-group
.devcontainer @department-of-veterans-affairs/backend-review-group @department-of-veterans-affairs/cto-engineers
app/controllers/appeals_base_controller.rb @department-of-veterans-affairs/backend-review-group
app/controllers/appeals_base_controller_v1.rb @department-of-veterans-affairs/backend-review-group
app/controllers/application_controller.rb @department-of-veterans-affairs/va-api-engineers @department-of-veterans-affairs/backend-review-group
Expand Down Expand Up @@ -777,6 +778,7 @@ docs/setup/native.md @department-of-veterans-affairs/backend-review-group @depar
docs/setup/rswag_setup.md @department-of-veterans-affairs/backend-review-group @department-of-veterans-affairs/va-api-engineers
docs/setup/running_docker.md @department-of-veterans-affairs/backend-review-group @department-of-veterans-affairs/va-api-engineers
docs/setup/running_natively.md @department-of-veterans-affairs/backend-review-group @department-of-veterans-affairs/va-api-engineers
docs/setup/codespaces.md @department-of-veterans-affairs/backend-review-group @department-of-veterans-affairs/cto-engineers
docs/setup/va_forms.md @department-of-veterans-affairs/platform-va-product-forms @department-of-veterans-affairs/backend-review-group @department-of-veterans-affairs/va-api-engineers
docs/setup/virtual_machine_access.md @department-of-veterans-affairs/backend-review-group @department-of-veterans-affairs/va-api-engineers
.github @department-of-veterans-affairs/backend-review-group
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ For frontend, see [vets-website](https://github.com/department-of-veterans-affai
- [Native setup](docs/setup/native.md) (OSX/Ubuntu)
- [Docker setup](docs/setup/docker.md)
- [Hybrid setup](docs/setup/hybrid.md)
- [Codespaces setup](docs/setup/codespaces.md)
## Running the app
Expand Down
8 changes: 1 addition & 7 deletions app/controllers/concerns/headers.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,7 @@
module Headers
extend ActiveSupport::Concern

included { prepend_before_action :block_unknown_hosts, :set_app_info_headers }

# returns a Bad Request if the incoming host header is unsafe.
def block_unknown_hosts
return if controller_name == 'example'
raise Common::Exceptions::NotASafeHostError, request.host unless Settings.virtual_hosts.include?(request.host)
end
included { prepend_before_action :set_app_info_headers }

def set_app_info_headers
headers['X-Git-SHA'] = AppInfo::GIT_REVISION
Expand Down
27 changes: 27 additions & 0 deletions docs/setup/codespaces.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Codespaces setup

## About codespaces

Github Codespaces provide an Integrated Development Environment (IDE) that is accessible entirely in a web browser. It is essentially a web based version of VS Code running on a cloud based virtual machine.

Codespaces is available for all members of the Department of Veterans Affairs organization on Github.

### More information

- [Platform documentation for codespaces](https://depo-platform-documentation.scrollhelp.site/developer-docs/using-github-codespaces)
- See the #codespaces channel in Slack for additional questions about using Codespaces.

## Creating a codespace

1. Go to [your Codespaces page](https://github.com/codespaces) on Github.
2. Click the [new Codespace](https://github.com/codespaces/new) button at the top right.
3. Select the vets-api repository and adjust other settings as desired, the defaults should work well.
4. Click the 'Create codespace' button

Your new codespace will open in Visual Studio Code if you have it installed locally, or otherwise in the browser. The vets-api repo and all dependencies will be installed, and it will be ready for use in about 5 minutes.

## Using your codespace

Your codespace will automatically start vets-api and forward port 3000 to your local machine if you have Visual Studio Code installed. The API can be accessed at http://localhost:3000/

For more information on running vets-api and specs, see the [native running instructions](running_natively.md).
14 changes: 7 additions & 7 deletions docs/setup/native.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Vets API requires:

- Ruby 3.2.3
- PostgreSQL 11.x (including PostGIS 2.5)
- PostgreSQL 15.x (including PostGIS 3)
- Redis 5.0.x

The most up-to-date versions of each key dependency will be specified in the `docker-compose.yml` [file](https://github.com/department-of-veterans-affairs/vets-api/blob/master/docker-compose.yml) and the `Dockerfile`.
Expand Down Expand Up @@ -114,14 +114,14 @@ All of the OSX instructions assume `homebrew` is your [package manager](https://

1. It is MUCH easier to use the [Postgres.app](https://postgresapp.com/downloads.html) which installs the correct combination of Postgresql and PostGIS versions.

- Download the Postgres.app with PostgreSQL 10, 11 and 12
- Download the Postgres.app with PostgreSQL 15
- Install Instructions here: https://postgresapp.com/
- `sudo mkdir -p /etc/paths.d && echo /Applications/Postgres.app/Contents/Versions/latest/bin | sudo tee /etc/paths.d/postgresapp`
- `ARCHFLAGS="-arch x86_64" gem install pg -v 1.2.3`
2. Alternatively Postgresql 11 & PostGIS 2.5 can be installed with homebrew
- `brew install postgresql@11`
- `brew services start postgresql@11`
- Install the `pex` manager to add your Postgresql 11 extensions from [here](https://github.com/petere/pex#installation)
2. Alternatively Postgresql 15 & PostGIS 3 can be installed with homebrew
- `brew install postgresql@15`
- `brew services start postgresql@15`
- Install the `pex` manager to add your Postgresql 15 extensions from [here](https://github.com/petere/pex#installation)
- Install the `postgis` extension along with a number of patches using the instructions summarized [here](https://gist.github.com/skissane/0487c097872a7f6d0dcc9bcd120c2ccd):
- ```bash
PG_CPPFLAGS='-DACCEPT_USE_OF_DEPRECATED_PROJ_API_H -I/usr/local/include' CFLAGS='-DACCEPT_USE_OF_DEPRECATED_PROJ_API_H -I/usr/local/include' pex install postgis
Expand Down Expand Up @@ -181,7 +181,7 @@ All of the OSX instructions assume `homebrew` is your [package manager](https://
2. Install PostGIS

```bash
sudo apt install -y postgresql-11-postgis-2.5
sudo apt install -y postgresql-15-postgis-3
sudo -i -u postgres
createuser postgis_test
Expand Down
21 changes: 0 additions & 21 deletions lib/common/exceptions/not_a_safe_host_error.rb

This file was deleted.

9 changes: 0 additions & 9 deletions spec/controllers/application_controller_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -478,15 +478,6 @@ def append_info_to_payload(payload)
expect(controller.payload[:user_uuid]).to eq(user.uuid)
end

context 'with a virtual host that is invalid' do
let(:header_host_value) { 'unsafe_host' }

it 'returns bad request' do
get :test_authentication
expect(response).to have_http_status(:bad_request)
end
end

context 'with a credential that is locked' do
let(:user) { build(:user, :loa3, :idme_lock) }

Expand Down
28 changes: 0 additions & 28 deletions spec/lib/common/exceptions/not_a_safe_host_error_spec.rb

This file was deleted.

0 comments on commit df914a8

Please sign in to comment.