We’re glad you’re thinking about contributing to this open source project! If you’re unsure about anything, ask us — or submit the issue or pull request anyway. The worst that can happen is that we’ll politely ask you to change something.
We welcome all friendly contributions, and we welcome your ideas about how to make our online presence more user friendly, accessible, and elegant.
- Have Docker Toolbox installed, which includes Oracle VM
- You will need admin rights to setup the docker environment. A registry key can disable the admin rights check. Ask IT to disable.
- Open Oracle Virtualbox and verify you only have 1 Host Only Adapter
- In the Oracle Virtualbox, select Global Tools
- Select Host Network Manager
- You should be in the Properties view.
- Verify only 1 or 0 adapters are listed.
- if more than 1 disable extra adapters. You should only have 1 enabled to avoid potential errors later.
- Also install 'bash' by installing git for Windows.
- Clone the repo to a directory in C:\Users. This is automatically shared by Docker.
- Launch Docker Quick Start which should be a shortcut on the Desktop
- Once launched switch to your source code directory. "doi-extractives-data"
- Although Quick start creates a VM we are going to create our own.
- On windows the initial machine usually results in errors. Such as a certificate errors.
- Also the machine is not very powerful.
- Sometimes the docker terminal crashes. Just restart it if it does.
- Make sure you are in your source code directory and run the following commands
docker-machine rm default
docker-machine ls # lists all machines, verify default is not listed
docker-machine create --driver virtualbox --virtualbox-cpu-count 2 --virtualbox-memory 4046 default
eval "$(docker-machine env default)"
docker-machine ls # Note your IP address
- Your docker-machine should now be ready to setup the development environment
- If you get a certificate error run this command from your source code directory
docker-machine regenerate-certs default
Instead of installing dependencies yourself and running different commands in separate terminal sessions, you should use Docker, which only requires installing Docker Community Edition and running one command in one terminal window.
Start by installing Docker Community Edition.
If you are on Windows, you'll also need bash, which you can probably
get most easily by installing git for Windows.
To get up and running with Docker, run the following in the project directory.
docker-compose build
docker-compose run --rm jekyll bash scripts/update-deps.sh
docker-compose up
Then visit http://localhost:4000/ in your browser. On Windows machines the URL is generally http://192.168.99.100:4000/.
Whenever you make changes to any files, the proper static assets will be rebuilt, and your changes will show up on the site.
We sometimes observe a behavior where docker-compose up will result in an error the first time it is run, but will work the second time.
The error usually looks like this (expand to see):
jekyll_1 | module.js:540
jekyll_1 | throw err;
jekyll_1 | ^
jekyll_1 |
jekyll_1 | Error: Cannot find module 'to-regex'
jekyll_1 | at Function.Module._resolveFilename (module.js:538:15)
jekyll_1 | at Function.Module._load (module.js:468:25)
jekyll_1 | at Module.require (module.js:587:17)
jekyll_1 | at require (internal/module.js:11:18)
jekyll_1 | at Object.<anonymous> (/doi/node_modules/chokidar/node_modules/braces/index.js:7:15)
jekyll_1 | at Module._compile (module.js:643:30)
jekyll_1 | at Object.Module._extensions..js (module.js:654:10)
jekyll_1 | at Module.load (module.js:556:32)
jekyll_1 | at tryModuleLoad (module.js:499:12)
jekyll_1 | at Function.Module._load (module.js:491:3)
doiextractivesdata_jekyll_1 exited with code 1
If you run into this error, then you should:
- Use
Control-Cto bring down the docker containers - Make sure everything is turned off with
docker-compose down - Use
docker-compose upagain
Docker is used for local development but not for production. Docker will run containers for jekyll and webpack and will start a small ExpressJS web server to make all of the URLs resolve as they would on production.
We have a difference in the process between local/development builds and production builds because the latter occurs on Federalist in a predictable order and does not need to be set up to watch files and rebuild on changes. However, we do want a development set up that does watch for changes and runs the relevant build process.
If you want to run commands like npm, make, or sqlite3, the easiest
way to do this is by running a shell inside the main container:
docker-compose run jekyll bash
Once you do this, you'll be in an interactive shell within the main container, and can run any commands you need.
Whenever you update the repository using e.g. git pull, run
docker-compose build jekyll again to rebuild the Docker container and
fetch any new dependencies. Building a single -compose.yml service at first caches all of the build steps and makes it faster to build the remaining services.
If you decide that Docker isn't for you, or if your Docker setup somehow
becomes broken and you're not sure how to fix it, run
docker-compose down -v.
The data catalog explains what most of the data is and where it came from. See the data directory for more detailed info and instructions on updating the data.
Data for the site is populated via data files in the _data directory. These are primarily yml files that are generated from commands in the Makefile.
To create the database locally, make sure that you have sqlite and run make db.
If you would like to query the local database instance:
- Open a new terminal shell and run
sqlite3 - Run
.open data.db - You can now run sqlite queries from the local instance.
- Run
.tablesto see the available tables you can query.
To update site data, run make site-data.
Gatsby is a JavaScript framework for building static websites. We're currently migrating from Jekyll to Gatsby. The migration is over a long time span, and there are some gotchas about making Jekyll and Gatsby work together.
Our approach is to replace Jekyll pages with Gatsby pages in batches. Since
Jekyll is happy to take static pages and assets and copy them over to the
_site build directory, Gatsby can output compiled pages and then we just drop
them in the right location in the Jekyll source. E.g. for the about page, after
Gatsby builds it, we copy the about.html to the Jekyll source directory and then
the Jekyll build will pick it up.
Gatsby expects to own the entire website, so ...
Gatsby supports a path prefix which we use for the Federalist build previews. Gatsby expects to be the entire site.
This site is deployed on Federalist whenever a commit it pushed to GitHub. Changes are deployed automatically to the production site when commits are pushed to the master branch.
If deploying the site to a production environment, make sure to minify the JS files:
- Set the $NODE_ENV to
prod:export NODE_ENV=prod - Package js files with webpack:
webpack --watch - Re-run the web server:
bundle exec jekyll serve
We are currently moving our Fractal Styleguide to a Gatsby-based Pattern Library.
Any new components should be implemented in the Pattern Library. The Fractal Styleguide should be considered read-only and removed once content is migrated.
Setup the docker environment with the steps from above. Then run the styleguide container to start the styleguide server.
docker-compose up styleguideOpen your web browser to localhost:3000.
Setup the docker environment with the steps from above. Then run the styleguide container to start the styleguide server.
docker-compose up patternsOpen your web browser to localhost:8000.
The pattern library lives in /pattern-library, but the project-level
package.json contains npm scripts to run the pattern library.
Install the dependencies and do an initial build.
npm run release-patternsNow you can run the development server.
npm run patternsAnd open your web browser to localhost:8000.
The JavaScript tests currently cover all datasets. You can run them with [Node]:
npm install --dev
npm testWe have created a set of custom Jekyll filters that can be used for templating. The filters in plugins/eiti*.rb are tested with rubydoctest.
You can run the unit tests as follows:
npm test-rubyAs the following example demonstrates, test cases are written in comment blocks immediately preceding a testable function. The test description is on the first line, with an empty comment block below it. Use >> syntax to invoke a test case, and follow it with hash rocket syntax, =>, to define the expected outcome of the invocation.
Inline unit test for to_i:
# attempt to look up a term in a hash, and return the value if that
# key exists; otherwise, return the key
#
# >> EITI::Data.lookup('hi', {'hi' => 'hello'})
# => 'hello'
# >> EITI::Data.lookup('yo', {'hi' => 'hello'})
# => 'yo'
def lookup(term, hash)
hash.key?(term) ? hash[term] : term
endWe are using CircleCI to test our code as we push it to Github.
As specified in our circle.yml configuration file, we are running our JavaScript and Jekyll filter tests to ensure that functions are working as expected.
We use Hound CI to enforce SCSS and JavaScript formatting conventions on new commits. You can run both of the linters with:
npm run lintThis runs both of the linters below in series.
Hound uses jshint, which you can install as part of the
npm package's devDependencies with:
npm install --devOr you can install it globally with npm i -g jshint. Then, to lint the
JavaScript, run:
npm run lint-jsHound uses scss-lint, which you can
install with gem install scss_lint if you haven't already run bundle install to get Jekyll and its dependencies. To lint the SCSS files, run:
bundle exec scss-lint -c .scss-lint.ymlor simply:
npm run lint-scssWhen you open an issue, fill out all relevant fields in the issue template and include links to any prior related pull requests or issues.
We welcome contributions from the open source community! If you would like to contribute, please direct your pull requests to the contribution branch (instead of the default dev branch). This enables us to create a preview link for your contribution.
- Create pull requests for all commits, even typo fixes. This helps us track work and increase visibility into current work going on.
- When you open a pull request, complete the template to make sure reviewers have all the information they need.
- Assign reviewers to notify specific team members who should review a pull request.
- While working, submit
[WIP]pull requests liberally.
- Anyone may informally review a pull request and make comments or suggestions.
- For more about how to responsibly review pull requests, see How to review a PR
- Don’t merge your own pull request. Ask a colleague to review your code and merge. This helps ensure that at least two people have verified the quality of the code and content.
By submitting a pull request, you agree to comply with the policies on our LICENSE page:
This project is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the CC0 1.0 Universal public domain dedication.
All contributions to this project will be released under the CC0 dedication. By submitting a pull request, you are agreeing to comply with this waiver of copyright interest.