Community server: Making a release

This page describes how to make a release of the Hackage server and how to update the instance running on http://hackage.haskell.org/.

Notes on the server setup

• Note: You must be part of the hackage and hackage-build groups. Ask Austin, Duncan or Herbert about this (or email [email protected].)

hackage.haskell.org is a Debian 7 machine. The server instance is predominately located in /srv/hackage. It's proxied by nginx on the frontend. Here's the directory overview:

/srv/hackage
 +-- bin/  # A set of bash scripts, which conveniently start the server binaries in lib/ with pre-configured arguments. 
 |         # bin/ itself is a symlink to /srv/hackage/git/private/bin
 +-- git/  # Hackage server source code, extra scripts
 |   +-- private/  # Some private scripts facilitating deployment
 |   +-- public/   # The public source code from http://github.com/haskell/hackage-server
 |                 # central-server branch, used for the live server's static data files.
 +-- lib/  # Library directory, containing the raw binary executables
 +-- log/  # Server, builder, and mirror log files
 +-- var/  # State for the hackage server, builder, and mirror

/etc/init.d/
 +-- hackage-server   # Init script for hackage-server process
 +-- hackage-mirror   # Init script for hackage-mirror process
 +-- hackage-builder  # Init script for hackage-builder process

The init scripts automatically run /srv/hackage/bin/hackage-{server,builder} for you, so there's no special configuration needed.

Updating the central-server branch

We use a special branch of the public git repo for the central hackage.haskell.org instance. We keep it as a public branch so that everyone can see and contribute to any bits that are specific to the central instance (like doc pages, special config etc).

So the first step is to bring the central-server branch up to date with the master branch. Get a checkout of the hackage-server git repo and merge from the master branch into the central-server branch. We try to avoid differences in the code, just the docs and other ancillary stuff, so there should not be any tricky conflicts. Please commit your merge change to the upstream repo, especially if you had to do any non-trivial merge.

Building the release

Of course best practice is to build on one machine and deploy to the server. You're welcome to do that if you have a matching system. Otherwise, you're welcome to build on the server. The instructions below are for building on the server.

The current version of ghc the on the server (in /usr/local/bin/) is 7.6.3. A recent version of cabal is also kept in /usr/local/bin/. If you need to update either/both of these then see below for additional important notes.

Get yourself a checkout of the central-server branch in your home dir on the server

cd ~/build/hackage-server  # or wherever you want build it
cabal update               # if necessary

nice cabal install --only-dependencies [--dry]
nice cabal configure --prefix=/srv/hackage --bindir=/srv/hackage/lib/
nice cabal build
cabal copy --destdir=./image/

Deploying the release

We currently deploy the binaries and the data files in different ways.

Currently this is not a very useful distinction since the server caches the the templates, but if we just add a feature to notify the server to re-read them then we could deploy updated data files without updating or restarting the server.

The /srv/hackage/var/server/datafiles is just a symlink to /srv/hackage/git/public/datafiles. So to update the data files, do a git pull in /srv/hackage/git/public (which is the the central-server branch of course).

cd /srv/hackage/git/public
git pull

For the binaries, we have a simple install script:

cd ~/build/hackage-server  # or wherever you were building it
/srv/hackage/git/private/scripts/install.sh

This will:

• Copy over the new binaries to /srv/hackage/lib.
• Stop hackage-build
• Restart hackage-server, wait 2min.
• Restart hackage-build

Which should perform a full upgrade.

The server will give a 503 page while its starting. If you're paranoid, you can follow the startup progress via tail -f /srv/hackage/log/server/log.

Testing

TODO

Starting a testing instance

To run on http://beta.hackage.haskell.org.

hackage-server run \
  --port=8082 \
  --ip=127.0.0.1 \
  --base-uri=http://hackage.haskell.org:82 \
  --state-dir=/home/hackage-test/state/ \
  --static-dir=/srv/hackage/var/server/datafiles \
  -v \
  --tmp-dir=/home/hackage-test/state/tmp

Uploading the package to Hackage

TODO

Tagging the release in Git

TODO