-
Notifications
You must be signed in to change notification settings - Fork 3
Process for making changes
Changes should be made to the iFEED site by using the following tools:
- a conda installation to set up the environment
- a local hosting of the site for testing
- a personal fork of the repo
- a text editor (good editors for python include notepad ++, atom, and visual studio code)
The stages are:
- Set up a local version of the code for editting, and make changes
- Create a pull request
- Get the site updated with the help of UoL IT
These three stages have numerous substages, which will be explained below:
The workflow for editting a repo using github forks and branches has been recently described excellently in the cemac-generic wiki here. As such, it is recommended that this page be referenced while going through these instructions.
Instructions for making your own fork of a repo are given in the cemac-generic wiki here, so these instructions will be brief. The process for setting up your own fork of the repository and assigning the original repo to be upstream remain unchanged.
The important thing to note here however is that the new feature branch that you will be making once your fork is set up will be branched from the development branch. This can be done in two ways:
git checkout development
git branch my_feature_branch
git checkout my_feature_branchor, more concisely
git checkout -b my_feature_branch developmentOnce you have your feature branch set up on your local machine, you can set up your conda environment
WARNING - Development of the ifeed site was carried out on a linux machine, and so local running has only been partially tested on a windows machine. If using windows, you can use the anaconda navigator to build and manage the environment. An alternative is that you can also use WSL2 and install and run conda from there.
There are multiple guides for how to use anaconda to manage python environments, including from cemac-generic and the anaconda docs. As such, that information will not be replicated here. Neither will information about how to install anaconda on your machine, as that can be found on the anaconda download page.
The conda environment for iFEED can be found in the environment.yml file. It has been designed to be as portable as possible, and so doesn't assume any particular version number or operating system. On linux you would set up the environment with:
conda env create -f environment.ymland on windows you can use the anaconda navigator to import environment from the file.
To locally host the site, activate the iFEED conda environment and navigate to the repo directory. Before you will be able to run the site locally, you will need to edit the file Applications/Flask_App/flask_app.py, on the line where the parameter DATABASE is defined. When running on ifeed01, this file is expected to be found at /var/www/production/iFEED.db, however if running locally this file is instead found in the program root (ie current working directory).
A second difference you would need to make would be to download some of the contents of the b0113 data drive and create a symlink to it. If you are wanting to look at changes in the data exploration pages you will need some of the data to explore, and you may also need the documents directory. If this is needed it is recommended that you download the malawi data, as the malawi files are the smallest among the iFEED data files, and together with the docs directory only take up 4.7GiB. On a linux system, the commands to obtain data and create a symlink would be:
cd ${HOME} #or wherever you want to download the data to
rsync -aP <uol_username>@foe-linux.leeds.ac.uk:/nfs/b0113/Data/iFEED/docs ./data
rsync -aP <uol_username>@foe-linux.leeds.ac.uk:/nfs/b0113/Data/iFEED/malawi* ./data
cd /path/to/iFEED/repo
ln -s ~/data Applications/static/dataThis assumes you have set up SSH port forwarding to allow you to directly log in to the UoL system. Instructions exist on the cemac-generic wiki for how to set this up.
Once the data and documents have been obtained and the symlink created, you should be ready to run the site locally. This can be done by the following
conda activate iFEED # if not already done
cd /path/to/iFEED/repo
python testing.pyOnce this is done, you should be able to open your browser, enter 127.0.0.1:5000 in the address bar (this may be different on windows, but the address will be shown in the terminal as shown below)
Any changes made should be made on your branch, as detailed above. If you are unsure of where the text to make changes is in the repo, see the page in this wiki regarding the site map. Once your changes have been made and you have confirmed them to be working on your local version of the site, save them and commit your changes to your local repo by running
git add -u #will stage any changes to pre-existing files. If you have created a new file run `git add newfilename`
git commit -m "Some description of the changes made"
git push origin my-feature-branchNow that you have made a change to your copy of the code, this change needs to be brought in to the main repo. This can be done by making a pull request from your local copy of the code to cemac:development (note by default the pull request will go to the master branch). This is done through the github website, by going to the address https://github.com/your-github-username/iFEED_Prototype/pull/new/my-feature-branch. There are excellent instructions on how to do a pull request in a forked repo on the github docs
It is important to try to be descriptive with your pull request, stating what has been changed and why. Especially if there are any structural changes this could help inform the reviewer of any possible issues
It is generally advisable to have someone else review a pull request, especially if the changes you have made are programmatical rather than just textual. This is true even for those with admin rights to the upstream repo. Once the changes have been reviewed and confirmed to be functional, they can be merged into the upstream repo development branch. At this point, it is usually best practice to delete your feature branch in your fork, and run an upstream pull of the original repo, using
git checkout development
git branch -d my-feature-branch
git pull upstream development
git commit -m "Merged feature branch from upstream"
git push
Once this is done, your local and remote repos will be up to date
Getting the changes up on the website is a two-stage process. First the changes need to be confirmed as working on the development server, then they need to be brought in to the production server.
iFEED has a development server which was set up to be able to test changes to the iFEED site before they went live. The site on the development server can be seen by navigating to http://ifeed01.leeds.ac.uk from a computer on the UoLeeds network - it is not visible from outside the University.
The files for the site on the development server are on the ifeed01 machine (ssh ifeed01.leeds.ac.uk) at /var/www/development, and use the development branch. The files in this directory do not require root access, so you are able to run the command
git pullin that directory to update the files there.
The changes that you have made will not be visible on the internal iFEED site however until the httpd service has been restarted. This will cause a refresh of the website. This cannot be done by a user however, and must instead be carried out be a member of IT staff.
Once IT have restarted the httpd service, the changes should be visible on the internal site.
Once the changes have been confirmed as working on the ifeed01 setup, they can be merged into the main branch on the iFEED repo. This is done again through a pull request, either from development into main in the same repo or from your fork of the repo in the same way that the PR was made for the development branch.
After the changes are brought into the main branch of the upstream repo, the external site is ready to be updated. The production site files live on ifeed01 at /var/www/production, however root access is needed to be able to change these files. As such, IT will have to run the git pull from that directory to update the iFEED sute and then restart the httpd service again.
Once the httpd service has been restarted a second time, you should confirm again that the changes are working and that there have been no unintended side effects.
Putting in an incident ticket with the following information is generally sufficient, and has a lead time of around 48-72 hours:
Incident: Restart of httpd on ifeed01 needed
Codebase for ifeed01 development app on ifeed01.leeds.ac.uk has been modified. A restart of the httpd service is needed to make those changes live and visible on the internally visible version of the site.
Following this and after confirmation that the changes are working correctly on the University systems a git pull from /var/www/production and httpd restart will be needed to make these changes live on the externally visible site.
After the first response from IT, the turnaround time can be as little as minutes if everything works correctly, as once httpd has been restarted the member of IT staff handling your incident will ask for confirmation that everything is working, after which they will be able to do the git pull and second restart straight away. Putting in separate tickets for each of the two restarts will increase the lead time to around 1 working week.