-
Notifications
You must be signed in to change notification settings - Fork 0
/
setup_server.txt
171 lines (117 loc) · 5.29 KB
/
setup_server.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
Installing on the server
Connect to server as ssh [email protected]
sudo apt update
sudo apt upgrade
sudo apt install make
Install Docker engine: (installed for ubuntu Lunar 23.04):
https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository
To install Docker, “sudo” was used, it would then be needed to run Docker with sudo.
As user bench, I don’t have the rights to use “sudo”. But
I connected as user “ubuntu”, but conbench needs to be installed as user “bench”
So user “bench” needs to be added to group docker, this way I don’t need to use “sudo”:
Do this as user ubuntu:
To see a list of groups:
cat /etc/group
To add “bench” user to group docker:
sudo adduser bench docker
Change user to “bench”
sudo su - bench
From now on, do everything as user “bench”:
mkdir conbench
cd conbench
Install mamba with: https://github.com/conda-forge/miniforge
curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
bash Miniforge3-$(uname)-$(uname -m).sh
(hit escape or q and then yes)
Exit bench user and login again with sudo su - bench (so mamba is activated)
Enter this so the environment base is not activated all the time you login:
conda config --set auto_activate_base false
Create environment:
mamba create -n conbench-env
mamba activate conbench-env
mamba install python
python -m pip install 'benchadapt@git+https://github.com/conbench/conbench.git@main#subdirectory=benchadapt/python'
python -m pip install 'benchalerts@git+https://github.com/conbench/conbench.git@main#subdirectory=benchalerts'
python -m pip install 'benchclients@git+https://github.com/conbench/conbench.git@main#subdirectory=benchclients/python'
python -m pip install 'benchconnect@git+https://github.com/conbench/conbench.git@main#subdirectory=benchconnect'
mamba install numpy python-dotenv pandas
environment is in:
/home/bench/miniforge3/conbench-env/
Clone my conbench version in branch “server”
git clone https://github.com/DeaMariaLeon/conbench.git
Or
From conbench repository, and then edit Makefile, docker-compose.yml and conbench/api/index.py
Edit Makefile in order to use port 5000 on server: USE BRANCH “server”
cd conbench/conbench (where the conbench clone is):
Go to line 13 an change
export DCOMP_CONBENCH_HOST_PORT=127.0.0.1:5000
to:
export DCOMP_CONBENCH_HOST_PORT=0.0.0.0:5000
Edit docker-compose.yml
APPLICATION_NAME: "pandas-conbench-PoC"
FLASK_ENV: ""
FLASK_DEBUG: "false"
REGISTRATION_KEY: “innocent-registration-key" CHANGE THIS
At the end of the file, in order to use volumes:
db:
image: library/postgres:15.2-alpine
volumes:
- db-data:/var/lib/postgresql/data
environment:
POSTGRES_DB: "postgres"
POSTGRES_USER: "postgres"
POSTGRES_PASSWORD: "postgres"
healthcheck:
test: [ "CMD-SHELL", "pg_isready -U postgres" ]
interval: 2s
timeout: 5s
retries: 5
ports:
- "127.0.0.1::5432"
volumes:
db-data:
Edit conbench/api/index.py line 209 so the database can’t be cleaned up:
#empty_db_tables()
Clone my repo conbench_toy
git clone https://github.com/DeaMariaLeon/conbench.git
ADD machine.json and benchmarks.json with its path, to this file:
asv_processed_files_server
Provide environment variables:
touch server_env.yml
Edit server_env.yml and set up these variables:
CONBENCH_URL=http://0.0.0.0:5000
CONBENCH_EMAIL=“set-your-email”
CONBENCH_PASSWORD=“set-your-password” -THIS IS MINE, when I sign up
CONBENCH_RUN_REASON=commit
PANDAS_ASV_RESULTS_PATH= Path to asv .json files
BENCHMARKS_FILE_PATH= Path to a file called benchmarks.json generated by asv
ASV_PROCESSED_FILES= Path and name to file that stores file names posted to web app
ALERT_PROCESSED_FILES= Path and name to file that stores file names analyzed by alert
[email protected]:pandas-dev/pandas
GITHUB_REPOSITORY=DeaMariaLeon/algos2 #temporary used for alerts
SLACK_TOKEN=
GMAIL_PASSWORD= Password of email account that sends alerts
GITHUB_APP_ID= Alerts app
GITHUB_APP_PRIVATE_KEY= For alerts app
Set the GITHUB_API_TOKEN:
Run export GITHUB_API_TOKEN="{token}" in your current shell.
Run the following inside conbench/conbench (The root of conbench clone):
Use nohup so it keeps running:
nohup command >/dev/null 2>&1 & # runs in background, still doesn't create nohup.
nohup make run-app 2>&1 & this is no hungup
You can see nohup.out to see the logs - look for 200, when this happens the conbench server should be running.
At that point open conbench server and create an account. Use the values of server_env.yml for that.
Once you have registered an account, you need to login.
Use this to:
make teardown-app - TO REMOVE CONTAINER
make run-app TO RUN - YOU NEED TO BE IN ~/conbench/conbench-clone/ THIS HUNGS UP ON YOU
Under subdirectory conbench/conbench_toy, make sure your environment is activated.
Run client.py with nohup:
nohup python3 client.py 2>&1 &
The last file should send all the asv results files to conbench server.
To run the alerts:
nohup python3 alerts.py
Make sure you have two files: alert_processed_files_server and asv_processed_files_server.
The web app, client.py and alerts.py run separately.
Set up cron job to run the alert every day at 11:45:
45 11 * * * cd /home/bench/conbench/conbench_alert/conbench_toy && ./run_alert.sh