Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Install pipeline and tests #429

Merged
merged 143 commits into from
Oct 3, 2024
Merged

Install pipeline and tests #429

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
143 commits
Select commit Hold shift + click to select a range
514653d
Adds scripts to import and export 1.x data
cbaxley Aug 20, 2024
cbdbb85
Modifies the import script to use podman
cbaxley Aug 20, 2024
7d62873
Adds the dashboard importer for 1.x to 2.0
cbaxley Aug 20, 2024
943c6f2
Updates the import and export scripts to add mappings
cbaxley Aug 21, 2024
7b3fbca
Updates the field limit on winlogbeat index upon import
cbaxley Aug 21, 2024
bb8b5fa
Moves the upgrade scripts to a folder and requires directory on import
cbaxley Aug 23, 2024
02dc878
Adds ability to remove the old docker volumes
cbaxley Aug 23, 2024
bcabb10
Puts the volume remover in the upgrade directory
cbaxley Aug 23, 2024
4e7a0c5
Makes the volume remover executable
cbaxley Aug 23, 2024
e0e3c0f
2x readme
cbaxley Aug 26, 2024
97a04f7
Increase default maximum field limit
cbaxley Aug 27, 2024
002f526
Alter title of imported dashboards to indicate 1x import
cbaxley Aug 27, 2024
f27f0ed
Clarify some points in the upgrade readme
cbaxley Aug 28, 2024
6f634af
Save this intermediary version of the docker files
cbaxley Sep 3, 2024
f22b4ff
Updates docker and linux only workflow for 2.0
cbaxley Sep 4, 2024
2c5d6cd
Updates the paths for the linux only containers
cbaxley Sep 4, 2024
dfaea36
Fixes the clean up script for linux only build
cbaxley Sep 4, 2024
cbc0990
Get the logs from broken container
cbaxley Sep 4, 2024
04aecce
Use root for the docker-compose.yml file
cbaxley Sep 6, 2024
2c710d7
Use azure for our installs on pipeline
cbaxley Sep 9, 2024
be7006d
Installs python modules before azure install
cbaxley Sep 9, 2024
c584375
Pass azure env vars to the docker azure install script
cbaxley Sep 9, 2024
2470df3
Updates the paths to the installer variable files
cbaxley Sep 9, 2024
a72fb9b
Changes the paths for the environment vars for installer
cbaxley Sep 9, 2024
c4b47c5
Change the password argument for the installer
cbaxley Sep 9, 2024
730e4e5
Comment out group removal for debugging
cbaxley Sep 9, 2024
957b2be
Make sure the containers are using the same id
cbaxley Sep 9, 2024
940af4d
Add the resource group prefix to the environment variables files
cbaxley Sep 9, 2024
ed1d90b
Leave out special chars in password generation
cbaxley Sep 9, 2024
82ec99c
Put in a pause to wait for the linux machine to be ready
cbaxley Sep 9, 2024
92e4d1f
Increase azure test machine size
cbaxley Sep 10, 2024
9ab9582
Speeds up pipeline docker creation
cbaxley Sep 10, 2024
9d90f26
Add sshpass to the apt packages in the Dockerfile
cbaxley Sep 11, 2024
950b071
Sleep after making ssh key
cbaxley Sep 11, 2024
6471a85
Show output of generating key
cbaxley Sep 11, 2024
93af11e
Adds the openssh-client to the doccker build
cbaxley Sep 11, 2024
6a61550
Run the tests remotely
cbaxley Sep 11, 2024
3c45432
Quote ssh commands and escape environment vars $
cbaxley Sep 11, 2024
07fd814
Install chromium for tests
cbaxley Sep 11, 2024
03ff246
Separate installing requirements from the test step
cbaxley Sep 11, 2024
b42e93c
Change default variables for tests
cbaxley Sep 11, 2024
450cff6
Skip the tests that don't work with 2.0
cbaxley Sep 12, 2024
e98d40d
Clean up azure resources when pipeline is done
cbaxley Sep 13, 2024
80e85fe
Update the cluster build to use the new installers
cbaxley Sep 13, 2024
4f389d9
Update unique id and branch name
cbaxley Sep 13, 2024
7ca105b
Check permissions on folder for config files
cbaxley Sep 13, 2024
94196ba
Rebuild container with correct uid
cbaxley Sep 13, 2024
3c32617
Check if directories are writable
cbaxley Sep 13, 2024
2141ceb
Puts the env file in the proper directory
cbaxley Sep 13, 2024
0aae9ae
Skips data insertion example tests
cbaxley Sep 16, 2024
26a791c
Change the default password for selenium tests
cbaxley Sep 16, 2024
d2d93a4
Skip selenium tests that point to old dashboards
cbaxley Sep 16, 2024
6e563a9
Skip failing tests
cbaxley Sep 16, 2024
b30b9d7
Install minimega
cbaxley Sep 16, 2024
2f24407
See if selenium tests pass without minimega
cbaxley Sep 16, 2024
5c5bfe4
Skipped failing test.
cbaxley Sep 16, 2024
7252e8b
Install linux in minimega
cbaxley Sep 17, 2024
38e9f07
Quote minimega arguments correctly
cbaxley Sep 17, 2024
b182d8e
Runs minimega as root
cbaxley Sep 17, 2024
9168831
Provide full path to minimega
cbaxley Sep 17, 2024
4d97d68
Runs minimega on the remote machine
cbaxley Sep 17, 2024
6064655
Remove the local call to minimega
cbaxley Sep 17, 2024
2d4bbc7
Get the azure and minimega ips in a variable for gh actions
cbaxley Sep 17, 2024
04b27de
Better method to get the minimega IP
cbaxley Sep 17, 2024
3d59ae9
Escape the arguments to getting the ip on minimega
cbaxley Sep 17, 2024
91b6fe0
Attempt escaping again
cbaxley Sep 17, 2024
c98ced4
Get ip of the linux vm using lib function
cbaxley Sep 18, 2024
da46470
Updates development files and workflows for the pipeline
cbaxley Sep 18, 2024
cd3e39a
Fail if the minimega ip isn't found
cbaxley Sep 18, 2024
9e9a774
Increase the size of the cluster azure instance
cbaxley Sep 18, 2024
48e5f81
Check if tests pass without minimega
cbaxley Sep 18, 2024
5a68ce8
Install minimega first because it restarts machine
cbaxley Sep 18, 2024
0041e3d
Uses the machine name of a running vm
cbaxley Sep 18, 2024
6fe7559
Output the reason for not getting the minimega ip
cbaxley Sep 18, 2024
8be34e0
Escapes the azure ip $ sign
cbaxley Sep 18, 2024
ef0f833
Checking the ssh command
cbaxley Sep 19, 2024
57f0f97
Echo IP early
cbaxley Sep 19, 2024
8e224f4
Gets the ip for minimega and doesn't check errors
cbaxley Sep 19, 2024
c701915
Get the vm info for the vm in minimega
cbaxley Sep 19, 2024
461aaed
Filter the ip outside of the remote command
cbaxley Sep 19, 2024
11a2762
Filter the ip inside of the ssh command
cbaxley Sep 19, 2024
235c021
Use single quotes to quote the jq query
cbaxley Sep 19, 2024
d98fbe6
Waits for an ip to be assigned to the minimega vm
cbaxley Sep 19, 2024
3b2a046
Get the policy and token for elastic agent
cbaxley Sep 19, 2024
cb28cae
Retrieve token after installing LME
cbaxley Sep 19, 2024
07bb391
Wait for the services to come up before running set-fleet
cbaxley Sep 20, 2024
8583eea
Put the check service command in the ssh command
cbaxley Sep 20, 2024
584baab
Run set fleet as sudo because it has podman available
cbaxley Sep 20, 2024
c0ff23b
Source bashrc for podman path
cbaxley Sep 20, 2024
924e5df
Try getting path to podman
cbaxley Sep 20, 2024
f963ec6
Echo path variable
cbaxley Sep 20, 2024
cb8e0cc
Check for podman path
cbaxley Sep 20, 2024
3700d41
Merge branch 'release-2.0.0' into cbaxley-390-install-pipeline
cbaxley Sep 20, 2024
5254b52
Put in absolute path to podman
cbaxley Sep 20, 2024
89aa8ba
Remove install fleet
cbaxley Sep 20, 2024
e361572
Attempt running set fleet in the pipeline
cbaxley Sep 23, 2024
e1a1ddd
Fix the typo in the pipeline docker build
cbaxley Sep 23, 2024
84140bf
Turn on debug for set fleet
cbaxley Sep 23, 2024
2be8d8d
Add a script to check the variables and results of set fleet
cbaxley Sep 23, 2024
3ed8e17
Run the check fleet script before installing
cbaxley Sep 23, 2024
224d40d
Update the fleet check script
cbaxley Sep 23, 2024
790dab5
Print debug info from kibana
cbaxley Sep 23, 2024
c9feaac
Prints out the fleet api response.
cbaxley Sep 24, 2024
b2e0aa8
Waits for fleet to be ready
cbaxley Sep 24, 2024
0937ccd
Turn off debugging for the fleet installation scripts
cbaxley Sep 24, 2024
dbd7f1d
Take out some debugging and sleeps
cbaxley Sep 24, 2024
a0536a1
Run a command in a minimage virtual machine
cbaxley Sep 24, 2024
bcf080a
Ssh to the virtual machine using non root
cbaxley Sep 24, 2024
75da2d2
Use the env vars to connect to the ssh instances
cbaxley Sep 24, 2024
2180d24
Ignore strict host checking in ssh
cbaxley Sep 24, 2024
807ee70
Don't shut down instance so we can debug
cbaxley Sep 24, 2024
fcd27a5
Test running sudo in minimega virtual machine
cbaxley Sep 24, 2024
a350ea3
Have pipeline ignore the certs when getting token and policy
cbaxley Sep 25, 2024
d602aaf
Use unique container names
cbaxley Sep 25, 2024
99e2118
Try running in a different azure zone
cbaxley Sep 25, 2024
064d3a3
Updates the ip in the config file
cbaxley Sep 25, 2024
e5b672f
Fix the password for azure machine
cbaxley Sep 25, 2024
642d857
Sleep a little after azure machine creation
cbaxley Sep 25, 2024
e7cc788
Keeps azure resources in place after pipeline run
cbaxley Sep 27, 2024
ec1b246
Fix yaml error in workflow file
cbaxley Sep 27, 2024
9178320
Fix error in cluster.yml
cbaxley Sep 27, 2024
2a08250
Echo enrollment token for debugging
cbaxley Sep 27, 2024
421ed61
Repllace the vars in the config file for the local IP
cbaxley Sep 27, 2024
0a720ef
Copy the install_agent_linux.sh script to Minimega
cbaxley Sep 27, 2024
ca60a0e
Test the install_agent_linux.sh script in Minimega
cbaxley Sep 27, 2024
9432ece
Try running the Elastic Agent installer in Minimega
cbaxley Sep 27, 2024
521117d
Make the install_agent_linux.sh script executable and run it in Minimega
cbaxley Sep 27, 2024
a2c25a3
Run the chmod and install_agent_linux.sh script in separate steps
cbaxley Sep 27, 2024
73746c2
Run the agent installer with automatic "yes" response
cbaxley Sep 27, 2024
04baa50
Quiet the untarring command
cbaxley Oct 1, 2024
2cad197
Reduce logging for pulling the elastic agent
cbaxley Oct 1, 2024
6477c99
Pass the enrollment token to the agent installer
cbaxley Oct 1, 2024
d0a4953
Try enrolling after installation
cbaxley Oct 1, 2024
c0e1d8c
Allow insecure enrollment
cbaxley Oct 1, 2024
ada485a
Start the agent from /opt and restart the service after enrolling
cbaxley Oct 1, 2024
c7147f2
Run enroller non interactively
cbaxley Oct 1, 2024
2d4b478
Force enrollment
cbaxley Oct 1, 2024
a00a5e1
Build the entire run again to test manually
cbaxley Oct 2, 2024
560941b
Checks if the elastic agent is reporting
cbaxley Oct 2, 2024
5b5bd3b
Sleep a little while waiting for results from agent
cbaxley Oct 2, 2024
9b07925
Try to separate out installation, config, and enrollment of agent
cbaxley Oct 2, 2024
b9a68cf
No need to run config. Enroll will do it
cbaxley Oct 2, 2024
1837591
Clean up the azure resources after the run
cbaxley Oct 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .devcontainer/python_development/devcontainer.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
{
"name": "Python Development",
"dockerComposeFile": [
"../../testing/development/docker-compose.yml"
"../../testing/v2/development/docker-compose.yml"
],
"service": "ubuntu",
"shutdownAction": "none",
"workspaceFolder": "/lme",
"workspaceFolder": "/root/LME",
"customizations": {
"vscode": {
"extensions": [
Expand All @@ -15,5 +15,5 @@
]
}
},
"remoteUser": "admin.ackbar"
"remoteUser": "root"
}
18 changes: 0 additions & 18 deletions .devcontainer/python_tests/devcontainer.json
View file
Open in desktop

This file was deleted.

Loading
Loading