Rework GCE sandbox setup and e2e execution #56
Conversation
|
/hold Not ready yet. |
|
FYI, here is how I launch this to test it (you can get rid of the $ gcloud compute instances delete -q nephio-r1-e2e && \
gcloud compute instances create \
--machine-type e2-standard-8 \
--boot-disk-size 200GB \
--image-family=ubuntu-2204-lts \
--image-project=ubuntu-os-cloud \
--metadata=startup-script-url=https://raw.githubusercontent.com/johnbelamaric/nephio-test-infra/update-packages/e2e/provision/gce_init.sh,nephio-setup-debug=true,nephio-test-infra-repo=https://github.com/johnbelamaric/nephio-test-infra.git,nephio-test-infra-branch=update-packages \
nephio-r1-e2e && \
sleep 30 && \
gcloud compute ssh nephio-r1-e2e -- \
-o ProxyCommand='corp-ssh-helper %h %p' \
sudo journalctl -u google-startup-scripts.service --follow |
|
Issues - some fixed / worked around, some not yet:
Follow up items:
|
e2e/provision/gce_install_sandbox.sh
Outdated
| # I don't know how to make ansible do what I want, this is what I want | ||
| mkdir /tmp/mgmt-repo | ||
| /usr/local/bin/kpt pkg get --for-deployment https://github.com/nephio-project/nephio-example-packages.git/repository@repository/v2 /tmp/mgmt-repo/mgmt | ||
| # sudo because docker | ||
| sudo /usr/local/bin/kpt fn render /tmp/mgmt-repo/mgmt | ||
| /usr/local/bin/kpt live init /tmp/mgmt-repo/mgmt | ||
| /usr/local/bin/kpt live apply /tmp/mgmt-repo/mgmt | ||
|
|
||
| mkdir /tmp/mgmt-rootsync | ||
| /usr/local/bin/kpt pkg get --for-deployment https://github.com/nephio-project/nephio-example-packages.git/rootsync@rootsync/v2 /tmp/mgmt-rootsync/mgmt | ||
| # sudo because docker | ||
| sudo /usr/local/bin/kpt fn render /tmp/mgmt-rootsync/mgmt | ||
| /usr/local/bin/kpt live init /tmp/mgmt-rootsync/mgmt | ||
| /usr/local/bin/kpt live apply /tmp/mgmt-rootsync/mgmt |
There was a problem hiding this comment.
Those instructions seems to belong to the workload process, right?
So if that's the case the can be included as a pre_task here
- role: kpt
repo_uri: https://github.com/nephio-project/nephio-example-packages
local_dest_directory: /tmp/mgmt-repo/mgmt
pkg: repository
version: repository/v2
context: kind-kind
namespaces:
- defaultThat role will execute the following instructions on the package:
- creates the base folder if doesn't exists (`/tmp/mgmt-repo/mgmt)
- kpt pkg get (If it hasn't done)
- kpt pkg tree
- kpt fn render
- kpt pkg diff
- kpt live init
- kpt live apply (5 attempts)
There was a problem hiding this comment.
thanks, I tried that but it didn't work exactly like I wanted:
- We probably need to update that to use
--for-deployment - I need the package name to be "mgmt", so I need the actual kpt command that runs to be the one in the script - I think it did something a little different, I ended up with /tmp/mgmt-repo/mgmt/repository instead of /tmp/mgmt-repo/mgmt. The idea is that we rename the package during the "get" - that rename actually is meaningful, it triggers some automation.
| ## Licensed under the Apache License 2.0 | ||
| ## SPDX-License-Identifier: Apache-2.0 | ||
| --- | ||
| - name: systctl config |
There was a problem hiding this comment.
It seems like there is an ansible module for managing sysctl entries, so manybe this can be implemented like
- name: Set kernel parameters
ansible.posix.sysctl:
name: {{ item.name }}"
value: "{{ item.value }}"
state: present
loop:
- {name: fs.inotify.max_user_watches, value: 524288}
- {name: fs.inotify.max_user_instances, value:512}
- {name: kernel.keys.maxkeys, value:500000 }
- {name: kernel.keys.maxbytes, value:1000000}| - {pkg: porch-dev, version: porch-dev/v1, namespaces: [porch-fn-system, porch-system]} | ||
| - {pkg: nephio-controllers, version: nephio-controllers/v1, namespaces: [nephio-system]} | ||
| - {pkg: configsync, version: configsync/v1, namespaces: [config-management-monitoring, config-management-system, resource-group-system]} |
There was a problem hiding this comment.
I was planning to extract some of this values as configurable parameters. I did some of that work in this PR, more likely for R1+
There was a problem hiding this comment.
ok. I want to split out the "base install" vs the "sandbox install". I think it's close now but now quite there.
- Adds MetalLB base installation to management cluster - Adds Sandbox config for Metal LB to management cluster - Adds updated Gitea installation package with LB service - Adds Gitea secret in nephio-system namespace
Also updated the README
|
/hold cancel Ok, I think this is ready to merge. The comment above does include some additional follow up items. But those should become separate issues. Note that there is definitely flakiness. Some clusters may not fully come up or all the packages may not all auto approve. These are issues with the underlying subsystems that we need to fix, but they shouldn't hold up this PR. You can test this by following the README, except that you have to use this PR as the startup-script-url, as described in #56 (comment). |
|
/approve This will need some changes in |
|
/lgtm |
|
I was able to install using this method. A few things to consider.
@vjayaramrh you asked for things to check once this is completed. Here are some commands I run. ubuntu@nephio-r1-e2e: |
|
/approve |
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: electrocucaracha, henderiw, johnbelamaric, radoslawc, vjayaramrh The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
No description provided.