Add upgrade test case from 1.5.7 (#2603)

tests/manual-test-cases/Group7-Upgrade/7-12-Upgrade-1.5.7.md

@@ -0,0 +1,25 @@
+Test 7-12 - Upgrade 1.5.7
+=======
+
+# Purpose:
+To verify the VIC OVA appliance v1.5.7 upgrades to latest with auto upgrade process works as expected
+
+# References:
+
+# Environment:
+This test requires access to VMWare Nimbus cluster for dynamic ESXi and vCenter creation
+
+# Test Steps:
+1. Deploy a new vCenter in Nimbus that is a simple VC cluster
+2. Install an older version of the VIC OVA appliance
+3. Walk through completing the install and use the VCH creation wizard to create a VCH
+4. Run a variety of docker commands on the VCH appliance
+5. Install the latest version of the VIC OVA appliance
+6. Execute the upgrade script pointing at the old version of the VIC OVA appliance
+7. Walk through completing the install
+8. Run a variety of docker commands on the previously created VCH
+
+# Expected Outcome:
+The VCH and VIC appliance upgrade should succeed without error and each of the docker commands executed against it should return without error
+
+# Possible Problems:

tests/manual-test-cases/Group7-Upgrade/7-12-Upgrade-1.5.7.robot

@@ -0,0 +1,40 @@
+# Copyright 2020 VMware, Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#	http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License
+
+*** Settings ***
+Documentation  Test 7-12 - Upgrade 1.5.7
+Resource  ../../resources/Util.robot
+Suite Setup  Nimbus Suite Setup  OVA Upgrade Setup
+Suite Teardown  Run Keyword And Ignore Error  Nimbus Cleanup  ${list}
+Test Teardown  Run Keyword If  '${TEST STATUS}' != 'PASS'  Copy Support Bundle  %{OVA_IP}
+
+*** Variables ***
+${old-ova-file-name}=  vic-v1.5.7-7923-f6c04457.ova
+${old-ova-version}=  v1.5.7
+${old-ova-cert-path}=  /storage/data/admiral/ca_download
+${new-ova-cert-path}=  /storage/data/admiral/ca_download
+
+*** Keywords ***
+OVA Upgrade Setup
+    Setup Simple VC And Test Environment with Shared iSCSI Storage
+
+*** Test Cases ***
+Upgrade OVA 1.5.7
+    Auto Upgrade OVA With Verification  7-12-UPGRADE-1-5-7  ${old-ova-file-name}  ${old-ova-version}  ${old-ova-cert-path}  ${new-ova-cert-path}  dc1
+    ${rc}  ${output}=  Run And Return Rc And Output  govc about -u=%{TEST_URL}
+    Log  ${output}
+    Should Be Equal As Integers  ${rc}  0
+    ${status}=  Run Keyword And Return Status  Should Contain  ${output}  6.0
+    Run Keyword Unless  ${status}  Stop All Containers
+    Run Keyword Unless  ${status}  Delete All VCH Using UI

tests/manual-test-cases/Group7-Upgrade/TestCases.md

@@ -23,3 +23,5 @@ Group 7 - OVA Upgrade
 -
 [Test 7-11 - Upgrade 1.5.5](7-11-Upgrade-1.5.5.md)
 -
+[Test 7-12 - Upgrade 1.5.7](7-12-Upgrade-1.5.7.md)
+-

tests/manual-test-cases/Group8-Manual-Upgrade/8-12-Manual-Upgrade-1.5.7.md

@@ -0,0 +1,29 @@
+Test 8-12 - Manual Upgrade 1.5.7
+=======
+
+# Purpose:
+To verify the VIC OVA appliance works after upgrading from 1.5.7
+
+# References:
+[VIC appliance design
+document](https://github.com/vmware/vic-product/blob/master/installer/docs/DESIGN.md)
+
+# Environment:
+This test requires access to VMWare Nimbus cluster for dynamic ESXi and vCenter creation
+
+# Test Cases
+
+### Test Steps:
+1. Deploy and initialize a VIC appliance version 1.5.7
+2. Creat a VCH, running container and push an image to harbor
+3. Deploy a current VIC appliance version. Do NOT power on.
+4. Follow instructions for manually moving or copying `/storage/data`, `/storage/log`, and
+   `/storage/db` disks and adding them to current appliance.
+5. Power on the current appliance, but do NOT initialize it.
+6. Run the appliance upgrade script with `--manual-disks` flag
+
+### Expected Outcome:
+
+- Upgrade script completed successfully
+- Verify container created in step 2 is still available and running
+- Verify image pushed in step 2 can be pulled from harbor

tests/manual-test-cases/Group8-Manual-Upgrade/8-12-Manual-Upgrade-1.5.7.robot

@@ -0,0 +1,51 @@
+# Copyright 2020 VMware, Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License
+
+*** Settings ***
+Documentation  Test 8-12 - Manual Upgrade 1.5.7
+Resource  ../../resources/Util.robot
+Suite Setup  Nimbus Suite Setup  Test Environment Setup
+Suite Teardown  Run Keyword And Ignore Error  Nimbus Cleanup  ${list}
+Test Teardown  Run Keyword If  '${TEST STATUS}' != 'PASS'  Copy Support Bundle  %{OVA_IP}
+
+*** Variables ***
+${datacenter}=  ha-datacenter
+${busybox}=  busybox
+${sample-image-tag}=  test
+
+*** Keywords ***
+Test Environment Setup
+    Setup Simple VC And Test Environment with Shared iSCSI Storage
+    # Used by Install VIC Appliance Secret keyword
+    Set Global Variable       ${OVA_USERNAME_ROOT}  root
+    Set Global Variable       ${OVA_PASSWORD_ROOT}  e2eFunctionalTest
+
+*** Test Cases ***
+Upgrade from v1.5.7
+    ${old-ova-file-name}=        Set Variable  vic-v1.5.7-7923-f6c04457.ova
+    ${old-ova-version}=          Set Variable  v1.5.7
+    ${old-appliance-name}=       Set Variable  manual-upgrade-${old-ova-file-name}
+    ${new-appliance-name}=       Set Variable  manual-upgrade-from-1.5.7-LATEST
+    ${old-appliance-cert-path}=  Set Variable  /storage/data/admiral/ca_download
+    ${new-appliance-cert-path}=  Set Variable  /storage/data/admiral/ca_download
+
+    Set Global Variable  ${OVA_CERT_PATH}  ${old-appliance-cert-path}
+    # Deploy old ova, install vch, create container, push an image to harbor and deploy new appliance
+    Manual Upgrade Environment Setup  ${old-ova-file-name}  ${old-appliance-name}  ${new-appliance-name}
+    # Copy data disk and attach to new appliance
+    Copy and Attach Disk  ${old-appliance-name}  ${new-appliance-name}  ${datacenter}
+    # Power on new appliance and run upgrade script
+    Power On Appliance And Run Manual Disk Upgrade  ${new-appliance-name}  %{OLD_OVA_IP}  ${old-ova-version}  ${datacenter}
+    # verify container and image in harbor
+    Verify Running Busybox Container And Its Pushed Harbor Image  %{OVA_IP}  ${sample-image-tag}  ${new-appliance-cert-path}  docker-endpoint=${VCH-PARAMS}

tests/manual-test-cases/Group8-Manual-Upgrade/TestCases.md

@@ -23,3 +23,5 @@ Group 8 - Manual Upgrade
 -
 [Test 8-11 - Manual Upgrade 1.5.5](8-11-Manual-Upgrade-1.5.5.md)
 -
+[Test 8-12 - Manual Upgrade 1.5.7](8-12-Manual-Upgrade-1.5.7.md)
+-