openshift / assisted-test-infra Goto Github PK

+1
Isn't managing it with TF just make it more complex?

Originally posted by @YuviGold in #928 (comment)

This is the way we can cleanup after use.

The following branches are being fast-forwarded from the current development branch (master) as placeholders for future releases. No merging is allowed into these release branches until they are unfrozen for production release

• release-ocm-2.11

When executing the make command as an user with sudo privileges or via 'sudo make' the installation fails.

Executing with root, allows installation to continue, but the installer itself (when it reaches to pip commands), warns about NOT using pip for installing packages as root.

make should succeed when executed with a privileged user and not require using 'root' directly

Manifests API was introduced by openshift/assisted-service#215.
The user can add manifests to the cluster to be considered during installation phase.

test-infra can be extended to support accepting manifests from the user and add them to the cluster after the cluster was created.
A new parameter MANIFESTS_FOLDER can be introduced. It will specify the path to the folder that contains the manifests the user would like to include during cluster installation.

MANIFESTS_FOLDER will expect the manifests files to be stored under 'manifests' or 'openshift' folders:

MANIFESTS_FOLDER
      |
      +---- manifests/
      +---- openshift/

The 'create manifest' https://github.com/openshift/assisted-service/blob/master/swagger.yaml#L771 expects three parameters (in addition to the cluster) https://github.com/openshift/assisted-service/blob/master/swagger.yaml#L3302:

1. folder - will be determined based on the folder in which the files will be placed
2. file_name - will be the same as the file name
3. content - base64 encoded manifest content

The test-infra will iterate over the files in MANIFESTS_FOLDER/manifests/' and MANIFESTS_FOLDER/openshift/'
and for each will read the file, encode to base64 the content and will use the 'create manifest' API to add the manifest to the cluster.

Facing with a scenario where a fresh node with RHEL 8/Centos8 gives a successful install, but after I issue make destroy and try a re-installation cluster on the same system doesn't come up.

Underlying OS

cat /etc/os-release 
NAME="CentOS Linux"
VERSION="8 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Linux 8 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

What is the issue?

• Newly bought up OS brings up the assisted-installer cluster with VM nodes perfectly

• Cleaning up this cluster using make destroy

• Issuing a new make all command after cleanup fails the installation with

  File "./tools/deploy_assisted_installer_configmap.py", line 36, in main
    service_host = utils.get_service_host(SERVICE, deploy_options.target)
  File "/root/assisted-test-infra/bm-inventory/tools/utils.py", line 19, in get_service_host
    reply = check_output("{} service --url {}".format(MINIKUBE_CMD, service))
  File "/root/assisted-test-infra/bm-inventory/tools/utils.py", line 14, in check_output
    return subprocess.check_output(cmd, shell=True).decode("utf-8")
  File "/usr/lib64/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/lib64/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'minikube -n assisted-installer service --url bm-inventory' returned non-zero exit status 70.
make: *** [Makefile:115: deploy-service-requirements] Error 1
make: Leaving directory '/root/assisted-test-infra/bm-inventory'
make[1]: *** [Makefile:177: deploy_bm_inventory] Error 2
make[1]: Leaving directory '/root/assisted-test-infra'
retVal
make: *** [Makefile:55: all] Error 2

I will be happy to provide any other logs if needed.

The following branches are being fast-forwarded from the current development branch (master) as placeholders for future releases. No merging is allowed into these release branches until they are unfrozen for production release.

• release-ocm-2.6

The following branches are being fast-forwarded from the current development branch (master) as placeholders for future releases. No merging is allowed into these release branches until they are unfrozen for production release.

• release-ocm-2.7

The following branches are being fast-forwarded from the current development branch (master) as placeholders for future releases. No merging is allowed into these release branches until they are unfrozen for production release.

• release-ocm-2.8

The following branches are being fast-forwarded from the current development branch (master) as placeholders for future releases. No merging is allowed into these release branches until they are unfrozen for production release

• release-ocm-2.10

The following branches are being fast-forwarded from the current development branch (master) as placeholders for future releases. No merging is allowed into these release branches until they are unfrozen for production release

• release-ocm-2.9

Traceback while running make execution

sleep 5;  # wait for service to get an address
python3 ./tools/deploy_assisted_installer_configmap.py --target "minikube" --domain "" \
        --base-dns-domains """" --namespace "assisted-installer" --profile "minikube" \
          --enable-auth "false"  \
        --ocp-versions '{\"4.6\":{\"display_name\":\"4.6.16\",\"release_image\":\"quay.io/openshift-release-dev/ocp-release:4.6.16-x86_64\",\"rhcos_image\":\"https://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/4.6/4.6.8/rhcos-4.6.8-x86_64-live.x86_64.iso\",\"rhcos_version\":\"46.82.202012051820-0\",\"support_level\":\"production\"},\"4.7\":{\"display_name\":\"4.7-pre-release\",\"release_image\":\"quay.io/openshift-release-dev/ocp-release:4.7.0-rc.0-x86_64\",\"rhcos_image\":\"https://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/pre-release/4.7.0-fc.4/rhcos-4.7.0-fc.4-x86_64-live.x86_64.iso\",\"rhcos_version\":\"47.83.202101171239-0\",\"support_level\":\"beta\"}}' --public-registries "quay.io" \
        --check-cvo False  
error: unable to load --registry-config: error occurred while trying to unmarshal json
Traceback (most recent call last):
  File "./tools/deploy_assisted_installer_configmap.py", line 124, in <module>
    main()
  File "./tools/deploy_assisted_installer_configmap.py", line 46, in main
    verify_ocp_versions(json.loads(json.loads('"{}"'.format(deploy_options.ocp_versions))))
  File "/home/kni/assisted_installer/assisted-service/tools/handle_ocp_versions.py", line 59, in verify_ocp_versions
    verify_image_version(key, metadata["release_image"])
  File "/home/kni/assisted_installer/assisted-service/tools/handle_ocp_versions.py", line 67, in verify_image_version
    major, minor, *_other_version_components = get_oc_version(release_image).split(".")
  File "/home/kni/assisted_installer/assisted-service/tools/handle_ocp_versions.py", line 86, in get_oc_version
    f"oc adm release info '{release_image}' {registry_config} -o template --template {{{{.metadata.version}}}}")
  File "/home/kni/assisted_installer/assisted-service/tools/utils.py", line 65, in check_output
    return subprocess.check_output(cmd, shell=True).decode("utf-8")
  File "/usr/lib64/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/lib64/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'oc adm release info 'quay.io/openshift-release-dev/ocp-release:4.6.16-x86_64' --registry-config '/tmp/tmp2dly3jm9' -o template --template {{.metadata.version}}' returned non-zero exit status 1.
make: *** [Makefile:191: deploy-service-requirements] Error 1
make: Leaving directory '/home/kni/assisted_installer/assisted-service'
make: *** [Makefile:327: deploy_assisted_service] Error 2

Hi,

Would you have any examples of using
the podman-hosted OCP assisted installer to redeploy OCP
on an existing set of baremetal servers
without using IPMI?

https://github.com/openshift/assisted-test-infra/tree/master/ansible-bm-install supports
systems with an existing OS with ssh access.

In our case, that'd be CoreOS since we are redeploying on baremetal servers
that are currently running an older OCP version.

Currently stuck at:
TASK [okd.assisted_installer.boot_iso_existing_os : Mount and Extract vmlinuz,initramfs] ******************************************************************************
fatal: [worker0.ocp4....]: FAILED! => {"changed": false, "cmd": "mount -o loop,ro "/tmp/.../installer-image.iso" "/tmp/tmpvlatfz2e"", "dest": "/tmp/...", "files": [], "gid": 0, "group": "root", "image": "/tmp/.../installer-image.iso", "mode": "0755", "msg": "Failed to mount ISO image '/tmp/.../installer-image.iso' to '/tmp/tmpvlatfz2e', and we could not find executable '7z'.", "owner": "root", "rc": 1, "secontext": "unconfined_u:object_r:user_tmp_t:s0", "size": 6, "state": "directory", "stderr": "mount: only root can use "--options" option\n", "stderr_lines": ["mount: only root can use "--options" option"], "stdout": "", "stdout_lines": [], "uid": 0}

Facing this issue deploying Assisted Service from this repo and using the default vars for all:

[root@devscripts2ipv6 assisted-test-infra]# make run
scripts/utils.sh validate_namespace assisted-installer
scripts/run_minikube.sh
Configuring minikube...
😄  minikube v1.11.0 on Redhat 8.2
    ▪ KUBECONFIG=/root/.kube/config
✨  Using the kvm2 driver based on user configuration
🛑  The "kvm2" driver should not be used with root privileges.
💡  If you are running minikube within a VM, consider using --driver=none:
📘    https://minikube.sigs.k8s.io/docs/reference/drivers/none/
👍  Starting control plane node minikube in cluster minikube
🔥  Creating kvm2 VM (CPUs=4, Memory=8192MB, Disk=20000MB) ...
🐳  Preparing Kubernetes v1.18.3 on Docker 19.03.8 ...
❗  This VM is having trouble accessing https://k8s.gcr.io
💡  To pull new external images, you may need to configure a proxy: https://minikube.sigs.k8s.io/docs/reference/networking/proxy/
🔎  Verifying Kubernetes components...
🌟  Enabled addons: default-storageclass, storage-provisioner
🏄  Done! kubectl is now configured to use "minikube"
eval 
Cloning into 'assisted-service'...
remote: Enumerating objects: 10514, done.
remote: Total 10514 (delta 0), reused 0 (delta 0), pack-reused 10514
Receiving objects: 100% (10514/10514), 20.28 MiB | 27.95 MiB/s, done.
Resolving deltas: 100% (7698/7698), done.
[skipper] Using build container: test-infra:latest
mkdir -p assisted-service/build
 0 DEPLOY_MANIFEST_PATH="" DEPLOY_MANIFEST_TAG="" scripts/deploy_assisted_service.shectory
/bin/sh: cannot: command not found
make: *** [Makefile:225: deploy_assisted_service] Error 127

The repository is fully updated.

The underneth issue is that the script scripts/deploy_assisted_service.sh is not inherit the NAMESPACE_INDEX variable from the Makefile

The following branches are being fast-forwarded from the current development branch (master) as placeholders for future releases. No merging is allowed into these release branches until they are unfrozen for production release.

• release-4.8
• release-4.7

Contact the Test Platform or Automated Release teams for more information.

At:

The script tries to create a file using sudo, but only cat runs as root, redirection occurs as user running the script.
sudo tee works better, I'll test and create PR.