I would like to add a way to control the debugging level of the acceptance tests.
This will avoid having other people see a bunch of cryptic shell printouts, but allow developers of tests to debug them.

I propose keeping in line with using environment variables and add the variable ROBOT_DEBUG_LEVEL which would have the following levels:

0 - None -> no printouts except errors
1 - Low -> some informative traces for users of the acceptance-testing repo
2 - Medium -> what we have now
3 - High -> set -x for everything + any python debug printouts

The CI tests have started to to fail during the setup phase with the following error.
Refs:
#77
#78

which virtualenv
+ true
+ pip3 install --user virtualenv
Collecting virtualenv
  Downloading https://files.pythonhosted.org/packages/c1/61/7506ddd79ef6f09beeefb81c4c55bf395a8ad96b33ff1c6b06e40f8aa101/virtualenv-20.0.7-py2.py3-none-any.whl (8.0MB)
Collecting importlib-resources<2,>=1.0; python_version < "3.7" (from virtualenv)
  Downloading https://files.pythonhosted.org/packages/2f/f7/b4aa02cdd3ee7ebba375969d77c00826aa15c5db84247d23c89522dccbfa/importlib_resources-1.0.2-py2.py3-none-any.whl
Collecting filelock<4,>=3.0.0 (from virtualenv)
  Downloading https://files.pythonhosted.org/packages/93/83/71a2ee6158bb9f39a90c0dea1637f81d5eef866e188e1971a1b1ab01a35a/filelock-3.0.12-py3-none-any.whl
Collecting six<2,>=1.9.0 (from virtualenv)
  Downloading https://files.pythonhosted.org/packages/65/eb/1f97cb97bfc2390a276969c6fae16075da282f5058082d4cb10c6c5c1dba/six-1.14.0-py2.py3-none-any.whl
Collecting appdirs<2,>=1.4.3 (from virtualenv)
  Downloading https://files.pythonhosted.org/packages/56/eb/810e700ed1349edde4cbdc1b2a21e28cdf115f9faf263f6bbf8447c1abf3/appdirs-1.4.3-py2.py3-none-any.whl
Collecting importlib-metadata<2,>=0.12; python_version < "3.8" (from virtualenv)
  Downloading https://files.pythonhosted.org/packages/8b/03/a00d504808808912751e64ccf414be53c29cad620e3de2421135fcae3025/importlib_metadata-1.5.0-py2.py3-none-any.whl
Collecting distlib<1,>=0.3.0 (from virtualenv)
  Downloading https://files.pythonhosted.org/packages/7d/29/694a3a4d7c0e1aef76092e9167fbe372e0f7da055f5dcf4e1313ec21d96a/distlib-0.3.0.zip (571kB)
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    ModuleNotFoundError: No module named 'setuptools'

Please see helm/community#135

Test Helm features across shells (bash 4, bash 3, zsh, etc.)

Related to helm/helm#6120 cc @marckhouzam

For example the "open" command is a Mac command

I'd like to allow for some tests to be run with Helm v2. To achieve this it should no longer be acceptance.sh that aborts execution when detecting Helm v2 but it should instead be pushed down to each robot test suite to choose if it supports v2 or not.

Add a Dockerfile for building a container image with all the required tools needed to run these test (for CI purposes)

We should test against 1.16 - i tried but the deployment of the test chart fails due to the new API versions/kinds used for Deployments

Essentially we should just need to update the helm chart have the new api types, only installing them on 1.16 and vice versa?

Instead of shelling out to kubectl, should just import and use the Python library: https://github.com/kubernetes-client/python

Instead of hardcoding /tmp everywhere, we should instead make a temporary directory using mktemp.

Example:

TMPDIR="$(mktemp -d -p ${PWD} --suffix=-helm-acceptance)"
trap "rm -rf ${TMPDIR}" EXIT

With Helm v2 having reached EOL, I was wondering if we should remove v2 support from the acceptance-tests.
The idea is simply to simplify the maintenance of the acceptance-tests.

Any opinion @jdolitsky?
I can handle the coding, if you agree with the idea.

Add registries.robot to test Helm functionality across all supported local (Distribution, Harbor) and cloud-based registries (Docker Hub, Quay, ECR, GCR, ACR, etc)

As per Helm's governance rules, it's probably best to add an OWNERS file to the project so others know who in the Helm org owns this project. :)

I've become quite worried about the rm -rf line of scripts/acceptance.sh:
https://github.com/helm/acceptance-testing/blob/master/scripts/acceptance.sh#L45

If the user were to make a mistake (because of lack of understanding) and set ROBOT_HELM_HOME_DIR to $HOME, for example, we would wipe the user's entire home directory!

I think we need to improve this ASAP, but I don't have an idea just yet on how to do it.