saltstack / salt-ext-modules-vmware Goto Github PK
View Code? Open in Web Editor NEWSalt Extension Modules for VMware
License: Apache License 2.0
Salt Extension Modules for VMware
License: Apache License 2.0
Integration test failed
def test_advanced_config(service_instance):
"""
Test advanced config on esxi host
"""
> ret = esxi.set_advanced_config(
config_name="Annotations.WelcomeMessage",
config_value="testing",
service_instance=service_instance,
datacenter_name="Datacenter",
cluster_name="Cluster",
)
service_instance = 'vim.ServiceInstance:ServiceInstance'
tests/integration/modules/test_esxi.py:276:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
src/saltext/vmware/modules/esxi.py:639: in set_advanced_config
return set_advanced_configs(
cluster_name = 'Cluster'
config_name = 'Annotations.WelcomeMessage'
config_value = 'testing'
datacenter_name = 'Datacenter'
host_name = None
service_instance = 'vim.ServiceInstance:ServiceInstance'
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
config_dict = {'Annotations.WelcomeMessage': 'testing'}, datacenter_name = 'Datacenter', cluster_name = 'Cluster', host_name = None
service_instance = 'vim.ServiceInstance:ServiceInstance'
def set_advanced_configs(
config_dict,
datacenter_name=None,
cluster_name=None,
host_name=None,
service_instance=None,
):
"""
Set multiple advanced configurations on matching EXSI hosts.
config_dict
Set the configuration key to the configuration value. Eg: {"Annotations.WelcomeMessage": "Hello"}
datacenter_name
Filter by this datacenter name (required when cluster is specified)
cluster_name
Filter by this cluster name (optional)
host_name
Filter by this ESXi hostname (optional)
service_instance
Use this vCenter service connection instance instead of creating a new one. (optional).
.. code-block:: bash
salt '*' vmware_esxi.set_advanced_config config_name=Annotations.WelcomeMessage config_value=Hello
Returns:
.. code-block:: json
{
"host1": {
"Annotations.WelcomeMessage": "HelloDemo"
},
}
"""
log.debug("Running vmware_esxi.set_advanced_configs")
ret = {}
if not service_instance:
service_instance = get_service_instance(opts=__opts__, pillar=__pillar__)
hosts = utils_esxi.get_hosts(
service_instance=service_instance,
host_names=[host_name] if host_name else None,
cluster_name=cluster_name,
datacenter_name=datacenter_name,
get_all_hosts=host_name is None,
)
try:
for h in hosts:
config_manager = h.configManager.advancedOption
ret[h.name] = {}
if not config_manager:
continue
supported_configs = {}
for opt in config_manager.supportedOption:
if opt.key not in config_dict:
continue
supported_configs[opt.key] = opt.optionType
advanced_configs = []
for opt in config_dict:
> opt_type = supported_configs[opt]
E KeyError: 'Annotations.WelcomeMessage'
We need to extract the function arguments used for vmc connections from pretty much everywhere, and change them to be pulled exclusively from pillar data. Conveniently this will mostly be deletions with just a few additions to vmc_reques::call_api
:)
I suggest this structure, with these 6 parameters, but @waynew will have a good perspective on the normal naming and organization of pillar data.
vmc_connection_details:
console_host: hostname
api_key: key
org_id: UUID
sddc_id: UUID
nsx_host: hostname
verify_ssl: False
verify_ssl
: (Optional) True/False or string with path to CA bundle. Defaults to True.
We should also delete cert
at the same time.
We will need to add documentation on how to find the correct values - generating an api key, browsing the UI to find the org_id
and nsx_host
etc.
This documentation will need to be linked in the module-level docstring of all vmc files.
When I create a SDDC using a state file like this and then add arguments for num_hosts, hostname, etc:
create_sddc:
module.run:
- name: vmc_sddc.create
It creates the SDDC in VMC.
However if I make a change to an argument to change the config, like num_hosts and re-run the state it creates a new SDDC instead of updating the existing named SDDC. Is there a way to enforce a state of a SDDC vs. creating a new one yet? Thanks.
80% of the variables in vmc_constants.py
are only used one time, and 95% of the time the variables are re-used, they are like GET_REQUEST_METHOD = "get"
, and it's better to use "get" in the code directly.
"get" as a http method is not changing, and request(method="get"
is perfectly clear.
Fresh VMC_CONFIG file
@pytest.fixture(autouse=True)
def setup(nsxt_config):
hostname, username, password = _get_server_info(nsxt_config)
> ip_block_from_nsxt = _get_ip_block_by_display_name_using_nsxt_api(
hostname, username, password, "IP_Block_Salt_State_FT"
)
hostname = 'nsxt-hostname/ip'
nsxt_config = {'cert': 'path-to-nsxt-cert.pem', 'compute_manager_server': 'compute-manager-server-ip', 'credential': {'credential_ty...ompute-manager-server-thumbprint', 'username': 'compute-manager-server-username'}, 'hostname': 'nsxt-hostname/ip', ...}
password = 'nsxt-password'
username = 'nsxt-username'
def test_add(integration_test_config, service_instance):
"""
Test esxi add
"""
> if integration_test_config["esxi_manage_test_instance"]:
E KeyError: 'esxi_manage_test_instance'
suggestion (non-blocking) - probably better as (part of) another PR - can use "... -m json.tool | grep -A4 '"vcenter_detail"'
, to provide the full command like the other examples.
Originally posted by @waynew in #125 (comment)
Not sure what or why they're broken, but each time I've run the integration tests I'm seeing errors like this:
ERROR tests/integration/modules/test_vmc_dhcp_profiles.py::test_update_dhcp_profile_smoke_test - requests.exceptions.HTTPError: 400 Client Error: for url: https://...
ERROR tests/integration/modules/test_vmc_networks.py::test_update_network_smoke_test - requests.exceptions.HTTPError: 400 Client Error: for url: https://...
This error seems to be happening across PRs, and is blocking PRs with clearly unrelated changes like #217
Currently this is blocking:
Also probably
What it says on the tin. There are some bits of information that we're only exposing via a .get_info
or something to that effect, that really should be grains.
I don't have an exhaustive list or even a single thing that I'm currently confident about, but... we should be considering grains.
def test_esxi_get_lun_ids_should_return_lun_NAA_ids(service_instance, integration_test_config):
expected_lun_ids = integration_test_config["esxi_datastore_disk_names"]
actual_ids = esxi.get_lun_ids(service_instance=service_instance)
> assert actual_ids == expected_lun_ids
E AssertionError: assert ['mpx.vmhba0:...ba0:C0:T0:L0'] == ['mpx.vmhba0:C0:T0:L0']
E Left contains one more item: 'mpx.vmhba0:C0:T0:L0'
E Full diff:
E - ['mpx.vmhba0:C0:T0:L0']
E + ['mpx.vmhba0:C0:T0:L0', 'mpx.vmhba0:C0:T0:L0']
Might be worth creating a test fixture to specify/create the lun/NAA. Otherwise this is probably an issue with the scraper.
utils/vmware.py::get_datastore
uses some fancy traversal to make finding datastores more efficient.
Unfortunately, when searching "everywhere" it doesn't search nested folders.
This should probably be made to work with nested folders. In the meantime, modules/datastore.py
is using a wrapper function that gets all datastores and searches by name if nothing exists. This should work but be less efficient.
I could be wrong, but src/saltext/vmware/modules/dvswitch.py
doesn't seem to have the ability to delete switches.
This seems like a useful feature, even if it's just for testing.
Fresh VMC_CONFIG file
nsxt_config = {'cert': 'path-to-nsxt-cert.pem', 'compute_manager_server': 'compute-manager-server-ip', 'credential': {'credential_ty...ompute-manager-server-thumbprint', 'username': 'compute-manager-server-username'}, 'hostname': 'nsxt-hostname/ip', ...}
@pytest.fixture
def setup(nsxt_config):
"""
Sets up test requirements:
Queries nsx api for transport node profiles with display name
Deletes if exists
Also queries and deletes transport zones which will be used in the IT
"""
> transport_node_profiles = _get_transport_node_profiles_by_display_name_from_nsxt(
nsxt_config, _display_name
)
Failed with fresh VMC_CONFIG
@pytest.fixture
def get_licenses():
url = BASE_URL.format(management_host=hostname)
response = nsxt_request.call_api(
method="GET", url=url, username=username, password=password, verify_ssl=_verify_ssl
)
> assert "error" not in response
E AssertionError: assert 'error' not in {'error': 'Error occurred while calling NSX-T API https://nsxt-hostname/ip/api/v1/licenses. Please check logs for more details.'}
response = {'error': 'Error occurred while calling NSX-T API https://nsxt-hostname/ip/api/v1/licenses. Please check logs for more details.'}
url = 'https://nsxt-hostname/ip/api/v1/licenses'
tests/integration/modules/test_nsxt_license.py:60: AssertionError
These tests failed for me when running the full test suite (i.e. python -m tests-3
) with a fresh vcenter.conf and vmc.conf for testing:
tests.integration.modules.test_datacenter : test_get
tests.integration.modules.test_esxi : test_esxi_get_lun_ids_should_return_lun_NAA_ids
tests.integration.modules.test_esxi : test_manage_service
tests.integration.modules.test_esxi : test_acceptance_level
tests.integration.modules.test_esxi : test_advanced_config
tests.integration.modules.test_esxi : test_get_dns_config
tests.integration.modules.test_esxi : test_get_firewall_config
tests.integration.modules.test_esxi : test_add
tests.integration.modules.test_esxi : test_manage_disconnect
tests.integration.modules.test_esxi : test_move
tests.integration.modules.test_esxi : test_manage_connect
tests.integration.modules.test_esxi : test_manage_remove
tests.integration.modules.test_esxi : test_esxi_get
tests.integration.modules.test_nsxt_compute_manager : test_register_update_and_remove
tests.integration.modules.test_nsxt_ip_blocks : test_nsxt_ip_blocks_execution_module_crud_operations
tests.integration.modules.test_nsxt_ip_pools : test_nsxt_ip_pools_execution_module_crud_operations
tests.integration.modules.test_nsxt_license : test_apply_license
tests.integration.modules.test_nsxt_license : test_get_licenses
tests.integration.modules.test_nsxt_license : test_delete_license
tests.integration.modules.test_nsxt_manager : test_nsxt_manager_get_and_set_manager_config
tests.integration.modules.test_nsxt_transport_node : test_check_transport_nodes
tests.integration.modules.test_nsxt_transport_node_profiles : test_nsxt_transport_node_profiles_execution_module
tests.integration.modules.test_nsxt_transport_zone : test_check_transport_zone
tests.integration.modules.test_nsxt_uplink_profiles : test_nsxt_uplink_profiles_execution_module
tests.integration.modules.test_vm : test_ovf_deploy
tests.integration.modules.test_vm : test_ova_deploy
(same as ovf_deploy)tests.integration.states.test_datacenter : test_present
tests.integration.states.test_datacenter : test_absent
- same as test_presenttests.integration.states.test_datacenter : test_present_dry_run
- same as test_presenttests.integration.states.test_datacenter : test_absent_dry_run
- same as test_presenttests.integration.states.test_folder : test_move_folder
tests.integration.states.test_nsxt_compute_manager : test_register_update_and_delete
- path-to-nsxt-cert.pem issuetests.integration.states.test_nsxt_ip_blocks : test_nsxt_ip_blocks_state_module
tests.integration.states.test_nsxt_ip_pools : test_nsxt_ip_pools_present_and_absent_states
- same as above blocs_state_moduletests.integration.states.test_nsxt_license : test_nsxt_licenses_state_module
- path-to-nsxt-cert.pem problemtests.integration.states.test_nsxt_manager : test_nsxt_manager
- same as abovetests.integration.states.test_nsxt_transport_node : test_state_transport_node_verify
(probably same as above)tests.integration.states.test_nsxt_transport_node_profiles : test_nsxt_transport_node_profiles_present_and_absent_states
'error' not in error (probably cert problem)tests.integration.states.test_nsxt_transport_zone : test_state_transport_zone_verify
tests.integration.states.test_nsxt_uplink_profiles : test_nsxt_uplink_profiles_present_and_absent_states
- error not in error, probably cert issueFailed with a fresh VMC_CONFIG file from the scraper
@pytest.fixture(autouse=True)
def setup(nsxt_config):
hostname, username, password = _get_server_info(nsxt_config)
> ip_pools_from_nsxt = _get_ip_pool_by_display_name_using_nsxt_api(
hostname, username, password, "IP_Pool_Salt_FT"
)
hostname = 'nsxt-hostname/ip'
nsxt_config = {'cert': 'path-to-nsxt-cert.pem', 'compute_manager_server': 'compute-manager-server-ip', 'credential': {'credential_ty...ompute-manager-server-thumbprint', 'username': 'compute-manager-server-username'}, 'hostname': 'nsxt-hostname/ip', ...}
password = 'nsxt-password'
username = 'nsxt-username'
VMC is currently using name
for salt logging and not much else.
In salt it's normal for name
in state functions to be the static identifier for the object being modified. Something like the file name you are chmod-ing, or the host you are checking for connectivity, or the command line string you are executing.
As an example, right now we have dhcp_profile_id
as well as name
.
We need to delete dhcp_profile_id
from the function argument list, replacing its usages with name
, then update the docstring of name
to be something like: The dhcp profile id, any static unique string identifying the rule. Also used for the display_name by default.
Fixture should probably remove the folder that it's going to be moved to(?)
ย | pyVmomi.VmomiSupport.vim.fault.DuplicateName: (vim.fault.DuplicateName) { dynamicType = <unset>, dynamicProperty = (vmodl.DynamicProperty) [], msg = "The name 'top_folder' already exists.", faultCause = <unset>, faultMessage = (vmodl.LocalizableMessage) [], name = 'top_folder', object = 'vim.Folder:group-v41' }
def test_move_folder(patch_salt_globals_folder_state):
"""
test move folder
"""
> folder.manage("top_folder", "create", "Datacenter", "vm")
patch_salt_globals_folder_state = None
tests/integration/states/test_folder.py:85:
This iteration happens several times. Perhaps it would better be extracted to something like
src/saltext/vmware/utils/vm.py
boot_order_list = []
device_types = (vim.vm.device.VirtualCdrom, vim.vm.device.VirtualDisk,
existing_order = [device for device in vm.config.hardware.device if isinstance(device, (vim.vm.device.VirtualCdrom, vim.vm.device.VirtualDisk, vim.vm.device.VirtualEthernetCard, vim.vm.device.VirtualFloppy))]
for device_name in order:
...
Looking closer, though, I notice that for ethernet and disk we're only paying attention to the first item in the list ๐ค I wonder if there's a better way for this generally.
tests/integration/modules/test_datastore.py
and
tests/integration/states/test_datastore.py
both use integration_test_config["datastores"]
.
We should modify those tests to get a datastore name (cached using a pytest fixture?), then modify it/restore its state; something like:
@pytest.fixture(scope="session")
def datastore(service_instance, integration_test_config):
host = integration_test_config["esxi_host_name"]
datastores = datastore.get(
service_instance=service_instance,
host_name=host,
)
return datastores[0]
This test is failing
def test_get(vmware_datacenter, service_instance):
"""
Test scenarios for get datacenter.
"""
# Get a non existent datacenter. Should return False
dc1_name = str(uuid.uuid4())
> dc1 = datacenter.get(cluster_name=dc1_name, service_instance=service_instance)
E AttributeError: module 'saltext.vmware.modules.datacenter' has no attribute 'get'
def test_acceptance_level(service_instance):
"""
Test acceptance level on esxi host
"""
ret = esxi.set_acceptance_level(
acceptance_level="community",
service_instance=service_instance,
datacenter_name="Datacenter",
cluster_name="Cluster",
)
for h in ret:
assert ret[h] == "community"
ret = esxi.get_acceptance_level(
acceptance_level="community",
service_instance=service_instance,
datacenter_name="Datacenter",
cluster_name="Cluster",
)
> assert set(ret.values()) == {"community"}
E AssertionError: assert set() == {'community'}
E Extra items in the right set:
E 'community'
E Full diff:
E - {'community'}
E + set()
salt.exceptions.VMwareApiError: ['Host did not have any virtual network defined.']
def test_ovf_deploy(integration_test_config, patch_salt_globals_vm):
"""
Test deploy virtual machine through an OVF
"""
> res = virtual_machine.deploy_ovf(
vm_name="test1",
host_name=integration_test_config["esxi_host_name"],
ovf_path="tests/test_files/centos-7-tools.ovf",
)
Currently if config values aren't setup for creating the service instance, an unhelpful message is returned:
salt-call --local vmware_datacenter.list
Passed invalid arguments: expected string or bytes-like object.
Usage:
Returns a list of datacenters for the specified host.
.. code-block:: bash
salt '*' vmware_datacenter.list
If an argument is passed then it's uh... much worse ๐
File "/usr/sbin/salt-call", line 8, in <module> [20/1883]
sys.exit(salt_call())
File "/usr/lib/python3.9/site-packages/salt/scripts.py", line 449, in salt_call
client.run()
File "/usr/lib/python3.9/site-packages/salt/cli/call.py", line 58, in run
caller.run()
File "/usr/lib/python3.9/site-packages/salt/cli/caller.py", line 112, in run
ret = self.call()
File "/usr/lib/python3.9/site-packages/salt/cli/caller.py", line 219, in call
ret["return"] = self.minion.executors[fname](
File "/usr/lib/python3.9/site-packages/salt/loader.py", line 1241, in __call__
return self.loader.run(run_func, *args, **kwargs)
File "/usr/lib/python3.9/site-packages/salt/loader.py", line 2274, in run
return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
File "/usr/lib/python3.9/site-packages/salt/loader.py", line 2289, in _run_as
return _func_or_method(*args, **kwargs)
File "/usr/lib/python3.9/site-packages/salt/executors/direct_call.py", line 12, in execute
return func(*args, **kwargs)
File "/usr/lib/python3.9/site-packages/salt/loader.py", line 1241, in __call__
return self.loader.run(run_func, *args, **kwargs)
File "/usr/lib/python3.9/site-packages/salt/loader.py", line 2274, in run
return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
File "/usr/lib/python3.9/site-packages/salt/loader.py", line 2289, in _run_as
return _func_or_method(*args, **kwargs)
File "/usr/lib/python3.9/site-packages/saltext/vmware/modules/datacenter.py", line 41, in list_
return utils_datacenter.list_datacenters(service_instance)
File "/usr/lib/python3.9/site-packages/saltext/vmware/utils/common.py", line 648, in list_datacenters
return list_objects(service_instance, vim.Datacenter)
File "/usr/lib/python3.9/site-packages/saltext/vmware/utils/common.py", line 299, in list_objects
item_list = get_mors_with_properties(service_instance, vim_object, properties)
File "/usr/lib/python3.9/site-packages/saltext/vmware/utils/common.py", line 219, in get_mors_with_properties
content = get_content(*content_args, **content_kwargs)
File "/usr/lib/python3.9/site-packages/saltext/vmware/utils/common.py", line 89, in get_content
container_ref = get_root_folder(service_instance)
File "/usr/lib/python3.9/site-packages/saltext/vmware/utils/common.py", line 37, in get_root_folder
return service_instance.RetrieveContent().rootFolder
AttributeError: 'str' object has no attribute 'RetrieveContent'
That's undesirable. A good approach would be for https://github.com/saltstack/salt-ext-modules-vmware/blob/main/src/saltext/vmware/utils/connect.py to detect missing values and alert on the missing config values.
def test_manage_disconnect(integration_test_config, service_instance):
"""
Test esxi manage disconnect task
"""
> if integration_test_config["esxi_manage_test_instance"]:
E KeyError: 'esxi_manage_test_instance'
fresh vmc_config file
def test_nsxt_manager_get_and_set_manager_config(nsxt_config, salt_call_cli):
"""
nsxt_manager.get_manager_config
nsxt_manager.set_manager_config
"""
hostname = nsxt_config["hostname"]
username = nsxt_config["username"]
password = nsxt_config["password"]
> response_json = _get_manager_config_from_nsxt_api(hostname, username, password)
...
def _get_manager_config_from_nsxt_api(hostname, username, password):
response_json = nsxt_request.call_api(
method="GET",
url=BASE_URL.format(hostname),
username=username,
password=password,
verify_ssl=False,
)
> assert "error" not in response_json
E AssertionError: assert 'error' not in {'error': 'Error occurred while calling NSX-T API https://nsxt-hostname/ip/api/v1/configs/management. Please check logs for more details.'}
hostname = 'nsxt-hostname/ip'
password = 'nsxt-password'
response_json = {'error': 'Error occurred while calling NSX-T API https://nsxt-hostname/ip/api/v1/configs/management. Please check logs for more details.'}
username = 'nsxt-username'
def test_manage_service(service_instance):
"""
Test manage services on esxi host
"""
SSH_SERVICE = "TSM-SSH"
ret = esxi.manage_service(
service_name=SSH_SERVICE,
service_instance=service_instance,
state="start",
datacenter_name="Datacenter",
cluster_name="Cluster",
)
assert ret
ret = esxi.list_services(
service_name=SSH_SERVICE,
service_instance=service_instance,
datacenter_name="Datacenter",
cluster_name="Cluster",
)
for host in ret:
> assert ret[host][SSH_SERVICE]["state"] == "running"
E KeyError: 'TSM-SSH'
Not sure what happened here, but... clearly failing
def test_check_transport_zone(nsxt_config, salt_call_cli):
ret_create = salt_call_cli.run(
"nsxt_transport_zone.create",
hostname=nsxt_config["hostname"],
username=nsxt_config["username"],
password=nsxt_config["password"],
host_switch_name="nsxDefaultHostSwitch",
transport_type="OVERLAY",
display_name="Test_Create_Transport_Zone-IT",
verify_ssl=False,
)
assert ret_create is not None
result_as_json_create = ret_create.json
> assert result_as_json_create["display_name"] == "Test_Create_Transport_Zone-IT"
E KeyError: 'display_name'
Fresh VMC_CONFIG file
def test_state_transport_zone_verify(nsxt_config, salt_call_cli):
response_create = salt_call_cli.run(
"state.single",
"nsxt_transport_zone.present",
name="create transport zone",
hostname=nsxt_config["hostname"],
username=nsxt_config["username"],
password=nsxt_config["password"],
host_switch_name="nsxDefaultHostSwitch",
transport_type="OVERLAY",
verify_ssl=False,
display_name=_display_name,
description=_description,
)
result_create = dict(list(response_create.json.values())[0])["result"]
comment_create = dict(list(response_create.json.values())[0])["comment"]
> display_name_response = dict(dict(list(response_create.json.values())[0])["changes"])["new"][
"display_name"
]
E KeyError: 'new'
Currently we're using the test_value_scraper.py as a workaround.
We need to replace test_value_scraper.py with proper fixtures that take the service instance and ensure that specific datacenters, vms, etc. are all existing.
This should potentially be fixtures using other fixtures, probably with scope='session'
๐ค
Failed with:
failed on setup with "OSError: Could not find a suitable TLS CA certificate bundle, invalid path: path-to-nsxt-cert.pem"
@pytest.fixture
def delete_compute_manager(nsxt_config):
"""
Sets up test requirements:
Queries nsx api for compute manager 10.206.243.180
Deletes compute manager if exists
"""
hostname = nsxt_config["hostname"]
username = nsxt_config["username"]
password = nsxt_config["password"]
cert = nsxt_config.get("cert", False)
compute_manager_server = nsxt_config["compute_manager_server"]
url = "https://{management_host}/api/v1/fabric/compute-managers".format(
management_host=hostname
)
> response = requests.get(url=url, auth=(username, password), verify=cert)
This is probably an issue with the test value scraper, or the tests
def test_move(integration_test_config, service_instance):
"""
Test esxi move
"""
> if integration_test_config["esxi_manage_test_instance"]:
E KeyError: 'esxi_manage_test_instance'
Currently the implementation of esxi.get
in #129 does not quite match up with the API in Salt's grains.core. For now this is fine but at some point in the future we want to bring these in sync so that it's trivial for a user of this extension module to upgrade to either a deltaproxy approach or some other as-yet-to-be-developed style of accessing esxi grain-style information.
I tried to follow the installation instructions listed here on a vRealize Automation SaltStack Config 8.6.1 appliance:
https://github.com/waynew/salt-ext-modules-vmware/blob/quickstart-docs/docs/quickstart.rst
Installation went fine, but the command to successfully test it didn't find the module:
salt-call vmware_datacenter.list
After some research we found out that the module directory has to be specified in the minion configuration file e.g. by creating /etc/salt/minion.d/module_dirs.conf file with following content:
module_dirs:
After "service salt-minion restart" above command was executed successfully.
@shaikres not sure who is responsible for this section, but I came across the issue when reviewing #22
Feel free to assign whoever would be best for fixing this.
I used this state
blerp:
vmc_security_rules.present:
- hostname: "blerp"
- refresh_key: "werd"
- authorization_host: "okay"
- org_id: "nope"
- sddc_id: lolwat
- domain_id: imperial dominion
- rule_id: 13
and rather than giving me a helpful message such as "that refresh key doesn't exist" or "unable to get access token" I got a much less helpful message:
local:
----------
ID: blerp
Function: vmc_security_rules.present
Result: False
Comment: An exception occurred in this state: Traceback (most recent call last):
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn
conn = connection.create_connection(
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/urllib3/util/connection.py", line 73, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/lib/python3.9/socket.py", line 953, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
conn.connect()
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/urllib3/connection.py", line 353, in connect
conn = self._new_conn()
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/urllib3/connection.py", line 181, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7fa7f862ca60>: Failed to establish a new connection: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/urllib3/util/retry.py", line 574, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='okay', port=443): Max retries exceeded with url: /csp/gateway/am/api/auth/api-tokens/authorize?refresh_token=werd (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fa7f862ca60>: Failed to establish a new connection: [Errno -2] Name or service not known'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/salt/state.py", line 2171, in call
ret = self.states[cdata["full"]](
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/salt/loader.py", line 1235, in __call__
return self.loader.run(run_func, *args, **kwargs)
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/salt/loader.py", line 2268, in run
return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/salt/loader.py", line 2283, in _run_as
return _func_or_method(*args, **kwargs)
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/salt/loader.py", line 2316, in wrapper
return f(*args, **kwargs)
File "/home/wayne/programming/salt-ext-modules-vmware/src/saltext/vmware/states/vmc_security_rules.py", line 222, in present
get_security_rule = __salt__["vmc_security_rules.get_by_id"](
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/salt/loader.py", line 1235, in __call__
return self.loader.run(run_func, *args, **kwargs)
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/salt/loader.py", line 2268, in run
return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/salt/loader.py", line 2283, in _run_as
return _func_or_method(*args, **kwargs)
File "/home/wayne/programming/salt-ext-modules-vmware/src/saltext/vmware/modules/vmc_security_rules.py", line 188, in get_by_id
return vmc_request.call_api(
File "/home/wayne/programming/salt-ext-modules-vmware/src/saltext/vmware/utils/vmc_request.py", line 110, in call_api
headers = get_headers(refresh_key, authorization_host)
File "/home/wayne/programming/salt-ext-modules-vmware/src/saltext/vmware/utils/vmc_request.py", line 54, in get_headers
access_token = get_access_token(refresh_key, authorization_host)
File "/home/wayne/programming/salt-ext-modules-vmware/src/saltext/vmware/utils/vmc_request.py", line 38, in get_access_token
response = requests.post(url, params=params, headers=headers)
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/requests/api.py", line 119, in post
return request('post', url, data=data, json=json, **kwargs)
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/home/wayne/programming/salt-ext-modules-vmware/env/lib/python3.9/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='okay', port=443): Max retries exceeded with url: /csp/gateway/am/api/auth/api-tokens/authorize?refresh_token=werd (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fa7f862ca60>: Failed to establish a new connection: [Errno -2] Name or service not known'))
Started: 18:52:23.095975
Duration: 3680.228 ms
Changes:
Summary for local
------------
Succeeded: 0
Failed: 1
------------
Total states run: 1
Total run time: 3.680 s
(typically when one sees a stack trace, that's a sign that something is broken -- and it's definitely broken here)
We cut a bunch of pre-release/rc versions of this module, we need to go back and document what was added to each of those versions.
Failed with fresh VMC_CONFIG file
@pytest.fixture
def delete_license():
"""
Sets up test requirements:
Queries nsx api for licenses
Deletes license if exists
"""
url = BASE_URL.format(management_host=hostname)
licenses_dict = nsxt_request.call_api(
method="GET", url=url, username=username, password=password, verify_ssl=_verify_ssl
)
> if licenses_dict["result_count"] != 0:
E KeyError: 'result_count'
licenses_dict = {'error': 'Error occurred while calling NSX-T API https://nsxt-hostname/ip/api/v1/licenses. Please check logs for more details.'}
url = 'https://nsxt-hostname/ip/api/v1/licenses'
tests/integration/modules/test_nsxt_license.py:34: KeyError
ย | TypeError: expected string or bytes-like object
def test_present(vmware_datacenter):
"""
Test scenarios for datacenter.present state run
"""
# datacenter.present on an existing datacenter. No changes should be made.
> ret = datacenter.present(name=vmware_datacenter)
def test_esxi_get(service_instance):
"""
Test get configuration on ESXi host
"""
ret = esxi.get(
service_instance=service_instance,
datacenter_name="Datacenter",
cluster_name="Cluster",
)
assert ret
for host in ret:
assert ret[host]["cpu_model"]
assert ret[host]["capabilities"]
> assert ret[host]["nics"]
E assert {}
Currently vmc_request.call_api
returns {"error": "reason"}
on errors.
Errors should be thrown up the stack. We need to define a few Exception types and raise those.
def test_get_firewall_config(service_instance):
"""
Test get firewall configuration on ESXi host
"""
ret = esxi.get_firewall_config(
service_instance=service_instance,
datacenter_name="Datacenter",
cluster_name="Cluster",
)
> assert ret
E assert {}
ret = {}
In order to actually focus on automating the release tasks, we need an issue for it.
This issue is to track the automation of part of the release process
The high level design is that we should have a single command that can be executed, and the end result is that we should have a pass/fail, and especially for pass we MUST have the accompanying release artifact. We should follow the release process, but basically this looks like:
I think we should wait on automating creating the tag on pass - but that's something that we could do once we have towncrier for building the changelog, since the annotation on the tag should just be the contents of the changelog.
This means that we could do something like:
$ create-release 2021-11-10
...
PASS - dist/saltext.vmware-21.11.10-py3-none-any.whl
$ git tag -a 21.11.10
$ twine upload dist/saltext.vmware-21.11.10-py3-none-any.whl
$ git push salt 21.11.10
Followed by creating a release on GitHub.
def test_manage_remove(integration_test_config, service_instance):
"""
Test esxi manage remove task
"""
> if integration_test_config["esxi_manage_test_instance"]:
E KeyError: 'esxi_manage_test_instance'
For example, see Salt module - file
is for managing files, apache
is for managing apache servers.
It recently came to my attention that some of our states/modules have plural names, so we need to check the list of existing names and deprecate the plural ones unless it makes sense (i.e. it actually has to operate on a collection of things)
def test_manage_connect(integration_test_config, service_instance):
"""
Test esxi manage connect task
"""
> if integration_test_config["esxi_manage_test_instance"]:
E KeyError: 'esxi_manage_test_instance
Might be an issue in the scraper
@pytest.fixture(autouse=True)
def setup(nsxt_config):
hostname, username, password, cert = _get_server_info(nsxt_config)
> ip_block_from_nsxt = _get_ip_block_by_display_name_using_nsxt_api(
hostname, username, password, "IP_Block_Salt_FT"
)
cert = 'path-to-nsxt-cert.pem'
hostname = 'nsxt-hostname/ip'
nsxt_config = {'cert': 'path-to-nsxt-cert.pem', 'compute_manager_server': 'compute-manager-server-ip', 'credential': {'credential_ty...ompute-manager-server-thumbprint', 'username': 'compute-manager-server-username'}, 'hostname': 'nsxt-hostname/ip', ...}
password = 'nsxt-password'
username = 'nsxt-username'
Fresh VMC_CONFIG file
def test_check_transport_nodes(nsxt_config, salt_call_cli):
# Create of the transport node
ret_create = salt_call_cli.run(
"nsxt_transport_node.create",
hostname=nsxt_config["hostname"],
username=nsxt_config["username"],
password=nsxt_config["password"],
verify_ssl=_verify_ssl,
transport_zone_endpoints=[],
maintenance_mode="DISABLED",
node_deployment_info=_node_deployment_info,
is_overridden=False,
resource_type="TransportNode",
display_name="Create-Transport-Node-IT",
)
assert ret_create is not None
result_as_json_create = ret_create.json
> assert result_as_json_create["node_id"] is not None
E KeyError: 'node_id'
fresh VMC_CONFIG file
def test_state_transport_node_verify(nsxt_config, salt_call_cli):
response_create = salt_call_cli.run(
"state.single",
"nsxt_transport_node.present",
name="create transport node",
hostname=nsxt_config["hostname"],
username=nsxt_config["username"],
password=nsxt_config["password"],
verify_ssl=False,
display_name=_display_name,
description="Create-Transport-Node",
node_deployment_info=_node_deployment_info,
)
result_create = dict(list(response_create.json.values())[0])["result"]
comment_create = dict(list(response_create.json.values())[0])["comment"]
> assert result_create is True
E assert False is True
comment_create = 'Failed to get the transport nodes : Error occurred while calling NSX-T API https://nsxt-hostname/ip/api/v1/transport-nodes. Please check logs for more details.'
Like mentioned in that issue, when a device is created with disks we should increment. Alternatively if we really need random instead of sequential, we should just shuffle the list and pop items off instead.
def test_get_dns_config(service_instance):
"""
Test get dns configuration on ESXi host
"""
ret = esxi.get_dns_config(
service_instance=service_instance,
datacenter_name="Datacenter",
cluster_name="Cluster",
)
assert ret
for host in ret:
> assert ret[host]["ip"]
E assert []
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.