elmiko / okd-camgi Goto Github PK

View Code? Open in Web Editor NEW

6.0 6.0 4.0 159 KB

OKD Cluster Autoscaler Must Gather Investigator

License: GNU General Public License v3.0

Python 60.59% HTML 38.86% Dockerfile 0.55%

okd-camgi's People

Contributors

Stargazers

Watchers

Forkers

lobziik 2uasimojo fortinj66

okd-camgi's Issues

add a resize function to html page

the size of the content section of the page is determined dynamically to enable the scroll bars. when the window is resized it should also be resized to keep the user experience consistent.

add machine-config-operator logs

this can help when machines are provisioning but not joining the cluster.

large log files are slow to open

some log files are quite large and slow the browser down when opening the collapsible. perhaps limiting the size of the div for the log file would help this.

add an automated container build

this would be a container image that can run the okd-camgi tool, thus bundling all the dependencies.

add kubelet logs

these could be associated with the node entries, similar to how the mapi stuff is organized.

render should add "about" information on the generated page

it would be nice to have some easily accessible information about where to log bugs or get help. adding an about modal, or some sort of comment text would help out. an interactive component is probably better as it would be more visible to users.

expand encoded CSR block

when there are pending CSRs on a cluster it can be very useful to inspect how the certificate requests are formed. it would be nice if camgi could decode the request blocks inside of the csr before displaying.

for example, instead of seeing this:

apiVersion: certificates.k8s.io/v1 
kind: CertificateSigningRequest                                                                                                           
spec:
  request: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQkVEQ0J0d0lCQURCVk1SVXdFd1lEVlFRS0V3eHplWE4wWlcwNmJtOWtaWE14UERBNkJnTlZCQU1UTTNONQpjM1JsYlRwdWIyUmxPakY2ZGpKNWFEZzFMVFF6TTJWa0xXNTRaSEpzTFcxaGMzUmxjaTB4TG01dmRtRnNiMk5oCmJEQlpNQk1HQnlxR1NNNDlBZ0VHQ0NxR1NNNDlBd0VIQTBJQUJBSmplblNZKzR6cE03L3pYODNFUXBKV1BYYTgKTEFwYkluaTBFdzhFVVpZRlZFUXJocDk1S1pqQ1ZwdGJnelh2Sm1yQW5JSTN1Q0hEdzcrQXpWVkFFMU9nQURBSwpCZ2dxaGtqT1BRUURBZ05JQURCRkFpQjVobGltc0JXc1AyZW9ZVG0zZHdQWC8rNUp3MVg3TkZKbWg1dDJ1WG1QCnBBSWhBTmo3ME5SYmVZL1NWWHVJamVRM1c2REZTWko3c1BjU3FvREJ2bk9MVnc4RgotLS0tLUVORCBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0K

the output would show this:

apiVersion: certificates.k8s.io/v1 
kind: CertificateSigningRequest                                                                                                           
spec:
  request: |
    Certificate Request:
        Data:
            Version: 1 (0x0)
            Subject: O = system:nodes, CN = system:node:1zv2yh85-433ed-nxdrl-master-1.novalocal
            Subject Public Key Info:
                Public Key Algorithm: id-ecPublicKey
                    Public-Key: (256 bit)
                    pub:
                        04:02:63:7a:74:98:fb:8c:e9:33:bf:f3:5f:cd:c4:
                        42:92:56:3d:76:bc:2c:0a:5b:22:78:b4:13:0f:04:
                        51:96:05:54:44:2b:86:9f:79:29:98:c2:56:9b:5b:
                        83:35:ef:26:6a:c0:9c:82:37:b8:21:c3:c3:bf:80:
                        cd:55:40:13:53
                    ASN1 OID: prime256v1
                    NIST CURVE: P-256
            Attributes:
                a0:00
        Signature Algorithm: ecdsa-with-SHA256
             30:45:02:20:79:86:58:a6:b0:15:ac:3f:67:a8:61:39:b7:77:
             03:d7:ff:ee:49:c3:55:fb:34:52:66:87:9b:76:b9:79:8f:a4:
             02:21:00:d8:fb:d0:d4:5b:79:8f:d2:55:7b:88:8d:e4:37:5b:
             a0:c5:49:92:7b:b0:f7:12:aa:80:c1:be:73:8b:57:0f:05

basename of working dir should populate correctly

regardless of how camgi is called (. or some/path), it should display the basename of the directory containing the must gather data.

add routes for individual components

it would be cool to use something like vue router to make routes for each yaml file to explore the possibility of opening new tabs with just the data.

Detect must-gather directory structure / subdirectories

Thanks for making this tool available. On my first try using the tool I encountered the following error when specifying a must-gather directory:

$ okd-camgi --webbrowser workspace/cases/01234567/must-gather.local.817596368927932123/
Traceback (most recent call last):
  File "/home/skrenger/.local/bin/okd-camgi", line 8, in <module>
    sys.exit(main())
  File "/home/skrenger/.local/lib/python3.10/site-packages/okd_camgi/main.py", line 49, in main
    content = load_index_from_path(path)
  File "/home/skrenger/.local/lib/python3.10/site-packages/okd_camgi/main.py", line 27, in load_index_from_path
    index_context = IndexContext(mustgather)
  File "/home/skrenger/.local/lib/python3.10/site-packages/okd_camgi/contexts.py", line 274, in __init__
    mapipods = sorted([PodContext(pod) for pod in mustgather.pods('openshift-machine-api')], key=lambda p: p['metadata']['name'])
  File "/home/skrenger/.local/lib/python3.10/site-packages/okd_camgi/interfaces.py", line 99, in pods
    for podname in os.listdir(pods_path):
FileNotFoundError: [Errno 2] No such file or directory: '/home/skrenger/workspace/cases/01234567/must-gather.local.817596368927932123/namespaces/openshift-machine-api/pods'

The issue is that okd-camgi is expecting the path to the inner directory in the must-gather, as the following will work as expected:

$ okd-camgi --webbrowser workspace/cases/01234567/must-gather.local.817596368927932123/quay-io-openshift-release-dev-ocp-v4-0-art-dev-sha256-415e56a6d8d2b8576c59d08a7c57a7d081fa59e7dc3cd8782ed7bcd6080c7c12/
file:///tmp/tmpg77gu4la/index.html

I believe it would be a good user experience if okd-camgi would try to see if the given path is valid and if there is a single subdirectory in there use that one.

There are some files present in all must-gather directories that you can use to more easily detect this (version in the inner directory and timestamp in the outer directory for example). Khizer Naeem has uses the following mechanism for omg: https://github.com/kxr/o-must-gather/blob/master/omg/cmd/use.py#L7

Add a command to export to HTML

Exporting to static HTML would allow us to merge this with https://github.com/openshift/must-gather or add a new CI step to produce the report as an artifact.

Most likely we'd have to rip out the logs and do some kind of lazy loading to keep the report small though

add a current size column to machinsets table on summary page

knowing the current replica counts for each machineset participating in autoscaling would be helpful

sometimes it seems to take a double click to open data

noticed some strange behavior when clicking on the machine and node accordians to open the manifests. it took a double click to open, and sometimes didn't close. not sure what the issue here is.

add machinesets

perhaps add these as part of the machine page to show which machines are in which machinesets

add certificate signing requests

highlighting any that are pending

add current gpu usage to cluster summary

similar to the cpu and memory stats, the summary page should show the current allocatable and capacity values for gpus.

error when processing must-gather when finding invalid token: `<unicode string>`

Hi, nice report tool! My first time using. =)

Problem

I got this error when running from a random MG:

$ okd-camgi --webbrowser $MUST_GATHER_PATH
[...]
DEBUG:root:loading core/nodes yaml from ${MUST_GATHER_PATH}/cluster-scoped-resources/core/nodes/ocp-infra-hm.yaml
Traceback (most recent call last):
  File "/home/myuser/.local/bin/okd-camgi", line 11, in <module>
    sys.exit(main())
  File "/home/myuser/.local/lib/python3.6/site-packages/okd_camgi/main.py", line 45, in main
    content = load_index_from_path(args.path)
  File "/home/myuser/.local/lib/python3.6/site-packages/okd_camgi/main.py", line 27, in load_index_from_path
    index_context = IndexContext(mustgather)
  File "/home/myuser/.local/lib/python3.6/site-packages/okd_camgi/contexts.py", line 110, in __init__
    nodes = NodesContext([NodeContext(node) for node in mustgather.nodes])
  File "/home/myuser/.local/lib/python3.6/site-packages/okd_camgi/interfaces.py", line 67, in nodes
    nodes = self.resources('nodes', 'core')
  File "/home/myuser/.local/lib/python3.6/site-packages/okd_camgi/interfaces.py", line 133, in resources
    resource = self.resource_or_none(f[:-5], kind, group, namespace)
  File "/home/myuser/.local/lib/python3.6/site-packages/okd_camgi/interfaces.py", line 122, in resource_or_none
    resource = yaml.load(man_file.read(), Loader=yaml.FullLoader)
  File "/home/myuser/.local/lib/python3.6/site-packages/yaml/__init__.py", line 114, in load
    return loader.get_single_data()
  File "/home/myuser/.local/lib/python3.6/site-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
  File "/home/myuser/.local/lib/python3.6/site-packages/yaml/composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "/home/myuser/.local/lib/python3.6/site-packages/yaml/composer.py", line 55, in compose_document
    node = self.compose_node(None, None)
  File "/home/myuser/.local/lib/python3.6/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/myuser/.local/lib/python3.6/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/home/myuser/.local/lib/python3.6/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/myuser/.local/lib/python3.6/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/home/myuser/.local/lib/python3.6/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/myuser/.local/lib/python3.6/site-packages/yaml/composer.py", line 127, in compose_mapping_node
    while not self.check_event(MappingEndEvent):
  File "/home/myuser/.local/lib/python3.6/site-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File "/home/myuser/.local/lib/python3.6/site-packages/yaml/parser.py", line 428, in parse_block_mapping_key
    if self.check_token(KeyToken):
  File "/home/myuser/.local/lib/python3.6/site-packages/yaml/scanner.py", line 115, in check_token
    while self.need_more_tokens():
  File "/home/myuser/.local/lib/python3.6/site-packages/yaml/scanner.py", line 152, in need_more_tokens
    self.stale_possible_simple_keys()
  File "/home/myuser/.local/lib/python3.6/site-packages/yaml/scanner.py", line 292, in stale_possible_simple_keys
    "could not find expected ':'", self.get_mark())
yaml.scanner.ScannerError: while scanning a simple key
  in "<unicode string>", line 466, column 1:
    4ef
    ^
could not find expected ':'
  in "<unicode string>", line 467, column 1:
    
    ^

I don't know why this char is inside the YAML collected by must-gather:

$ tail -n1 ${MUST_GATHER_PATH}/cluster-scoped-resources/core/nodes/ocp-infra-hm.yaml
4ef

Resolution

I just removed it and run again and everything works.

Next steps:

Some exception handling could check some kind of stuffs to prevent stop processing.

refactor server command

currently the server is nominally useful but it only works for a single must-gather and doesn't provide much. it would be nice to refactor the server mode so that a user could specify a url that the server would pull the must-gather from. it would then use the tar unpacking method to create the static page to serve. this will allow the server to be useful on a wide level in remote deployment scenarios,

the url might be specified through a path parameter, or perhaps as a form that could allow uploading.