Git Product home page Git Product logo

pso-analytics's Introduction

pso-analytics

UPDATED: PSO Explorer!

This tool has been significantly improved and relaunched as PSO Explorer.

PSO-Analytics

Storage analytics for Kubernetes: for a detailed description see my accompanying blog post.

Problem Statement

When consuming storage in Kubernetes through PersistentVolumeClaims and using a CSI provisioner, there is no visibility into the storage layer itself. The only information visible from Kubernetes is the capacity allocation per PersistentVolume, which leaves almost all useful storage administration questions unanswered:

  • How much storage does this cluster use on each backend device?
  • How much storage does each statefulset use in total?
  • How much storage usage in each namespace?
  • What is the data reduction across any of the above dimensions?
  • What is the performance across any of the above dimensions?

The origin of this problem is that Kubernetes has only half the necessary info, e.g. labels and statefulsets. And the storage system has the other half: space usage, data reduction, and performance stats. To have full visibility into the storage usage of your kubernetes applications, you need to merge these two views.

PSO-analytics v0.1

PSO-analytics is a deployment that periodically correlates storage usage and data reduction rates across multiple dimensions:

  • statefulset
  • storageclassname
  • backend device (FlashArrays and FlashBlades)
  • label
  • namespace

To install in the 'pso-analytics' namespace:

kubectl apply -f https://raw.githubusercontent.com/joshuarobinson/pso-analytics/master/pso-analytics.yaml

Download and modify the yaml to change the namespace or periodicity of output.

To view the output:

kubectl logs -n=pso-analytics deployment.apps/pso-analytics

PSO-analytics v0.2

Added support for a Prometheus endpoint via the command line flag '--prometheus' on port 9492.

Configure Prometheus to scrape pso-analytics as follows:

    scrape_configs:
      - job_name: 'pso-analytics-monitor'
        static_configs:
        - targets: ['pso-analytics:9492']

Limitations:

  • Needs ability to LIST all pods in all namespaces in order to auto-discover PSO
  • Output is rigid and not easy to parse, query, or visualize
  • Many other interesting aggregations and top 10s missing
  • Performance information not included

pso-analytics's People

Contributors

joshuarobinson avatar

Stargazers

 avatar

Watchers

 avatar  avatar

Forkers

menardorama

pso-analytics's Issues

Prometheus metrics are correctly labeled only for statefulesets

Hi,

First of all thank you for providing such tool that was lacking to monitor purestorage pvc.

I am currently trying to integrate on our prometheus stack (prom operator) and it seems that metrics are useful for statefulset but not for deployment.

I have lots of pvc that are not labeled and I can only see on which namespace they are defined.
Do you think it's feasable for you to add deployments as well ?

Global data reduction is wrong

Global datareduction display is wrong, on my flasharray I have a poor datareduction of 3 and your tool is displaying 92

Are you doing a sum instead of an average or something like this ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.