Git Product home page Git Product logo

pymeter's Introduction

pymeter

All Contributors

Simple JMeter performance tests API for python

Powered by JMeter-DSL and pyjnius

Version Generic badge Generic badge Documentation Status

License






Load testing with JMeter using python!

Read the documentation here
Read about performance benchmarking here

JMeter is one of the most popular and long standing load testing tools.
The original implementation is a gui based tool to script load test scenarios in a hierarchical structure, however this came with limitations and shortcomings.

For once, upgrading JMeter versions is painful, as it involved manually downloading and deploying executable files. This became very clear when log4j vulnerability was discovered, and software developers needed to instantly upgrade their log4j versions. With JMeter, this was even more painful without a proper package management system such as maven or gradle.

Other limitations include difficulty to share code between different projects, using source control management tools such as git or svn. It is quite difficult to extend JMeter and it requires a GUI editor which means to use additional development environment instead of using a single IDE for all needs.

The awesome folks at abstracta have put up an amazing amount of work to deliver JMeter-DSL, which allows developers to use plain Java to script their load test scenarios, and pretty much solve all the pain mentioned above.

pymeter project is aimed to capitalize on the success of JMeter-DSL and extend it to the python community! Using pyjnius developed by Kivy, it is possible to bridge between JMeter-DSLs classes written in Java and reflect them into python's runtime environment without spawning up java runtime and relying on costly inter-process communication.

Pre-requisites:

  1. python version 3.9 or higher - download
  2. Java version 8 or 11 - download
  3. JAVA_HOME environment variable set - read

Install pymeter

>>> pip install pymeter

simple pymeter script:

"""unittest module"""
from unittest import TestCase, main

from pymeter.api.config import TestPlan, ThreadGroupWithRampUpAndHold
from pymeter.api.postprocessors import JsonExtractor
from pymeter.api.reporters import HtmlReporter
from pymeter.api.samplers import DummySampler, HttpSampler
from pymeter.api.timers import UniformRandomTimer


class TestTestPlanClass(TestCase):
    def test_1(self):
        json_extractor = JsonExtractor("variable", "args.var")
        timer = UniformRandomTimer(1000, 2000)
        http_sampler = HttpSampler(
            "Echo",
            "https://postman-echo.com/get?var=${__Random(0,10)}",
            timer,
            json_extractor,
        )
        dummy_sampler = DummySampler("dummy ${variable}", "hi dummy")
        tg = ThreadGroupWithRampUpAndHold(
            10, 1, 60, http_sampler, dummy_sampler, name="Some Name"
        )
        html_reporter = HtmlReporter()
        tp = TestPlan(tg, html_reporter)
        stats = tp.run()
        print(
            f"duration= {stats.duration_milliseconds}",
            f"mean= {stats.sample_time_mean_milliseconds}",
            f"min= {stats.sample_time_min_milliseconds}",
            f"median= {stats.sample_time_median_milliseconds}",
            f"90p= {stats.sample_time_90_percentile_milliseconds}",
            f"95p= {stats.sample_time_95_percentile_milliseconds}",
            f"99p= {stats.sample_time_99_percentile_milliseconds}",
            f"max= {stats.sample_time_max_milliseconds}",
            sep="\t",
        )
        self.assertLess(stats.sample_time_99_percentile_milliseconds, 2000)


if __name__ == "__main__":
    main()

In this example, the standard python unittest was used to execute the test code, however pymeter is framework agnostic and can be used by any other testing framework

File Structure

|   .coverage
|   .gitignore
|   .pylintrc
|   cosmic-ray-config.ini
|   LICENSE
|   make.bat
|   Makefile
|   poetry.lock
|   pyproject.toml
|   README.md
|   tox.ini               
+---source
|   |   conf.py
|   |   index.rst
|   |   
|   +---_static
|   \---_templates
+---src
|   \---pymeter
|       |   __init__.py
|       |   
|       +---api
|       |   |   config.py
|       |   |   postprocessors.py
|       |   |   reporters.py
|       |   |   samplers.py
|       |   |   timers.py
|       |   |   __init__.py
|       |   |   
+---utests
|   |   test_postprocessors.py
|   |   test_reporter.py
|   |   test_sampler.py
|   |   test_test_plan.py
|   |   test_thread_group.py
|   |   test_timers.py
|   |   __init__.py
|   |   

Code styling

black used for auto-formatting code read,

pylint used for code linting and pep8 compliance read,

mypy used for type hinting read,

perflint pylint extension for performance linting read

cosmic-ray Python tool for mutation testing read

Contributors

Antonio Zaitoun
Antonio Zaitoun

๐Ÿ“–
Eldad Uzman
Eldad Uzman

๐Ÿš‡ โš ๏ธ ๐Ÿ’ป

links

  1. JMeter Dsl
  2. pyjnius

pymeter's People

Contributors

allcontributors[bot] avatar dependabot[bot] avatar eldaduzman avatar minitour avatar yanzkosim avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

pymeter's Issues

jmeter-java-dsl upgrade

Hi

It seems that pymeter uses jmeter-java-dsl 1.23.3 (as seen in pom.xml):

  <artifactId>jmeter-java-dsl</artifactId>
  <version>1.23.3</version>

Implement `Vars`

As a developer I would like to declare custom variables.

Here is my current implementation

from typing import Optional

from pymeter.api import TestPlanChildElement

class Vars(TestPlanChildElement):
    def __init__(self, values: Optional[dict[str, str]]) -> None:
        self._vars_instance = TestPlanChildElement.jmeter_class.vars()
        if values:
            for k, v in values.items():
                self.set(k, v)
        super().__init__()

    def set(self, key: str, value: str):
        # TODO: perform type checking
        self._vars_instance = self.java_wrapped_element.set(key, value)
        return self

Usage Example:

variables = Vars({'api': config.get('api.endpoint')})

Use of decorators for more idiomatic experience

This is more of a suggestion rather than an issue.

This is how I am defining my custom decorators so I can use them in my unit tests:

@inject
def http_get_performance_test(url: str,
                              threads=10,
                              duration=60,
                              html_reporter=Provide[Container.html_report]): # injected from dependency injection for a unified report
    def decorator(fun):
        def wrapper(*args, **kwargs):
            timer = UniformRandomTimer(1000, 2000)
            http_sampler = HttpSampler(
                fun.__name__,
                url,
                timer
            )
            tg = ThreadGroupWithRampUpAndHold(
                threads, 1, duration, http_sampler, name="Some Name"
            )
            tp = TestPlan(tg, html_reporter)
            fun(*args, **kwargs, test_plan=tp)

        return wrapper

    return decorator

When in use the unit tests look a lot cleaner:

class TestTestPlanClass(TestCase):
    @e2e.utils.decorators.http_get_performance_test(
        url="https://postman-echo.com/get?var=${__Random(0,10)}",
        threads=10,
        duration=30
    )
    def test_2(self, test_plan):
        stats = test_plan.run()
        self.assertLess(stats.sample_time_99_percentile_milliseconds, 2000)

I believe you can achieve a much nicer Developer Experience with decorators (you can also chain them)

Add support for the User Parameters preprocessor

Request:
Add support for the User Parameters preprocessor

Use case:
I have a JMeter test plan that uses User Parameters because the value needs to be different on each iteration. Due to the small size of the test, it performs better with User Parameters than with a csv data source.

create github actions workfllow

Need to perform mvn dependency:copy-dependencies to download all jar dependencies and then to append them in the python package

CSVDataSet With nested data types

Hi everyone!

I'm quite a newbie with Jmeter and pymeter. So, I'm trying to pass nested data via CSVDataSet.

What is a problem?
I have an application with an endpoint expecting list of string in body:

from typing import List

from pydantic import BaseModel
from fastapi import Request, FastAPI

app = FastAPI()

class Human(BaseModel):
    age: List[int]

# human: Human,
@app.post("/human/update")
async def update_human(human: Human, request: Request):
    a = await request.json()

And I have a pymeter TestPlan for such app:

local_data_schema = {
    "age": "${age}",
}

local_dcsv_data_set = CsvDataset("kek.csv")

http_sampler = (
    HttpSampler("echo_get_request", "http://0.0.0.0:8081/human/update")
    .post(local_data_schema, ContentType.APPLICATION_JSON)
)

thread_group = ThreadGroupSimple(
    1, 1, http_sampler, local_dcsv_data_set
)
test_plan = TestPlan(thread_group)
stats = test_plan.run()

with kek.csv like

,age
0, [1,2,3]

And all data that comes to andpoint is strings. As I see CSVDataSet uses , as delimeter, so even strings that comes to endoint are separated and value that comes first is '[1', so i can get the whole list. I've found the way to pass such nested data like python dict, but there is no option to send list of dicts, so I can't make my load test various.

I will be happy if you help me with that problem

Consider exposing `children` as a function

Consider exposing children as a function in BaseThreadGroup (and other places where applicable).

Sometimes I don't want to pass the children in the init because I don't have them yet and would rather add them later.

Current workaround is to do something like this:

tg = ThreadGroupWithRampUpAndHold(10, 1, 60, name="Some Name")
tg._ramp_to_and_hold_instance.children(*[c.java_wrapped_element for c in children])

What I would like to see is:

tg = ThreadGroupWithRampUpAndHold(10, 1, 60, name="Some Name")
tg.children(children)

Add support HTTP2 & gRCP

Hi

Nice, project, thanks for doing that.
I am curious, do you plan to add support for gRPC samples?
I will try to implement some custom samplers for POC.

Unable to access response data from HttpSampler

I am attempting to access the response data from a sampler (generic in this example) via the JsonExtractor:

extractor = JsonExtractor("variable", "args.vendorname")
http_sampler = (HttpSampler("echo_get_request", "[https://postman-echo.com/post"](https://postman-echo.com/post%22), extractor)
                        .post("vendorname: ${vendorname}, description: ${description}, catalognumber: ${catalognumber}, size: 30, threshold: 0.0", ContentType.APPLICATION_JSON))

In the example in the docs I see that variable is accessed inside another sample like this:

dummy_sampler = DummySampler("dummy ${variable}", "hi dummy")

I tried to access that variable as a python obj, but nothing worked. Is there a way to read the value or access the response(s) from the HttpSampler directly?

Cannot use .header() and .post() at the same time in HttpSampler

i have been trying to send auth token and post data at the same time using this code

http_sampler = HttpSampler("sampler 1",url, timer,).header("Authorization", f"bearer {idToken}").post({"name":"abc"},ContentType.APPLICATION_JSON)

but it gives an error

AttributeError: 'us.abstracta.jmeter.javadsl.http.DslBaseHttpSample' object has no attribute 'post'

i tried to solve it myself and found that the value of self.java_wrapped_element when .header() is called is us.abstracta.jmeter.javadsl.http.DslHttpSampler but when .post() is called immediately after it, then the value changes to us.abstracta.jmeter.javadsl.http.DslBaseHttpSampler

but i am not good with classes so i couldn't find any solution for it

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.