pragmaticflow / nbomber Goto Github PK

Example of proposed output message [XUnit]

[xUnit.net 00:00:16.5057600] Tests.My test [FAIL]
Failed Tests.My test
Error Message:

Assertion #1 FAILED for Scenario Test HTTP (https://github.com) with 100 concurrent users
Assertion #2 FAILED for Scenario Test HTTP (https://github.com) with 100 concurrent users
Assertion #6 FAILED for Scenario Test HTTP (https://github.com) with 100 concurrent users
Assertion #7 NOT FOUND for Test flow GET flow1
Assertion #8 FAILED for Test Flow Test HTTP (https://github.com) with 100 concurrent users
Assertion #9 FAILED for Step Test HTTP (https://github.com) with 100 concurrent users
Assertion #10 FAILED for Step Test HTTP (https://github.com) with 100 concurrent users
Assertion #11 FAILED for Step Test HTTP (https://github.com) with 100 concurrent users

Expected: True
Actual: False
Stack Trace:
at Tests.My test() in /Users/alexkondrashov/Projects/NBomber/examples/NBomber.Examples.FSharp.Assertions/Tests.fs:line 93

Total tests: 1. Passed: 0. Failed: 1. Skipped: 0.
Test Run Failed.
Test execution time: 17.4991 Seconds

Current calculating is very stupid

@alkondrashov istra bistra astra

stdDev, 95%

with ex -> //latencies.[stepIndex].Add(response,latency)

Windows task manager shows such info per process.
I assume that we also can retrieve such info

1. display common table of all results
2. display the graphs like latence < 800ms and latency > 1.2 sec

https://ci.appveyor.com/project/VIPLogic/nbomber/build/0.2.151/tests

let t1 = Task.Delay(scenario.Duration)
let t2 = runProgressBar(scenario.Duration) // this task can affect test results
Task.WhenAll(t1).Wait()

stopFlows(actorsHosts)

Steps to reproduce

• run the simple push scenario (c#)
• when run finishes and and you are asked to repeat the same scenario one more time - type 'y'
• observe exception

System.ArgumentException
  HResult=0x80070057
  Message=An item with the same key has already been added. Key: warm_up_flow
  Source=System.Private.CoreLib
  StackTrace:
   at System.ThrowHelper.ThrowAddingDuplicateWithKeyArgumentException(Object key)
   at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
   at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value)
   at <StartupCode$NBomber>[email protected](StepListener x) in F:\code\NBomber\src\NBomber\Domain\Scenario.fs:line 57
   at NBomber.StepListeners.Init(StepListener[] items) in F:\code\NBomber\src\NBomber\Domain\Scenario.fs:line 57
   at [email protected](ListenerStep listStep) in F:\code\NBomber\src\NBomber\Domain\Scenario.fs:line 112
   at NBomber.TestFlowModule.initFlow(TestFlow flow) in F:\code\NBomber\src\NBomber\Domain\Scenario.fs:line 112
   at [email protected](TestFlow flow) in F:\code\NBomber\src\NBomber\ScenarioRunner.fs:line 27
   at NBomber.ScenarioRunner.runScenario(Scenario scenario) in F:\code\NBomber\src\NBomber\ScenarioRunner.fs:line 27
   at NBomber.ScenarioRunner.Run(Scenario scenario) in F:\code\NBomber\src\NBomber\ScenarioRunner.fs:line 18
   at NBomber.Examples.CSharp.Program.Main(String[] args) in F:\code\NBomber\examples\NBomber.Examples.CSharp\Program.cs:line 18

Possible fix

I think that when initing the listeners, the listeners dictionary should be cleaned first.

Request has been sent and pending

https://github.com/Mpdreamz/shellprogressbar

http://scriptcs.net/

The example should be some business transactional flow, like buying a product via Amazon.
The HTTP server should be faked, like in PubSub sample.

rampUsers(500) over (10 minutes)).throttle(reachRps(100) in (10 seconds), holdFor(10 minutes)

1. Step's name should be unique
2. TestFlow's name should be unique