allure-framework / allure1 Goto Github PK
View Code? Open in Web Editor NEWAllure 1 isn't supported any more, please consider using Allure 2 https://github.com/allure-framework/allure2 instead
License: Apache License 2.0
Allure 1 isn't supported any more, please consider using Allure 2 https://github.com/allure-framework/allure2 instead
License: Apache License 2.0
Add ability to create custom step listener. It should have before & after method for each of:
We need to inform user that some resources are loading and he will have to wait.
This module will be suitable http://chieffancypants.github.io/angular-loading-bar/. @allure-framework, do you like?
Sometimes it needs to do smth like this:
@Step
public void step() {
if(...) {
//do
}
// else - nothing
}
It will be useful to throw some kind of exception, which ignores only this step, but dont fail test.
For example:
@Step
public void step() {
dontIf(object, matcher);
// do smth
}
I've prepared the demo to reproduce: https://github.com/smecsia/allure-camel-spring-test
[ERROR] Failed to execute goal ru.yandex.qatools.allure:allure-junit-plugin:1.2.2:allure (default) on project allure-spring-test: Unable to execute mojo: Compiler errors:
[ERROR] error at (no source information available)
[ERROR] /tmp/allure-spring-test/src/test/java/test/SimpleCamelSpringTest.java:0::0 The type org.springframework.test.context.junit4.SpringJUnit4ClassRunner cannot be resolved. It is indirectly referenced from required .class files
For JUnit:
Parametrized, Suites, Categories
Когда в степ передается null:
@Test
public void test() throws Exception {
step(null);
}
@Step("{0}")
public static void step(String array) {
}
возникает следующая ошибка
java.util.NoSuchElementException: null
at java.util.LinkedList.getLast(LinkedList.java:255)
at ru.yandex.qatools.allure.storages.StepStorage.getLast(StepStorage.java:21)
at ru.yandex.qatools.allure.Allure.fire(Allure.java:43)
at ru.yandex.qatools.allure.aspects.AllureStepsAspects.stepFailed(AllureStepsAspects.java:50)
at my.company.MyTest.step(MyTest.java:70)
at my.company.MyTest.test(MyTest.java:56)
Test mechanism Time elapsed: 0.19 sec <<< ERROR!
java.lang.NullPointerException: null
at ru.yandex.qatools.allure.Allure.fire(Allure.java:65)
at ru.yandex.qatools.allure.junit.AllureRunListener.testFinished(AllureRunListener.java:39)
at org.junit.runner.notification.RunNotifier$7.notifyListener(RunNotifier.java:191)
at org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:61)
at org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:188)
at org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:39)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:277)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray2(ReflectionUtils.java:208)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:158)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:86)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:95)
For view titles and descriptions in javadocs for tests, annotations should have java.lang.annotation.Documented
annotation
Проблема в том, что у нас сейчас степы хранятся в thread local storage. А JUnit запускает тесты с таймаутом в отдельном треде.
Write log to allure-report.log file to allure-results folder
When use
@Step("Step: {0}")
public void clickOn(HtmlElement... elements) {
...
}
Expects: Step: ["Element1", "Element2"]
But: Step: [Lru.yandex.qatools.htmlelements.element.HtmlElement;@42ed46a7
occurs.
It will be nice to print varags as list of String
Dont forget about primitive types (int... ints
)
For parameterized tests it is convenient to navigate in testsuite with Up and Down keys. Choice of Up and Down is an option, other hot keys are acceptable.
add variable {parametrizedName} to Title annotation
ref: allure.xsd
skipped
status is not exhaustive, there is two huge difference:canceled
, due to some previous fail, or environment settings (there is a kind of failure message)pending
, test is not yet implementedseverity
is very similar to labels, except that it is test-case only field. imho, it may be removed from model, and labels used instead. when distribution graph is been built, it make sense to use labels in general.parameters
and attachment
, maybe it may be unified to special attachment 'key-value-storage'. "other" is not the case, because key-value-storage attachment may be directly used in reports for dimensions.value
, it may be convenient just to make it optional. For reports this means no need to provide distribution of such dimension.Now all text use arial,sans-serif
, and it ugly.
It will be nice to use "Helvetica Neue",Helvetica,Arial,sans-serif
everywhere, because of it pretty! :)
"Helvetica Neue",Helvetica,Arial,sans-serif
Еще мне сложно придумать как написать это по английски, но простой arial не такой сглаженный и очень угловатый. При большом количестве буков это утомляет
This module must contain global allure config, for example name conventions
Now we use two big JSONs. Need to split it to more smaller files. For each tab should match at least one special JSON.
xUnit.json
- based on testsuite.json
. Info about testsuite and short info about testcasetestcases.json
- array of short info about all testcases (one for graph and timeline tab){uid}-testcase.json
- full info about single testcaseSometimes we need to launch a huge amount of test-cases at once (for example 40k test-cases) and it is almost useless to generate report with passed tests because nobody will look at them. Plus report becomes very heavy: it takes about 3 min to finish all tests and 10 min to generate a report which becomes so heavy that lags occur while browsing it
It would be nice to create a property like "allure.only.defects=true/false" and if it is true then build report only with failed and broken test-cases.
Генерируются одинаковые уиды у тескейсов, из-за этого артифакты отображения
Unsynchronized method createResultsDirectory() can throw AllureException when running tests in parallel
add feature.json
It seems need to implement Reporter with specific to
Allure functionality, taking in account that ScalaTest may have nested suites
I hope 'ru.yandex.qatools.allure.annotations' will work as expected without any actions.
This should allow to start using it with sbt by passing reporter param to Runner:
something like
val project = project(".")
.libraryDependencies(
"ru.yandex.qatools.allure" % "allure-scalatest-adaptor" % Versions.allure
)
#and for scopes...
testOptions in scope <+= Tests.Argument(TestFrameworks.ScalaTest, "-C", "ru.yandex.qatools.allure.scalatest.AllureTestReporter")
and then, incrementally,
new sbt-plugin might add this dependencies for all test, providing step that executed after test, testQuick, testOnly sbt command to generate report.
@bvenners, @cheeseng, you might be interested in this. If you have ideas how to do it better, please share your thought.
Inspired By: Selenium Camp 2014
Суть:
Есть тест со status='skipped'
со вложенным failure
:
<?xml version="1.0" encoding="utf-8"?>
<ns0:test-suite xmlns:ns0="urn:model.allure.qatools.yandex.ru" start="1392388427993" stop="1392388428016">
<description>
This module contains basic examples of tests
</description>
<name>demo.test_basic_examples</name>
<test-cases>
<test-case severity="normal" start="1392388428000" status="skipped" stop="1392388428000">
<name>test_skip</name>
<description>this test is skipped</description>
<failure>
<message>skipped</message>
<stack-trace>Skipped: for a reason!</stack-trace>
</failure>
<attachments/>
<steps/>
</test-case>
</test-cases>
</ns0:test-suite>
Но причина skip'а не показывается: вместо нее стандарная плашка This test has been skipped
:
Ожидается, что Failure так же будет показываться с причиной, которая берется из stack-trace
-- или как-то еще.
It will be nice to build log some kind of:
-> START STEP: SettingsPageSteps.clicksOnFolder(folder)
<- DONE STEP: SettingsPageSteps.clicksOnFolder
-> START STEP: DefaultSteps.shouldBeEnabled([Кнопка «Новая папка», Кнопка для папки «Создать правило»])
<- DONE STEP: DefaultSteps.shouldBeEnabled
With all args and name of method (and class where is it). Ability to turn on/off logging in view as in allure-report (with customization of step-names) also would be great.
It will be nice to see, if step not executed in TEST method, but in before or smth else
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.