Hi. Is there any support for putting @metatag featureName inside the .feature files so you can filter what tests should be run or not run based on the meta tags? I searched the code, but did not see anything like that.

.NET Core already has test filtering by attributes in the .cs files (https://docs.microsoft.com/en-us/dotnet/core/testing/selective-unit-tests) but support for meta tags inside the gherkin files is one of the key attributes of the BDD.

Don't get me wrong. Xunit.Gherkin.Quick is awesome!!!!!!!!! But, testers will not be happy with out support for meta tags inside the gherkin files

The README contains this helpful line:

Important: change feature file properties to ensure it gets copied into output directory. Set the value of Copy to Output Directory to Copy Always or Copy if Newer.

Maybe it might be useful to mention that when using the newer Microsoft.NET.Sdk project files that it is very easy to do this automatically for all feature files in the project by adding a glob pattern to the project file. For example:

  <ItemGroup>
    <None Update="Features\**\*.feature">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </None>
  </ItemGroup>

This has the advantage that you don't need to worry about forgetting to set the property on new feature files.
Of course this will only work if the project file starts with <Project Sdk="Microsoft.NET.Sdk">

Hi @ttutisani

Just pulled latest master to se the status of your project and if there might be something i could contribute.

I experience that 10 of the tests fail.

• Given When Then :: Simple parameter types [FAIL]
• Discounts :: All Deals :: Coupons :: 2 [FAIL]
• Discounts :: All Deals :: Coupons :: 4 [FAIL]
• Discounts :: All Deals :: Daily deals :: 1 [FAIL]
• Discounts :: All Deals :: Bundles :: 1 [FAIL]
• Discounts :: All Deals :: Coupons :: 5 [FAIL]
• Discounts :: All Deals :: Coupons :: 1 [FAIL]
• Discounts :: All Deals :: Coupons :: 3 [FAIL]
• UnitTests.StepMethodInfoTests.DigestScenarioStepValues_Sets_Primitive_Values [FAIL]
• UnitTests.PrimitiveTypeArgumentTests.DigestScenarioStepValues_Takes_Value_By_Index(index: 2) [FAIL]

the last two fail with a format exception
System.FormatException : String was not recognized as a valid DateTime.

This looks like it is related to the way that parameters are parsed to their specific type in the PrimitiveTypeArgument file.

Value = Convert.ChangeType(argumentValues[_index], _parameterInfo.ParameterType);

it looks like adding the CultureInfo.InvariantCulture parameter fixes the issue.

I've followed the instructions for use, and I am seeing all the individual steps showing up in the VS test runner, and none of them will run.

I've added a complete example here:
https://github.com/csurfleet/GherkinQuickIssue

Any help would be brilliant, I'm running out of .net core options for gherkin now!

Both ScenarioAttribute and KeyWordAttributeBase inherits from the Xunit.FactAttribute, this means that both the individual steps and the scenario gets picked up by the test runner and displayed in the Test Explorer.

The individual steps are instantiated and invoked in Feature.cs which means that the relationship between KeyWordAttributeBase and FactAttribute does not seem important.

I did a small test where is changed

public abstract class KeyWordAttributeBase : FactAttribute

to just use the Attribute type

public abstract class KeyWordAttributeBase : Attribute

The Scenario seems to run fine and the upside is that Steps no longer show up in the Test Explorer.

I know that several people have already mentioned about refactoring. Before anybody commits to do so, this topic is here to discuss it in advance.

If no actions are observed, I will post my thoughts around it later. Meanwhile:

Regardless of who does it, in my opinion, refactoring should make concepts of this framework more explicit, easier to digest and change further in the future.

Refactoring should be driven by readability and expressiveness, rather than by technical aspects only (such as dependency injection or splitting into smaller interfaces). I believe such focus will ensure successful growth of the project, instead of dragging it down long term and eventually slowing it down.

@ttutisani What is your plans for release(s) and scope?

While the Specifications and Scenarios show up brilliantly in the test runner, there is also an entry for each step. These extra entries are not runnable, but do clog up the interface, and will do so especially once many scenarios are implemented. I'd like these extras to be hidden if possible.

Test Explorer cannot find the test also when trying to create a new feature I get a file not found.
Using empty solution with only a single feature/scenario.
Target Framework: .NET Core 2.2
MS Visual Studio Enterprise 2017 Version: 15.9.7
Not sure if it is an issue with Gherkin Quick or local install issue.
Including test solution.
TestXunitGherkinQuick.zip

After creating a vanilla project that includes both Xunit.Gherkin.Quick and Xunit.DependencyInjection, Xunit.Gherkin.Quick throws an ArgumentNullException.

System.ArgumentNullException : Value cannot be null. (Parameter 'testOutputHelper')
  Exception doesn't have a stacktrace

This seems to be caused in Xunit.Gherkin.Quick.ScenarioOutput line 20.

In any c# class of the project I can inject the testOutputhelper just fine, any idea whats wrong here?

Minimal reproduction example can be found here:
https://github.com/fredericbirke/gherkin-bug

A recent pull request (#39) made me realize that Output property is null during the feature class constructor execution. However, since it's a derived property from base Feature class, developers may naturally expect that they can use the value in the constructor as well (as if the base class constructor initializes it, but it doesn't).

So, Output property is designed in a misleading way. I should probably make it internal so that derived classes don't even have access to it. If they need to acquire access to output, then they can simply add a constructor parameter of type IOuptutHelper, which will be injected by Xunit out of the box. Then, derived class will have an option to keep the instance of the output as a field/property for later use, or use it inside the constructor as well.

Since in some cases (when usage is needed in the constructor) derived class will need to add the parameter to the constructor, it makes sense to to make it the only way of getting the output instance.

So, Output property on the base class should be internal.

Keeping it here until time allows to do this change.
Also, after this change, major version will increase, because this is not a backwards compatible change (minor incompatibility, but still).

Scenario Outline example:

Scenario Outline: eating
  Given there are <start> cucumbers
  When I eat <eat> cucumbers
  Then I should have <left> cucumbers

  Examples:
    | start | eat | left |
    |    12 |   5 |    7 |
    |    20 |   5 |   15 |

I have not received much demand for report formats for test outputs but there may be some interest (somebody asked this question on Reddit).

If you are reading this issue and want to generate test output in a certain format, please reply here and explain your needs.

All responses will be considered for future development if there is demand and if it fits in the scope of this framework.

If you use "Xunit.Gherkin.Quick," please post here information around who you are (company/GitHub project name), how you use it (use case), why you like it (deciding factors), and so on. Everything is optional, provide as much or as few details as you want.

This discussion can help others decide to use or not to use the framework. Also, with more feedback, more people might be interested in considering this fantastic framework. As a result, the framework will be more popular, better tried through more use cases, with better quality, and more features too! Everybody benefits!

Thank you for your time!

Gherkin step keywords include * as well as Given, When, Then, And, But. It is normally used as a list of steps rather then precondition, action, result. Is it possible to implement this functionality?

References:
https://relishapp.com/cucumber/cucumber/docs/gherkin/using-star-notation-instead-of-given-when-then
https://stackoverflow.com/questions/27361647/cucumber-step-definitions-must-use-given-and-then-etc

If you reference the xunit.gherkin.quick nuget from a full-framework xunit project, the scenarios will never be discovered.

I'm not sure if this is an extra configuration step I'm missing, something that needs fixing in the nuget, or if it is simply not possible. I can throw up a branch with the consumer project converted to demonstrate too.

Hi,

I'm trying to work with a simple project in macOS, and when I run dotnet test, any test are discovered.

1. dotnet info

SDK .NET Core (reflétant tous les global.json) :
 Version:   2.2.100
 Commit:    51868761f2

Environnement d'exécution :
 OS Name:     Mac OS X
 OS Version:  10.14
 OS Platform: Darwin
 RID:         osx.10.14-x64
 Base Path:   /usr/local/share/dotnet/sdk/2.2.100/

Host (useful for support):
  Version: 2.2.0
  Commit:  1249f08fed

2. The csproj of my project

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netcoreapp2.2</TargetFramework>

    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.9.0" />
    <PackageReference Include="xunit" Version="2.4.0" />
    <PackageReference Include="Xunit.Gherkin.Quick" Version="3.4.0" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.0" />
  </ItemGroup>

   <ItemGroup>
    <None Update="features\bookvtc.feature">
      <CopyToOutputDirectory>Always</CopyToOutputDirectory>
    </None>
  </ItemGroup>

</Project>

1. The feature file and a simple cs file

Feature: VTC Testing

Given book a vtc

using Xunit.Abstractions;
using Xunit.Gherkin.Quick;
using Feature = Xunit.Gherkin.Quick.Feature;

namespace domainacceptance.steps
{
    [FeatureFile(@".\features\bookvtc.feature")]
    public sealed class bookvtc : Feature
    {
        [Given(@"book a vtc")]
        public void Test()
        {
        }
    }
}

I already check the feature file location in my bin folder, and it's as it place.

I also followed this quick tutorial, to check if my project is good :
http://bristoltech.net/using-cucumber-with-net-core/

And after some check, I decide to download the project located in github and to run dotnet test, but I have the same issue, any test was discovered.

Have you an idea ?

Hi,
In specflow we can access the same step definition using multiple steps. For example: A "Given" step in a test scenario can be used as a "When" step in another scenario. So instead of duplicating the code we can use:

[Given(@"I search for '(.)'")]
[When(@"I search for '(.)'")]
public void ISearchFor(string searchTerm)
{
...
}

It would be very useful as well in the case of "And" step in Xunit.Gherkin.Quick for example:

[And(@"I search for '(.)'")]
[When(@"I search for '(.)'")]
public void ISearchFor(string searchTerm)
{
...
}

The Features text files are very particular about text encoding.
I had an issue where a new Feature file and corresponding test class would not be discovered and would not run (locally or on build box). But many existing features existed and run without issue.
I created new files with new names and cleaned up the text and simplified the Feature and test to absolute basics. Nothing worked, the test would not be discovered.

So I took a Feature file that I knew worked and copied it as a starting point. Then manually edited it so that it eventually looked like the one the didn't work. But this time the test could be discovered and run, without issue.

Looking at the two files (Working.feature and NotWorking.feature) in Notepadd++ with special chars highlighted or in BeyondCompare revealed nothing useful. However, I don't doubt that there was something abnormal about the encoding.

Summary
If you have a Feature file that refuses to be discovered by the test runner, but have other files that are being discovered fine, copy one of the good ones as a starting place and avoid copy-paste from PDFs etc.

DoD for each code change.

Hi,

From what I can read from Gherkin document a scenario outline with examples should run one test for each rows :

https://cucumber.io/docs/gherkin/reference/#scenario-outline

Currently, the implementation only run the last row of the examples.
This is a problem because scenario output with examples have no longer interest.

Specflow supports predefined hooks that can be used to perform additional automatic logic on specific events, such as before/after executing a scenario. For example: [BeforeTestRun], [BeforeFeature], [BeforeScenario], etc. Is there a plan to support this as well?

Many people asked about hooks for before/after scenario execution. For this test class Constructor and Dispose can be used.
I need to document this so that people find this answer easier.

is there a way to go to step definition directly from feature file?
Like by pressing F12

We use Pickles (https://github.com/picklesdoc/pickles) to package up our results and feature files as a nice readable website as part of our CI/CD process. Currently, Pickles is able to match test results to scenarios correctly for everything but scenario outlines - these all end up blank in the Pickles output despite having entries in the test output.

I am generating a XML results file by running dotnet test --test-adapter-path:. --logger:xunit -r /results. Scenario outline results in the file look like this:

    <test name="Feature Name :: Scenario Outline Name :: Example Name :: #1" type="Path.To.Namespace" method="ScenarioOutline" time="0.5140000" result="Pass">
        <output>Given A: PASSED
When B: PASSED
Then C: PASSED

</output>
        <traits>
          <trait name="FeatureTitle" value="Feature Name" />
          <trait name="Description" value="Feature Name :: Scenario Outline Name :: Example Name :: #1" />
          <trait name="Category" value="@tag1" />
          <trait name="Category" value="@tag2" />
          <trait name="Category" value="@tag3" />
        </traits>
      </test>

I'm not sure if the issue is with Xunit.Gherkin.Quick or PicklesDoc not interpreting correctly (it claims to understand 'xunit2' results files, but I'm not sure if this is universal or just SpecFlow-specific xunit results.), so I figured I'd ask.

I have been struggling a bit with a strange issue since I added a constructor to my Features in order to have a before and after scenarios as per https://github.com/ttutisani/Xunit.Gherkin.Quick/blob/master/docs/before-after-scenario-hooks.md

I included a constructor that contains an asynchronous call that should be awaited.

[FeatureFile("./Features/MyFeature.feature")]
public class MyFeature
    : Feature
{
    public MyFeature()
    {
        var container = MyDIHelper.RegisterAllDependencies();
        var eventStoreManager = container.Resolve<IEventStoreManager>();
        eventStoreManager.StartAsync().GetAwaiter().GetResult();
    }

    [Given(@"whatever")]
    public async Task GivenWhateverAsync()
    {
        // whatever given
    }

    [When(@"something happens")]
    public async Task WhenSomethingHappensAsync()
    {
           // whatever operation
    }

    [Then(@"assertion")]
    public void ThenWhatever()
    {
          // whatever assertion
    }
}

I have several features working in a similar way. I think this is important to notice.

When I run the whole test project either with resharper (All tests from solution) or with
dotnet test MyApp.ComponentTests/MyApp.ComponentTests.csproj -c Release
or with
dotnet vstest *ComponentTests/bin/Release/**/*ComponentTests.dll

I ran the dotnet commands in Windows and Linux, it didn't make any difference.

The test execution gets stuck at the Starting test execution, please wait.. step. However if I run the Features one by one it executes properly and it succeeds. So I suspected it had something to do with a bunch execution (in parallel or not).

I tried adding a xunit.runner.json with

{
  "parallelizeTestCollections": false,
  "maxParallelThreads": -1
}

but the result was the same, test execution stuck, no errors, just stuck forever.

Then I decided to make the asynchronous call without awaiting inside my constructors, like this:
eventStoreManager.StartAsync()

and it worked properly although sometimes the tests fail as you can imagine because sometimes the event store has not yet started the connection by the time the connection is used. So it's not ideal.

I suspect there is some issue with this framework.

If you use a scenario:

Scenario Outline: Register fails for under 18s
Given 'Sue Smith' registers with the site
        | Age     |
        | <Age> |
When she registers
Then the registration fails
Examples:
    | Age |
    | 17   |

With c#:

[Given(@"'(\w+ \w+)' registers with the site")]
 public void Given_Registration(string name, DataTable data)
 {}

I would expect data to include '17' rather than '<Age>'

I'm happy to do the work for this if you agree with the way it should work @ttutisani ?

Hello Tengiz,

I didn't see/know how to contact you so I created this issue.

I really appreciate the work that you've put into this project, so I've created a visual studio extension to help get this thing up to par(or close to) the specflow project. I'll be looking to contributing here as well.
VS extension: https://github.com/mihaibanu/Gherkin.Xunit.Quick.VS.Extension

Right now it doesn't do much, it adds two item templates for the "Calculator.feature" and the Calculator.cs, and it automatically sets the "CopyToOutputFolder" flag to true.

Looking to add towards automatic navigation, step generation and code completion.

I use specflow extensively for .net framework projects and I'm wondering if there is some sort of IoC mechanism that this framework can take advantage on. For example, at the start of each scenario it would be cool if the application under test could create the containers and behave just like the main application would. If not, how can I help?

Regards,
Mihai

When I execute a test, I would like to see, in the test output, the steps of the BDD specification which have been executed, along with an indication of whether the step passed or failed

As of Gherkin 6, but I am not yet sure whether the AST produces Rule objects yet.

It would be great if the scenarios in the test runner behaved like normal tests - double clicking to take you to the test in code. Ideally this would open the scenario file, or if not the steps file would be reasonable

I'm not sure if this is even possible, but I'll take a look in the next few days.

Hi, was wondering if tables or multi-line text is supported in this implementation? My attempts to use them weren't working.

Thanks much.

Can you confirm that currently it is not possible to define reusable steps in a separate implementation file, to which several features can be mapped?

I see that there is an inheritance-based approach chosen, however it requires all steps to be in 1 file, and at a first glance does not allow to have several features "sharing" them ...

One of the main features of gherkin is the support for writing scenarios and features in your own language. It seems like this project only supports steps written in english.

Please add support for more languages.
A list of translations for the different keywords can be found here:
https://github.com/cucumber/cucumber/blob/master/gherkin/gherkin-languages.json

This started happening with visual studio 2019 I guess because I have not seen it before.

If I have a feature that has no scenario or scenario outline, then I can see the base Feature class respective method in the test explorer. It is grayed out and does not execute, but it's still not good user experience. For example, this is what I see for the feature without outlines in the feature file:

I should not be seeing ScenarioOutline method ideally.

I create a feature file with 2 scenarios, but when I looked at the VS Code/.NET Test Explorer I see only 1 test for the feature, not 1 for each scenario, as stated here:

"If the feature has multiple scenarios, add them to the same feature file. They will show up as additional tests in the test explorer. And they will need additional methods in the same feature class for execution."

Did something change recently, or I have done something wrong?

Wondering if you can give me any pointers on how to debug the base project so as to observe it discover and define the scenarios, etc. This appears to be something that's happening at compile time? Thanks much.

Nuget complains:

We found the following issue(s):
<licenseUrl> element will be deprecated, please consider switching to specifying the license in the package. Learn more.
We recommend that you fix these issues and upload a new package. Read more

Step definitions do not appear to support async methods. You can execute async methods just fine but scenario execution immediately continues to the next step, which obviously causes problems.

I think this issue is here:

This would need to be made a bit more clever to detect the return type of the step function.

Hi there, I'm having trouble getting the tests running in Visual Studio, both 2017 and 2019. I've followed the Getting Started and the test is marked as "ignored" with the message

"VsTest test case is missing. Rebuild the project and try again".

I've tried this person's repo and I get the same result:

https://github.com/sajohnstone/gurkin

This is more a questions than issues. I've being using the Xunit.Gherkin.Quick to run UI tests in .netcore and selenium. I have got this working but have these issues:

1. I use the tags on the scenario to perform various functions. e.g Set the browser windowsize. I need to be able to access the current scenerios tags and perform so code on start. Is this possible?
2. At the end of scenario, can we the determine in the steps code if the test has passed or failed. I want to keep the web browser open and just navigate home if the test passed and shutdown the browser if it failed. This help performance of running these tests. (approx 1500 scenarios)

I have used Cucumber with Java and Python in the past, but need to test a .NET console app. In the readme file there is a statement "If you use command line to run unit tests, simply run them as always. You should see the scenario full name in the results as a newly added unit test name."

Can anyone provide a specific example of how to do this?

If I run the command "dotnet test", I get a message "Starting test execution, please wait..." and then get a warning "No test is available in [DIRECTORY] Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again."

I installed the framework through nuget and I can't see the documentation for methods. Must be something with how the documentation generation is configured.

By default, when the [FeatureFile] attribute is missing, the framework finds the feature file in the base folder. By looking at the class namespace, it could also check the subfolder, assuming that the class namespace follows the folder structure.

This can be convenient when splitting various scenarios into separate feature files. In practice, need to split scenarios between different files comes when they have different backgrounds. Since there can only be one background per feature file, split will be necessary.
After splitting, same feature class cannot handle both feature files. If the feature class could support multiple feature file attributes, then this limitation would be removed.

e.g. currently in the test solution there are 2 feature classes: AddTwoNumbers, and AddNumbersTo5. These 2 feature classes could be consolidated into 1 if the above limitation is resolved.

Hi,
In specflow we can use the tag '@ignore' to skip a test. The use case for it, for example, if the test is failing for a known bug and you want to skip running it until the bug is closed. Are you going to support it?

Synopsis

On my team we use Specflow/Gherkin tests to describe the various white box test of our services.

The tests are perfect for release documentation as we can map them one to one to the release documentation/protocol we need to satisfy with every software update.

The tests are very comprehensive and can some times span multiple services to enable testing of the last link in a long chain of events.

One of the key functionalities we use is sharing steps between different features.

eg. most of the tests will start by authenticating with username and password before progressing to the next step in the process.

Specflow

Specflow has a implementation that is parsed along with every scenario that allows for grouping and sharing important information - namely the ScenarioContext.

Solutions

• Some kind of constructor injection
• Some implementation that relied on xUnit shared context

There are probably other options - but something more native to xUnit is probably the best solution.

Please share any suggestions - or if you dont feel like this belongs in the package at all.

Currently:

Copyright
2018 Tengiz Tutisani

Must be

Copyright
2018 - 2019. Tengiz Tutisani

Testing is working fine, however, the steps are showing as ignored tests. Is it possible to hide these?

If you write your code like this:

[Given("I do something")]
public async void Do_Something() => throw new NotImplementedException();

the step will always pass.

I think this is something to do with how we consume the Task object - I'm happy to go fix it