specflowoss / specflow Goto Github PK

I would like to be able to use the standard visual studio function:
Edit -> Advanced -> Comment Selection (Ctrl-K, Ctrl-C)
Edit -> Advanced -> Uncomment Selection (Ctrl-K, Ctrl-U)

In .feature files in Visual Studio 2010. This would insert a '#' symbol to comment out the selected lines.

Currently, the CreateInstance extension method on Table requires two columns, one titled "Field" and one titled "Value." Like this:

| Field | Value |
| FirstName | John |
| LastName | Galt |

Instead, allow any column header for these two columns (to allow for different languages). It will be assumed that if CreateInstance is passed a two-column table, the first column is the "Field" column and the second column is the "Value" column.

SpecFlow emits an erroneous suggestion for the When step in VB.NET - the table arguments have C# syntax and is missing the _ line continuation for VS2008 compatibility:

-> No matching step definition found for the step. Use the following code to create one:
<Binding>
Public Class StepDefinitions

    <TechTalk.SpecFlow.When("blah blah blah")>
    Public Sub WhenBlahBlahBlah(Table table)
        ScenarioContext.Current.Pending()
    End Sub

End Class

The sub should be:

    <TechTalk.SpecFlow.When("blah blah blah")> _
    Public Sub WhenBlahBlahBlah(ByVal table As Table)
        ScenarioContext.Current.Pending()
    End Sub

SpecFlow should support executing different scenarios in multiple threads. This would allow reusing SpecFlow for powerful stress suites.

To support multi-threaded execution, state information (such as ScenarioContext) should not be stored in static classes.

(Please note that state should not be stored in ThreadStatic variables either, since a single thread might actually be executing multiple scenarios concurrently, by means of asynchronous IO.)

A detailed discussion of this feature is available here.

SpecFlow.Assist helpers support smart column name resolution in Table.CreateInstance/CreateSet, so in case you have a table

| First name | Last name |
| Bill | Gates |

and the corresponding class

class Person
{
public string FirstName { get; set; }
public string LastName { get; set; }
}

then Table.CreateSet will manage to map columns "First name", "Last name" to properties FirstName, LastName.

Unfortunately this is not the case in comparison. Table.CompareToInstance/CompareToSet extension methods require strict match between table column names and class properties.

CompareToInstance/CompareToSet should allow spaces in column names and be case-insensitive. This will make them consistent with CreateInstance/CreateSet and give spec writers more liberty in choosing column names.

If you have a scenario step without binding, you have to run the test
first, and copy-paste the step definition skeleton from the test output.

This process could be automated somehow, like with a command in the context
menu of the scenario step (in the editor) or connected to the validation.

In the release 1.6. there is warning when the feature.cs is generated

Warning as Error: 'TechTalk.SpecFlow.GenerationTargetLanguage' is obsolete: 'Use ProgrammingLanguage enum'

There is currently no support for properties of type char in SpecFlow Assist. Could you please add it? I could do it myself, since it is pretty straightforward, but I still haven't managed to get the hangs of using git. Thank you!

Specflow 1.6, VS2010:
When opening a feature file from TFS history, only the first few lines are syntax colored. Syntax coloring works fine when opening a feature file from the solution explorer.

Hi,

I would like to report a unhandled exception with the step definition report generator. I've described the scenario below.

Given a solution
And a SpecFlow project
And a SpecFlow feature file
And the feature file contains some background "givens"
And the feature file contains a scenario
And the feature file' scenario's first given step uses the "and" keyword instead of the "given" keyword
When generating the step definition report
Then SpecFlow.exe should not throw an unhandled exception.

[email protected]

The code generated by SpecFlow for features decorate test classes with
[Microsoft.VisualStudio.TestTools.UnitTesting.TestClassAttribute()]

When using coded UI tests, you need it to generate:

[Microsoft.VisualStudio.TestTools.UITesting.CodedUITest()]

Can you please add an option in config for this?

the displayed method skeleton is not indented properly when displayed in a unit test runner (tested with ReSharper)

Only the first 'Scenarios' table has the header formatting applied. In the sample below, the name header for scenario A will be italic, but the name column for scenario B will appear as a standard row.

Scenario Outline:
Given ...

Scenarios: A
   | name |  
   | data |

Scenarios: B
   | name |
   | data |

Here is the issue.
I want to log some information in the tests. if I use Console.WriteLine. My test framework just eat the infromation, and did not display them. So what I did is that I implemented a special ITraceListener for my test frame. However, my step defintion file can not use the ITraceListener install since it does not expose to user. I don't want to use my TraceListener directly since I want my tests can run under different test framework.

I am writing UnitTestGenerator for our own test framework. We are facing issue which is that we need to change the class NameSpace, add specific attributes in each test method, etc.
If the IUnitTestGenerator can provide a method which allow me to modify the finally generated code block, it will be helpful.

Thanks
Qingsong

see http://groups.google.com/group/specflow/browse_thread/thread/09c7b2ce6fd5be68/607b7aaf6f4608a9?show_docid=607b7aaf6f4608a9

1. Create a new C# class library, reference NUNit.framework v2.5.5 and TechTalk.SpecFlow v1.3.5.2
   2.Delete the class1.cs, create a features folder, add new item "SpecFlow Feature File" into the features folder
   3.Open the project properties, under Application change "Target Framework" to .NET Framework 2.0
2. Build solution will generate 3 compile errors:
   "The type or namespace name 'TechTalk' could not be found..."
3. Change the Target Framework back to .NET Framework 4 and rebuild solution
4. Build will succeed

Running Windows XP SP3 and VS2010 Professional.

See details here: http://groups.google.com/group/specflow/browse_thread/thread/cc55931f2112b0e3/67af55f0b9e48ea7#67af55f0b9e48ea7

I run my tests using TestDriven.NET and SpecFlow produces loads of
text in my output window. I would like to run it in silent mode where
it only outputs failing tests. I don't need it to echo my test cases
to the output when they pass. It makes it much harder to see tests
that failed since I have to dig through all the output of the passing
tests.

In looking at the code I don't see a way to do this currently. I have
reduced the output using the configuration file and setting
traceSucessfulSteps="false", but I still get the step text in the
output.

Feature Request.

Given as a developer using Specflow
And I execute Specflow test containing example or outline
And I use Mstest as my test execution
And I run the mstestexecutionreport to format my report
Then I should see outlined scenarios uniquely

http://groups.google.com/group/specflow/browse_thread/thread/a189acfe8f4a76fa#

SpecFlow has various helpers to generate or validate collections, but unless I overlook something, I don't think it covers all of the following scenarios:

1. Validating that a query result contains a collection;
2. Validating that a query result equals to a collection;
3. Validating that a query result equals to a collection in an exact order;
4. Validating that a query result does not contain any element from a collection;

Examples:

1. The result should include a collection as a subset
   When user receives mails
   | Sender | Subject |
   | Ann | Hi |
   | Bill | Hello |
   Then user's mailbox should contain unread items
   | From | Subject |
   | Ann | Hi |
   | Bill | Hello |
2. The result should include a collection and no other items
   Given user's shopping cart is empty
   When user adds to the cart items
   | Name | Price | Qty |
   | Bike | 100 | 1 |
   | Ball | 10 | 2 |
   Then the shopping cart should only contain items
   | Name | Price | Qty |
   | Bike | 100 | 1 |
   | Ball | 10 | 2 |

etc.

I've found CompareToSet helper method, but it does not cover all cases above. In my project I have implemented some ad-hoc helpers that do the job, but I question if there is a need for a generalized approach. MsTest has AssertCollection helper methods to test exact match, subsets etc., so this can be used as example of method names.

According the gherkin wiki (section Multiline Strings
on https://github.com/aslakhellesoy/cucumber/wiki/multiline-step-arguments ), the indentation of lines within a PyString formatted multiline string argument is considered as it goes beyond the indentation of the triple-quotes.
With SpecFlow instead, the indentation is lost.

In a German gherkin file, when I type in "Gegeben sei" and select a given sentence from the list, it removes the "sei" part of the keyword. I guess the same issue could be present in other languages with multi-word keywords (but haven't tried to reproduce it).

Currently, the CreateInstance extension method on Table only works tables that list their values down, like so:

| Field | Value |
| FirstName | John |
| LastName | Galt |

CreateInstance should work with tables that list values horizontally, like so:

| FirstName | LastName |
| John | Galt |

It would be great if it were possible to write specification in other languages than English. It's difficult for me to involve users in this process when I can't write specifications in their language.

Use an IDisposable class for context injection.
In this case, SpacFlow manages the instances (creates/releases for each scenario)
Whenever it releases the objects at the end of the scenario, it should call the Dispose method.

If a step definition is scoped for a tag, it recognizes only the scenario-level tags, not the feature-level tags.

in the sl4-wp folder, the silverlight runtime is included instead of the wp7 runtime.

it is in the installer though.

Currently cannot generate the step definition report at all with .NET acceptance test projects compiled with .NET 4.

See http://www.marcusoft.net/2010/12/know-where-you-stepgenerate-step.html for a workaround but would be better if SpecFlow could do this out of the box. Perhaps like some unit testing frameworks, produce a .NET 4 runtime version of SpecFlow.exe

If the project has not built for any reason (including missing NUnit DLL), the Step Intellisense throws an exception the first time the IntelliSense would pop up. After that, hitting Ctrl+Space throws a NotImplementedException.

See discussion on the group:
http://groups.google.com/group/specflow/browse_thread/thread/75f0cc35d7964497?hl=en_US

I spent several hours trying to figure out why my method was not being detected on this one, it was very annoying.

If you have a Scenario like this

Scenario Outline: User Login
Given i am logging in as the user
And the password is
Then the login should

Examples:
| User | Password | Result |
| Bob | | Fail |

This will fail to find the method with the tag [Given("the password is (.*)" )] because the sting for the Password is null. to fix this you must enter a string such as "null" and detect that it should be a blank null field.

Edit a feature file for longer time (touch multiple scenarios).
After a while, the editor slows down (typing and scrolling).
Close/reopen the feature file solves the problem.

A really neat feature would be to have a smart tag (like Implement interface in C# for example) to pop up on steps that doesn't have a corresponding step definition.

I can think of three different options that would be applicable in the drop down for such a smart tag:

• implement step definition in [feature file name]Steps.[language extension] => creates the file if needed and implement the step definition in it
• implement step definition in Steps.[language exentension] => creates the file, if needed, and implement the step definition in it
• implement step definition to Clipboard => simply puts the code for the step definition on the clipboard to paste into a code file later

Also - a final and related feature is for a red squiggly line (ErrorTag, see http://msdn.microsoft.com/en-us/library/dd885240.aspx and http://msdn.microsoft.com/en-us/library/microsoft.visualstudio.text.tagging.errortag.aspx) for steps that doesn't have a corresponding step definition.

I'm sorry that I don't have the time (this time around) to commit any code to do this, but I've checked how to implement SmartTags and had great use of this walkthrough: http://msdn.microsoft.com/en-us/library/ee334190.aspx)

Thanks for a great tool - and the great work you put into it.

Hi,

Execution reports produced with xUnit-generated XMLs does not display the Gherkin text of the scenario. Instead, they display C# names of automatically-generated unit test functions. This makes them quite useless.

The reason for this is that specflow.exe expects a text file with test output, in addition to the NUnit.xml, which it then parses. Unfortunately, there is no simple way to make xUnit generate the test output file.

The best way to fix the situation is to support native xUnit XML logs, which have everything NUnit XMLs have, but also contain the test output. Unlike NUnit, no second file is necessary, and no parsing is required.

Thanks,
Te.

Feature request

As a specflow user I would like to be able to specify a different color for the parameters present in a step.

For example in a step like: Given I have entered 50 into the calculator
Then with the step definition like [Given(@"I have entered (.*) into the calculator")]
I would like to have in VS the value 50 colored differently than the rest of the step

this feature was present in the cuke4vs plugin in VS2008 and was really appreciated by our test writter. It is also a great feature to demo the use of BDD and specflow.

And thanks for your work and this great tool!

If step bindings are declared in a partial class, for example:

StepsGiven.cs:

[Binding]

public partial class Steps {}

StepsWhen.cs

public partial class Steps {}

StepsThen.css

public partial class Steps {}

The 'Go To Definition' feature is unable to locate step bindings which are in StepsWhen.cs or StepsThen.cs.

http://groups.google.com/group/specflow/browse_thread/thread/3fa4bae7ac26418e

see http://groups.google.com/group/specflow/browse_thread/thread/59fbae4f907de01c/ae93f68d548b413a?show_docid=ae93f68d548b413a

I was able to repro this in two situations:
A)

1. Add new feature file
2. Paste a new scenario from notepad (so with CRLF)
3. Save & Close
4. Reopen feature file -> will report inconsistent line ending in feature file

B)

1. Add new feature file
2. Paste a long (~20 lines) narrative text after the provided narrative from the addition feature
3. Save the file -> will report inconsistent line ending in code behind file

@darrencauthon
The FillInstance method currently stops as soon as property fails to convert by throwing an exception. This means you can not handle the exception for any properties which you would expect to fail but have the remaining properties still filled.

It would also be helpful for the exception to contain information about the failed property so you can easily identify the issue.

Add scope for Binding as for steps(StepScopeAttribute)

Use the tags I add to features to build a tree in the VSMDI file.
Example:

@regression
@positive
Scenario...

When the Custom Tool runs, this test is automatically added to the VSMDI file, so when I open the test list editor, I find this scenario under
List of Tests \ Regression \ Positive

This would save a lot of manual work.
I'm currently working on a project where I have over 1.000 tests. Some are "Regression", some "performance", some are slow, and therefore don't need to be executed every time, ...

FYI the site's down. I get the following error message:

System error
An error occured processing your request at URL /Default.aspx.

Click here to go back to the home page

Using SpecFlow v1.6, VS2010, C# and .Net 4.0

My step definition regex:
[Then(@"the target index for sequence element (\d+) on(?: the)?? AWG(\d?) should be (\d+)")]
public void ISetTheTargetIndexForSequenceElementTo(string element, string awg_number, string expected_value)
{...}

But when I use intellisense to find the step and insert it to a feature file I see (and get):
Then the target index for sequence element {e AWG{awg_number}

I would expect:
Then the target index for sequence element {element} on(?: the)?? AWG{awg_number} should be {expected_value}

This is a small one...

I've noticed that the step intellisense doesn't popup in the Background-section. This was a problem that I had in the orginal solution I sent to you... Sorry might have led you down the wrong path there ;)

You need to look to the start of the file not only to the start of the scenario (as I did at first) or at least also include Background to the allowed "places" where to show intellisense

SpecFlowContext.Set has no generic constraints, so I can put any type into the dictionary. Get has a constraint "where T : class", which means I can only get objects out. So, I can add a bool, but I can't retrieve it.

This applies to ScenarioContext and FeatureContext.

Thanks
Matt

After upgrading my project from v1.4 to v1.5 I noticed that my app.config looked like this:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow" />
  </configSections>
  <specFlow>
    <unitTestProvider name="xUnit" />
    <unitTestProvider name="NUnit" />
  </specFlow>
</configuration>

The second unitTestProvider element was added during the upgrade. Not a huge problem, but I'm sure it'll catch someone out! :)

What steps will reproduce the problem?

1. Create a class which inherits from MsTest2010GeneratorProvider class.
2. Write SetTestSetup method that adds some statements to the method. For example:
   public new void SetTestSetup(CodeMemberMethod memberMethod)
   {
   base.SetTestSetup(memberMethod);
   memberMethod.Statements.Add(new CodeCommentStatement("Hello from custom UnitTestGeneratorProvider"));
   }
3. Define your class in app.config:
   
4. Run custom tool on any feature file.

What is the expected output? What do you see instead?
I expected to see following in the generated file:
public virtual void ScenarioSetup(TechTalk.SpecFlow.ScenarioInfo scenarioInfo)
{
testRunner.OnScenarioStart(scenarioInfo);
// Hello from custom UnitTestGeneratorProvider
}
But I don't see my comment!
I also understand that TestInitialize attribute is not used and ScenarioSetup is called in the beginning of every test method. But TestInitializeAttribute should be added by base class:)

What version of the product are you using? On what operating system?
Version 1.3.4.

Please provide any additional information below.
It seems that you need to add statement
testGeneratorProvider.SetTestSetup(setupMethod);
in SpecFlowUnitTestConverter.GenerateTestSetup method. Also you need to change all implementations of IUnitTestGeneratorProvider.SetTestSetup.

Best regards, Koloto