Monday, December 13, 2010

Using tags in SpecFlow features

Since I first read about Cucumber in the excellent RSpec Book the concepts of tags has been the one that I haven’t really grasped. I liked the idea as outlined here, which states that you can use tags to organize your features and only run a subset of them. And the über-cool @wip-tag that allow you to limit the number of items in progress for the team. Yam for Kanban lovers.
OK – but when I got around to try it in SpecFlow I was a bit disappointed to learn that the only @ignore was supported… Or was it?

@ignore

@ignore is in other words the only tag that is supported by default and is translated to “IGNORE” in your test framework of choice (for example Ignore in NUnit or MsTest).
This can be used to disable features and/or scenarios that are not ready to be run yet.

Custom tags

You then have the liberty to make up any tag you want. For example Aslak Hellesoy list these;
@important @billing @bicker @annoy @qa_ready
You can also use tags for features and scenarios to refer to another system with requirements (TFS or JIRA for example) with the ID from that system. Love Aslak comment on that:
“if you have to deal with old school requirements in a different system”
Anyhow, in short – creativity is the limit. You can tag them with anything you want.

Using tags in SpecFlow hooks

But what for? What can I (or rather SpecFlow) do with them. There are two answers to my knowledge; hooks and test execution.

SpecFlow comes with a number of Hooks that in effect are events that is fired during the running of features and scenarios. SpecFlow can create a file with them all for you if you choose the SpecFlow Event Definition file template. The available hooks are and their running order are:

[BeforeTestRun]
  [BeforeFeature]
    [BeforeScenario]
      [BeforeScenarioBlock]
        [BeforeStep]
        [AfterStep]
      [AfterScenarioBlock]
    [AfterScenario]
  [AfterFeature]
[AfterTestRun]

See this specification to get it from the source

Luckily they are well-named so it’s easy to know when they run.
Also, they all take zero to more tags as an optional input that we can make use of to write specialized hooks. Like this:
[BeforeScenario("testTag1")]
public void BeforeScenario_testTag1()
So now you can write your own definitions on what’s going to happen when you tag a scenario (or feature) with a certain tag. Using this technique we have a @restoreDb that triggers a BeforeScenario-method with that tag ([BeforeScenario("restoreDb")] ) that does that, restore a new database in a virgin state for us.

Please note that you can use more than one tag in a Hook, like this:
[BeforeScenario("testTag1", "testTag2", "testTag3")]
public void BeforeScenario_testTags()

But then the tags are OR:ed together. So the above method (and the empty one) will be run for @testTag1, @testTag2 and @testTag3.

Integration with test frameworks

But wait, there is more. If the test framework supports it (as NUnit and MsTest 4.0 does) the tags will be converted to Categories which you can use to run certain parts of your test suite for example. That would map to the part on "Running a subset of scenarios” here.

Here’s a picture of Resharpers test runner grouping my features on the different tags I’ve used:

Conclusion

Of course this can be used on the command line, in your build script etc. to not have to run through all the tests on every check-in for example.

This have been a quick tour of the capabilities of tags and hooks in SpecFlow. I’ve uploaded my demo project here.

Please note that I’ve only used tags for Scenario-hooks in the demo-project, but they work the same for features.

5 comments:

Bobby Baird said...

Were you able to get the ignore tag to work in Resharper's TestRunner? I'm not having any luck with that.

Anonymous said...

Hmm i know that I got it to work with NUmit test, ignore and resharper, but I'm not sure how I ran my MsTests. Probably used the Vs2010 built-in testrunner.

You're having problems?

Have you checked the generated code? Does it contain the proper Ignore (or equivalent) attribute?

Mark Broadhurst said...

Ignore seems to work for me I just put @ignore on top of my Scenario and it added the Nunit Ignore attribite on to the generated test.

Anonymous said...

Below link is broken:
https://github.com/techtalk/SpecFlow/blob/master/Tests/FeatureTests/BeforeAfterHooks/BeforeAfterHooks.feature

Unknown said...

Thank you - updated my username at github which broke the links...

Link updated now. Should be https://github.com/marcusoftnet/DemoSpecFlowTags