Fit's native mechanism for grouping test pages is in suites. In the case of Folder Runner, a suite is represented by a file system folder, so test pages are organized in a suite hierarchy where every suite has three more or less explicit functions;
1. Group related test pages together and organize them in a logical manner
2. Allow for execution of a sub-set of tests, e.g. use -i to select a subfolder or single page of the entire suite
3. Act as a context for setup/teardown and suite setup/teardown, to provide test pages with reusable before/after steps.
Unfortunately, suite hierarchies are naturally one-to-many, i.e. a test page can only belong to one single suite.
Tagging, on the other hand, is more free-form. It's naturally many-to-many, i.e. many test pages can share a single tag, and a single test page can have many tags. This is primarily useful as an additional selection mechanism for #2 above. The tag selection and suite/page selection are orthogonal and can be used together to select tagged pages across multiple suites.
Tagging
You tag a page by adding an HTML element with id fit_tags. For example, to tag a page with "ShoppingBasket" and "Regression", add an element like this;
<p id="fit_tags">Tags: ShoppingBasket,Regression</p>The highlighted id attribute is what marks this as a tag specification. Note that you can use any element type; <p>, <h1>, <div>, <span>, etc, as long as it has an id of fit_tags. Also, you are free to add style attributes to highlight or hide the tag specification, so you can format pages to your taste. We prefer to keep tags visible, because it makes sense for users editing the page to know which tags apply to it.
The format of the element contents allows any prefix label (Tags: in the example above), so you can use a localized label as long as it ends with a colon. The prefix label is optional, if there's no colon in the contents, it's assumed that there is no prefix label, and the entire string will be interpreted as a tag list.
A tag list is simply a comma-separated list of tag names. The only rules for tag names are;
- They must not contain a comma
- They must not have leading or trailing whitespace
- They should not contain wildcard characters "*" and "?", because those are reserved for wildcard filtering
- They should not contain the ":" character, because the first colon is used to locate the optional prefix label. Colons in tags works well as long as there is a prefix label.
- The tag list must not contain HTML markup, it is expected to be plain text
For an example of a tagged page, see SampleSimpleTest.
Filtering
The Folder Runner command-line allows you to specify a tag list, which is used as a filter to only execute the pages that match the command-line argument. Wildcards are supported in filters, so you can match partial tag names easily.
Given the example with "ShoppingBasket, Regression" above, the page will be executed if;
- No tag filter list is provided on the command-line
- "ShoppingBasket" is provided
- "ShoppingBasket, Regression" is provided
- "shoppingbasket, regression" is provided, because matching is case insensitive
- "Reg*" is provided, because matching supports wildcards
But it will not be executed if;
- "Checkout,Regression" is provided, because it doesn't have the Checkout tag
- "ShoppingBasket, WorkInProgress" is provided, because it doesn't have the WorkInProgresstag
That is, all tag filters provided on the command-line must match the page for it to be executed.
If you take care with tag naming, you can use wildcard filtering to do an OR-match of pages, which is otherwise not supported. A common tag prefix can be used to allow tag filters to group these tags together. For example, pages tagged;
- "feature.notifications.email"
- "feature.notifications.twitter"
- "feature.notifications.phone"
Note that there are no filtering mechanisms for executing entirely untagged pages or pages without a specific tag.