These tips are advice taken from years of hard work and learning about testing, test design, software design, usability design, and development process. These are not rules that should be applied blindly. They're guidelines that are effective in the vast majority of cases.
# Read the Test Output
There's a good reason that modern test frameworks allow the expression of tests using the hierarchical context/specification style: The tests are intended to provide human-readable output that can be consumed by developers and other stakeholders to review the understanding of the system and the requirements expressed in the test output.
Take time to read and review the output printed when tests are run. You may be surprised to find how often this activity can expose not only misunderstandings of requirements and expectations, but also de facto design mistakes, and poor test structuring.
# Write Test Descriptions to be Read
This tip is the other side of the code of the Read the Test Output tip.
The reason to write context descriptions and test descriptions is to have them be read. When you're writing the test descriptions, read and re-read the descriptions to make sure that the appropriate meaning is conveyed.
# Or Don't Write Test Descriptions at All
If you just can't come up with good test descriptions, or if you don't have the mental energy to come up with the right language, then skip it for now and come back later.
You can write
context blocks and
test blocks without providing any description at all.
context do test do end end
# Separate Automated Tests from Interactive Tests
Some tests should always be executed with the intention of having an operator eye-balling the output. Usually these are "final inspection" tests where the operator isn't really expecting to find any problems, but that final step of having a human operator perform interactive tests is critical to being able to certify the software as being de facto verified.
Interactive testing is a necessary and healthy part of testing practice. It's neither something that should be eliminated or that can be eliminated.
Separate automated tests from interactive tests by putting them in separate directories under your principal
|-test | |-test_init.rb | |-automated.rb | |-automated | | |-automated_init.rb | |-interactive | | |-interactive_init.rb
Note that interactive tests are not intended to be run as a batch in an automated fashion, and therefore, there's no
interactive.rb batch runner on the
# Use Anonymous Context Blocks for Scoping
You may find that you need to use Ruby's lexical scoping to isolate test state.
You can use anonymous context blocks to create lexical scopes to ensure that the state of a test isn't accessible to another test.
context "Some Context" do context do value = true test "Something true" do assert(value) end end context do value = false test "Something false" do refute(value) end end end
# Specify the Experience Rather than the Implementation
When writing test descriptions, try to describe the experience of the software that you're specifying, rather than the implementation.
context "Calculator" do sum = calculator.sum(1, 11, 111) test "#sum" do assert(sum == 123) end end
context "Sum" do sum = calculator.sum(1, 11, 111) test "Addends are added" do assert(sum == 123) end end
The test implementation itself will already document the implementation that is being tested.
The natural language part of the test descriptions should be used to document the system from the vantage point of a user of the system. This is true even if the user of the system is other code, rather than a human user.
Remember that BDD is about describing observations of the behavior. The names of methods, and such mechanical implementation concerns, aren't behavior. Such things are implementation details, and they should not appear in test descriptions.
# Only Assertions in Test Blocks
Avoid putting any code in test blocks other than assertions.
Code that declares variables, or any other supporting code, should be outside of the test block in the context block that encompasses it.
# Literate and Scannable Code
Readability is a less desirable quality of code than scannability.
Code should be understood at a glance. Avoid forcing users to have to read code in detail rather grasp its essence at a glance. Avoid forcing users to seek out the details of the test implementation, and write test code so that knowledge leaps off the screen.
Except for the most trivial of assertions, move meaningful things to the left margin of the code and avoid nesting significant facts about a test into an assertion.
Use explaining variables to highlight meaningful and significant elements.
context "Sum" do sum = calculator.sum(1, 11, 111) added = (sum == 123) test "Addends are added" do assert(added) end end
# Consistent Language
The descriptions and the implementation should reflect each other.
The test descriptions are the English (or whatever language) representation of the test's meaning, and the Ruby is the programming language representation of the same meaning.
Your goal is to provide interpretations of the same idea in both natural language and programming language.
The language of the test description and the language of the assertion code should be consistent with each other.
test "Addition of two numbers" do assert(two_numbers_added) end
# Create Knowledge Hierarchies
Knowledge hierarchies, or taxonomies, are a critical characteristic of creating navigable tests.
Write tests so that outer contexts are more general, and inner contexts are more specific.
context "Calculator" do context "Sum" do sum = calculator.sum(1, 11, 111) test "Addends are added" do assert(sum == 123) end end end
# Use Small and Focused Test Files
Avoid lengthy test files, and avoid anti-patterns like having a single test file per class, with many unrelated contexts and concerns.
Use a single file for a given test context. For some object or concern that can be used in different ways, create a different test file for each context, and organize those files into a directory.
Don't be afraid to use the file system to organize test concerns. There's little value in reducing the number of test files in a test suite. Just organize them sensibly.
# Use the File System as a Table of Contents
The file system's hierarchy of directories and files is a natural table of contents.
Leverage the file system to create a navigable table of contents that the user can browse and explore to familiarize themselves with the concerns of the system under test, as well as the tests themselves.
# Use Empty Tests to Create To-Do Lists
Contexts and tests can be written without blocks. You can leverage this aspect of the TestBench API to write empty tests that act as to-do lists for tests that have yet to be written.
context "Some Context" do test "Something" test "Something Else" end
# Temporarily Deactivate Contexts and Tests While Debugging
It can be useful occasionally when debugging a test to deactivate some of the tests in a test file or context.
Contexts and tests can be deactivated by adding an underscore prefix to their declaration.
context "Calculator" do context "Sum" do sum = calculator.sum(1, 11, 111) # This test will not be executed _test "Addends are added" do assert(sum == 123) end end # This context will not be executed _context "Subtract" do # ... end end
# Exclude Files from Execution
While TestBench offers no dedicated feature to exclude certain files from execution, you can use the batch runner's exclusion capability to skip files that match a pattern.
For example, you can use a pattern that skips files that start with the underscore pattern. When you want to deactivate a file, rename it and prefix the filename with
# test/automated.rb TestBench::Run.('test/some_directory', exclude_file_pattern: /\/_|_init\.rb\z/ )
# Distribute Fixtures with a Library
If you're a developer of a library that other people are going to use, you can provide a set of fixtures that your users can leverage in their own tests that use your library.
A library that ships a set of fixtures doesn't have to take a dependency on the entirety of TestBench. The
TestBench::Fixture namespace is its own self-contained package. That package is safe to be packaged with production code, whereas TestBench, being a test framework, should not be distributed with production code.