The Tests are the Requirements

February 5th, 2008

I get an amazing amount of pushback to this idea which I see as a perfectly natural chain of logical deduction. Let’s try it, shall we?

  1. Specifications are typically written before code as instructions for what the code SHOULD do.
  2. Tests, (written before or after the code), are a set of instructions detailing what the code MUST do. If we can’t pass the tests, then we cant release the product.
  3. Specifications have no enforcement mechanism beyond human beings.
  4. Tests may be automated and the enforcement of them may be as well. Without passing a suite of regression tests, for example, the software under test (SUT) may not be deployed to the drop point on a network.

Tests act as a gate keeper. Regardless of how few or poor our tests are, they serve to ensure the software meets the specifications. This is why we refer to passing a test as PASSING. Passing a test means that we can get beyond it. Functional requirements written in a document have no such enforcement ability.

Ideally, tests are an expression of the functional requirements such that we have a validation tool for the functional requirements. If this is true, are not tests the functional requirements in an executable form?

This is why tools like Fitnesse exist. Can we as developers accomplish the same level of testing with a Unit Testing framework? You bet, but the frameworks don’t typically provide the nice syntax and user experience business users need to create the tests. This is exactly what Fitnesse strives to provide, a way for business users to specify requirements in an automat-able form.

Some Other Observations

  • Tests, while ideally written before code, still rarely are. Most tests are written after the code is written. These tests include regression tests, acceptance tests, unit tests, etc. All kinds of tests are at play here.
  • A MS Word document can’t break the build.

If these things are all true, then how can we possibly consider the tests anything BUT the functional requirements. They may not have that label at the top of the page, but tests fundamentally define acceptance of the system.

I just don’t understand how this doesn’t compute for some folks.

David Starr

  • http://elegantcode.com Chris Brandsma

    Something I have said when teaching TDD, and Jeremy Miller recently reiterated as well: TDD helps your code architecture.

    If you write your code with an eye towards testability you will naturally head down numerous best practices; like Separations of Concerns, Single Responsibility, Inversion of Control, etc.

    So, not only do you get better code because it is tested, but you get better code because you are forced to write better code in order for the code to be tested. (that might be confusing to read — oh well, it is late).

    Viva la TDD.

  • http://elegantcode.com dcarver

    Interesting. Where does a traceability matrix fit in? How do you ensure all of your requirements are realized in code (or some other form)?. Also your fitnesse link seems to be pointed to the wrong place. I think you want http://www.fitnesse.org although that site seems very slow.

  • http://elegantcode.com David Starr

    We have 2 options to ensure realization of requirements in code.

    1. Hand test the requirements by creating a test plan from them.
    2. Automate the requirements as tests themselves by expressing requirements as tests in the first place.

    I think we are focusing on simply skipping the part where we translate requiremetns into a test plan, and then into (maybe) manual tests for regression, and then automating the tests if possible.

    This simply means “let’s express the requirements in an automatable way in the first place, and skip the human translation.”

  • http://elegantcode.com/2008/03/15/everything-i-ever-wanted-to-know-about-agile-development-i-learned/ Elegant Code » Everything I ever wanted to know about Agile Development I learned…

    [...] Verification.  In the simplest form this is a way to have the tests match the requirements (as David put it).  In a broader form it is to verify program program correction overall.  To many [...]

blog comments powered by Disqus