David Ross's Blog Random thoughts of a coder

Its almost a year since the TDD talk I did last year at Skillsmatter with Chris Roff.  Of the topics we discussed Deployment Verification Tests seemed to be the least understood and 12 months later the majority of projects still don’t have a suite of automated tests to determine if a deployment has been successful.  The following is a summary of a introduction to DVTs that I recently wrote for one of my clients.  Unfortunately there is no code attached.

Introduction

Deployment Verification Tests (DVTs) determine if an installation of a component has been successful.  Typically they are executed as a manual set of checklists, by the environment support team, or as a smoke tests by the QA team.  A better approach is to automate the tests and have the development team verify each of the assumptions about the environment that the component is deployed to.

In particular:

• That all dependent systems (including databases, Web Services, Message Queues, remote file shares etc) can be accessed
• That all dependent systems have been deployed correctly
  • Has the database upgrade script run correctly/Verify that new the tables, columns or stored procedures exist?
• That all local resources can be accessed
  • Is the account running the new Windows Service able to write to its log file?

Best Practices

Deployment Verification tests should follow the same practices as unit and integration tests hence they should be:

• Rerunnable
• Focussed – Each test should verify a single assumption which quickly helps the investigation process when a test fails
• Part of the build – By writing the tests in NUnit enables the DVTs to be executed as part of the build process

Unlike the Unit and Integration tests, Deployment Verification Tests are executed in production. 

They must be:

• Idempotent for Users - Never change customer visible data stores within an environment (i.e. any database records)
• Clean for administrators - Minimise the impact on production staff visible data stores (i.e. log files)
• Independent of other tests – They should exist within their own assembly to ensure that Integration or Acceptance tests aren’t accidently executed in production (these tests will often change the contents of the database)

Don’t Repeat Yourself – Sharing Configuration

Since DVTs are used to verify that a deployment is successful it is imperative that the test uses the same configuration information as the System Under Test (SUT).  A simple way of achieving this is to have the test extract any environment specific configuration item, such as a connection string, out of the SUTs configuration file.

In the following example the DVT assembly has been deployed into the same folder as the website that is being tested.  The OracleDatabaseAssert class uses an XPath to drill into the web.config to retrieve the connection string. 

   1: [TestFixture]

   2: public class verify_can_connect_to_different_services

   3: {

   4:     [Test]

   5:public void can_connect_to_oracle()

   6:     {

   7:         OracleDatabaseAssert.CanConnectUsingConfiguration

   8:             ("web.config", "/configuration/appSetting[@name='db']/@value", 

   9: "Can not connect to the Oracle database");

  10:     } 

  11: }

The test will fail fail when:

• The web.config is missing the appropriate appSetting tag
• There is a firewall blocking access to the Oracle database server
• The user account accessing the database hasn’t been configured

Its very simple to use this example to create a set of libraries to cover the most common testing scenarios including:

• Database Asserts
• Networking/Firewall Asserts
• File System Permissions Asserts
• Web Services Asserts

Example Assertion Library - OracleDatabaseAssert

Used to perform a DVT against an Oracle database.

CanConnectUsingConfiguration	Used to determine if the connection details can be used to connect to the database
TableExistsUsingConfiguration	Used to determine that table exists within a database
ColumnExistsUsingConfiguration	Used to determine if a column within a database table exists
StoredProcExistsUsingConfiguration	Used to determine if a stored procedure exists in the database

Summary

With a small amount of effort it is very simple to convert manual smoke tests into automated Deployment Verification Tests.  By implementing the tests using NUnit enables the tests to be easily executed by the environment support team.