Charles Colbourn | Site / Testing

Automated testing

Every new deployment of an application or infrastructure component needs some degree of testing, otherwise you are risking a nasty surprise when it gets into production. Just how much of a problem you are risking by not testing can be determined using some sort of risk assessment.

Many organisations still use only manual testing procedures - a test analyst identifies 'use cases' (normally from the design documentation, which should include use cases as a matter of course) and works through them in a test system exercising the code/config to ensure that it performs as expected. Whilst this is valid, and a vital part of testing, it's only part of testing. Manual testing is too labour intensive to provide a thorough test of error cases, unexpected data formats, and many other nasty surprises that can occur. Automated test tools such as Testdirector, xUnit, and the various testing modules in CPAN, can perform a large amount of regular testing for you. In particular, this intercepts the sort of issues where a change to code/config has an entirely unexpected effect.

I have extensive experience in manual and automated testing, both from a design and test execution point of view. If you have a requirement to institute testing procedures in your organisation and require expert assistance, please see my contact page. Types of Testing

Black box

'Black box' testing treats the code/component as a 'black box' - i.e. inputs are fed in, and the resulting outputs examined to ensure that they meet expectations.

White Box

'White box' testing is the converse - the code is internally accessible to the test utility. This is normally done by the developer, who has knowledge of the internal structure of the code.

Unit testing

Unit testing tests individual components in isolation, prior to assembly into a larger, more complex whole.

Integration testing

Integration testing tests the larger application after individual units are assembled. This is often an appropriate place for manual testing

UAT

User Acceptance Testing is often less thorough than earlier phases of testing, and serves to assure the end customer that the product does what they have specified.

Perl testing

Perl is the ideal language for testing. It has a strong testing culture in the development community which has given rise to a number of well developed libraries for developing tests. It's also very flexible in its ability to read/write from a wide range of data sources (databases, LDAP, etc) making it quick and easy to develop tests for infrastructure. Why use multiple test environments? A lot of smaller organisations don't perform testing in isolation from their production environment, if they perform any formal testing at all. This is a serious risk to data integrity. Buggy applications or configurations can result in data loss or worse, data corruption. Human error can also result in data loss - it only takes a momentary mistake to delete the entire customer database when you intended to delete a block of test data. In these days of freely available virtualisation products, it's easy and cheap to set up isolated test environments. If you're interested in setting up test environments in your organsiation and don't know where to start, I'm available for consultancy. Test coverage Facilities such as Devel::Cover in Perl permit automated examination of a test suite, and evaluation of the proportion of the code that is 'covered' by the tests. This can be a huge help in maintaining a large code base. Developing new tests for existing code can be a background task that is handy to fill ten spare minutes, and over time your infrastructure can become a stable, reliable system.