Testing Frameworks Using XP. Patterns and Strategies
Ing. Santiago Valdarrama del Pino.
(This pattern is the english translation of "Pruebas en Esqueletos de Software utilizando XP. Patrones y Estrategias")
Abstract.
The process of writing unit and acceptance tests is one of the most important practices in the application of Extreme Programming in the development of software projects. In order to apply this process to the development of frameworks, some changes should be applied. This paper proposes a set of practices for the development of software unit and acceptance tests in the creation of frameworks using Extreme Programming techniques. In the same way, a set of patterns are modified and adapted in order to be used to guide the test process of systems with the above mentioned characteristics.
Keywords: Extreme Programming, Tests, Frameworks, Patterns
Introduction
Together with the evolution of the Internet, there has proliferated the creation of applications for different domains for the construction, exposition and actualization of hierarchical information via the Net. Each time the necessity arises for an application with these characteristics, it is necessary to define and implement each one of the structures involved in this process, implying an enormous wasted time and the creation of a lot of potential errors. Due to these problems, the decision was made to generalize, design and implement a framework for the manipulation of resource catalogs, gaining this way, a flexible and accessible environment to manage this type of structures.
To implement the above mentioned framework, were used techniques of Extreme Programming (XP). Due to the characteristics that software frameworks present, XP cannot be strictly applied as was expounded in [1], but must be slightly modified in order to be correctly applied to the development of software applications that have these characteristics [6].
This paper attempts to propose a set of practices that should be considered when creating and applying unit and acceptance tests in the development of frameworks using Extreme Programming techniques. These practices were identified during the creation of the framework used in the manipulation of the resource catalogs previously mentioned. Similarly, a set of patterns to guide the development of one of the most important practices stated in the XP manifest [1] were selected and adapted through this paper.
Frameworks
A framework in software development is a high level design term expressed in the reutilization of code. A framework provides functionality with fixed aspects that never change, and variable aspects that will be subject to change. Different systems can be created starting from a framework depending on the configuration of its variable aspects.
A framework consists of a set of classes that cooperate between each other [5]. The interactions between these classes are, frequently, part of the fixed aspects of the framework. The high level design of the system, which defines the components of the solution, its interfaces and interactions, are reutilized within the framework, and as classes and methods become part of the fixed aspects, the code and the detailed design will also be reused [3]. Combining the use of high level designs, detailed designs and code, the framework promises high grades of reutilization.
Frameworks and XP
The development of frameworks using XP varies a little from the normal application of Extreme Programming. XP can be used for the development of frameworks with the objective of providing certain functionality to multiple, or many times, anonymous clients, but for this, it is necessary to slightly modify its structure [6].
The creation of a framework using XP presents characteristics that differ from the development of traditional applications. Amongst these we have, the loss of a single client, which provokes the absence of the role destined to determine the system requirements, and the priorities on its development. Another difference when applying XP to this type of systems is the loss of the concrete character of the user stories, arousing the necessity that these become as general as possible, satisfying the necessities of a greater number of clients. Lastly, in the development of traditional applications using XP, a mechanism of progressive development is used, not taking into consideration the order in which the characteristics of the system are implemented. However, in the development of frameworks, an early analysis must be done to determine the points in which similarities can be found to be modeled by the framework.
In [6], the authors expound a group of variations to the application of XP in the construction of frameworks. In order to create them, it cannot be considered only the set of user stories, as these could be only related to one business in particular, which could limit the reach of the final product. A valid technique in this case, is the realization of an analysis of the user stories defined by various clients in similar systems, so that it can be determined which of these could be a general character, so that the principal candidates are implemented within the framework. This is the form by which, the authors of [6] classify the stories of the clients in two fundamental groups, user stories, which contain the specific aspects for one particular system, and framework stories, which define those aspects that must be implemented according to the system to be developed.
Unit and Acceptance Testing in XP
During the development of software, unit testing increases exponentially the quality of the produced code. The tests are divided into two groups during the application of Extreme Programming; the first of these is where the writing of the unit testing is done, while the second is the writing of acceptance tests [1]. The application of unit tests and acceptance tests are some of the fundamental practices of XP [1].
The developers write the unit tests meanwhile the application source code is written. The client will write the acceptance tests while the user stories are defined. Maintaining a set of tests during the construction of the project, and executing them at regular intervals to validate the results obtained, will help in the development of the application [4].
For each one of the methods that could fail, a corresponding unit test should be written before writing the code that will pass the test. On the other hand, the developers should write the acceptance tests as soon as the user stories are written. The acceptance tests can be automatic or manual, being that the first are the preferred ones at all times, nevertheless that, on many occasions, is quite difficult to automate them.
Testing Frameworks
The tests for a framework differ a bit from the tests utilized in the development of traditional applications making use of XP. For the particular case of the frameworks, there must be taking into account two levels of acceptance tests: acceptance tests for user stories and the acceptance tests for the framework stories.
The acceptance tests for the user stories will be responsible for the verification of specific business aspects for each potential client in particular. In other words, these tests will verify the validity of each user story for each one of the clients in a set of systems that utilize the framework. The acceptance of the set of tests of the user stories will demonstrate the generality of the implemented framework. Hence, the greater the number of satisfactory acceptance tests, the greater is the guarantee that the created framework can be generalized for a greater number of case studies.
With the acceptance tests for the framework stories, an analogy can be made to the traditional acceptance tests for the user stories in the construction of systems making use of common Extreme Programming techniques. On this point, it can be analyzed from a normal application perspective, testing each one of the implemented functional aspects.
In spite of that the two tests mentioned before are a good starting place for the validation of a framework, the implementation of the example applications and the utilization of the framework in real applications will give the final word in that respect. As the framework is being applied to problems of this nature, it will gain maturity, being able to detect those aspects that were omitted in the initial planning that were not taken into account, in that moment, with the group of ideal aspects to be modeled.
In the construction of a framework, due to the absence of real applications that make use of this, it is advisable the construction of a group of example applications of a different nature, which will include the greatest number of possible functional characteristics, this way, creating test scenarios that permit the correct validation of the developed product. These applications will be the main weapon for test creation in a framework.
Patterns on Framework Testing
The utilization of patterns in the design and implementation of software is extremely important due to the advantages that they offer. A group of related patterns exists for the application of tests during the implementation of these patterns. Each of these patterns could be adapted to the implementation of frameworks making use of Extreme Programming techniques. Following a set of patterns which were modified for their application will be enumerated and discussed.
Problem Area
The Problem Area pattern, originally expounded in [2], deals with the areas of the system that should receive special attention in the realization of tests and the control of source code. The solution is to keep a list of the persistent problems areas. It should be reinforced the severeness in the construction of the unit tests, that verify these areas and prioritize the execution of such controls.
A different approach in which this pattern could be applied is when a problem arises in the execution of client applications of the implemented framework.
Due to the existent division in the documentation of requirements, defined in [6], such as user stories and framework stories, it is necessary to determine the location of the problem within one of these fields. If the problem is located in the user story, emphasis must be placed in the generality of the control of the framework, but if it is found in the framework stories, the implementation controls should be reinforced. As a result, common errors can be controlled which brings about a more reliable system free from uncertainties.
Strange Behavior
The Strange Behavior pattern, originally expounded in [2], deals with the decisions to be made when a characteristic of the implemented system is malfunctioning. Any unusual behavior should be taken as a possible problem and keep track of it. This should be done even when the identified problem is not related to the test case in question. It is necessary to be alert when the tests give acceptable results but different from the expected.
Just like the aforementioned pattern, the Strange Behavior Pattern can be analyzed from two points of view: user stories and framework stories. It is necessary to pinpoint the location of the undesired behavior within one of these fields. If the problem exists in the user story, it should be analyzed the generality of the framework as it may happen that it is not prepared to bear with the desired requirements; but if it located in the framework stories, the problem should be profoundly analyzed, in order to identify the internal algorithm that could be causing the unexpected results.
Killer Test
The Killer Test Pattern, originally expounded in [2], deals with the way in which the quality of a system in development can be determined. In order to do this, a test case should be built which can be executed anytime under any circumstances. This test case should be designed such that failure is imminent. In order for it to be constructed, all the critical and extreme test cases should be included, in the framework implementation. As the complexity of the test case increases, better is the evaluation of the quality of the system to be developed.
As the requirements of the framework are determined, a Killer Acceptation Test can be constructed at the same time. Far from being the correct implementation of the characteristics of the framework, it would center its application on the generality desired. The application of the variant of the Killer Test makes possible that the final product meets the necessities of a greater number of clients. The result of this control will be very appropriate to measure the stability of the system. When found and corrected this type of problem, the application will be more stable and efficient.
Document the Problem
The Document the Problem pattern, originally expounded in [2], deals with the way in which potential problems found during the implementation of a system should be communicated and documented. To apply the pattern, a report should be written with the identified problems. One cannot rely on memory, and promises that everything will be alright cannot be accepted; the problem should be stated in documents or error control systems. In this way, it is made sure that the problem is corrected as soon as possible, building the corresponding test case.
In the case that the problems are identified by the developers themselves, a strategy much better than the documentation of this, is the immediate writing of the corresponding test case for the correction of the problem. Violation of these considerations could lead to forget an error which could arise later on and cause the developed framework to malfunction.
Conclusions
The application of Extreme Programming in the development of frameworks should be modified to achieve a correct adaptation to systems that present these characteristics [6].
Along with this, some new techniques might be identified for the development of unit and acceptance tests of applications in the context aforementioned. The same way, the application of patterns to guide the design and implementation of such tests could lead to a dramatic improvement in the process organization of the test and control of the produced code.
During the course of this paper, basing it on the acquired experiences during the development of a framework to implement resource catalogs, a set of practices were identified and documented for the realization of tests, as well as a group of patterns utilized to guide the control process by using unit and acceptance tests in frameworks making use of Extreme Programming.
References
[1] Beck, K. Extreme Programming Explained: Embrace the Change. Addison - Wesley. (1999)
[2] DeLano, D. E. and Rising, L. System Test Pattern Language. "The Pattern Handbook. Techniques, Strategies, and Application". Cambridge University Press. (1996). pp. 97-119.
[3] Jacobson, I., Griss, M., et al. Software Reuse. Architecture, Process and Organization for Business Success. Addison - Wesley. (1997)
[4] Jeffries, R. Essential XP: Unit Tests at 100. http://www.xprogramming.com/xpmag/expUnitTestsAt100.htm. (2001). Last confirmed: November 05, 2003.
[5] Johnson, R. E. and Foote, B. Designing Reusable Classes. Object-Oriented Programming. Vol. 2. (1988). pp. 22-35.

