English | EN Dear Testing Community,
The Sogeti Testlab Hub Stuttgart wants to give you a short and practical guideline for the standard test levels used in agile SW-Projects.
Diagram 1: The Test Pyramid – Unit testing should create a solid fundamental of the all over test coverage
The hints and recommendations presented here are based on the experience of several testing workshops done with developers and test automation Experts in international projects with different Sogeti clients in the last year.
As shown in the Test pyramid let’s start with the essential and basic test level, the unit test.
This blog doesn’t regard the structural aspect but will discuss the methodological aspects of the functional unit test as listed in the test framework below (Diagram 4).
Diagram 2: the 2 essential test types of unit testing
As seen in the diagram above unit testing should comprise of a structural test executed for example by a structural code analyzer like SonarQube following the rules of the structural testing and functional testing using Frameworks out of the X-Unit framework catalogue. This blog talks about a methodological toolkit the developer should know and use in order to write trustworthy and purposeful test cases with his automation-framework.
Definition of Unit Testing:
Let’s follow Roy Osherove, 2015 who defines the unit test as follows:
Diagram 3: The unit test in the context with the user’s view
The unit test will prepare and control the GUI-based testing.
This definition allows distinguishing the unit test level from the integration test level by indicating the fact that the unit tests build on a total data control organized by the developer and executed by the test framework. Each run of a test case has to produce the same result based on the same input test data. The exception will be discussed later. 
Diagram 4: Framework for the functional unit test
Test Objects:
What are now the basic test objects to be verified by unit testing?
For the functional tests the following three test objects will be mainly addressed:
Classes: Class calls (internal flow and dependencies)
Methods: Member tests (generated data as return values)
Objects: Object states (state-based testing)
The classes will be tested concerning their call dependencies as it is designed in i.e. UML sequence diagrams. The methods will be tested for correctness and completeness of their target business data. The objects will be tested concerning their correct content and states and state transitions.
A risk-based approach can be used by analyzing the test objects concerning their complexity and criticality.
Sociable or solitaire Tests:
Diagram 5: Martin Fowler
According to Martin Fowler you may find two “schools” of unit testers in the programming community.
The classical school:
The “classicists” accept the dependencies of the tested classes to other classes and open the data interfaces. So they don’t use test doubles and are working with the test data provided by the neighbor-classes. They are doing sociable tests. (i.e.: price method invokes functions from the product and customer class). Prerequisite: The data resource must be stable in order not to violate the definition given by Roy Osherove.
The “mockist” school:
Following strictly the definition of the unit test and using stubs and mocks to isolate the class tests and keeping a total control of the test data.
Risk based Unit Analyses:
A risk-based approach can be used in order to analyze, evaluate, control and manage the test objects. Two characteristics of the test objects have to be understood: Complexity and Criticality.
Complexity:
According to Robert Binder, 2000, the complexity of a class consists of the
Runtime execution dependencies between methods and between methods and objects in a class. As we can see in diagram 5 four types of complexity will be differentiated:
- non modal: Methods can be executed randomly i.e. Date time
- uni-modal: Methods can only be executed in a certain order i.e. TrafficLight
- Quasi modal: Methods can be executed in dependency of the Object state i.e. Stack operations (First in, Last out)
- Modal: Methods can only be executed in a certain order and in dependency with the object state i.e. banking account
Diagram 6: Framework for the functional unit test
Criticality and Risk matrix:
The criticality describes the degree of the significance of effects of the defect from a business point of view. As we see in the following risk matrix the developer should evaluate for example on the method level the business criticality in order to derive the test intensity. The complexity can serve as a scale for the probability of occurrence.
Diagram 7: Risk matrix for criticality evaluation
For the Practitioner, it is worthwhile to design a risk matrix in order to do a small risk assessment of his classes and methods under test in order to know how deep the test of the objects should be and not to waste time for useless testing preparation. In one of the next blog, a short intro into a risk assessment and risk evaluation scheme will be given.
Basic Rules for Automated Unit Tests:
Good written test cases should follow these basic rules:
During our unit-test workshop, the following six systematic and methodic steps to achieve unit testing ready to be automized has been proven as a Best Practice Approach:
Diagram 8: 6 Steps to the automated functional unit test
Use or write requirements…
For the test design of the functional test cases, specification-based test methods can be used as shown in the following diagram. Above all, for the implementation of business critical processes (BCP) it is worth to define explicit requirements written by a requirements engineer or the developer himself to have a clear and solid test base.
Diagram 9: Requirements workbook for unit tester
Write logical test Cases first…. A basic test technique is the equivalence class technique, where test data are classified in all general cases that the system has to cover. In one of our examples, we used this test technique in order to test data file handling. As you see in the following table we define 1 plus 7 logical test cases to cover all scenarios the test object has to deal with.
The first test case (first row) contains valid test data (“1”) of 7 different equivalent classes. The seven other test cases are invalid (“0”). The invalid equivalence class is indicated by the zero in the row. As we know, to test an invalid system behavior just one equivalence class should contain an invalid test data.
Diagram 10: logical test case based on equivalence class technique
Conclusion and Prospect: The unit test is the essential test level as the test pyramid shows us and the developer should get aware of this fact and his responsibility. Test cases should be prepared by a systematic approach. A methodic 6 step approach as shown in this article is the appropriated best practice to assure a solid functional quality of the programming code.
In the next blog, we want to discuss a practical approach to a user story acceptance test which is the complementary test level to a unit test within a sprint cycle.
References
1. Binder, R.: Testing Object-Oriented Systems: Models, Patterns and Tools. Reading, MA: Addison-Wesley, 2000 [ISBN 0-201-74868-1].
2. Sneed,H.M., Winter, M.: Testen objektorientierter Software – Das Praxishandbuch für den Test objektorientierter Client-/Serversysteme. Karl Hanser Verlag, München 2002 [ISBN 978-3-446-21820-8].
3. Liggesmeyer, Peter: Software-Qualität – Testen, Analysieren und Verifizieren von Software.
4. Spektrum Akademischer Verlag Heidelberg, 2009 [ISBN 978-3-8274-2056-5].
5. Goll, Joachim, Müller-Hofmann, Frank, Heinisch, Cornelia, Java als erste Programmiersprache-Vom Einsteiger zum Profi, Viehweg+Teubner Verlag, Wiesbaden, 2011, [ISBN 978-3-8348-0656-7].
6. Beck, Kent: J-unit, Pocket Guide, O‘Reilly Media, Gravenstein Highway North, Sebastopol,2004,[ISBN 978-3-8266- 9712-8].
7. Osherove, Roy: The Art of Unit Testen – with examples in C#. Mitp-Verlag GmbH 2015, South Hill Drive, Westampton, NJ, USA; 2015 [ISBN 978-3-8266- 9712-8].
8. Majer, Damir: Unit Tests mit ABAP. dpunkt.verlag, Heidelberg 2009 [ISBN 978-3-89864-539-3].
