DevOps Oriented Test Architecture

Test architecture set up correctly: Practical report on the efficient testing of agile software products with a high degree of automation

Part 1: Introduction

Agile process models coupled with the use of Continuous Integration and Deployment technologies are increasingly being used in the automotive and its supplier industry for the software product development.

Thus, the organizations promise the following advantages in addition to breaking up silo-shaped barriers:

• More flexibility in the realization of market and customer-oriented software requirements;
• increasing the production rate in the software development process;
• Acceptance of the costly delays of “semi-finished products”;
  • and connected with it
• A higher innovation and thus the market effect of the software functions to be delivered.

Above all, the projects for the creation of individual software play a pilot function in the agile approach whose structure and process organization does not have to fit into the functional pattern of the IT landscape (for example, SAP as the backbone of the business core functions). The introduction of the SAFe concept (Scalable Agile Framework) by Dean Leffingwell showed the evidence of a holistic agile successful approach including the portfolio and program level even in “large scale” projects and huge companies.

However, it can be seen in many projects that the organization of agile processes has only reached the team level and that companies are still a long way from being able to integrate the Portfolio- and Program-levels.

This entails the risk that the development process with several distributed agile teams and a high degree of coordination will not guarantee an efficient agile flow with a sustainable velocity.

Diagram 1: Software development process with low maturity

This loss in terms of agile effects results at least in the lack of Quality in Process (QiP), which affects the operational readiness and initial Maturity of the product and the user’s satisfaction.

This article describes the Quality Assurance perspective of the software development of an intralogistics Software application in the automotive supply industry.

We, the test architects, want to show you how to develop and implement a testing implementation model with a high degree of test Automation on the base of a normative classical Test Reference Model.

The Solution had to interoperate with the DevOps-oriented software approach, with agile service and platform teams, micro-server architectures and infrastructure based on continuous integration and continuous delivery pipeline.

Diagram 2: DevOps appropriated Value Chain with DevOps enabler

The test architecture description presented to the reader in this report is the result of a collaboration between three experts in the areas of:

• System architecture and capabilities
• Test automation
• Process Integration, Management, Quality and Testing

Diagram 3: Integration of test tools and -methods

Furthermore, we want to show how a systematic combination of the relevant expert knowledge can be used to design a test architecture that can meet the following requirements:

• Separation into organizational, methodical and implementation-technical levels of the test architecture.
• Construction of an implementation model derived out of the normative test concept respecting the DevOps principles with the following goals:
• Scalable test coverage along the development and deployment chain
• Early feedback loops to report quality in the System under Test (SuT)

Diagram 4: The three principles: Flow, Feedback and experimental Learning (2/3)  (source: Gene Kim et al, 2016)

• Supporting the production flow by fitting the test automation sequences optimally along the “IT production line”.

Diagram 5: The three principles: Flow, Feedback and experimental Learning (1/3) (source: Gene Kim et al, 2016)

• Adjust the specified, partly manual test processes in the agile sprint-driven process flow.

We believe that these three expert perspectives are necessary and complementary to each other in order to shape a trustworthy, reliable and resilient test process.

Diagram 6: DevOps oriented Capabilities and Processes within the Architecture

Part II: Reference Model of the Test architecture with four Action Levels

• Norm-based Planning
• Quality construction
• Test Level based Test Strategy
• Diagnostic: automated Test Pipeline

Part III: Implementation Model and Tooling

• Structure of the deployment pipeline
• The 4-stage continuous integration process
• Integration test of an SAP interface
• Integration of a corresponding process schema
• Setup of an automatic system test

Part II and III will be published under SogetiLabs in the next few weeks.

This blog was a collaboration between Stephan Schramm and Sabine Reißenweber (technical test expert of Sogeti Germany).

The blog is the Translation of a professional article in applied Software Engineering authored by Stephan Schramm and Jochen Eberle.