QA in the cloud age

Let me start with a few snippets on the adoption of Cloud from our latest World Quality Report 2020-21 insights

• Ever increasing demand on cloud adoption
  • Based on 2018 survey, 73% of all applications are based in the cloud
• Rise of Multi cloud
  • To prevent vendor lock in, Companies are hosting different applications with different cloud service providers
• IoT driving edge computing
  • As IoT and its associated technologies mature, more companies will be adopting a mixture of edge computing and cloud hosting in a manner that optimizes results
• Key CIO concerns
  • Security requirements
  • Performance management
  • Risk management
  • Third party integration

While for now most of the testing has revolved around “cloud based” or “cloud enabled” application testing, its more important for Quality engineering community to understand and differentiate the testing requirements for the “cloud native” applications.

Creating Cloud-Native applications refers to the approach of design, architect, and build distributed software applications in such a way that they can take complete advantage of the underlying PaaS (Platform-as-a-Service) and IaaS (Infrastructure-as-a-Service) service models offered by the cloud service providers. Most often these applications are built as a suite of small Microservices (Reference: public domain definition)

Hence it’s important that the we redefine the approach towards the testing for “Cloud Native” apps. Following could be some key areas that one needs to keep in mind while building the right strategy for such testing

• Apply microservices testing approach for Unit, Integration and End to end testing
  • Given there could be a large number of tests due to multiple combination and permutations, optimize the same via leveraging risk based approach to testing
  • The right strategy and insights on the readiness of services is most important to avoid wastage due to non readiness or flakiness in testing due to non availability
• Contract Testing approach
  • It’s a standard approach used in Microservices testing; automation of such contract test coupled with integration pipeline will enable early detection of defects
• Non functional testing especially Chaos engineering/ Failure mode testing
  • The need to ensure software meets the non-functional requirements for scalability, flexibility, and resilience is of equal importance as to ensure the product meeting its business requirement
  • As compared to old fashioned monolithic applications, identifying potential failure modes is challenging due to very nature of microservices architecture.
  • Chaos engineering is helpful by injecting small controlled failures and making it possible to detect and analyse those and taking corrective actions
  • Its important to note – its not always to be done at staging environment but in live environment where you can take the risk of crashing the server on purpose. If the system is built right, another server will take over. But its important to understand the overall consequence and accordingly design your test strategy
• Extend to post production tests – Observability and Log analytics
  • Analysis of monitoring logs and metrics can enable information related to the application state, behaviour or interaction between the services in production. These can give good insights to evaluate and debug any issues quickly
  • A good combination of observability tools and monitoring tools will compliment each other.
• Version controlled aspects in testing
  • Given we are working with technologies like Kubernetes that will dynamically change the versions of containers, the roll back challenges needs to be taken care of
  • Its utmost important that one keeps track of changes from one version to another while testing a multi-version scenario.
• Shift in your App security approach
  • Given there are so many underlying components in cloud native app development, its more challenging and important to look at the security aspects differently
  • Everything is a code and scanning through code or configuration or integrations is all an important aspects as part of the strategy
  • Growing need for Infrastructure as code and the challenges to do scan of the same and establishing connection between code, infrastructure and configurations
  • Bottom line test from first line of code written.

One must understand, we can plan for extensive functional, non-functional testing to improve the quality of cloud native applications, end users might still face issues. Our strategy should enable us to reduce the risk of failures and in event of failure, should enable us to quickly analyse and fix the issues and avoid in further releases.