English | EN 
No, I’m not leaving. Yet. That I know about.
I’m harping back to one of those immortal phrases I used to love hearing as a tester, as it inevitably was proven not to be so time and time again. The phrase is, the legendary
“It’ll never happen in production”
Whilst it’s featured a few times in my career, one instance stands out in my halcyon memories of my early career but not because of what happened in production.
I raised as a defect but it was duly rejected in triage by those in authority because, guess what, “it’ll never happen in production”. For those interested, it related to the location of an account number in an authorisation message and there were, to the best of my recollection, two or three possible locations the account number could be populated in the message. The defect found an error when the account number was placed in one specific position in the message and triggered an error.
The next stage of testing prior to releasing the code was a series of accreditation tests which I conducted in collaboration with a representative of the target system. The lady, I forget her name, drive into her office in Dallas through heavy snow for a 5 am start. The second test case focused on the account number being in the location which would never happen in production, and we immediately failed accreditation, waiting another 5 weeks until we got back to, and subsequently, passed accreditation.
A second, similar instance meant I found a feature which was already in production, and whilst extremely unlikely to ever happen, would have effectively disabled all PIN number checks had it ever occurred. It was fixed very shortly after it was detected. Sorry.
I do though feel a huge amount of professional pride in those instances. In the first one because I identified the risk and it was validated by a second party, overturning the triage decision. And in the second one because, through luck as much as judgement, I found something significant which had been missed, possibly for many years. And in both cases, because the resulting production software contained a little less risk and was of a little higher quality.
What concerns me is that with a shift away from Quality Control there is an increased risk of defects oozing, albeit unintentionally, into production. As we transition to Quality Assurance how do you build quality into a process which ensures the requirements are complete and development hasn’t missed anything? There is something about the “I wonder what happens if I do this…?” mentality of a professional tester.
Depending on what you believe, agile has this all covered. Nothing gets missed. Until I collected up my new smart railcard, a chip-based card to store my tickets on. Because as a consultant my work location varies between my base office, client sites, pre-sales visits and a few days working from home, this means I can neither buy a long-term season ticket (easily), and I also try to optimise the cost as much as possible. The smart ticket offers both weekly and ten return tickets, called a “Carnet” ticket, to use over a two-month period. Which I thought was fantastic for my needs.
But I had one nagging doubt. What happens if I have 8 remaining return tickets and I decide to use a season ticket one week? I asked a nice man at the station and he said “Good question”. A further enquiry to the rail company produced, eventually, a rather sheepish “Well, currently, you’ll pay twice. Once from your weekly ticket, and once for a Carnet ticket.”
It seems this test case was simply missed. Whether this is a product of a more agile development lifecycle I simply do not know. I am just glad I asked the question before I started unwittingly paying twice from some train journeys, as from the tone of the response, it may not be easy to fix this one fast now the fail has been uncovered in production.
With ever-increasing agility and automation supporting a move to continuous deployment, the role of the tester is becoming increasingly important as we transition from control to assurance. Whilst increased agility is enhancing software engineering practices and increasing quality, in many cases, we’re still not there yet. And I would argue the increased frequency of code drops represents an increased risk of something being found in production.
Sogeti’s test professionals deliver success by constantly asking “What happens if I do …?” to safeguard production, whilst our extensive experience of digital transformation will help your business transition move to a more quality assurance-centric model supporting continuous development.
