English | EN FAMOUS LAST WORDS
July 16, 2019
No, I’m not leaving. Yet. That I know
about.
I’m harping back to one of those immortal
phrases I used to love hearing as a tester, as it inevitably was proven not to
be so time and time again. The phrase is, the legendary
“It’ll never happen in production”
Whilst it’s featured a few times in my
career, one instance stands out in my halcyon memories of my early career but
not because of what happened in production.
I raised as a defect but it was duly
rejected in triage by those in authority because, guess what, “it’ll never happen in production”. For those
interested, it related to the location of an account number in an authorisation
message and there were, to the best of my recollection, two or three possible
locations the account number could be populated in the message. The defect
found an error when the account number was placed in one specific position in
the message and triggered an error.
The next stage of testing prior to releasing the code was a series of accreditation tests which I conducted in collaboration with a representative of the target system. The lady, I forget her name, drive into her office in Dallas through heavy snow for a 5 am start. The second test case focused on the account number being in the location which would never happen in production, and we immediately failed accreditation, waiting another 5 weeks until we got back to, and subsequently, passed accreditation.
A second, similar instance meant I found a
feature which was already in production, and whilst extremely unlikely to ever
happen, would have effectively disabled all PIN number checks had it ever
occurred. It was fixed very shortly after it was detected. Sorry.
I do though feel a huge amount of professional pride in those instances. In the first one because I identified the risk and it was validated by a second party, overturning the triage decision. And in the second one because, through luck as much as judgement, I found something significant which had been missed, possibly for many years. And in both cases, because the resulting production software contained a little less risk and was of a little higher quality.
What concerns me is that with a shift away from Quality Control there is an increased risk of defects oozing, albeit unintentionally, into production. As we transition to Quality Assurance how do you build quality into a process which ensures the requirements are complete and development hasn’t missed anything? There is something about the “I wonder what happens if I do this…?” mentality of a professional tester.
Depending on what you believe, agile has this all covered. Nothing gets missed. Until I collected up my new smart railcard, a chip-based card to store my tickets on. Because as a consultant my work location varies between my base office, client sites, pre-sales visits and a few days working from home, this means I can neither buy a long-term season ticket (easily), and I also try to optimise the cost as much as possible. The smart ticket offers both weekly and ten return tickets, called a “Carnet” ticket, to use over a two-month period. Which I thought was fantastic for my needs.
But I had one nagging doubt. What happens
if I have 8 remaining return tickets and I decide to use a season ticket one
week? I asked a nice man at the station and he said “Good question”. A further
enquiry to the rail company produced, eventually, a rather sheepish “Well,
currently, you’ll pay twice. Once from your weekly ticket, and once for a
Carnet ticket.”
It seems this test case was simply missed.
Whether this is a product of a more agile development lifecycle I simply do not
know. I am just glad I asked the question before I started unwittingly paying
twice from some train journeys, as from the tone of the response, it may not be
easy to fix this one fast now the fail has been uncovered in production.
With ever-increasing agility and automation supporting a move to continuous deployment, the role of the tester is becoming increasingly important as we transition from control to assurance. Whilst increased agility is enhancing software engineering practices and increasing quality, in many cases, we’re still not there yet. And I would argue the increased frequency of code drops represents an increased risk of something being found in production.
Sogeti’s test professionals deliver success
by constantly asking “What happens if I do …?” to safeguard production, whilst
our extensive experience of digital transformation will help your business
transition move to a more quality assurance-centric model supporting continuous
development.
About the author