English | EN What do we mean by performance testing and what is the real purpose?
Jan 6, 2016
For many years the generally accepted approach has been load testing, with the general intent of determining that capacity planning has been sufficient. When the dynamic web content is largely gernerated server side, using JavaScript primarily to show and hide content this was enough. It gave us a general picture of performance and life was good.
Most of the new web development I see utilizes some form of MV* JavaScript framework. This creates a whole new world for performance testing. We are no longer serving up dynamic content. We are now serving up data, usually in JSON format, although the format really doesn’t matter. What matters is that much of the processing that used to take place on the server now happens in the browser. With the Single Page Application (SPA) pattern the network traffic generally becomes more chatty, but very little traffic is actually going across the network.
Does the traditional performance test still meet our needs? The simple answer is “in part”. I say in part because we still need to make sure we are insuring sufficient server capacity to handle the network traffic. Despite the fact that we aren’t typically sending as much data across the network, we are using more connections. Where we used to pull down one dynamically generated page and the associated static content (e.g. image files, JavaScript files, CSS) all of which are typically cached by the browser, we typically pulled down one page. Now we are making several AJAX calls to REST services per “page”. These REST services are generally nothing more than a light weight HTTP wrapper around a database call.
But there is one thing that we are missing completely. We aren’t testing user facing performance. When we push all that processing to the browser, that’s where our user performance occurs. I’ve seen cases where the network transfer for all the data on an SPA view takes less than 500 milliseconds, but the page took 16 seconds to render. With traditional testing this would be ignored unless someone notices it through anecdotal evidence. We can’t rely on this happening as we will typically hear, “that isn’t a production environment”, harkening back to the days when nearly everything was server side.
The state of web development as 2015 comes to a close should remind us that any paradigm shift must be viewed holistically. If development has undergone a paradigm shift the way we must review whether or testing approach also requires a shift. Developer testing (e.g. TDD and ATDD) will obviously have to change if we are to start pushing much of the processing to the browser. Developers will have to write many more tests for the front end JavaScript code than they ever have before. Fortunately most of these MV* frameworks have been written to embrace TDD and ATDD.
We must start looking at how long the browser takes to render the view. When we do this we must also look at each of the common browsers we can expect to use this view. Each JavaScript engine will execute at different speed. What works well in one browser may be incredibly slow in another. When we have a responsive design, performance can be even more problematic.
So what do we do? Most performance testing tools still focus on server response times. Some may have add-ons that will test rendering time, generally only on one browser. ATDD tools typically don’t measure performance, but can be adapted to do so. But is this the best approach?
When dealing with any paradigm shift there are three key components which need to be addressed, People, Process and Tools. For this shift the people need to change to see that server response times are only a part of the equation. The process needs to address that a significant portion of the processing is moving to the browser and that processing must be performance tested. Finally tools need to be improved or adapted to address this need.
The point is, performance testing just got a lot harder!
About the author
