Acceptable application response times vs. industry standard

Scott Barber

It feels like hardly a single day has passed in thepast six years that someone hasn't asked me this questions: "What isthe industry standard response time for a Web page?"
And in the past six years, the answer hasn't changed, not even alittle bit. So if the answer hasn't changed, why am I still gettingasked the question on virtually a daily basis?
The answer is simple. It's because there are no industry standards.How could there be? Think about how you use the Web. How long were youwilling to wait for this page to load? How long are you willing to waitto view your family's online photo album? How long are you willing towait for your tax software to confirm that your return has beensubmitted successfully? Are those numbers the same when you are at homeas when you are at work? How about when you are using the wirelessconnection in an airport?
Your actual numbers don't really matter. The point is that no onenumber could possibly be the answer -- at least until Web pages startregularly having response times fewer than .25 seconds. Until then,what you are measuring is a combination of your current expectationsabout Web page response time and your determination to accomplish tasksvia the Web.
This is because by the early 1980s cognitive psychologists hadalready determined that a delay of longer than one quarter of a secondbetween an action and a response, on a computer or otherwise, wouldnoticeably impact humanperformance, increasing error rate andincreasing the probability of switching to a competing task. So, as faras I'm concerned, until our Web sites make it to that .25 secondbarrier, what matters more than agreeing on a standard is staying aheadof the expectations of our users.
For years, the most commonly quoted standard was the so-called"8-second rule." This was based on some research Nielsen Mediaconducted in the late 1990s, which concluded that most Internet userswouldn't give up on the task they were trying to accomplish as long asthe Web site responded in 8 seconds or fewer. While that was certainlyan interesting piece of research, it had nothing to do with usersatisfaction nor was it ever intended as an industry standard. What itdid measure was the degree to which people had come to accept that, ifthey wanted to accomplish a task on the Web, 8 seconds was how long itwas bound to take over their 33.6 kbs modems. I can assure you that ifthose users had been presented with an option of one site with an8-second response time and a competing site with a 3-second responsetime, they would have flocked to the 3-second site without a secondthought.
In November of 2006, a new study popped up that almost immediatelyreplaced the "8-second rule" with a "4-second rule." The title of thepress release is"Akamai and Jupiter Research Identify '4 Seconds' as the New Threshold of Acceptability for Retail Web Page Response Times"and its first line reads as follows:
"CAMBRIDGE, MA — November 6, 2006 --Four seconds is themaximum length of time an average online shopper will wait for a Webpage to load before potentially abandoning a retail site."
As this claim cut the existing "rule" in half, I found it to be anintriguing finding, so I downloaded the whole report, only to find outthat this "new rule" was determined by collecting 1,058 responses tothe following survey question:
"Question: Typically, how long are you willing to wait for a single Web page to load before leaving the Web site? (Select one.)
A. More than 6 seconds.
B. 5-6 seconds.
C. 3-4 seconds.
D. 1-2 seconds.
E. Less than 1 second."

Clearly, this "new rule" is no more an industry standard than theNielsen research from nearly a decade before. The Nielsen research wasat least observationally accurate, if misused; this research simplydemonstrates that we all learned the same rule for taking multiplechoicetests in junior high school: "When you have no idea what thecorrect answer is, pick C; you might get lucky."
Try it yourself. Ask the person in the office next to you thisquestion and see what his or her answer is. Then ask your guinea pig tosurf the Web and find a Web page that loads in the same time bracket ashis or her answer. Use your watch to see how close he or she is toestimating the load time. Do that with 10 people and see what kind ofaccuracy you get.
I have been doing performancetestinglong enough to know that Websurfers have no idea how long 4 seconds is. In fact, I promise that ifsomeone were to sit down with those respondents and ask them toidentify how many seconds various pages took to load, *most* of themwould not get it right, and we would find that *most* of the wrong ones*think* a page takes longer to load than it actually does.
The real question is not "What is the industry standard?" but rather"What response time will the users of my Web site or application findacceptable?" The challenge is that determining what your users aregoing to deem "acceptable" is both difficult and subject to significantchanges over short periods of time. Software development shops don'twant to do regular usability studies with groups of representativeusers because it is time-consuming and expensive. For the most part,they don't have the resources or the training to conduct thoseusability studies even if they wanted to, which is why so many folkskeep latching onto narrowly conducted anecdotal research andproclaiming a standard. The real problem is that defaulting to a faultystandard is actually more likely to lead people to develop and releaseWeb sites that users find frustrating due to poor performance than ifthose same people just sat down and used the site, deciding whetherperformance was good enough based on how it felt.