American Marketer


New: Q3 2014 mobile site performance benchmark report

November 18, 2014

Site to see for Bloomingdale's Site to see for Bloomingdale's


In a new exclusive series, this publication will report on quarterly mobile site performance as monitored by Catchpoint Systems. Find out in this first installment how leading brands in the retail, banking and travel sectors fared.

Here, in its entirety, is the report as penned exclusively for this publication by Leo Vasiliou, director of Web performance engineering at Catchpoint Systems, New York.

In the third quarter, mobile site performance for retailers was strong overall: the average document complete time, or the time it takes for an end-user to begin interacting with the page, was at an acceptable 2,357.7 milliseconds, or 2.4 seconds.

Still, with the holidays right around the corner, there is room for improvement.

Many mobile sites in retail feature an excess of data and requests, particularly compared to other industries such as banking.

Some retailers do not even have dedicated mobile sites, and accessing their site on a mobile device simply brings you to their desktop version. As a result, these companies are generally at the bottom of our benchmark rankings.

But even if a retailer does have a separate mobile site, its performance can degrade due to factors such as too many high-resolution images or third-party requests.

A site such as Gap, for example, was around 3MB of data for about two weeks due to several high-res images on the page.

On the other hand, sites that had fewer bytes and objects on the page were significantly faster.

The foremost example of this would be, whose mobile site was both the lightest page – eight objects and 55K downloaded bytes – and the fastest in the measurements of document complete, which is the time it takes for the user to be able to click around the page, and Webpage response, which is the time it takes for all site elements to load, including those below the fold.

Compared to Sears, Gap was roughly 29 times heavier on average, with a Webpage response time that was nearly five times slower.

In our view, Sears should be positively recognized for the investments they it is clearly making in its mobile site performance.

Leo Vasiliou is director of Web performance engineering at Catchpoint Systems Leo Vasiliou is director of Web performance engineering at Catchpoint Systems

Considering the news of store closings in recent weeks – right before the lynchpin holiday season – this seems to indicate that the company is heavily focused on driving online sales and, more specifically, mobile Web sales. The retailer has done this by using the URL to house just a search function and one featured deal.

Meanwhile, Best Buy and Lands’ End have taken note of this trend, as their sites were considerably lighter from the previous quarter, and saw a marked improvement in their performance as a result.

Top sites (Webpage response):

1. Sears

2. Grainger

3. Newegg

Top sites (document complete):

1. Sears

2. Office Depot

3. Bloomingdale’s

In general, mobile sites in banking are performing slightly better than mobile sites in retail due to fewer objects on the page and bytes that need to be downloaded.

The average document complete time for banking mobile sites was 2,096 milliseconds, or 2.1 seconds – 11.1 percent faster than mobile sites in retail.

According to a recent report by Bank of America, mobile banking is really starting to go mainstream.

While it does not look like mobile banking will replace branch visits any time soon, it can drive major cost-savings for banks. This is why many banks are focusing on delivering strong end-user experiences that will help promote adoption of this channel.

Banks clearly understand that speed and reliability trump feature-richness when it comes to creating happy customers in the mobile realm.

In line with this, the banking sites we tested did not have a lot of re-targeting or marketing initiatives on their login pages, which translated to lighter and faster pages.

This was not the case for all pages, however, as certain sites such as Capital One and HSBC had more objects than their competitors and, as such saw Webpage response times that were 27 and 50 percent slower, respectively, than the industry average.

Top sites (Webpage response and document complete):

1. Citizens Bank

2. US Bank

3. Citibank

The travel sites that we monitored generally registered slower times, but better availability than the ecommerce sites.

The average document complete time for mobile sites in travel was 4,002 milliseconds, or 4 seconds, due to the fact that they are, by and large, heavier than mobile sites in retail.

Obviously, mobile sites in travel are slower than banking as well, and with good reason.

Since travel companies traditionally are feature-rich, image-heavy sites, they cannot get away with being as slim as the banking sites.

While overall the travel industry has some work to do when it comes to driving mobile Web speed, there are some bright spots.

For instance, Kayak came in as the fastest in terms of document complete time, which is not surprising given that it has made a concerted effort through the years to position itself as one of the fastest booking sites around.

With the critical Thanksgiving travel period approaching, it will be interesting to see what steps are taken to lighten mobile sites and enhance speed.

Top sites (Webpage response):

1. Expedia

2. Delta

3. United

Top sites (document complete):

1. Kayak

2. Southwest

3. Delta

As the data shows, the lighter a site is, the better it will generally perform on mobile devices.

It is important to remember that mobile devices have limited resources and far less caching ability than desktops, so by limiting the amount and size of requests, the faster the page will load at the last mile.

Therefore, it is important to have a dedicated mobile site that is as lean as possible rather than just mirroring the desktop version.

One way to do this is limit the amount of images on the mobile site, and for those that must be there, to use lower resolution versions than what you have on the desktop site.

Third-party requests can be more difficult to manage, particularly for retailers, due to the need for a compromise between speed, usability and marketing initiatives, but any efforts that can be made to limit them or manage them in real-time will lead to faster response times and fewer single points of failure.

With peak traffic periods coming up, organizations across industries should also have emergency contingency plans that allow them to remove any problematic third-party services immediately when a performance issue is identified.

IT IS NOT worth having the latest and greatest third-party functionality if it is going to slow your whole mobile site down and hurt revenues in your make-or-break season.

We also recommend using design techniques that are specifically designed for mobile sites.

Responsive Web design is the most popular version due to the cost of implementing it, but adaptive Web design has proven to lead to much faster mobile performance, specifically, 49.5 percent faster, on average, based on a recent study we completed. This is often the case even when adaptive sites are heavier, speaking to the strength and versatility of adaptive design in filtering out unnecessary content for mobile devices, while retaining feature-richness for desktop users.

Leo Vasiliou is director of Web performance engineering at Catchpoint Systems, New York.