11Jun2012
Due to the increasing popularity of smartphones and tablet computers in recent years, more and more people are accessing the web using mobile devices and expecting an experience as polished and usable as the desktop experience. As a result, it is now more important than ever that web designers, developers and user experience experts take into account these users when creating their websites. This can be achieved by creating a single responsive site or distinct desktop and mobile sites.
And when something is developed, it needs to be tested! A recent project at Auros included mobile-specific versions of a client’s main, desktop website.
The Auros test department had the task of testing both versions of the site. Typically, we test on the most popular desktop browsers: Internet Explorer, Firefox, Chrome, and so on. This includes different versions of a browser e.g. IE9, IE8 and IE7.
Narrowing it down
With such a wide variety of mobile devices out there, with different operating systems and browsers, it can be daunting considering all the possible permutations. As is always the case with testing, a balance has to be reached between testing everything ever (and thus taking forever) and completing the project within time and budget constraints. With clients often having important deadlines they need to meet, such as a website going live to coincide with a big marketing campaign, it is therefore imperative that testing focuses on the major features of the product and concentrates on the parts of the product most likely to yield bugs.
Hand in hand with this is the consideration of the platform on which the majority of users will access the product (in this case, a website). From a potential list of hundreds of devices and the many browsers compatible with them, we narrowed it down to a manageable selection. In reviewing site analytics, we identified the most popular devices. This, together with the latest default browsers, defined the testing baseline. Therefore, we tested on platforms including iOS (iPad, iPhone), Android, Windows Phone, and BlackBerry.
Emulators and simulators
iPhone Drift in action
Previously, we had explored the possibility of testing using emulators and simulators, rather than actual physical devices. The advantage of this approach was that, potentially, the website could be tested on a wider range of platforms with a lower setup cost. A quick search turns up helpful lists of such tools. There were many web-based simulators but, as our test sites were being hosted on our internal network, the challenge was to find downloadable, standalone simulators and emulators that could connect with these internal test sites. A number of products were considered, including the following:
We found that testing using these simulators/emulators was not a viable alternative as, for a truly accurate representation of the user’s experience, you need to develop and test using the very devices they will be using. As Stephanie Rieger notes on her blog, some issues are only apparent when using the actual device, rather than a software imitation of one.
Our mobile test devices
It’s useful to replicate the user experience of handling the physical device with physical actions such as multi-touch gestures. Therefore, we decided to test using a collection of actual mobile devices. For most people, the biggest barrier to putting together a good selection of mobile test devices is the initial cost of buying all these things. Brad Frost provides a practical guide on his blog.
The best things come in small packages (as well)
Testing the mobile version of the site required a different approach from that applied to the desktop site. Our designer had defined the web pages to fit a smaller format, with, for example, collapsed menus and resized elements.
As well as testing these changes, we had to check that the website worked in both portrait and landscape view. Some features needed to be tested to see whether they were just as easy to use (or indeed usable at all) with a touchscreen interface. One thing to consider is the differing user interfaces employed by a touchscreen device and a screen/keyboard/mouse combo. For a slideshow, you might have buttons to click through slides. But would a touchscreen user expect to be able to swipe through slides using their fingers? Alternatives to Flash controls were added to the mobile site, to cater for iOS and Windows mobile devices.
Besides the above differences, the mobile site has the same content and functionality as the desktop site. Part of the testing cycle involved re-running the tests carried out on the desktop site; for example, a feature which generated a personalised PDF brochure based on selections made in an interactive tool. Overall, mobile users have a comparable experience with that of desktop users.
By the end of this project, Auros had designed, developed and tested a website that was viewable on a wide range of devices, from desktop PCs to iPhones. Our investigations made clear that there is no substitute for the physical device as used by the end user to access a site. To test real experiences, you need real devices.
Najeeb is an ISEB qualified test professional and has 8 years’ experience as a Test Analyst in the IT industry spanning the commercial and public sectors. He joined Auros to coordinate the testing processes, and is responsible for day-to-day testing activities.
Read more...
Testing real experiences with real devices
Tags: Comment, Mobile, Testing
Posted by: Najeeb Ahmed
And when something is developed, it needs to be tested! A recent project at Auros included mobile-specific versions of a client’s main, desktop website.
The Auros test department had the task of testing both versions of the site. Typically, we test on the most popular desktop browsers: Internet Explorer, Firefox, Chrome, and so on. This includes different versions of a browser e.g. IE9, IE8 and IE7.
Narrowing it down
With such a wide variety of mobile devices out there, with different operating systems and browsers, it can be daunting considering all the possible permutations. As is always the case with testing, a balance has to be reached between testing everything ever (and thus taking forever) and completing the project within time and budget constraints. With clients often having important deadlines they need to meet, such as a website going live to coincide with a big marketing campaign, it is therefore imperative that testing focuses on the major features of the product and concentrates on the parts of the product most likely to yield bugs.
Hand in hand with this is the consideration of the platform on which the majority of users will access the product (in this case, a website). From a potential list of hundreds of devices and the many browsers compatible with them, we narrowed it down to a manageable selection. In reviewing site analytics, we identified the most popular devices. This, together with the latest default browsers, defined the testing baseline. Therefore, we tested on platforms including iOS (iPad, iPhone), Android, Windows Phone, and BlackBerry.
Emulators and simulators
iPhone Drift in action
Previously, we had explored the possibility of testing using emulators and simulators, rather than actual physical devices. The advantage of this approach was that, potentially, the website could be tested on a wider range of platforms with a lower setup cost. A quick search turns up helpful lists of such tools. There were many web-based simulators but, as our test sites were being hosted on our internal network, the challenge was to find downloadable, standalone simulators and emulators that could connect with these internal test sites. A number of products were considered, including the following:
We found that testing using these simulators/emulators was not a viable alternative as, for a truly accurate representation of the user’s experience, you need to develop and test using the very devices they will be using. As Stephanie Rieger notes on her blog, some issues are only apparent when using the actual device, rather than a software imitation of one.
Our mobile test devices
It’s useful to replicate the user experience of handling the physical device with physical actions such as multi-touch gestures. Therefore, we decided to test using a collection of actual mobile devices. For most people, the biggest barrier to putting together a good selection of mobile test devices is the initial cost of buying all these things. Brad Frost provides a practical guide on his blog.
The best things come in small packages (as well)
Testing the mobile version of the site required a different approach from that applied to the desktop site. Our designer had defined the web pages to fit a smaller format, with, for example, collapsed menus and resized elements.
As well as testing these changes, we had to check that the website worked in both portrait and landscape view. Some features needed to be tested to see whether they were just as easy to use (or indeed usable at all) with a touchscreen interface. One thing to consider is the differing user interfaces employed by a touchscreen device and a screen/keyboard/mouse combo. For a slideshow, you might have buttons to click through slides. But would a touchscreen user expect to be able to swipe through slides using their fingers? Alternatives to Flash controls were added to the mobile site, to cater for iOS and Windows mobile devices.
Besides the above differences, the mobile site has the same content and functionality as the desktop site. Part of the testing cycle involved re-running the tests carried out on the desktop site; for example, a feature which generated a personalised PDF brochure based on selections made in an interactive tool. Overall, mobile users have a comparable experience with that of desktop users.
By the end of this project, Auros had designed, developed and tested a website that was viewable on a wide range of devices, from desktop PCs to iPhones. Our investigations made clear that there is no substitute for the physical device as used by the end user to access a site. To test real experiences, you need real devices.
About Najeeb Ahmed
Najeeb is an ISEB qualified test professional and has 8 years’ experience as a Test Analyst in the IT industry spanning the commercial and public sectors. He joined Auros to coordinate the testing processes, and is responsible for day-to-day testing activities.
Read more...
Leave a comment