Executive Summary
The Internet and the Web continue to evolve to deliver new customer experiences and increased application utility. The label "Web 2.0," while imprecise, signifies the newest and best examples of this evolutionary process.Organizations are now adopting these Web 2.0 technologies and design methods to enable the creation of richer and more responsive interactions. But to be effective, the resulting applications must also be significantly more complex than traditional Web sites, complicating performance management and imposing new requirements on performance measurement tools.
The Web Application Landscape
The Internet and the Web have become the primary vehicle for business communications, evolving to subsume and replace older technologies. As software technologies exploit steady advances in the Internet hardware platform, the Web continues to evolve to deliver new user experiences and increased application utility. The most advanced example of the Web becoming a platform is the rich Internet application (RIA), reflects the gradual transition of Web applications from the simple thin-client Web browser to a richer distributed-function paradigm that behaves more like the desktop in a client / server model.
Measuring RIA Performance
This architecture complicates performance measurement, which goal is to understand the customer's experience. In an RIA, the time to complete a Web page download may no longer correspond to something a user perceives as important, because (for example) the client engine may be prefetching some of the downloaded content for future use. Standard tools that measure the time for web page downloads to complete can record misleading data for RIAs. To implement RIAs successfully, enterprises must re-evaluate their approach to performance management. Instead of relying on the definition of physical Web pages to drive the subdivision of application response times, RIA developers or tool users must break the application into logical pages. Measurement tools must recognize meaningful application milestones or markers that signal logical boundaries of interest for reporting, and subdivide the application's response time accordingly.
The Importance of Web Performance Management
What does web application performance mean to you? Most business executives would evaluate the success of a Web application by looking at business performance metrics such as revenue, costs, and customer satisfaction. Because an application may be created to serve customers, partners, members of an organization, or employees, the relative importance of those metrics may vary. For any Web application, effectiveness means simply fulfilling the planned design and delivery objectives, delivering online experiences that lead to satisfied customers, and so meeting the intended business performance goals.
Common Goals, Divided Responsibilities
In many organizations, responsibility for these four aspects of effectiveness is divided among four groups or departments.
Divide and Conquer:
Business – experts provide the content or specify the behaviors that are the site's purpose (Utility).
Design and usability – professionals specialize in making it easy for customers to navigate the site (Clarity).
Site developers – build the site in ways that determine download speed (Responsiveness).
IT staff – manage the systems that keep the site up and running (Availability) and maintain a consistent level of performance (Responsiveness).
Measurement Challenges of Web 2.0
The characteristics of Web 2.0 applications highlighted earlier-the network as a platform, collaborative environment, social networking, mashups, and rich media interfaces-create several additional challenges for all measurement tools. In increasing degree of complexity, these are:
o Measuring an increasingly diverse customer community
o Measuring the quality of collaborative interactions, in which customers both upload and download information.
o Measuring applications that incorporates a much richer variety of media and interfaces.
o Measuring application forms that no longer correspond to one (or more) HTML pages
o Reporting data in ways that provide insight into Web 2.0 applications.
Getting the Right Data
To sum up the importance of these issues, consider the popular saying covered by Tom DeMarco: You can not control what you do not measure. Measuring the wrong things, or basing key management decisions on reports that contain incomplete data, is as bad, if not worse, than not measuring at all. So it's not wise to measure Web 2.0 applications using only the tools and approaches developed for traditional Web sites. Inaccurate data undermines the effectiveness of any program of systematic performance management and causes performance-tuning skills and resources to be applied in ways that are not optimal. It can also lead to unproductive interdepartmental conflicts and disputes over service-level agreements with internal or external service providers when staff question the accuracy of the data, or discover discrepancies in data from different sources.
Keynote Web Performance Management Solutions
Success in each one of these five performance management activities depends critically on an organization's ability to gather and report meaningful, timely, and accurate measurement data with the focus on the right metrics. Since a key idea of Web 2.0 is enhancing the user's experience, it is vital to measure actual customer experience proactively.
Measuring the Diverse Customer Community
On Web 2.0 sites, personalization options allow customers to tailor their experience of a site to their individual preferences, and sites are carefully designed to download and display content efficiently and successfully in all major browsers. Because customers' experience depends on their Internet connectivity, sites may even adjust their content based on the browser's connection speed. Measurement data must reflect this diversity.
Measuring Upload and Download Performance
In traditional Web applications, customers consume content, so all performance measurement efforts have focused on download times as the key metric. But as Web 2.0 applications add collaboration and social networking features, customers also supply content. To ensure the quality of a customer's experience, it's therefore necessary to measure and report upload performance as well.
For example, the user of a collaborative application may navigate to a product or member directory, complete a login or authentication dialog, search for a certain subject or interest area, browse the results, select a particular area of interest, proceed to an upload page or dialog, complete a browse dialog to select content to be uploaded from their laptop or desktop, entering some additional descriptive metadata as appropriate for the application, and click the Upload button. Through this interaction download activities are minimal, and may respond rapidly. But if, after all this work, the upload stage is painfully slow, or fails altogether, because of congestion at the server end, that customer may be lost forever.
For an organization needing to measure and ensure the quality of its customers' experience when contributing content to a collaborative or social networking application.
Conclusion
When considering the need for Web testing and measurement, it can be tempting to view a Web application simply as a collection of Web pages. Adopting that simplifying view makes performance measurement and reporting tasks easier. But doing so ignores the dimension of Web performance most affected by Web 2.0, and may the most important feature of a web application-the paths taken by Web users. For every Web application, a user's ability to navigate smoothly through a relatively small set of business transactions-the "money paths" -is the key to business success. It is only by completing these key transactions that visitors are converted into customers, triers into buyers. Therefore, in addition to tracking key performance indicators like conversion rate, most organizations consider it essential to identify, track, and measure their key business transactions. By doing so, they can better understand and improve each application's conversion rate-the second key to site effectiveness and therefore the health of the business.