Back

Usability Analysis & Evaluation

White Paper
  |  
Published: 
July 1, 2008

An increasing number of non-accredited certification programs offered by industry firms suggests a growing demand for usability evaluation skills. While some specialized advanced degrees exist within formal academic programs such as Human-Computer Interaction (HCI) within computer science, human factors within psychology and physiology and, to some degree, information architecture within library sciences, there are limited...

View Report  
Usability Analysis & Evaluation

An increasing number of non-accredited certification programs offered by industry firms suggests a growing demand for usability evaluation skills. While some specialized advanced degrees exist within formal academic programs such as Human-Computer Interaction (HCI) within computer science, human factors within psychology and physiology and, to some degree, information architecture within library sciences, there are limited options for specialized study related to Web-based systems. At this time, colleges may be in a position to lead national or international efforts to formalize credentials and curriculum related to Web-based usability evaluation, analysis and design. Usability evaluation is not limited to the computer world, as it is critical in auto design, architecture, medical devices, system integration, and many other fields. As usability is a multi-disciplinary craft, it is another example of the developing trend toward knowledge and skill mergers as indicated in several previous TSTC TechBriefs and Emerging Technology publications.

According to Human Factor International, usability analysts earn between $40,000 and $120,000 per year depending on experience.1  Applicants with industry experience and usability evaluation skills such as survey design, statistical analysis, information architecture, small group communication and presentation skills will have a strong advantage in the workplace. Specialized usability jobs have increased in demand and recognition in recent years; however, much of the training is happening on the job, in commercial programs of training and certification or in specialized advanced degrees.2 Some independent small firms, often non-accredited, who specialize in usability design services offer non-credentialed usability evaluation training. Although existing community and technical college programs such as digital design, Web design and information technology incorporate usability as a topic of study, these courses tend to focus on usability design with less attention to analytics. Colleges should investigate strengthening usability analytics within existing Web, application development, computer science, video game development and other programs involving human/machine interaction, and explore the market potential for specialized certifications dedicated to usability evaluation within corporate training and continuing education programs. Such certifications may be well suited to enterprise employees with three-to-ten years of experience. Current “Interface Design” courses (IMED 1041, 1341, 1441) do mention usability but the actual curriculum is focused predominantly on usability design with little focus on actual usability evaluation.3 This approach is important and valid; however, there is a need for skilled and even credentialed “usability evaluation” professionals who specialize in the evaluation of new and existing Web-based systems. Usability evaluators can be programmers or artistic designers; however, the core of the skill set is statistical evaluation, survey design, and communications rather than specific computer-based artistic or technical skills.

Usability evaluation is a research discipline incorporating observational studies and user interviews to inform product development and enhancement.

Usability Defined

ISO 9241-11 defines usability as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.”4 Usability testing evaluates the effectiveness of products and systems to more closely match user needs and requirements. This process should take place before, during and after the production phase.

Good usability is typically a product of a user-centered design process. User-Centered Design (UCD) situates people in the center of the product design process in order to optimize ease of use.5 The Elements of User Experience.) Although this may sound like common sense, many Web-based systems (as well as products and services in general) are designed without significant consideration of how people actually use such systems.

Usability is also a critical factor in the design of a system’s user interface as well as the supporting data architecture. For example, the iPhone uses “gestures” that enable users to navigate, select and read content by tapping, flicking and pinching the screen. This innovative interface is extremely intuitive and has set a new design standard that other phone companies are quickly duplicating. While gestures enable end-users to easily access the iPhone’s features, the back-end architecture provides developers with sets of tools and logical frameworks. The iPhone’s accessible data architecture, frameworks, and tools empower developers to create new applications that increase the platforms functionality and strengthen adoption.

Usability Evaluation Model (Source: Enhance.ie)

Usability evaluation is a research discipline incorporating observational studies and user interviews to inform product development and enhancement.6 The Usability Toolkit (http://www.stcsig.org/usability/resources/toolkit/toolkit.html) provided by the Society for Technical Communication gives examples of the process and tools of usability evaluation. Employment opportunities exist for those who incorporate usability into the production process as well as for people who specialize in usability evaluation.

Employability

In a 2005 survey, Human Factor International found that usability analysts earn between $40,000 and $120,000 per year, depending on experience.7 Information and communications technology-based usability jobs may be found across industries. Examples of specific usability evaluation job titles include usability analyst, usability tester, usability engineer and customer experience designer.

Other job categories and academic disciplines which may benefit from usability practices include software developer, programmer, Web developer, application developer, information architect, user interface designer, graphic designer, technical writer, business analyst, market researcher, quality assurance, project manager, documentation specialist, librarian, metadata analyst and trainer.

Competencies

Usability analysts are responsible for experimental design, usability testing, statistical interpretation and reporting.8 Usability analyst tasks include analyzing markets and competition, observing and interviewing users, interpreting data from user feedback sessions, creating prototypes and conducting prototype feedback sessions, capturing the design and the rationale for design decisions, communicating the design to others, and making design trade-offs based on schedules, resources, cost and the priorities of other designers.9  Skills also include group facilitation and interpersonal skills, research, critical thinking and analytical skills, and proficient technical and business communication skills (i.e. writing and speak ing). As such, the usability analyst is often distinguished from production staff (such as graphic designers and programmers) who practice usability design as a function of product development.

The Usability and User Experience Community of the Society for Technical Communication[10."Usability Toolkit." Society for Technical Communication. Last visited 12 August 2007] has created a usability “toolkit” which includes informational tools and models for usability evaluation and design including:

Trends

New products and processes, such as Apple’s iPod and the Staples “Easy” business processes, are examples of how businesses are adopting usability as a means to differentiate brands. The practice of usability is not yet widely accepted throughout the business world (Fisher); therefore, it is very easy to find examples of products and systems that confuse and frustrate end users.10

As user-optimized products, brands and companies win in the marketplace, we can expect to see an increase in the demand for usability skills-sets in traditional production staff as well as for specialized usability analysts. Information and communications technology-based projects and firms are also likely to increase emphasis on usability to enhance customer interaction.

Drivers

The pace of ongoing integration of Web-based applications into all facets of daily organizational “transactions” continues to increase. Specific areas of transformational growth include:

  1. the proliferation of Web-based systems within all types of organizations since the emergence of the commercial Internet;
  2. the need for commerce optimization to enhance user experience with Web-based transactional systems;
  3. the split between Web-based artistic design and technical design creating a space for an intermediary to bridge artistic, technical and customer world views; and
  4. the relationship between brand experience and Web experience which drives competition and the need for customer satisfaction in Web-based customer interactions.

Constraints

Many college programs teach usability as part of Web development, application development and graphic design; therefore, the perception is that usability is adequately covered within existing curricula. Web-based usability evaluation is, however, an emerging discipline which may be suited to continuing education and corporate training programs rather than a new degree within a college degree.

Ergonomics and usability professionals with bachelor’s, master’s or doctoral degrees may resist movement toward college programs specializing in Web-based usability evaluation; however, many usability companies are offering similar programs today.

Timing

Formal usability evaluation education is an outgrowth of more traditional university programs, including Human-Computer Interaction (HCI), human factors, computer science and cognitive psychology. The Board of Certification for Professional Ergonomics offers general certifications for ergonomics and human factors which require a master’s degree and/or related experience.11

In the U.S., many firms that provide usability consulting services also offer usability certifications. For example, Human Factors International has certified 1,300 usability analysts.12 There is much controversy surrounding usability certification. In 2002, the European Union UsabilityNET project worked with an international group to research and define the scope of usability certification; however, financial support was withdrawn.13 Today, usability analysts who can demonstrate prior work experience are often hired without a specialized degree because of their unusual skill set.14

As user-optimized products, brands and companies win in the marketplace, we can expect to see an increase in the demand for usability skills-sets in traditional production staff as well as for specialized usability analysts.

Relevance

Today, some colleges offer usability courses such as the WECM course “Interface Design” (IMED 1041, 1341 and 1441). These courses focus on usability predominately as a function of design. Colleges should consider integrating more formal and rigorous usability evaluation instruction into these courses. The recently developed course GAME 1027 is a step in this direction. In this course, various games are analyzed and evaluated and students are responsible for improving a user interface in need of re-design.15 Iterative user interface design and usability testing are emphasized.

In cases where local firms express the need for usability analysts who function separately but related to production staff, colleges should investigate continuing education and related certification and/or degree options. Colleges considering usability programs should consider the past work of study groups who proposed frameworks for certification including: “Technical Competence Definition for Usability Professionals”16 and “Accreditation of Usability Professionals”.17

The relatively new field of “information architecture” is intently focused on front-end user interface designs and the supporting back-end data architectures of various systems. The University of Texas at Austin has one of the first master’s degree programs in this new field.

Sources
  1. Usability Professionals’ Association, 2005 Member and Salary Survey. April 2006
  2. Whitaker, Barbara. “Technology’s Untanglers: They Make It Really Work.” The New York Times. 8 July 2007. Last visited 12 August 2007
  3. IMED Course 1041, Interface Design.” Workforce Education Course Manual 2006-2007. Texas Workforce Education. Last visited 16 June 2008
  4. What is Usability.” Usability Net. Last visited 12 August 2007
  5. Katz-Haas, Raïssa. “What is User-Centered Design?” Society for Technical Communication. Last visited 12 August 2007
  6. What is a Usability Professional?” Usability Professionals’ Association. Last visited 12 August 2007
  7. UPA, op. cit.
  8. Adams, Carol. Phone Interview. Ergosoft Laboratories. 6 August 2007.
  9. Fisher, L. as quoted in Berni, K. and M.T. Davis. “Usability.” 29 April 2002. Last visited on 12
  10. Gadney, M. “The Secret of Making Things Work.” BBC News. 1 November 2005. Last visited on 12 August 2007
  11. About BCPE.” Board of Certification in Professional Ergonomics. Last visited 12 August 2007
  12. Complete Certification Package for Usability.” Human Factors International. Last visited 12 August 2007
  13. Bevan, N. “Accreditation of Usability Professionals.” Usability Net. 2003. Last visited on 12 August 2007
  14. Nash, Betsy. Phone Interview. Sphiere Recruiting. 3 August 2007
  15. Austin Community College. Video Game Development Art Course Descriptions. Last visited 16 June 2008.
  16. Bevan, N. “Technical Competence Definition for Usability Professionals.” Usability Professionals’ Association. 2002. Last visited on 12 August 2007
  17. Bevan, op. cit.
View Report  

More from C4EO

Download the Report

Enter your information to access the full publication.

By clicking "Submit" you give C4EO permission to contact you with updates and news related to our initiatives. We do not spam.

Your download is ready!

Thank you for your submission, click below to download your file.
Download Now
Oops! Something went wrong while submitting the form. Please try again.