Usability uation with the Cognitive Walkthrough 自己整理.docx

/?p=1569UX O/Usability Evaluation with the Cognitive Walkthrough John Rieman,* Marita Franzke,* and David Redmiles* *MRC Applied Psychology Unit 15 Chaucer Rd Cambridge CB2 2EF England *US WEST Technologies 4001 Discovery Dr. Boulder, Colorado, 80303 *Department of Information and Computer Science University of California Irvine, California 92717-3425 ACM AbstractThe cognitive walkthrough is a technique for evaluating the design of a user interface, with special attention to how well the interface supports exploratory learning, i.e., first-time use without formal training. The evaluation can be performed by the systems designers in the early stages of design, before empirical user testing is possible. Early versions of the walkthrough method relied on a detailed series of questions, to be answered on paper or electronic forms. This tutorial presents a simpler method, founded in an understanding of the cognitive theory that describes a userUs interactions with a system. The tutorial refines the method on the basis of recent empirical and theoretical studies of exploratory learning with display-based interfaces. The strengths and limitations of the walkthrough method are considered, and it is placed into the context of a more complete design approach. Keywords: Cognitive walkthroughs, usability inspections, exploratory learning, software engineering. IntroductionOne of the basic lessons learned in the area of HCI is that usability evaluation should start early in the design process, optimally in the stages of early prototyping. The earlier critical design flaws are detected, the greater the chance that they can and will be corrected. Empirical usability testing, still the most comprehensive evaluation technique, however, is expensive and requires at least a working prototype. Traditionally, it is used at the end of the design cycle, where changes to the interface can be costly and difficult to implement. Unfortunately, usability recommendations given at this time are therefore often ignored. The cognitive walkthrough was developed as an additional tool in usability engineering, to give design teams a chance to evaluate early mockups of designs quickly. It does not require a fully functioning prototype, or the involvement of users. Instead, it helps designers to take on a potential userUs perspective, and therefore to identify some of the problems that might arise in interactions with the system. THEORY UNDERLYING THE WALKTHROUGHThe cognitive walkthrough is practical evaluation technique grounded in Lewis and Polsons CE+ theory of exploratory learning 3,4,5. The CE+ theory is an information- processing model of human cognition that describes human- computer interaction in terms four steps: 1) The user sets a goal to be accomplished with the system (for example, check spelling of this document). 2) The user searches the interface for currently available actions (menu items, buttons, command-line inputs, etc.). 3) The user selects the action that seems likely to make progress toward the goal. 4) The user performs the selected action and evaluates the systems feedback for evidence that progress is being made toward the current goal. For most realistic tasks that a user would attempt with a system, these four steps are repeated many times to achieve a series of subgoals that define the complete task. The cognitive walkthrough examines each of the correct actions needed to accomplish a task, and evaluates whether the four cognitive steps will accurately lead to those actions. THE WALKTHROUGH PROCEDUREPrerequisites to the walkthrough include: (1) a general description of who the users will be and what relevant knowledge they possess, (2) a specific description of one or more representative tasks to be performed with the system, and (3) a list of the correct actions required to complete each of these tasks with the interface being evaluated. The walkthrough is typically performed by the interface designer and a group of his or her peers. Small-scale walkthroughs of parts of an interface can also be done by individual designers as they consider alternative designs. In a group situation, one of the evaluators usually takes on the duties of scribe, recording the results of the evaluation as it proceeds, and another group member acts as facilitator, to keep the evaluation moving. With the prerequisites assembled and duties assigned, the walkthrough process involves examining each individual step in the correct action sequence and trying to tell a believable story about why the prospective user would choose that action. Note that this is not an open forum approach of predicting what activities the user might engage in, given this interface and task. It is specifically limited to considering whether the user will select each of the correct actions along the solution path. In many cases, the group of evaluators will readily agree that the user will select the correct action, and no further analysis is required. For example, the first action in using a Macintosh program may be to double-click its icon; the evaluators could readily agree that experienced Mac users would have little trouble with this step. Other cases, however, may be less clear. To assess the ease with which the correct action will be selected, the walkthrough process suggests four criteria for evaluating the stories told about the users actions. The four criteria for evaluating the stories directly reflect the information-processing model that underlies the walkthrough. They ask the evaluators to consider the users goal, the accessibility of the of the correct control, the quality of the match between the controls label and the goal, and the feedback provided after the control is acted on. RECENT EMPIRICAL RESULTSRecent experimental work has provided support for the theoretical assumptions underlying the cognitive walkthrough method 1. New users of display-based (GUI) applications employ a strategy of first scanning the interface for a well-labeled action, and then quickly narrowing their search by selecting that action. If further options are displayed as a result, the scan-search cycle will be continued until the guiding goal has been accomplished. The success of this strategy is dependent on the saliency of the next correct move in the interaction. Four design features determine whether an action will pop out at a first-time user. (1) Subjects will try label-guided actions first (menu items, buttons, etc.) before they experiment with direct manipulations of unlabeled objects (tools, double clicking, moving of objects). (2) A well-labeled action will be especially salient. (3) Providing few actions in the search set can help to narrow the search if labeling cannot be provided, or if criteria for a good label are difficult to establish. (4) Set effects may prevent users to try untypical actions. (5) Users are reluctant to extend their search beyond the readily available menus and controls. Frequently used interfaces techniques may bias users to search for them rather than for less frequent techniques. These findings suggest that evaluators should check for the type of interaction, the quality of the label, the number and grouping of alternative choices, and consider the overall flavor of the interaction techniques when evaluating the availability of actions and label matches. A SIMPLIFIED WALKTHROUGH METHODEarly evaluations of the cognitive walkthrough method criticized the tedium of form-filling, and the narrow band of the problems noted 5,6. The current version and recommendations for its application therefore rely on a minimal form. We suggest involving small groups of evaluators, rotating record-keeping and other duties, evaluating simple tasks first, keeping track of all problems identified in the process of the walkthrough (even though not discovered by it), and loosening the forms orientation once the evaluators are familiar with it. EXTENSIONSAs described above, the cognitive walkthrough procedure supports software developers in the upstream activities of identifying and refining requirements and specifications. It can be combined with other user-centered evaluation techniques to yield software products that more closely match users work contexts. These techniques affect the software development process by specifically incorporating the assumption that requirements will change incrementally. Under this assumption, developers must plan for change, along with the additional costs and reduced predictability that change implies. One avenue for developers to reduce the cost of evaluation and simultaneously make it more helpful in workplace settings is through program instrumentation. An approach using expectation agents adapts the cognitive walkthrough procedure to support the evaluation of prototypes with real end users in their work places 2. Expectation agents monitor users working with the prototype and report mismatches between developers expectations and a systems actual usage. Simultaneously, the agents provide end users with an opportunity to communicate with developers, either synchronously or asynchronously. SUMMARYThe cognitive walkthrough is an usability evaluation method based on cognitive theory. The tutorial presents the basic methodology and indicates how it fits into the software development cycle. References1. Franzke, M. (1994). Exploration and Experienced Performance with Display-Based Systems. Ph.D. Dissertation, Department of Psychology, University of Colorado; to be submitted as CHIU95 Technical Paper. 2. Girgensohn, A., Redmiles, D., Shipman, F. (1994) Agent-Based Support for Communication between Developers and Users in Software Design. Proceedings of the 9th Annual Knowledge-Based Software Engineering (KBSE-94) Conference (Monterey, CA), IEEE Computer Society Press, Los Alamitos, CA, September 1994. 3. Lewis, C., and Rieman, J. (1993). Task-Centered User Interface Design Q A Practical Introduction. Distributed via anonymous ftp (Internet address: ). 4. Polson, P.G., Lewis, C., Rieman, J., and Wharton, C. (1992). Cognitive walkthroughs: A method for theory- based evaluation of user interfaces. International Journal of Man-Machine Studies 36, 741-773. 5. Wharton, C., Rieman, J., Lewis, C., and Polson, P. (1994). The Cognitive Walkthrough Method: A Practitioners Guide. In Usability Inspection Methods, J. Nielsen and R.L. Mack (Eds.), New York: John Wiley & Sons, pp.105-141. 6. Wharton, C., Bradford, J., Jeffries, R., and Franzke, M. (1992). Applying cognitive walkthroughs to more complex user interfaces: Experiences, issues, and recommendations. Proceedings CHIU92 (Monterey, CA, 3-7 May, 1992). holder.The JobsBlogUser Experience Qualifications - Top 10, Kind OfPosted Monday, April 30 2007 by The JobsBloggers Many of you have asked about what attributes are important to showcase on a User Experience (UX) resume and are curious about what classes are out there to prepare for this compelling career path. Interestingly enough, many schools are adding these programs if they didnt already exist in their curriculum, so there seems to be a surge of interest in the field. There are many user-focused degrees and certifications available out there, sowell write about those in a future post.But for now, Frank and I (with the help of newly-hired Researcher, Tazin Shadid) have attempted to put together a TOP 10 LIST of what a UX Hiring Manager looks for in a UX Designer and UX Researcher. Okay, okay - we were short a few on each list BUT wed love to hear comments from those of you who know more so we can fill in the list further.For a Designer:1. Portfolio of various work in Interaction Design and/or Information Architecture2. Prior design experienceand/or an internship in a Win32 or web-based environment3. Degree (or equivalent work experience) in Design or Computer Science4. Mastery of design tools and applications5. Excellent interpersonal, communication, negotiation and leadership skills6. Experience with design & research methodologies7. Working knowledge of Winforms, WPF, C#, XAML are also big plusesFor Researchers:1.Advanced degree in Human Factors, Human Computer Interaction (HCI) or Cognitive Psychology (or equivalent work experience)2.Prior experience in a research capacity3.Deep knowledge of various research and lab assessment methodologies (specifically RITE)4.Working knowledge of design principles and methodologies5.Experience working with Interaction Designers6.Technology background7.Excellent interpersonal, communication, negotiation and leadership skillsIn our opinion, these are things that would definitely jump out in a resume and might form the core of someones qualifications for a job in this area.Of course theyrenot theonly things that are important though - so whatwould you add?- Angela如何进行可用性评估可用性评估，它的主要目的是为了找出可用性方面的问题，即用户能否很好地使用产品的各项功能。用户研究是制作原型和可用性评估的基础和前提，只有通过用户研究，设计人员才知道用户对界面的特定需求和使用特点，才能在这些知识的基础上制作原型，同时在对原型评估时，也必须根据用户自身及使用时的特点来对原型的好坏进行判断。根据在评估过程中是否有用户参与，可用性评估可以分为两大类。第一类是没有用户参与的评估方法，第二类是用户参与进行评估的方法，为什么会存在没有用户参与的方法呢？以下是两个主要原因：第一， 用户以及用户的时间都不是免费的、何时想要何时就能得到的一种资源，大部分用户都有自己的工作要做，他们只能抽出很少的时间来帮助你进行可用性评估。因此，即使是不考虑经济方面的成本，有些时候你仍可能无法找到用户来进行评估。第二， 有时请用户进行评估也并不总是能发现所有问题的，这仍是因为可供测试的用户数量有限，只有很少功能能被评估工作覆盖到。另一方面，用户的熟练程度也决定了 只有特定的问题会被发现。例如，新手用户进行评估很可能发现产品在易学习性方面的问题，但却不容易发现使用效率方面的问题，而熟练用户的情况则相反，如果 由设计人员进行评估，就可以系统地对所有的功能在可用性的所有方面进行评估。下面我们首先来讨论两种不需要用户参与评估的方法：认知走查和启发式评估认知走查(Cognitive Walkthrough)认知走查法试图想象出人们在第一次使用某个产品时的想法以及所采取的动作，它的大致流程是这样的，你已经有一个原型或对于界面的详细描述，或者就是 一个真正的产品。同时，你知道可能的用户是谁。此时，你可以选择产品所能支持的某个功能来进行评估。评估的具体过程就是把用户在完成这个功能时所做的所有 动作进述成一个令人可以信服的故事。为了使得这个故事可信，针对于用户所做的每一个动作，你必须要能够证明，根据用户的知识水平以及界面上的各种信息提示 及反馈，用户做出该动作是合情合理的。认知走查前的准备工作需要有一个可供评估的原型或真正产品，或者对于界面使用方法的详细描述，产品的使用手册也可以，这些描述或使用手册越详细越好，因为只有这样才能发现更多的问题。需要一个对于要完成的任务的描述，这个任务最好是用户在使用产品时的典型任务。需要有一个用户在完成上述任务时所执行的一系列动作的完整而详细的描述。有关用户自身生理、心理特点以及他们的知识和经验的描述。怎样在认知鼠查中发现可用性问题在上述准备工作的基础上，认知走查的核心部分就是对假定的用户所采取的每一个动作进行质疑，看看它的发生是否合乎情理，我们可以从以下四个方面加以考虑：第一、 用户能否想到去做某个动作。第二、 用户能否找到执行某个动作的控件在哪里。第三、 用户能否看出操作控件可以产生他们想要的效果。第四、 在用户执行操作后，界面是否提供了适当的反馈，用户能否较好地理解这些反馈来有效地指导的后续的操作。认知走查的局限性认知走查只适合于评估一个产品的易学习性，因为它考虑的是用户在第一次使用界面时的想法和行为，但不太容易发现使用效率方面的可用性问题。启发式评估启发式评估又称经验性评估，它是指根据一些界面设计方面的经验性准则来对界面进行系统的审查，以便来发现其中的可用性问题。在这方面最有名的的是由Jakob Nielsen和他的同事提出的10个经验性准则，它们是： 简洁而自然的对话。用户界面应当尽可能简洁，屏幕上的功能和信息越多，用户就需要学习越多的东西，出错的可能性就会增加，从中查找信息就会更困难。 使用用户的语言。用户界面中所出现的用语应当使用用户所熟悉的问题领域的语言，而不是计算要领域的术语。 将用户的记忆负担降低到最小。人的短时记忆和长时记忆能力都是有限的，应当尽可能利用计算机的计算和存储能力所弥补人在这方面的局限性，以使得人可以更轻松地使用计算机。 一致性。同样的命令或操作应当产生同样的结果。 反馈。系统应当向用户表示出，用户已经执行了什么操作或发出了什么命令，这些命令或操作的执行情况如何，距离用户的目标还多远。 清楚地标识退出。应当让用户始终对系统掌握完全的控制权。 使用灵活性及效率。熟练用户应当可快速地完成各种操作，尤其是频繁使用的操作。 良好的出错信息。出错信息不仅仅要明确地、以用户可以理解的方式指出错误的原因，还应当指出可能的解决方法。 采取各种措施来防止错误的发生。 帮助和文档。向用户提供易于检索、便于用户逐步学习的帮助信息。可用性测试可用性测试是指通过让实际用户来使用产品或原型方法来发现界面设计中的可用性问题。下面介绍进行可用性测试时主要包括的活动。1.招募测试用户在招募测试用户进的主要原则是，这些用户要能尽可能地代表将来真实的用户，比如说，如果系统的主要用户将会是新手用户，那么就应当选择一些以有对于系统不熟悉的用户。在实际操作中，也可以委托一些专门负责可用性测试的咨询公司来负责招募测试用户。2选择测试地点和记录方法3测试前的准备测试人员在测试前需要准备好一些要求用户完成的任务，这些任务应当是一些实际使用中的典型任务。另外，在开始之前，可用性的主持人应当明确地告诉用户，这 个测试目的是发现软件产品中的问题，而不是要测试用户是否有能力来很好地使用软件。清楚地说明这一点将有助于减轻用户的压力，使得他们能使在真实环境一样 来使用软件。4.测试过程可用性测试的基本过程就是用户通过操作软件来完成所要求的任务，同时观察人员在一旁观察用户操作的全过程，并把发现的问题记录下来。可用性测试的主持人应 当要求用户在操作软件的过程中采用一种名为“发声思唯”的方法，即在使用软件的同时说出自己的思维过程，比如为了完成某个任务，用户想先做什么，后做什 么，为什么要做某个动作。等等。在测试的过程中还需要注意，除非用户完全无法继续下去，否则不要给用户任何提示或暗示，任务这样的信息都有可能帮助用户找到正确的操作方法，从而无法暴露出原本可以发现的问题。5.测试结束时的活动在测试结束时，主持人或观察人员可以询问用户对于产品整体的主观看法或感觉。另外，如果用户在测试的过程中没有完全把思维的过程说出来，此时也可以询问用户他们当时的想法和思维，询问他们为什么做出那些操作。6.事后的研究和分析在可用性测试结束之后，所有的观察人员把各自的记录进行汇总并加以分析，产生出一分产品的可用性问题列表，并对可个可用性问题的严重程度进行分级，以使设计人员根据项目进度来优先选择严重的问题首先处理。PPT:如何开展可用性评估和研究(2006-09-17 22:46:27)转载前一段时间，由于工作上有变动，一直没有时间来写点东西。新单位的事情也比较多，想着能否拿点东西出来与大家共享一下，正好有一个给原来单位做报告时做的一个PPT，不管是否成熟，是否有条理，贴出来，供大家交流吧。如何进行可用性评估和研究报告框架什么是可用性评估？理解可用性为什么要做评估？探明评估目标评估哪些方面？确定评估指标选择哪类评估？选择评估方法评估前需要哪些准备？评估准备如何实施评估？评估实施如何撰写评估报告？评估报告什么是可用性评估？理解可用性可用性定义（ISO9241-11）：产品在特定环境下特定用户用于特定用途时所具有的效果、效率和用户主观满意度。为什么要做评估？探明评估目标研究导向：证实与证伪产品导向：发现问题，改善设计为什么要做评估？研究导向我发明了一个全新的技术，我想知道用户对这个创新技术的反应，以确认它是否有价值。验证性评估我发明了一个可替代现有技术的新技术，我想知道它是否比现有技术更有价值（对比）。对比性评估为什么要做评估？产品导向（1）战略上的目标1 使我的产品所提供的功能用户真正“想要”和“想用”，建立起清晰的产品定位。2 使我的产品在同类产品中更具核心竞争力。功能是产品的核心价值，当同类竞争产品之间的功能相差不大时，可用性和用户体验就升格为核心价值。Idea：可用性/用户体验是产品竞争的最后一座“堡垒”。3 使我的用户满意我的产品信赖我的产品的品牌成为我的产品的“骨灰级粉丝”为什么要做评估？产品导向（2）具体目标（1）建立可用性标准对当前版本进行可用性评估，为下一版本的产品提供可用性标准。（2）控制开发成本在开发周期的早期就能够发现设计上的问题（原型测试） VS Coding的成本非常高（3）降低开发风险等待产品发布后再获得用户的反馈，风险太高（4）降低技术支持和维护成本用户容易学习和使用产品，自然就很少打技术支持的“热线电话”，也无需太多的时间去维护产品（5）提高销售额好用的产品自然就会有良好的口碑，良好的口碑成就更高的销售额要评估哪些方面？确定评估指标选择哪类评估？ 选择评估方法1、用户参与（1）对比实验：用户数：多组设计的可用性比较；用户数：每组12人以上。（2）可用性观察测试：通过观察找出界面可用性问题，提出设计改进建议；可以是正式的实验室观察，也可以是非正式的观察。用户数：2人以上，一般612人。2、用户不参与（1）启发式评估可用性专业人员根据已有的可用性原则，对界面进行逐一评估（2）认知走查法可用性专业人员将自己“扮演”成为用户，通过一定的任务对界面进行检查评估。（3）GOMS模型预测（4）击键层模型预测选择哪类评估？ 选择评估方法评估方法的选择，取决于：1 我的评估目标如前面所述研究导向：验证我的设计Idea是否有利于用户（观察测试等）；比较两个或多个设计Ideas（对比实验）产品导向：发现问题，改善设计（观察测试、启发式评估、认知走查等） ；证实我的产品达到可用性目标（观察测试、对比实验等）2 我希望获取的数据的类型量化数据（对比实验）质性数据（其他评估方法）3 我关于评估和与用户打交道的价值观念 用户参与的评估评估前需要哪些准备？评估准备一、人员组成二、人员培训三、观察者安排四、测试计划五、测试任务设计六、招募测试用户评估前需要哪些准备？评估准备用户参与的评估一、人员组成一个可用性评估组中至少要有两位实施人员. 虽然一个人也可能完成可用性测试，但他/她必须是经过严格训练的可用性专家，必须能够即时处理所有的任务。他/她需要：1 照看好测试测试用户 (使所有测试用户都感觉良好并完成得比较顺利) 观察测试中发生的事情，安排管理测试场景 (作为测试的促进者)，并提出问题(如果需要) 2 观察