Category: Pedagogy

Collaborative Usability Study in an Undergraduate Information Design Class

When we teach information design, we need to include user-centered design and usability in our curriculum. We need to address the designer’s responsibility to ensure that the deliverable is audience appropriate and to observe how the audience is using that deliverable. The class assignment discussed in this article introduced my undergraduate students to usability testing, allowing them to serve a nonprofit client and concurrently to learn about user-centered design.

In “Information Design and Usability” (IDU), we study search engine optimization, navigation on a digital page, the inverted-pyramid organization, tags and keywords, and online presence and security in addition to the mechanics and style of the English language. We also have integrated into the curriculum an introduction to usability testing. At the end of the semester, students are not usability experts, but they are familiar with usability testing and methods after this assignment. These students learn usability testing because their field of study is animation and many of them graduate and go on to design, create, and test virtual games and online products. However, this assignment would be useful to a range of technical communication students, such as Web designers, graphic designers, documentation writers, and others concerned with user experience.

Planning the Assignment

Last semester, I taught two sections of IDU and, upon identifying a client, I assigned my students to conduct usability tests on the client’s Web site. I divided students into teams of four, and each team chose one interviewer, two note takers, and one official timekeeper (the tasks of the note takers and timekeepers on the teams overlapped).

I requested and obtained IRB approval for the study, ensuring the board that all participants (testers and subjects) would remain anonymous outside the classroom. My university requires minimal review for any research that involves any human subjects: to ensure that the research subjects’ rights are protected and the research methods are ethical and per university standards. Therefore, because the study was not only an in-class activity but also a study that we formalized and reported, I obtained IRB approval. (I recommend this step so you can ensure and report that you conducted the tests ethically and so you can release the data from your students’ tests to your client without any concern of liability.)

I teach two sections of the course, so I allowed each section to create and conduct its own test. Thus, section one had four teams of students providing test results, and section two had five teams providing their test results. This allowed the students to work within their class and collaborate face-to-face. This also allowed me to provide the client with results from TWO tests—one from each section.

Executing the Assignment

The following timeline shows how we proceeded through the four classes for the study.

Prior to assignment—Introduce students to the qualitative and quantitative methods for usability testing and instruct them to read several articles including some from the Nielsen group. (You can find a list of articles at the end of this description.)

Class 1—Introduce the Web site to the students and record all of their responses. Provide a list of tasks to the teams—e.g., registering for the site, finding certain links, posting—and document the students’ initial responses as they complete the tasks. (Student feedback was valuable when I compared their responses to their test findings. Therefore, I considered this step as a way to triangulate findings—students’ initial responses, usability tests, and post-test surveys.) At the end of the introduction (10–15 minutes), ask students to suggest tasks that they could test in a usability study.

Class 2—Provide students with a draft of their task list and have them revise the list after they work through the tasks on the Web site. As a class, draft a script for the usability test. (We had already discussed how to write instructions, so the students were prepared to script instructions for the usability test. As we drafted, we discussed how to script an introduction, how to obtain the subject’s consent to participate, how to encourage subjects to “think aloud,” and how to encourage subjects if they became frustrated with the process—a skill that 4 of the 9 teams needed during the actual usability tests).

Class 3—Provide students with the script and have them work through the test 3–4 times to ensure that they are comfortable and prepared. Practice timing, note taking, and speaking to the subject. Have teams practice in front of each other and let students suggest improvements. Finalize the script. (I also had a student who decided to create a time log, a .doc that he shared with his classmates.)

Class 4—Prepare for the test. Distribute a copy of the script, the time log (if appropriate), and the follow-up survey to each team. (I took on the responsibility to create the follow-up survey so the students could focus on the actual usability test.) Conduct the usability tests.

Students designed the test to take approximately 15 minutes; however, 2 teams had technical difficulties with their computers in the lab, and 2 teams had students who did not rush through the tasks, so some of our times for tasks were skewed. We allowed 30 minutes of class for the usability tests, so all teams were able to complete their data collection before class ended.

Reporting Findings

After the students conducted their tests, they had out-of-class and in-class time to compile their findings and to write a team report of their findings. I did not require a standard format for the report other than to instruct students to provide me with all notes from the actual test (to verify and clarify information) and to provide times in minutes and seconds. I was aware of who was on each team, but the students were reminded not to include their names on their team reports per our IRB approval.

While the students wrote their team reports, I gathered the post-test surveys and statistically analyzed that data. In addition, I began to write a compiled report for our client.

Attached to this article, you can find a copy of one of the scripts that the students’ created and the post-test survey that subjects took after the test.

Experiencing Complications and Successes

The greatest complication we had in completing this assignment was student attendance and participation. One of the classes had several students who were consistently late, and those students missed some of the in-class instruction. We also had one student who did not show up for the usability testing—a complication that frustrated the student’s team.

Wanting to allow the students to evaluate each other for this team project, I provided students with a confidential team evaluation in which they could grade their team members for the assignment. Students also evaluated their own performance. I then averaged those scores and awarded those points to each student as their team evaluation grade. Only 2 of 36 students complained—1 of the 2 being the student who did not attend class for the testing.

The students obtained a variety of skills through this exercise. They learned:

  • to consider the client’s needs;
  • to evaluate the rhetorical situation for a variety of documents related to the usability test, including the tested Web site, the script for the study subjects, and the final report for the client;
  • to collaborate with team members to plan, conduct, and report the usability test;
  • to conduct a well-designed usability test;
  • to consider human subjects’ rights and their responsibility as researchers to study ethically;
  • to communicate professionally with study subjects; and
  • to design a deliverable for the client.

Overall, the assignment was a great success, and I will seek to assign a similar collaborative project next semester.

Suggested Readings for Students

Jerz, D. G. (2011, April 11). Usability testing: Top 8 tips for designing usability tests. Jerz’s Literacy Weblog. Retrieved fromhttp://jerz.setonhill.edu/writing/technical-writing/usability-tips

Nielsen, J. (1997, October 1). How users read the Web. Neilson Norman Group. Retrieved from http://www.nngroup.com/articles/how-users-read-on-the-web

Nielsen, J. (2001, August 5). First rule of usability? Don’t listen to users. Nielsen Normal Group. Retrieved fromhttp://www.nngroup.com/articles/first-rule-of-usability-dont-listen-to-u…

Nielsen, J. (2012, January 16). Thinking aloud: The #1 usability tool. Nielsen Normal Group. Retrieved fromhttp://www.nngroup.com/articles/thinking-aloud-the-1-usability-tool

Rohrer, C. (2008, October 6). When to use which user experience research method. Nielsen Normal Group. Retrieved fromhttp://www.nngroup.com/articles/which-ux-research-methods

Author

Carie S. Lambert, PhD
Clinical Assistant Professor of Communication
Arts & Humanities
The University of Texas at Dallas

 

Advertisements

Infographics in Technical Writing and Communication

By Lee Brasseur  Department of English, Illinois State University

Infographics seem to be the “in thing” in information design these days, and more technical writing instructors are beginning to include them as assignments in their classes.  I first became interested in infographics when I started to see how the genre of graphs and charts had shifted from simplistic representations to ones embellished with graphics (as those originally shown in USA Today). I then saw this trend move to even more complex visual and verbal presentations of quantitative and qualitative information in newspapers, websites, and books.

Infographics were a new kind of genre, offering a more complex, more encapsulated way of understanding quantitative information for a new generation.  In this article, I will discuss the genre of infographics, its relationship to both cognitive and contextual theories, and using them as a technical writing assignment. Continue reading “Infographics in Technical Writing and Communication”

Groups, Groups, and More Groups: Using Groups Throughout the Semester (Part two of a two-part series)

Meg Morgan, Department of English

UNC Charlotte

Last March Meg reported on a semester-long group project experiment in her beginning technical writing classes. In Part 2, she finishes her tale with a reflection on the experience after the term.

It is now the end of summer, and I have almost fully recovered from my experiment with the “less work” collaborative model last spring.  Overall, I was pleased with some aspects of the course design.  Here is a brief summary of what happened and what I learned. Continue reading “Groups, Groups, and More Groups: Using Groups Throughout the Semester (Part two of a two-part series)”

Groups, Groups, and More Groups: Using Groups Throughout the Semester (Part one of two)

Meg Morgan, Department of English

UNC Charlotte

I have been working with students in collaborative writing groups since the mid-1980s, when as a graduate student at Purdue, I worked with four fellow graduate student colleagues to write a research article on − collaboration.  With that experience and many others over the next 35 years or so, I thought I knew everything there was to know about using collaborative writing groups in a technical writing classroom.  Was I wrong! This semester I’m learning that you are never too old or too experienced to learn something new or to make some mistakes in the process. Continue reading “Groups, Groups, and More Groups: Using Groups Throughout the Semester (Part one of two)”

Second Language Students in Technical Writing Classrooms

Meg Morgan
UNC Charlotte

Spring 2010

According to the US Census Current Population Report, the numbers of Hispanic students attending US colleges and universities increased from 443,000 in 1980 to 2,131,000 in 2007.  In addition, in 1980, 286,000 international students enrolled in US colleges and universities; in 2008, over 624,000 enrolled.  These numbers indicate an increasing numbers of students whose first language is not English and suggest, if things stay the same, that second language students will continue to be a significant presence in our classrooms. Continue reading “Second Language Students in Technical Writing Classrooms”

Hazardous Rhetoric

Paul Dombrowski, University of Central Florida, Spring 2009

Rhetoric and ethics are related, Aristotle noted long ago. Our rhetorical choices reflect our values and our purposes. Especially in our activities to engage the public in complex technical issues, appropriate and effective rhetorical choices are vitally important.

In my junior-senior honors course on technical communication, we examine the rhetorical techniques employed in the website of an organization concerned about a particular controversial chemical. This examination ranges broadly to include the web site’s color choices, visuals, formats, and above all language choices and associated rhetorical implications. The results of our analysis are then applied to websites about other chemicals and environmental issues in order to understand how they work both rhetorically and ethically, yielding heightened critical sensitivity in the students. Continue reading “Hazardous Rhetoric”

Ethics in words: The conundrum of nuclear safety

Paul Dombrowski
University of Central Florida
Spring 2011

Nuclear energy is becoming an increasingly important component of the overall national energy picture. In a technical communication course for juniors and seniors, both majors and non-majors, we discuss the ethical challenges of representing nuclear energy realistically and fairly in discourse. We examine how everyday language can become confused, misused, and misunderstood when applied to technical and scientific information. We also discuss how this  difficulty is compounded by the emotional, political, and world-view dimensions of the discourse context.   Continue reading “Ethics in words: The conundrum of nuclear safety”

Exploring Online Teaching and Learning

Meg Morgan, ATTW Teaching Committee
UNC Charlotte
Fall 2009

I recently read an article in the August 11, 2009 electronic edition of Inside Higher Ed about online teaching written by Jonathan Kaplan, President of Walden University, an online university.  Kaplan summarizes a recent report published by the U.S. Department of Education “that looked at 12 years’ worth of education studies, and found that online learning has clear advantages over face-to-face instruction.”  In his article, Kaplan cites the report which stated: “students who took all or part of their class online performed better, on average, than those taking the same course through traditional face-to-face instruction.” Continue reading “Exploring Online Teaching and Learning”

Evaluating Textbooks: A Recommendation Report Assignment for Introductory Technical Communication Courses

Michael J. Albers
East Carolina University
Spring 2009

In many introductory technical communication classes, students may have trouble finding recommendation report topics which lend themselves to a reasonable assignment length. If left to themselves, students tend to come up with variation on the infamous “new parking garage” report that frequently does not fit within 5-7 pages because the problem is too large. With other topics, the nature of collecting and analyzing the information results in a highly artificial environment with students unable to collect enough information to report. As a result, they end up making stuff up. Continue reading “Evaluating Textbooks: A Recommendation Report Assignment for Introductory Technical Communication Courses”