Spring 2008 - Midterm
Update: Comments and solutions are given below.
This take-home midterm exam asks you to apply some of the topics that we've discussed this far this semester to a hypothetical design challenge. You are to answer the questions below as clearly and succinctly as possible: in other words, keep it short and sweet. Quantity is not a substitute for quality or clarity: if I can't tell where you're going, I might not read to far and too hard to figure out where you're going.
For some of the questions, you may be tempted to provide pictures or diagrams. If you feel that this would help clarify matters, please do so, but you should not feel compelled to make production-quality graphics: just get your pont across.
Your grades will be dependent upon the appropriate application of material from the course to the questions below. Most of the questions will involve process vs. product: thus, if I ask you to describe a potential interface design, I will be interested in hearing about how you arrived at certain aspects of the design. I will be less interested in seeing beautiful pictures.
Some ground rules:
- This is an "open book, closed internet" test - you can use your textbook and lecture slides, but all other resources are off-limits.
- The work you hand in must be your own. I reserve the right to use search engines to verify originality of content. Plagiarism or cheating in any form will be grounds for a failing grade in the course.
- Your assignment is due via email by 5PM EDT, April 1.
- All assignments must be typed or otherwise prepared on the computer.
- Assignments that are late will be docked 15% per day.
- You may email me with questions regarding clarification of any aspects of the assignment. Any answers will be shared with the rest of the class.
- I may revise the content of this exam as need be. I will try to clarify changes. Please check this page frequently to make sure that you have accounted for any clarifications. This version - Version 1 - was posted on March 23 at 9pm.
Take as much space and time as you need, but no more. If your answers take 2 pages, you've likely not done enough work. If you're tempted to write 40 pages of prose, you've probably done too much.
The content of the exam is given below.
You're working as a usability specialist at a software company. You've just returned to work after a long weekend taken in celebration of your completion of a major project, when your manager calls yo in to talk about a new product that she's interested in developing. You know that you're boss can be more than a bit wacky when it comes to new ideas - what could possibly be next?
Your boss explains that her latest brainstorm involves solving a problem that she's had at home. Her teenage children use the computer all of the time, downloading and installing software, music, videos, and who knows what else. She's happy to have them use the computer, but she doesn't want them to install software that could expose them to viruses, spyware, or worse, and she certainly doesn't want file-sharing software to expose her checkbook files to the whole world. At the same time, she doesn't want to spy on her children's activities: this would betray their trust.
Given this challenge, your boss has decided to investigate a non-intrusive safeguarding system. This would let her understand how the computer is being used, the potential security threats are, how which files are being shared, and what steps should be taken to minimize or reduce risks.
At the same time, her children would use this tool to provide information about the sorts of activities they are participating in - without providing any private detail. This voluntary disclosure would let her teenagers show their ability to act responsibly, thus earning her trust.
The system must be clear and usable, even for parents (and teenagers) who don't understand much, if anything at all, about computing and networking, and certainly not about computer security.
Your assignment is to help with the HCI aspects of this project beginning with the conceptualization and requirements-gathering phases, and continuing through evaluation.
- Requirements Gathering (35%): Your boss' description is merely a sketch of a possible product.How would you go about turning this sketch into a coherent set of requirements that could form the basis of a product? Who would you talk to? Which techniques would you use to understand user needs and requirements? What sort of participants would you want? Keep in mind that both parent s and teenagers will be using this tool, but in different ways.
- Functionality & Design (30%): After you've been
working for a few weeks on data gathering for requirements, your boss
comes into your office saying " Change of plans: we've just found
out that our competitor is working on this same idea. We don't have
time to talk to potential users to gather requirements. Based on the
conversations we've already had, we've got to design a tool and build
it immediately so we can beat our competitor to market".
Fortunately, the tool that must be built immediately need not be as general as the original plan. Specifically, you are to focus on anti-virus functionality.
What are the major components, concepts, and operations that this tool should support? What are the models/metaphors that you might use to help users achieve their desired goals? How would you help users understand the complexities of potential viruses, their sources, and their impacts.
Note that since you're pressed for time, you answer will be necessarily preliminary and incomplete. Please don't spend a great deal of time describing every little detail. Instead, describe the major concepts that the interface will need to cover.
When answering these questions, be sure to keep in mind the need to support both teenagers and parent, some of whom may be completely unfamiliar with anything than the most basic concepts in computer use.
- Evaluation (35%): Your crackerjack coding team has come
up with a prototype. How would you evaluate the success of the tool?
Keep in mind that this means success both for teenagers and
parents.
Each set of users must feel confident that they have made appropriate use of the tool without harming either privacy or trust. For teenagers this would mean that they told parents what they were willing to tell them, without giving away any information that they might have considered sensitive. For parents, they need enough information to make sure that they are able to understand what is being done with the computer, and to be able to make appropriate decisions regarding security and privacy. Accuracy of perceptions is important in both cases: if users think that they have appropriate information, but they are, in fact, incorrect,this should be considered at least a partial failure of the system.
What sort of techniques, tests, questionnaires, or other techniques and materials would you use to evaluate the usability of this interface?
Comments and Possible Solutions
This midterm exam was challenging and ambiguous by design. Your graded solutions should have some indications of shortcomings that I found. The biggest problem that I found involved lack of sufficient detail - if you're going to say that you'll run a certain kind of test, or to do some interviews, provide details. Which questions should you ask? Which tasks, etc.?
People also lost points for inappropriate inclusion of irrelevant concepts, unclear text, overly verbose answers, and other stylistic items.
Grading is (as always) somewhat subjective. I tend to reserve full-credit for answers that are really outstanding. The average score on this exam was just under 89.
The following answers give an example of the spirit of what I was looking for. These answers are not necessarily definitive - you may have had some good ideas that I have not included here. That said, these answers may have some items and/or text are taken from some of the answers that were submitted. If you recognize some pieces of some of your answers, consider this as evidence of your good work.
- Requirements Gathering:
- Market Research - identify existing products.
- Contact high-schools to identify parents and children
- Parent questionnaires, with questions including: Do you have computer security concerns regarding your teenager's computer use? If so, please explain your concerns. What do you do to keep your teenager's computer secure? What does your teenager do to keep his/her computer secure? What information about your teenager's computer use would make you feel better about his/her usage with regards to security?
- Sketches: Parents and teens might be asked to sketch out their own understanding of concerns and designs of ideal interfaces.
- Parent Discussions/Focus Groups: to help parents learn more and then express concerns.Subsequent questionnaire to clarify changes in needs and preferences.
- Teen peer discussions about computers and security, including questions regarding Computer activities in which teenagers participate; What they currently tell their parents about their computer use; Why they choose to share certain details about their computer use; and Why other details are not shared. Subsequent questionnaires with similar questions to gather additional detail.
- Teens computer usage might be observed to identify potentially problematic behahvior.
- Use of a limited-functionality prototype to observe interaction and see problems and reactions
- Functionality & Design:
- Components: parent interface, teen interface, activity logs, information/education screens and feedbacks.
- Concepts: virus, threat level, system scan, blocking access,
- Operations: Block a site, allow access to a site, report an activity, download a file, provide reports, justify a decision. alert users of possible harm, identify suspicious or potentially harmful actions, grant/revoke downloading and file-sharing privileges.
- Models: Threat of varying risk levels (none-high)
- Metaphors: traffic light for risk level
The user interface must present virus warnings that both parents and teenagers can understand. Technical terms, computer jargon and error codes should be avoided in the warning messages. Warnings should be presented in natural language to promote user understanding. Warnings should be informative so that teenagers and parents can make wise decisions about downloads. The warnings may include examples of the damage viruses can cause.
The warnings should be presented such that they grab the user's attention and for serious viruses should not be easily closed or dismissed. The system should not allow teenagers to download potentially harmful files. Teenagers should be given the option to add the files to a list for later review/approval by their parents. Files should not be automatically added to this list to respect the teenage users' privacy.
The system should provide reports to parents that show the types of security threats that have been presented to the computer without providing details regarding the source of the threat, if known. The system should allow parents to review the list of blocked downloads as requested by the teenage users. Parents are able to allow the download, or permanently block the download so that it is not presented on the request list again.
As teenage users demonstrate more responsible and safe computer usage, parents should be able to grant more file sharing and downloading privileges to the teenage users. The system should also allow parents to revoke the extra privileges when necessary.
The use of recognizable icons and/or colors may help users identify safe or harmful downloads or activity. For example a green traffic light may indicate safe computer use. A yellow traffic light may indicate some questionable use. The user may be able to click the yellow light for additional details about suspicious activity. A red traffic light may indicate that harmful activity has been identified and blocked. As with the yellow light, the user is able to click the red light for additional details.
As mentioned above, warnings provided to both parents and teenagers may include examples of the damage viruses and potentially harmful downloads can cause. These warning messages will teach teenagers and parents more about computer security and the risks associated with exposing the computer to such vulnerabilities. As the parents review the download/share lists, the system should provide suggestions based on the risks associated with the item under review. These suggestions should help parents decide whether or not to permit the download to proceed.
- Evaluation:
- Expert usability evaluations: Usability experts evaluate interface for adherence to eight golden rules or other interface guidelines.
- Installation and Configuration Tests: Parents will be asked to install and configure the software. Usability problems and difficulties will be noted
- Usability activity in lab. Particular attention will be given to
both correct completion of tasks and building understanding of
underlying issues.
Parent/child teams will be invited to the lab to try the software. Parents will be asked to configure the software to meet their needs and preferences. Teens will then be asked to browse, download/share files, and engage in other potentially risky behavior. Tasks performed by teens will involve a combination of both their own tasks and tasks presented by the experimenters. Parents will then examine the activity logs and take other actions in response to those actions. Data regarding task completion and usabilty problems will be collected. Post-test questionnaires with teens and parents will be used to address usability issues.
The teen discussions and questionnaires should address topics such as: What have the teens learned about computer security? Allow the teens to review the reports generated by the tool. With respect to their privacy, ask if the teens feel the reports reveal too much information? Too little? Do the teens feel the reports provided enough information to make informed computer security decisions? If not, what additional information are they willing to provide?
Parents complete questionnaires and engage in another parent group discussion. Parent discussions and questionnaires should address topics including: Does the parents' interpretation of the system reports reflect the teenagers' true online behavior? Do the parents feel that they have more control over the security of the computer? Did they feel the reports provided enough information to make informed computer security decisions? If not, what additional information would help?
- Field tests: A set of families will use the tool in their home for a month. Weekly interviews with parents and teens - together and separately - will address concerns regaarding usability, identify problems, and otherwise investigate any difficulties or shortcomings that might exist.
- Ongoing email, focus groups, and bulletin boards can provide ongoing feedback post product release.