Due Thursday, March 31
Overview
For this project you will need to design, implement, and review an evaluation for your project.
Purpose
This user study will help to establish the validity of your project and its functionality. Product testing through user studies is a critical part of the iterative design process. This is the first time that potential users will officially be using your functional prototype, so construct your user study carefully. Your team should use this study to test interactions and functionality. Your findings will help you to design your final functional prototype.
Format
Your team will design one or more scenarios relevant to the use of your system. You are encouraged to present these scenarios to your mentors prior to conducting your user study in order to get extra feedback. Before you conduct your study, you will need to create a relevant consent form or modify the one used in your previous study.
Next, your team will conduct the actual user study based on the developed scenario(s). You must have at least 5 participants for this user study. Students on other teams are viable study participants, so consider helping each other out as needed. The formative components of your study should focus on user opinions or their ability to understand and use your interface(s). Decide which data collection techniques you will employ (surveys, interviews, etc.). The summative components of your study should serve as an overall assessment of your design. As this relates to quantitative data, your team will need to come up with both independent variables that you can control and dependent variables that you can measure. Develop a central hypothesis for this user study that you will attempt to prove.
Each team will turn in a 2-4 page writeup over this user study in the ACM SIGCHI Publications Format. The writeup should include the following main sections. Be sure to touch on each of the listed questions.
Introduction:
- Describe your project and the motivation for your project
Motivation:
- Describe the motivation for your evaluation.
Your motivation section should answer the following questions: What is the overall goal of a working system?
What are the characteristics of a good system? (On a per project basis) Your goal of the evaluation should be two find the answers to how well your project fulfills the overall goals and the characteristics of a good system.
Implementation:
- In this section describe the design of your evaluation. The design should be such that the evaluation will successfully answer the question as to how well the system succeeds in fulfilling the goal and how much it contains the characteristics of a working system. Characteristics should be measured both through qualitative (such as freeform questions) and quantitative (such as Likert scales) measures, and should be measured through subjective (such as perceived run time) as well as objective means (versus actual run time). Make sure to clearly state your hypothesis for this user study. Clearly describe any tasks did you have users conduct or simulate and how they relate to the goals of the system Clearly discuss your independent and dependent variables and how they relate to your system. Also turn in a copy of any surveys or questionnaires.
Fully describe the methods of your evaluation. Who were your participants (number of, gender, major, level of expertise with area of focus, etc.)? Include those facets of information that are most relevant. What data did you collect? How did you collect this data? For example, you may have employed the use of surveys, interviews, or focus groups. Make sure that your consent form reflects the data collection methods used!
Results:
- What does the data show? This is more in relation to your quantitative data. Make sure to include at least two charts/graph to convey meaningful summative data.
Analysis:
- What did you learn from this user study? Include at least 3 main points of discussion.
- How will what you learned impact your design? This is an application of your knowledge.
- What study would you conduct in the future if you had more time?
Grading Rubric
You will be graded on the quality of your results and on how well you followed the outline provided. An emphasis will be placed on how well you articulate your findings. You must include both formative and summative findings in order to receive the maximum amount of points for this assignment. Again, make sure to include the information as outlined above.