When it comes to planning a user study various steps need to be executed, summarized in this subsection.
General Preparation for User Studies
- What is the goal of your study? So what are your overarching research questions?
- What are your particular hypotheses and why?
- Which experimental design is beneficial for your investigations: between-subjects or within-subjects design?
- Which VR setup shall be used? CAVE, HMD w/wo eye-tracking?
- Which tasks do participants have to fulfill and in which virtual environment?
- Is a familiarization phase required to get used to the hardware, the scene, the controls, and the task? (Highly recommended!)
- Which data needs to be logged by the study framework? (Advice: Log as much as possible; it is better not to use data in the end instead of missing data.)
- Which questions need to be asked to gain meaningful insight?
- Which tool to use for the questionnaires? We recommend SoSciSurvey as an online platform. See some documentation here. However, it is sometimes useful to do the questionnaires directly in VR - to do so, have a look here at out LikertScale Plugin
- How many participants do you need? Do they need to be experts in a certain field or do they need to be novice users?
- When do you want to conduct your study?
- How is the concrete study procedure?
- Do we have funding to pay participants? In this case contact [email protected] to setup the "Werksvertrag", you get the money transferred and also a specific Quittungsliste which every participant has to sign (twice).
Study Procedure
After you designed your study it is time to set it up. The following section will thereby focus on the supporting material, aka questionnaires, instruction texts, etc.
- Scheduling
- Block the equipment in the lab for the course of your study.
- Set up a Foodle or Doodle or something similar to schedule appointments with your participants.
- Send an [invitation mail](StudyProcedure/Invitation Mail) to potential participants, share it on social media, and via posters.
- Introductory Text
- Write a [text](StudyProcedure/Explaining Texts) informing participants about the general procedure of the study. Do not give away details that might bias the outcome. It is also totally fine if users do not know what to pay attention to.
- Informed Consent
- Prepare the [informed consent](StudyProcedure/Informed Content) which participants need to sign up front after reading the general introductory text. Keep all the signed copies of it for 10 years!
- Demographics
- Prepare a demographics questionnaire as the first questionnaire to be filled out prior to the actual study.
- Task Description
- If not mixed with the introductory text, [prepare a task description](StudyProcedure/Explaining Texts) explaining the interaction and navigation controls in detail as well as the actual user task. Use figures to underline your descriptions.
- Questionnaires
- Prepare questionnaire to be asked before, between, and after the various conditions, you test.
- Whenever possible use standardized questionnaires cited in the literature.
- Add your own precise questions focusing on your very technique/investigation object.
- Add free text fields to ask for reasons for a special vote as well as additional thoughts, recommendations, and ideas.
- Interview
- Decide whether you need a structured or semi-structured interview (probably in the end) to gain more insight or if the questionnaires are sufficient.
- Debriefing
- After finishing the study, you can and should tell participants what the focus of your study was. However, make sure that they are aware that this information should not be disclosed to people who might still want to participate in the study, to avoid any bias.
Study Evaluation
Some ideas on how to evaluate your gathered data and what tools to use can be found in the Statistics section