When is Virtual Reality real enough? Using VR in spatial cognition research

Abstract

Although the use of Virtual Reality in spa­tial cog­ni­tion research is becom­ing more and more wide­spread, there is little guid­ance on what kind of VR setups and exper­i­men­tal par­a­digms are most suit­able for spe­cif­ic sit­u­a­tions and research ques­tions. This might result in a number of issues, such as sub­op­ti­mal use of equip­ment and insuf­fi­cient trans­fer­abil­i­ty of results to the real world.

The aim of this half-day work­shop is to dis­cuss and raise aware­ness of these issues, and take first steps in tack­ling them. The work­shop will start with a brief intro­duc­tion of the cur­rent state of VR in spa­tial cog­ni­tion research, fol­lowed by indi­vid­ual talks on spe­cif­ic research projects in this area. The work­shop will con­clude with a gen­er­al panel discussion.

Attendees will leave with an improved under­stand­ing of the advan­tages and chal­lenges of using var­i­ous VR setups to address dif­fer­ent research ques­tions. No prior VR knowl­edge or expe­ri­ence is required for attendees.

Important dates:

Workshop date: August 31, 2012, from 8:30–12:00, as part of the Spatial Cognition 2012 conference

Organizers:

Bernhard Riecke (Simon Fraser University)

Introductory short pre­sen­ta­tion: “When is Virtual Reality Real Enough? Embodied vs. Disembodied Spatial Cognition in VR”

 

 

Klaus Gramann (Technical University Berlin)

Introductory short pre­sen­ta­tion: “Disembodiment in VR Spatial Cognition Research: a Neuroscience Perspective”

 

 

Presenters:

Frank Steinicke (University of Würzburg)

“Making Small Spaces Feel Large: Self-Motion Perception, Redirection and Illusions”

 

 

 

Maria Kozhevnikov (Harvard Medical School, University of Singapore)

“Understanding immer­siv­i­ty from cog­ni­tive neru­o­science perspective”

 

 

 

 

Christian Doeller (Radboud University Nijmegen)

“Virtual real­i­ty in neu­ro­science: bridg­ing between cells, sys­tems and behav­iour”

 

 

 

Nicholas Giudice (University of Maine)

“Moving beyond the vision-only bot­tle­neck: Using mul­ti­modal infor­ma­tion in vir­tu­al real­i­ty research”

 

 

 

 

---

Tentative Timeline

8:30 - 10:00, Part 1: Presentations

8:30: Bernhard Riecke (Simon Fraser University)
Introductory short pre­sen­ta­tion: “When is Virtual Reality Real Enough? Embodied vs. Disembodied Spatial Cognition in VR”

Propose gen­er­al guidelines:

• as we have lim­it­ed time we’ll need to reduce ques­tions during talks — instead, please jot down your thoughts&questions you’d like to dis­cuss in during second part of the workshop
• • we’ll hand out post-it notes, and invite you to put them on the wall before/during coffee break, so we can start gath­er­ing, cat­e­go­riz­ing & pri­or­i­tiz­ing poten­tial dis­cus­sion questions
• use coffee break to start brain­storm­ing & gather around dis­cus­sion topics etc.

8:45: Klaus Gramann (Technical University Berlin)

Introductory short pre­sen­ta­tion: “Disembodiment in VR Spatial Cognition Research: a Neuroscience Perspective”

8:55 Frank Steinicke (University of Würzburg)
“Making Small Spaces Feel Large: Self-Motion Perception, Redirection and Illusions”

9:10 Maria Kozhevnikov (Harvard Medical School, University of Singapore)
“Understanding immer­siv­ity from cog­ni­tive neru­o­science perspective”

9:25 Christian Doeller (Radboud University Nijmegen)
“Virtual real­ity in neu­ro­science: bridg­ing between cells, sys­tems and behav­iour”

9:40 Nicholas Giudice (University of Maine)
“Moving beyond the vision-only bot­tle­neck: Using mul­ti­modal infor­ma­tion in vir­tual real­ity research“

9:55: wrap-up & out­look of ses­sion 2 (Bernhard)

10:00-10:30 coffee break

Suggestion: use coffee break to put up your post-it notes with dis­cus­sion ques­tions & start brain­storm­ing & gath­er­ing around dis­cus­sion topics…

10:30-12:00, Part 2: Brainstorming, Breakout group discussion & Final Plenum Discussion

10:30 Overview of pos­si­ble dis­cus­sion ques­tions based on your post-it notes

10:35 Brainstorm more ques­tions, gather (e.g., Bernhard using MindMap soft­ware?), decide on most inter­est­ing dis­cus­sion topics (open questions).

10:40 break out into groups of about 5 people with one main focus question

• decide on note­tak­er (should have laptop or use flipchart) to sum­ma­rize essence of discussion. 
• decide on group reporter (who will briefly present essence of discussion/insights to plenum: New insights? Suggestions? Tips&Tricks? New ques­tions? What stood out for you or was surprising/most exciting/stimulating/useful? How could we use the insights in the future? Ideas for future projects/collaborations?)
• make sure all team mem­bers contribute
• if inter­net access is avail­able: write direct­ly into shared google doc, else use memory stick or pass net­work cable around…

11:30 break-out groups present results in plenum

11:50–12:00 Conclusions, out­look, poten­tial next steps

---

---

General Structure

Part 1: Introduction and Presentations

This half-day work­shop will be divid­ed into two parts. First, Bernhard and Klaus will briefly intro­duce the over­all topic, pro­vide a bit of a back­ground on embod­ied vs. dis­em­bod­ies spa­tial cog­ni­tion in VR, and con­clude by rais­ing some ques­tions and cur­rent chal­lenges, thus fram­ing the upcom­ing pre­sen­ta­tions and dis­cus­sions. This is fol­lowed by 4 experts (see above) pre­sent­ing and dis­cussing their research and expe­ri­ence with using VR setups in the con­text of spa­tial cog­ni­tion research, as indi­cat­ed above. Further dis­cus­sion topics might include:

• The role of phys­i­cal motion cues in spa­tial orientation
• Sensory con­flicts and simulator/motion sickness
• Different dis­play para­me­ters and their effects
• The use of self-motion illu­sions to “fake” phys­i­cal motion
• Different strate­gies for and under­ly­ing neural rep­re­sen­ta­tions of spa­tial ori­en­ta­tion in VR

Part 2: Interactive Brainstorming and Discussion Workshop

The second half of the ses­sion will be more inter­ac­tive and work­shop-style: After a short brain­storm­ing ses­sion, we will col­lect and orga­nize the most inter­est­ing and rel­e­vant dis­cus­sion topics, and will spend the remain­ing time dis­cussing these in small­er groups (depend­ing on the over­all work­shop size). Topics might include 

• the cur­rent state and future of VR in spa­tial cog­ni­tion research
• expe­ri­ences, strate­gies and chal­lenges in using VR as a research tool
• guide­lines for its opti­mal usage.

One or more note­tak­ers will direct­ly doc­u­ment these dis­cus­sions, and we will put the out­come of the work­shop online soon after­wards (at least for par­tic­i­pants). If there’s suf­fi­cient inter­est, energy, and excite­ment, we hope this might kick­start fur­ther dis­cus­sion and collaboration.

Organization and goal:

Virtual Reality can be a highly useful research plat­form in spa­tial cog­ni­tion. It pro­vides a unique oppor­tu­ni­ty to study human per­cep­tion and behav­ior in immer­sive, multi-modal and nat­u­ral­is­tic vir­tu­al envi­ron­ments under repro­ducible and con­trol­lable exper­i­men­tal conditions. 

For an exam­ple, imag­ine a researcher study­ing human nav­i­ga­tion in city envi­ron­ments. A typ­i­cal exper­i­ment in this field might mea­sure par­tic­i­pants’ ori­en­ta­tion and sense of direc­tion as they nav­i­gate in an actual city. This presents some chal­lenges. For one, the researcher must select an exist­ing envi­ron­ment to use. As the selec­tion pool is usu­al­ly lim­it­ed by geo­graph­i­cal loca­tion, the researcher might have to settle for an envi­ron­ment that is less suit­able than what she had in mind. Also, she must relin­quish con­trol of sev­er­al pos­si­ble con­founds that can affect the valid­i­ty of her study. These include prior knowl­edge, weath­er, vis­i­bil­i­ty, sound, ori­ent­ing cues like the direc­tion of sun­light, wind, or global land­marks, and other inter­fer­ence or con­founds from the environment.

Under the right cir­cum­stances VR pro­vides an ele­gant solu­tion to these prob­lems. Rather than set­tling for an exist­ing envi­ron­ment, a vir­tu­al city can be built accord­ing to exact spec­i­fi­ca­tions. Environments can even be adjust­ed and changed on-the-fly as needed. Confounding vari­ables can be sig­nif­i­cant­ly reduced, as more fac­tors can be direct­ly con­trolled and mea­sured. Finally, depend­ing on the sit­u­a­tion, a VR-based study can be safer and more cost-effec­tive than a cor­re­spond­ing study per­formed in the real world, such as research into spa­tial cog­ni­tion during flight or poten­tial­ly dan­ger­ous sit­u­a­tions like evac­u­a­tion scenarios.

However, the use­ful­ness and util­i­ty of VR in research crit­i­cal­ly depends on the spe­cif­ic cir­cum­stances. That is, it can be ben­e­fi­cial or detri­men­tal to study valid­i­ty and real-world trans­fer­abil­i­ty, depend­ing on the task, envi­ron­ment or research focus. Correctly eval­u­at­ing under what con­di­tions which kind of VR setup might be most suit­able can be chal­leng­ing, espe­cial­ly as tech­nol­o­gy advances. It is often simply assumed that a give VR setup is suit­able and pro­vides real-world trans­fer­abil­i­ty, simply because it is a costly and state-of-the-art setup. This is, how­ev­er, not nec­es­sar­i­ly the case, as will be dis­cussed in the workshop. 

Our goal with this work­shop is to raise aware­ness for these issues, share and dis­cuss rel­e­vant find­ings and expe­ri­ences, and thus enable researchers to make more informed deci­sions regard­ing the usage and util­i­ty of dif­fer­ent types of VR setups and inter­ac­tion par­a­digms in their research. Participants will leave with a better under­stand­ing of when they can use VR to their advan­tage and how they can go about imple­ment­ing it. In addi­tion, we hope to strength­en a net­work of researchers in this area that will extend beyond the time of the workshop.

Potential Participants:

This work­shop will be most ben­e­fi­cial to researchers who active­ly design and carry out stud­ies in spa­tial cog­ni­tion and cur­rent­ly use or plan on using VR tech­nol­o­gy to do so. This includes stud­ies of spa­tial nav­i­ga­tion, ori­en­ta­tion, memory, learn­ing, wayfind­ing, and other fields that rely on exper­i­men­tal paradigms.