Improving 3d visualisation: Near-correct focus cues in 3d displays

Lead Research Organisation: Bangor University
Department Name: Sch of Psychology

Abstract

Stereoscopic displays enable one to artificially create three-dimensional (3d) images by providing each eye with a slightly different view of a scene (this happens naturally in the real world because your eyes are separated laterally in your head). Such displays/referred to hereafter as 3d displays/are increasingly used in society, in a wide range of applications that includes operation of remote devices, medical imaging, scientific visualization, surgical training, design, virtual prototyping, and vision research. A useful 3d display should be comfortable and natural to use, and must create a faithful impression of the 3d structure of the object or scene being portrayed. Unfortunately, current 3d displays typically fail on both counts. They often induce significant discomfort and fatigue, and the resulting perception of depth appears to be flattened, compared to equivalent real scenes. Both these problems are caused in part, if not completely, by the fact that almost all existing 3d displays present each eye's image on a single surface, at a fixed viewing distance. This means that the eyes try to focus at the display surface, irrespective of the depth variations in the portrayed scene. This affects depth perception because the eye's focusing response is a signal to depth. It causes fatigue and discomfort because viewers must decouple the natural relationship between their focusing response and the eye movements they make to look at near and far objects: this process is effortful and often unpleasant. I am proposing a programme of experiments to develop a novel 3d display technique that has been designed to rectify this problem. The display presents a near-correct stimulus to the eye's focusing response by presenting each eye's image as the sum of multiple planar images at different distances from the eye. I call these multi-plane images. All the other visual information is presented correctly to each eye. A difficult situation to simulate is when an object lies at a distance in-between two image planes. To do this, the object is presented as a blend of light at the two image planes. The core experiments will evaluate whether this technique is effective in stimulating the eye to focus in-between image planes. I will also measure eye movements to multi-plane images, to see whether they are equivalent to viewing real world scenes. Furthermore, I will investigate the relationship between eye movements discomfort in 3d displays, and determine whether multi-plane images can reduce this problem. The display technique and experiments are motivated by a theoretical model of how the eye's focusing response is controlled. This emphasis on basic science will allow me to develop a display that is better suited to the properties of the human visual system. In turn, the theoretical model, in conjunction with the unique properties of the multi-plane display, will allow me to address some long-standing research questions regarding how the eye focuses. I will continue to develop and refine the model throughout the research programme. The research findings will aid the development of 3d displays that are more comfortable, natural to use, and produce more accurate 3d perception than current displays. This should be of significant benefit to the growing number of users of this important technology.

Publications


10 25 50
Banks MS (2012) Stereoscopy and the Human Visual System. in SMPTE motion imaging journal
Miles H (2012) A review of virtual environments for training in ball sports in Computers & Graphics
 
Description Stereoscopic 3d displays are increasingly used in society in a wide range of applications such as entertainment, scientific visualisation, and medical imaging (including surgical planning, and surgical robots). Typically, viewers must converge their eyes at the depicted stereoscopic depth (vergence) while focusing the eye (accommodation) at the screen surface. Decoupling these responses is effortful, and causes a number of aversive side-effects, including fatigue, discomfort, and poor perception of stereoscopic depth.



Our goal was to develop and evaluate an approach to 3d visualisation that is better suited to the properties of the human visual system. The above problems could be solved if, like the real world, 3d displays presented continuous variations in focal distance. The primary objective of our work was to evaluate a proposed solution to achieving this called 'depth filtering'. Here, each eye sees the sum of images presented on multiple focal planes, at different distances, and continuous variations in focal distance are approximated by distributing image intensity across planes. Our secondary objective was to use our display to advance basic knowledge of how the eye focuses, and how accommodation and vergence interact.



We first constructed a novel multi-plane 3d display that (uniquely, to our knowledge) allows us to present depth-filtered images while simultaneously measuring the eyes' accommodation and vergence responses.



We carried out a series of experiments demonstrating that depth filtering does create continuous variations in the eye's accommodation response. Critically, we then determined the maximum image-plane separation that supported accurate accommodation and vergence, while preserving good stereoscopic depth perception. This is key to determining if depth filtering offers a practical solution: placing image planes close together improves the accuracy of the approximation to the real world, but with current technology displays can only be built with a small number of image planes.



Our key findings are that using a display with just four or five planes, spaced relatively widely (0.6 - 0.9 dioptres), depth-filtered images can create vergence and accommodation responses, and perception of stereoscopic depth, that do not differ from viewing equivalent real-world stimuli. Thus, this approach appears to offer a practical solution to the aversive side-effects inherent in viewing conventional 3d displays.



We also used our display to better understand the conditions under which conventional 3d displays cause aversive side effects. The multi-plane solution is not practical for mass entertainment applications (cinema and TV), and these experiments have provided valuable information about where, in stereo depth, content should be placed in conventional 3d displays.



In addition, we have used the unique properties of our display to test a new model of how the eye focuses. This has provided significant insights into this basic question, and appears to unify two apparently contradictory and long-standing theories. Overall, the results suggest that fine-scale image information contributes less to the accommodation response than has previously been thought. The model has practical value because it allows us to predict how accommodation responses might break down in our display, and in new designs.
Exploitation Route Our findings provide a useful basis for the development of next-generation stereoscopic displays, including head-mounted displays, and virtual reality, where it will be important to present near-correct focus cues.
Sectors Aerospace, Defence and Marine,Creative Economy,Digital/Communication/Information Technologies (including Software),Other
 
Description Thus far, the principle contribution of the work outside academia has been to communicate how a better understanding of the interaction between the human visual system and stereoscopic media can significantly improve how natural/comfortable they are to use, and how effective they are. Evidence that this has been at least in part achieved comes from a large number of invitations to present the work to professional/industry forums, including technical (IBC, International Meeting on Information Display), creative (3Dstereomedia), and standards-setting agencies (SMPTE; the Society for Motion Picture and Television Engineers). In future we expect the specific findings on contracting multiple-focal-planes displays to influence the design and construction of next-generation stereoscopic displays.
First Year Of Impact 2009
Sector Creative Economy,Digital/Communication/Information Technologies (including Software),Other
Impact Types Societal
 
Description Stereoscopic HDR displays 
Organisation Bangor University
Country United Kingdom of Great Britain & Northern Ireland (UK) 
Sector Academic/University 
PI Contribution Visual perception expertise, particularly stereoscopic vision, with respect to display technology
Collaborator Contribution Expertise in display design and evaluation, particularly high dynamic range displays, software, conducting studies
Impact Multi-discipinary: Computer Science, Psychology (vision science) 10.1145/2628257.2628258
Start Year 2013
 
Description Virtual Rugby Simulator 
Organisation Bangor University
Country United Kingdom of Great Britain & Northern Ireland (UK) 
Sector Academic/University 
PI Contribution We secured a knowledge transfer PhD studentship (Knowledge Economy Skills Scholarships; KESS) in collaboration with Computer Science and Sports, Health and Exercise Sciences at Bangor University, and an external partner SME (North Wales Rugby) to develop and build a Rugby Simulator. I contributed expertise in stereoscopic vision, building stereoscopic displays, and designing perceptual experiments.
Collaborator Contribution Expertise in sports science, technical virtual-reality expertise, physics modelling of rugby ball, conducting experiments (academic partners), and expertise in playing and coaching Rugby (SME partner)
Impact Multidisciplinary: Psychology (vision science), Computer Science, Sports Science Efficacy of a Virtual Environment for Training Ball Passing Skills in Rugby; 10.1016/j.cag.2012.04.007; 10.1109/CW.2013.45;
Start Year 2010
 
Description 3-D TVs for industry: $64B; health research: zero Market projections rosy, but health effects are still unknown 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact The PI was interviewed for, and quoted in, an article on possible heath effects of 3d stereoscopic television for the online industry journal 'EE Times'. The possible solutions to some of these problems, explored in the grant, are referred to in the article, published on March 3rd 2010.

Not possible to identify specific impacts, due to nature of communication (internet media)
Year(s) Of Engagement Activity 2010
 
Description A vision-science perspective on stereoscopic 3d media 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The PI gave an invited seminar to the German film and TV post-production company 'Cinepost', at the BavariaFilmPlatz, Germany.

After my talk I was contacted by several professional practitioners seeking vision science-related advice.
Year(s) Of Engagement Activity 2012
 
Description Focusing and Fixating on Stereoscopic Images: What We Know and Need to Know 
Form Of Engagement Activity Scientific meeting (conference/symposium etc.)
Part Of Official Scheme? No
Type Of Presentation keynote/invited speaker
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The PI gave an invited address to the Society of Motion-Picture and Television Engineers' (SMPTE's) 2nd-annual International Conference on Stereoscopic 3D for Media & Entertainment. Based on work in the grant, he outlined the principle problems associated with eye movements and focusing on 3d media to industry and media delegates.

SMPTE is the main international standards-setting agency for the broadcast and film industry, so it is valuable to have communicated vision science issues in 3D media directly to this audience.
After my talk several professional practitioners contacted me asking for advice on vision science issues in 3D media. The presentation was also reported on popular 3D media blogs.
The vision science symposium (four talks) lead to an award-winning paper in the SMPTE motion imaging journal (doi: 10.5594/j18173)
Year(s) Of Engagement Activity 2011
 
Description Focusing and fixating the eyes on stereoscopic images 
Form Of Engagement Activity Scientific meeting (conference/symposium etc.)
Part Of Official Scheme? No
Type Of Presentation keynote/invited speaker
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The PI gave an invited address presenting latest findings from the grant and reviewed the principle problems caused by incorrect focus cues in 3d stereo media, at the first annual 3dStereo Media meeting (2009), in Liege, Belgium, to an audience of researchers and industry delegates.

After my talk several professional practitioners contacted me asking for advice on vision science issues in 3D media
Year(s) Of Engagement Activity 2009
 
Description Getting to Grips with Stereo 3D 
Form Of Engagement Activity Scientific meeting (conference/symposium etc.)
Part Of Official Scheme? No
Type Of Presentation keynote/invited speaker
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The PI gave an invited address to the international broadcast industry conference IBC 2011, based on the work on the grant, and outlining the principle vision science issues in working with stereo 3d media. The address was aimed at industry delegates.

After my talk I was contacted by several professional practitioners seeking vision science-related advice.
Year(s) Of Engagement Activity 2011
 
Description Improving stereoscopic displays: Why vergence-accommodation conflicts matter and how they might be solved 
Form Of Engagement Activity Scientific meeting (conference/symposium etc.)
Part Of Official Scheme? No
Type Of Presentation keynote/invited speaker
Geographic Reach International
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact An invited address to the 16th European Conference on Eye movements. The PI reviewed the purpose and findings from work on the grant.

stimulated discussion from researchers studying eye movements, but not in stereoscopic viewing.
Year(s) Of Engagement Activity 2011
 
Description Improving stereoscopic displays: why focus cues matter and how to get them (nearly) correct 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact The PI gave an invited colloquium talk to the University College London inter-departmental talk series, Vision@UCL.

sparked stimulating discussion
Year(s) Of Engagement Activity 2012
 
Description Improving stereoscopic displays: why focus cues matter and how to get them (nearly) correct 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact The PI gave a department colloquium at Hull University Psychology/computer science departments.

sparked discussion of cross-discipinary science
Year(s) Of Engagement Activity 2012
 
Description Lessons from Vision Science for producing audience-friendly stereoscopic 3D 
Form Of Engagement Activity Scientific meeting (conference/symposium etc.)
Part Of Official Scheme? No
Type Of Presentation keynote/invited speaker
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The PI was invited to coordinate and chair a session from vision scientists examining the latest research on the principle vision-science-related issues in 3d stereo media, at the fourth annual 3dStereo Media meeting (2012), in Liege, Belgium, to an audience of researchers and industry delegates.

After my talk I was contacted by several professional practitioners seeking vision science-related advice.
Year(s) Of Engagement Activity 2012
 
Description Member of Programme Committee for 3d Stereomedia 2011 
Form Of Engagement Activity Scientific meeting (conference/symposium etc.)
Part Of Official Scheme? No
Type Of Presentation workshop facilitator
Geographic Reach International
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact The PI (Watt) was invited to join the Programme Committee for the third 3d StereoMedia meeting, in Liege, Belgium, a European 3d-stereo summit for science, technology and digital art. . Awarding Body - n/a, Name of Scheme - n/a

Contributed to publication of conference proceedings on a topic of societal value
Year(s) Of Engagement Activity 2011
 
Description Member of Programme Committee for 3d Stereomedia 2012 
Form Of Engagement Activity Scientific meeting (conference/symposium etc.)
Part Of Official Scheme? No
Type Of Presentation workshop facilitator
Geographic Reach International
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact The PI (Watt) was invited to join the Programme Committee for the scientific conference at the fourth 3d StereoMedia meeting (in Liege, Belgium), a European 3d-stereo summit for science, technology and digital art. . Awarding Body - 3d stereomedia, Name of Scheme - 3d stereomedia

Contributed to publication of conference proceedings on a topic of societal value
Year(s) Of Engagement Activity 2012
 
Description Member of Programme committee for 3d Stereomedia 2013 (international conference on 3d; IC3D) 
Form Of Engagement Activity Scientific meeting (conference/symposium etc.)
Part Of Official Scheme? No
Type Of Presentation workshop facilitator
Geographic Reach International
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Contributed to publication of conference proceedings on a question of societal value

n/a
Year(s) Of Engagement Activity 2013
 
Description Stereopsis and stereoscopy: insights from vision science 
Form Of Engagement Activity Scientific meeting (conference/symposium etc.)
Part Of Official Scheme? No
Type Of Presentation keynote/invited speaker
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The PI was invited to coordinate and chair a session from vision scientists examining the latest research on the principle vision-science-related problems in 3d stereo media, at the second annual 3dStereo Media meeting (2010), in Liege, Belgium, to an audience of researchers and industry delegates. He also presented his own work from the grant.

After my talk several professional practitioners contacted me asking for advice on vision science issues in 3D media
Year(s) Of Engagement Activity 2010
 
Description Stereoscopic displays without the headaches: why focus cues matter and what to do about them 
Form Of Engagement Activity Scientific meeting (conference/symposium etc.)
Part Of Official Scheme? No
Type Of Presentation poster presentation
Geographic Reach National
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact The PI presented principle findings from the grant to a joint meeting of the Applied Vision Association (AVA) and the British Machine Vision Association (BMVA), in Cardiff, 26.05.2011.

Presentation initiated discussion about multi-disciplinary projects and funding, with EPSRC representatives.
Year(s) Of Engagement Activity 2011
 
Description The causes of discomfort in stereo 3D displays: a view from vision science 
Form Of Engagement Activity Scientific meeting (conference/symposium etc.)
Part Of Official Scheme? No
Type Of Presentation keynote/invited speaker
Geographic Reach International
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact The PI gave an invited workshop to the International Meeting on Information Display, Daegu, Korea, Organised by the Korean Information Display Society (KIDS) and the Society for Information Display (SID), August 2012.

After my talk I was contacted by several professional practitioners seeking vision science-related advice.
Year(s) Of Engagement Activity 2012
 
Description The good, the bad and the ugly truth about 3D movies 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Undergraduate students
Results and Impact Invited talk to University of East Anglia Psychology Dept. series, "Psychology and the Creative Arts".

Talk series aimed predominantly at undergraduate students.

Sparked discussion with undergraduate students about range of work within psychology
Year(s) Of Engagement Activity 2012
 
Description The good, the bad, and the ugly truth about 3d movies 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact The PI gave a public lecture to members of his local community, as part of an organisation called the Science Cafe, on the principle vision science issues/concerns with 3d stereo media such as 3d movies and 3d television.

Many discussions arose, in particular around the fact that the public and other academics who attended were (pleasantly) surprised that someone in psychology department work on such topics, and that there was a valuable scientific angle to what many view as a creative activity (making 3D media)
Year(s) Of Engagement Activity 2011
 
Description The good, the bad, and the ugly truth: what S3D may and may not do to your eyes and brain 
Form Of Engagement Activity Scientific meeting (conference/symposium etc.)
Part Of Official Scheme? No
Type Of Presentation keynote/invited speaker
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The PI was invited to coordinate and chair a session from vision scientists examining the latest research on the principle vision-science-related problems in 3d stereo media, at the third annual 3dStereo Media meeting (2011), in Liege, Belgium, to an audience of researchers and industry delegates. He also presented his own work from the grant.

After my talk I was contacted by several professional practitioners seeking vision science-related advice.
Year(s) Of Engagement Activity 2011
 
Description The stimulus to accommodation Reconciling theoretical and empirical accounts of the contributions of different spatial frequency components 
Form Of Engagement Activity Scientific meeting (conference/symposium etc.)
Part Of Official Scheme? No
Type Of Presentation paper presentation
Geographic Reach National
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact The PI gave a talk on the control of the eye's focusing response, based on work completed under the grant., to the Annual General Meeting of the Applied Vision Association, UK. Published abstract [edited]:



There have long been conflicting views over the role of different spatial frequency components in accommodation control. Fine-tuning models posit that accommodation is guided by increasingly high spatial frequencies as optimal focus is approached. This makes sense because signals from high

My talk sparked interesting discussions.
Year(s) Of Engagement Activity 2013
 
Description member of Programme committee for 3d Stereomedia 2014 (international conference on 3d; IC3D) 
Form Of Engagement Activity Scientific meeting (conference/symposium etc.)
Part Of Official Scheme? No
Type Of Presentation workshop facilitator
Geographic Reach International
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Contributed to publication of conference proceedings on a question of societal value

n/a
Year(s) Of Engagement Activity 2014