Learning from the expert: Can observing the oculomotor behaviour of expert face processors improve training of face matching?

Lead Research Organisation: University of Portsmouth
Department Name: Psychology


Face matching is crucial to tasks such as checking passports at border control, searching CCTV footage for suspects of a crime and trying to identify people who have been missing either a short time or for many years. Face matching of unfamiliar people is surprisingly inaccurate, so it is important to identify methods to improve it. We will test a cross-section of the public to identify people particularly skilled at face matching. We will then observe how they compare faces, recording the eye movements they make, and ask them about the strategies they use.

Training will be developed which includes both explicit strategies to be used and implicit training through “oculomotor demonstration”, which involves showing the trainee a dynamic mapping of an expert’s gaze mapped onto the faces being inspected. Previous research in other perceptual tasks has suggested that oculomotor demonstration can improve the rate of learning. The effectiveness of this simple technique may be because people innately attend to other people's gaze to understand what in the environment those other people consider important. We will test whether such training is more effective than previously developed training and whether the effects last for up to one month.

Planned Impact

Face matching errors can have devastating effects in criminal justice. For instance, a surveillance officer's mis-comparison of a photo of a terrorist suspect to the CCTV image of Jean Charles de Menenez is thought to be one of the reasons for the tragic shooting of him by police in 2005. Further, it is important to block known terrorists from entering the country, and one of the steps to doing that is to ensure that border security personnel can recognise them from their photos. Likewise, at a local level, the effectiveness of drinking banning orders is critically dependent on distribution of photos of the banned person and accurate identification of the person from those photos by security personnel at bars and off-licences.

This work will also impact criminal prosecutions. Identification evidence often represents a substantial amount of the evidence supporting a prosecution (Brewer, et al., 2005), and influences decision making throughout the forensic setting. Consequently, reliable identification evidence is essential for effective criminal investigations and the successful prosecution of criminals. Thus, our proposed research can improve the efficiency of criminal investigations and will therefore be of interest to forensic investigators attempting to maximise the reliability of identification evidence.

If the training method proves successful, this work will have an impact on society by providing the forensic community a new training method that would improve face matching. Improving forensic investigations and the effectiveness of border control will benefit society by reducing crime and terrorism, both of which have cultural and economic impacts. The forensic community includes security firms, police, UKBA, and the Home Office. The public should be safer, which will benefit quality of life.

This research would also benefit other training providers in that it tests a way of transferring the skills of an expert to those of a trainee without one on one contact. Historically, apprenticeships have been used to transfer practical skill from an expert to a trainee. As business becomes more distributed, even globalised, it is more difficult for experts to be geographically present with trainees, and so alternatives to apprenticeships are needed. If showing the eye movements of an expert provides some of the benefit of having the expert present, this could be an invaluable addition to training methods used by businesses.

If a beneficiary decided to implement the training, they would only need to identify the face-matching tasks that they use, identify one or more people particularly good at the task, use an eye-tracker to record eye movements of the expert doing the task, and develop a short training course where the trainee watches the video and practices the task. It is easy to envisage that these steps could be done in less than 6 month, which means that the timescale of impact is relatively short.

We will be advertising the project to the wider community. As the topic is engaging, this should also benefit society by giving the general public an opportunity to become involved in research.

The research assistant will gain invaluable research experience and, in particular, learn how to use an eye-tracker. That particular tool would be of benefit for a research career either in academia or in other domains (e.g., marketing).


10 25 50
Description FINDING 1: The core scientific impact was that, contrary to previous work in related fields, modelling good inspection behaviour (i.e., showing the eye movements of skilled matchers) is not necessarily a better training method than those previously used. Inspection modelling is no better or worse than other training methods. This is of interest because (a) we believe this is the first test of the method that tried to improve performance on a task that was already practiced by observers, (b) this is one of the first tests of the method that trained with one set of stimuli and tested with another set of stimuli (and hence our evaluation of the training method was more rigorous than most past evaluations), and (c) there is to date no particularly effective method to train face matching whereas there are many jobs in which it would be helpful to have an effective training method.

FINDING 2: We found some evidence that for a set of face-pairs that was relatively easy to judge, inspection modelling improved performance better than the other training methods, and that the improvements generalised to other relatively easy matching trials and endured for at least three weeks. This finding needs to be replicated because of weaknesses in the evidence.

FINDING 3: We have developed a test that is unusual and useful in that faces differ considerably in terms of superficial characteristics and documented age.

FINDING 4: We learned that accuracy in matching faces does not vary consistently with the difference in age between the faces. Our stimulus set was small enough that this will require replication, but there was no hint of an effect of age difference on matching accuracy.

FINDING 5: We replicated some previous findings about differences between experts and novices in the way they do their work (experts thought longer before starting to inspect) and in their confidence about their behaviour (experts were more aware of whether they were accurate).
Exploitation Route The principle way to take our results forward is to continue pursuing ways to train face-matching. The method we were keen to test - modelling of inspection - needs to be tested more rigorously by lengthening the amount of training and by giving feedback to people about their accuracy. The most promising result (successful training of relatively easy comparisons) needs replication. If it proves reliable, we need to understand why the training method worked for easier comparisons but not for more difficult comparisons.

If modelling of inspection is unsuccessful for face-matching, after this extension of research, it is worth pursuing why the method works for other tasks but not for this one.

Outside of academia, there are many sectors that need to check people's identities by comparing two photographic identifications or comparing one photographic identification with a live person. These sectors are identified in the next question. Once a successful training method is identified, it should be conveyed to all of these sectors.
Sectors Communities and Social Services/Policy,Education,Financial Services, and Management Consultancy,Healthcare,Leisure Activities, including Sports, Recreation and Tourism,Retail,Security and Diplomacy
Description The project consisted of the development of a face-matching skill test, efforts to find differences between skilled and unskilled face-matchers, and an evaluation of three methods for training face matching. One of the training methods was specifically targeted, as it was a new method that involved having trainees observe the way a skilled matcher inspected faces. The trainees were shown where the matcher fixated on the images of the face-pairs in the training stimuli. It was hoped that this observation would be a form of implicit training. By the end of the funding period, the project found no strong evidence that the new method was better than the other methods. There was, however, some promising weak evidence that in some limited cases it would be better. For full understanding of these cases, our research is continuing even today. There has been no societal or economic impact of the project, because the training method was not particularly effective.
First Year Of Impact 2013
Description Invitation to small Home Office workshop about identity checks in airports
Geographic Reach National 
Policy Influence Type Participation in advisory committee
Title Face Matching Skill Test 
Description We have a set of pairs of faces that made up our face-matching skill tests and training materials, along with the performance of our population of participants on the tests. The faces themselves are not our property, so we do not share them directly with interested parties. However, we are happy to share the list of which faces were used and where to source them. This will be added to UK Data Service once our results are published. 
Type Of Material Database/Collection of data 
Provided To Others? No  
Impact None 
Description Multi-centre study 
Organisation University of Adelaide
Country Australia, Commonwealth of 
Sector Academic/University 
PI Contribution A member of our team is working with partners at University of Adelaide to further investigate face matching. The entire group is collecting and analysing data.
Collaborator Contribution The entire group is collecting and analysing data.
Impact 3 x conference presentations: 1. Stephens, R., Semmler, C., & Sauer, J.D. (2013, April). Confidence-accuracy calibration for positive and negative face matching decisions. 40th Australasian Experimental Psychology Conference, Adelaide, Australia; 2. Stephens, R., Semmler, C., & Sauer, J.D. (2013, February). Confidence-accuracy calibration for positive and negative face matching decisions. Australian Mathematical Psychology Conference, Sydney, Australia. 3. Semmler, C., Stephens, R., & Sauer, J.D. (2015, June). The confidence-accuracy relationship in identity verification settings: Base rates, task orientation and the positive-negative asymmetry. 11th Biennial Conference of the Society for Applied Research in Memory and Cognition, Victoria, Canada). Also, about to submit a manuscript.
Start Year 2013