COMMANDO-HUMANS: COMputational Modelling and Automatic Non-intrusive Detection Of HUMan behAviour based iNSecurity

Lead Research Organisation: University of Surrey
Department Name: Computing Science

Abstract

This project addresses mainly the Human Factors challenge of the joint Singapore-UK call, and it has an interdisciplinary team with expertise in cyber security, cognitive psychology, and human-computer interface (HCI). It aims at producing direct evidence that human behaviour related insecurity can be detected automatically by applying human cognitive models to model and simulate humans involved in security systems. A key outcome of the project will be a working software system that can be used for this purpose by researchers and practitioners. The project will focus on human user authentication systems as a representative use case and will produce new knowledge on the role of human behaviours in such systems and security systems in general. Both the software framework and new knowledge on human behaviours can also help address other challenges of the call (e.g., detection of intruders/extremists requires knowledge on how they behave; protection of user privacy require knowledge on how human users handle personal data; policy makers need to understand behaviours of their organisations' employees and human attackers targeting their organisations to make more informed decisions).

It has been well known that human factors are a very important aspect of cyber security, as recognised by governments all over the world e.g., in the UK Cyber Security Strategy (2011), in Singapore's National Cyber Security Masterplan 2018 (2013), and in the US Federal Cybersecurity Research and Development Strategic Plan (2011). Human related insecurity is often related to intended or unintentional (maybe subconscious) insecure human behaviours. To conduct research on human behaviours (in cyber security, HCI, psychology and other related fields), researchers normally depend on involvement of real human users via surveys, interviews, simulated scenarios, observations of real cases, interactive games, or other specially designed user studies. Such approaches are often time-consuming and costly, and suffer from other issues like limited and/or biased samples, questionable ecological validity, difficulties in reproducing results, and impossibility of running some studies due to ethical/privacy/legal concerns.

This project aims at developing the first (to the best our knowledge) general-purpose computational framework and supporting software tools that will enable automatic detection of human behaviour related insecurity at the HCI level without the need to involve real human users. The framework will be built on computational models of human cognitive processes, HCIs, human behaviour related attacks and (in)security measures. The framework will be non-intrusive: instead of evaluating the running system itself, the framework will evaluate an abstract executable model of the system and humans involved. Removing real human users from the process allows faster and more objective inspection of potential insecurity of a given security system. The automated process can still be combined with traditional user studies to make better use of limited resources in automatically detecting potential insecurity problems deserving further manual analysis.

The framework and software tools developed will be of great value for cyber security researchers, security system designers/developers and security industry to deliver securer systems to end users. As a natural byproduct, they will also allow easier evaluation of usability of security and non-security related computer systems with an HCI. As we mentioned above in this summary, people having concerns on other challenges of the call can benefit from the project's outcomes as well.

In this project we will focus mainly on HCI-level ("micro") human behaviours, but possible extensions to higher-level ("macro") behaviours (e.g., how human users adapt their behaviours over time via rehearsals and learning) will be looked at as well to pave the way for our future research.

Planned Impact

The "Academic Beneficiaries" field of the Je-S form explains the expected academic impact in detail, so here we focus on economic and societal impact.

While the project is targeting mainly researchers, we will make the software framework accessible to non-researchers as well so it can help security system designers and developers, and security industry in general to check human behaviour related insecurity problems at the HCI level in the design stage of their security products and services. Even when user studies are still needed to evaluate their products and services' performance, the software framework can help identify key areas they need to pay more attention to and thus making a better use of the limited resources. This, on one hand, can help enhance the research capacity, knowledge and skills and efficiency of security industry to deliver securer security products and services, and on the other hand can improve the overall experience and quality of life of end users by reducing security incidents that can be avoided before such products and services are introduced into the real world. If it is possible to collect more realistic (and anonymous) information about human users using a deployed security product or service, the vendor/provider can also identify more potential insecurity problems that exist for a particular group of users only and find ways to serve them better.

We also expect that the software framework developed will help organisations' policy makers and IT managers to get more information about behaviours of their employee's and human attackers targeting their organisations, and the usability-security trade-off of their security systems (deployed and those under consideration for purchase), which will allow them to make more informed decisions on things like what security systems to use, how to use them, what security policies should be enforced, and if any training or educational programmes are needed for their staff and customers. We understand policy makers and IT managers will have more interests in macro human behaviours and more systems beyond human user authentication, so they can be potential users of the planned extensions of our research in future.

Like most IT systems, there are two types of end users of security products and services: 1) non-security service providers using such products and services developed by other companies to serve their customers (e.g., banks); 2) end human users who are actually using the products and services. In addition to indirectly benefiting from the software framework we will develop, both groups of end users can actually use the software framework to conduct independent evaluation of security products and services they use, which can help increase transparency of the security industry and eventually benefit security industry by giving more credits to better products and services. This may also foster a new service on independent security and usability evaluation of IT systems (e.g., like what Virus Bulletin Ltd is currently doing on anti-malware products). We will exploit the possible commercialisation of the software framework developed towards this direction.

As can be expected, our proposed research on human behaviours at the HCI level will create new knowledge on how human users and attackers behave and interact with computer systems. Such knowledge is not only useful for researchers, but equally so for practitioners and end users. This is particularly important for security education and training purposes, e.g., in designing and implementing cyber security awareness campaigns for the general public. The focused cyber security systems, human user authentication systems, are also a very good use case here as passwords are widely used in security education and training.

It deserves mentioning that the human and HCI modelling parts of our software framework are independent of security, so can be used for evaluating usability of any IT systems.

Publications


10 25 50
 
Description We have studied existing software tools for cognitive modelling and found out one particular tool (CogTool) is the best in terms of supporting the further development of the software framework we proposed for the project. We also discovered many parameters that we did not previously know that should be incorporated in our software framework.

We found out that descriptions of user interfaces of some user authentication systems (and wider cyber security systems) require algorithmic parts rather than just static descriptions, which led to a new way of describing the user interface by having both static descriptions and interpreted computer programs.

We discovered eye-tracking is a useful technology to identify better ways to model human behaviours at the human-computer interface level, and has proved this through a use case on Undercover, which also led to a resesarch paper accepted to a conference (not published yet so will not be added to the publications until next year). The eye-tracking element was added to the software framework as a new component previously we did not include.

Based on the original plan and the above new discoveries, we have designed a more complicated software framework for modelling and simulating human behaviours in user authentication systems at the human-computer interface level. The framework is still being developed, and some components have been done or partly done. Part of the components have been tested using an example user authentication system called Undercover.

In addition to the design and development of the software framework, we also clarified human behaviour data we need to support the modeling tasks. Particularly, we identified a major gap in existing cognitive modelling tools: visual search. We have designed an experiment to conduct some user studies to get raw data we need to build human behaviour templates of visual search for the software framework.

Finally, the PI was also supported to work on another password related research which led to a conference paper accepted (not published yet so will not be added until next year).
Exploitation Route The software framework we developed will help both researchers and practitioners who are using cognitive modelling tools such as CogTool to do more automated analysis with less efforts. While our software framework will be tested more on user authentication systems, most components we are developing will be universal for general modelling of user interfaces on computers. We expect our software tools (named CogTool+) will be able to attract all users of CogTool and other similar software tools. Since our tools will allow automated detection of some human behaviour related security problems, designers of user authentication systems and wider cyber security systems will find them useful. Our study on visual search in cognitive modelling will help psychologists and computer scientists to understand how human users behave to visual tasks on graphical user interface, thus gaining more insights on how to design such interfaces better. Our work can also clearly benefit cyber security education since it will provide new insights on complicated attacks caused by insecure human behaviours. We envisage our work will benefit many different sectors since user authentication and cyber security systems are used everywhere nowadays.

The research on humn cognitive modelling in cyber security has inspired the PI to improve other research work and start new research activities, which include an accepted paper on password visualisation and several new research projects on passwords and human-assisted data loss prevention.
Sectors Communities and Social Services/Policy,Digital/Communication/Information Technologies (including Software),Education,Electronics,Financial Services, and Management Consultancy,Healthcare,Government, Democracy and Justice,Other
URL http://www.commando-humans.net/
 
Description This research has helped inspire the PI to co-develop a new user authentication technology with his PhD student (who is not funded by the project). The new technology has been named Pass8 (PassInfinity) and a patent application has been filed by the University of Surrey (using its own tech transfer funding). A prototype of Pass8 has been produced and some external funding has been secured from DCMS and Innovate UK through the SETsquared Partnership's Cyber Security ICURe (Innovation to Commercialisation of University Research) Programme for market research. Pass8 can bring new angles to the planned research in the project (as user authentication systems can now be designed in a very different and much more complicated way), but itself has the potential to create very high potential non-academic impacts as it can be used very widely by organisations and users to save costs and increase security of user devices and organisational networks. Pass8 can help policy makers as well because it supports much more flexible and agile policies on user authentication. Pass8 has been publicised by the University of Surrey and has generated great interests from the general public including BBC World Serivice. Since the technology just started creating impacts, it has not been actually used yet. The main line of research in the project is still in its early stage of developing the software framework so at this moment no non-academic impacts are expected.
First Year Of Impact 2017
Impact Types Societal,Economic
 
Description ACCEPT: Addressing Cybersecurity and Cybercrime via a co-Evolutionary aPproach to reducing human-relaTed risks
Amount £278,000 (GBP)
Funding ID EP/P011896/1 
Organisation University of Cambridge 
Department Engineering and Physical Sciences Research Council EPSRC
Sector Public
Country United Kingdom of Great Britain & Northern Ireland (UK)
Start 04/2017 
End 03/2019
 
Description Eyes Can Tell: Applications of Eye-tracking Devices in Cyber Security Research
Amount £19,392 (GBP)
Organisation Government of the UK 
Department Government Communications Headquarters (GCHQ)
Sector Public
Country United Kingdom of Great Britain & Northern Ireland (UK)
Start 10/2016 
End 03/2017
 
Description H-DLP: Human-assisted machine learning for bootstrapping DLP (data loss prevention) systems
Amount £192,003 (GBP)
Funding ID KTP010417 
Organisation Government of the UK 
Department Innovate UK
Sector Public
Country United Kingdom of Great Britain & Northern Ireland (UK)
Start 01/2017 
End 12/2020
 
Description PassInfinity: An "All in One" user authentication framework
Amount £28,968 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Academic/University
Country United Kingdom of Great Britain & Northern Ireland (UK)
Start 04/2017 
End 09/2017
 
Description Collaboration with Clearswift Ltd 
Organisation Clearswift Ltd
Country United Kingdom of Great Britain & Northern Ireland (UK) 
Sector Private 
PI Contribution The University of Surrey's Dr Shujun Li initialised the conversation with Clearswift Ltd in 2014 which led to an Innovate UK KTP application. The KTP application was successful in 2016 and the project officially started in 2017. Dr Shujun Li provided a potential technology to solve a problem facing Clearswift and other DLP (data loss prevention) vendors. Dr Shujun Li and Dr Ben Shenoy of University of Surrey play the roles of academic supervisors in the KTP project. The University of Surrey is in charge of managing HR matters around a KTP associate, and provided needed training.
Collaborator Contribution Clearswift Ltd provided the problem for the KTP project to attack, participated in the project proposal writing, provided match funding per KTP rules, and is hosting the KTP associate to work full-time at its main office in Theale, Reading.
Impact The project just started in January 2017, so no concrete outcomes yet.
Start Year 2014
 
Description Collaboration with Crossword Cybersecurity plc 
Organisation Crossword Cybersecurity
Country United Kingdom of Great Britain & Northern Ireland (UK) 
Sector Private 
PI Contribution The University of Surrey resercher Dr Shujun Li initialised collaboration with Crossword Cybersecurity plc on tech transfer of two new inventions from his research project.
Collaborator Contribution Crossword Cybersecurity plc has been a partner of an ongoing project on Pass8 (PassInfinity) and will be the partner of another forthcoming project. They provided and will provide in-kind support for both project. The figure reported above is for the forthcoming project only.
Impact A spin-out is being discussed but at this moment has not been formed yet.
Start Year 2014
 
Description Collaboration with Data61, CISRO, Australia 
Organisation Commonwealth Scientific and Industrial Research Organisation
Country Australia, Commonwealth of 
Sector Public 
PI Contribution This was continuation of our previous collaboration with NICTA, Australia after its merger into CISRO's Data61 department. CISRO supported this project proposal as an unfunded partner and participated in all WPs.
Collaborator Contribution Two researchers and some interns from CISRO have contributed to this project by conrtributing to all WPs, attending meetings to discuss research plan and to provide data on a new user authentication system for timing attack analysis. A joint user study on eye-tracking for the user authentication system CISRO developed is being designed and to be conducted.
Impact Some data collected from a new user authentication system and a report on an enhanced timing attack are produced and sahred with all members of the COMMANDO-HUMANS project.
Start Year 2016
 
Description Collaboration with Singapore Management University 
Organisation Singapore Management University (SMU)
Country Singapore, Republic of 
Sector Academic/University 
PI Contribution The project allowed researchers at the University of Surrey to collaborate with five researchers at the Singapore Management University. The work proposed in the project is split between the two research teams and both sides helped each other.
Collaborator Contribution The Singapore Management University is in charged of WP3 and contributed to WP2. They contributed to management of the project as well.
Impact A joint publication on timing attack against PIN entries is being prodcued. A joint software CogTool+ is being co-developed.
Start Year 2016
 
Description Collaboration with University of Split, Croatia 
Organisation University of Split
Country Croatia, Republic of 
Sector Academic/University 
PI Contribution This is a continuation of collaboration between Dr Shujun Li and two researchers of the University of Split since 2010. The collaboration was broadened to cover all memebrs of of the COMMANDO-HUMANS project.
Collaborator Contribution Two researchrs from the University of Split contributed to all WPs and attended all quarterly meetings of the COMMANDO-HUMANS project. They have been working with other partners espcially CISRO in an enhanced timing attack.
Impact A joint research report between CISRO and the University of Split has been produced with some software and data.
Start Year 2011
 
Description Consortium for project ACCEPT 
Organisation Neighbourhood and Home Watch Network
Country United Kingdom of Great Britain & Northern Ireland (UK) 
Sector Charity/Non Profit 
PI Contribution The University of Surrey led the formation of the consortium and won a research bid for EPSRC's Human Dimensions of Cyber Security call, which led to the project ACCEPT to start in April 2017.
Collaborator Contribution Other partners helped form the consortium by bringing their expertise into the project proposal.
Impact The project is to start, so no outcomes yet.
Start Year 2016
 
Description Consortium for project ACCEPT 
Organisation Transport Research Laboratory Ltd (TRL)
Country United Kingdom of Great Britain & Northern Ireland (UK) 
Sector Private 
PI Contribution The University of Surrey led the formation of the consortium and won a research bid for EPSRC's Human Dimensions of Cyber Security call, which led to the project ACCEPT to start in April 2017.
Collaborator Contribution Other partners helped form the consortium by bringing their expertise into the project proposal.
Impact The project is to start, so no outcomes yet.
Start Year 2016
 
Description Consortium for project ACCEPT 
Organisation University College London (UCL)
Department UCL Genetics Institute
Country United Kingdom of Great Britain & Northern Ireland (UK) 
Sector Academic/University 
PI Contribution The University of Surrey led the formation of the consortium and won a research bid for EPSRC's Human Dimensions of Cyber Security call, which led to the project ACCEPT to start in April 2017.
Collaborator Contribution Other partners helped form the consortium by bringing their expertise into the project proposal.
Impact The project is to start, so no outcomes yet.
Start Year 2016
 
Description Consortium for project ACCEPT 
Organisation University of Warwick
Department Department of Statistics
Country United Kingdom of Great Britain & Northern Ireland (UK) 
Sector Academic/University 
PI Contribution The University of Surrey led the formation of the consortium and won a research bid for EPSRC's Human Dimensions of Cyber Security call, which led to the project ACCEPT to start in April 2017.
Collaborator Contribution Other partners helped form the consortium by bringing their expertise into the project proposal.
Impact The project is to start, so no outcomes yet.
Start Year 2016
 
Title Improved Authentication 
Description This is a patent applicaiton filed by the University of Surrey to protect Pass8 (PassInfinity), a new user authentication technology developed in the context of the COMMANDO-HUMANS project as a byproduct. It was filed in January 2017 and is currently evaluated by UK IPO. It was also the result of the broader work funded by the EPSRC funded ACE-CSR at the University of Surrey. 
IP Reference 1700649.5 
Protection Patent application published
Year Protection Granted 2017
Licensed No
Impact Not yet.
 
Title CogTool+ 
Description It is an extended tool based on CogTool (https://github.com/cogtool) supporting meta-modelling and automated simulation of a large number of models of the same meta-model. It is still being developed and the first beta version is expected to be released in summer 2017. 
Type Of Technology Software 
Year Produced 2017 
Open Source License? Yes  
Impact The tool has not been completed yet so its impact is to be seen. It is however designed to make WP4 of the COMMANDO-HUMANS project possible. 
 
Description Human/User-Centric Security 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact It was an invited talk given at the Fast Stream Conference 2017 (Digital: Definition Unknown), organised by UK Government's Civil Service Fast Stream. The audience was mainly members of the UK Government's Civil Service Fast Stream. The talk was also advertised to general public through LinkedIn and Slideshare.net.
Year(s) Of Engagement Activity 2017
URL http://www.slideshare.net/hooklee/humanusercentric-security
 
Description Observer-Resistant Password Systems: How hard to make them both usable and secure? 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Undergraduate students
Results and Impact This was a talk given to a mixed audience of students, researchers and industry, as part of a half-day workshop on Human Factors in Cyber Security, Surrey Centre for Cyber Security and Department of Computer Science, University of Surrey, UK. It was also publicised through a blog article to the general public.
Year(s) Of Engagement Activity 2016
URL http://blogs.surrey.ac.uk/sccs/2016/03/31/from-shoulder-surfers-and-keyloggers-to-mitm-and-malware-c...
 
Description Pass8 (PassInfinity) 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact This was an interview broadcast via BBC World Service's Tech Tent programme. Dr Shujun Li was interviewed for his new technology Pass8 (PassInfinity). This interview was triggered by a press release of the University of Surrey and itself generated further media reports on the techonology.
Year(s) Of Engagement Activity 2017
URL http://mms.tveyes.com/Transcript.asp?StationID=7195&DateTime=2%2F17%2F2017+3%3A24%3A02+PM&Term=Unive...