National Centre for Research Methods 2014-2019

Lead Research Organisation: University of Southampton
Department Name: School of Social Sciences

Abstract

Social science forms and transforms our understanding of the world we live in, underpins the development, delivery and evaluation of public policy, as well as commercial and business practice, and informs national and international debate on matters of social, economic and political controversy. Research evidence from the social sciences permeates our lives, from the production, analysis and interpretation of economic indicators, through documenting stability and change in social attitudes, to understanding the factors which lead to educational attainment, community cohesion, national well-being, and inter-state conflict, to select just a few pertinent examples. It is essential, therefore, that social scientific research is as robust as that produced in any other area of scientific endeavour, and that research carried out under the banner of social science is of the highest possible standard in the academic sector and beyond. While the quality of research depends on a number of different factors, the most important is methodology; the ways in which social scientists collect, analyse and interpret social and economic data.

The proposed new National Centre for Research Methods (NCRM) will address long-recognised problems relating to under-capacity in the UK social science research community. It will build on the successful programme of work that has already been undertaken in the Centre's first ten years. However, much has changed since NCRM was founded in 2004, including the emergence of new types and volumes of data to be handled, new modes of communication and networking (such as Facebook, and Twitter), as well as new perspectives on data accessibility and the use of various kinds of administrative data (such as tax records and GP patient records) for research purposes. The sorts of questions that social scientists are seeking to address have also changed, reflecting important shifts in the national and global environment. For instance, social scientists are now addressing questions relating to climate change, food security, online behaviour and identity, international terrorism and the costs of higher education, all of which were either low priority or non-existent issues when NCRM was established a decade ago.

The new Centre will be a partnership between three institutions with leading international reputations in undertaking methodological research and training in the social sciences: the Universities of Southampton, Manchester, and Edinburgh. Together the partner institutions will deliver three core activities:
1. A programme of methodological research which will develop innovative ways of collecting and analysing new and traditional forms of social science research data. The programme will comprise a set of inter-related work-packages to be undertaken by the collaborating partners and also a set of commissioned projects to complement and integrate with the Centre's research work-packages.
2. A programme of face-to-face training events in advanced areas of methodological practice, delivered by leading national and international experts at locations which are accessible to social scientists throughout the UK.
3. Develop and expand NCRM's existing heavily used website to incorporate a range of new features and functions, such as tailoring presentation of material to user-specified interests and developing a range of new online resources to enable social scientists to learn about new methods at their own pace and convenience.

The Centre will consult widely with organisations in the public and private sectors to ensure that the work it undertakes is in line with their needs and interests. It will collaborate with leading scholars and institutions from overseas to ensure that the Centre's work meets the highest international standards and to promote its work on the international stage.

Planned Impact

Who will benefit? Social science research is undertaken and commissioned by a broad range of non-academic groups and all stand to benefit from the methodological research development and training resources that will be produced and made available through NCRM in its next stage of funding. This includes researchers working in central and local government and the Office for National Statistics, in public sector organisations such as the NHS, and charitable foundations such as the Wellcome Trust and the Nuffield Foundation, as well as smaller third sector organisations. Researchers in commercial and not-for-profit data collection agencies such as NatCen, TNS-BMRB, GfK-NOP and Ipsos-MORI will benefit from the research developments and training opportunities NCRM will provide, as will researchers in the broader commercial and market research industries.

How will they benefit? Our intention is for the Centre's impact on these non-academic groups to manifest in three primary ways. Firstly, through the Centre's own programme of methodological research (work packages 1 to 6 in case for support). The focus of our research programme has been selected in order to address areas which have previously been identified as being of strategic importance for the broader social science research community by NCRM's own research needs assessments, in the National Data Strategy 2013-2018, and by the 2012 ESRC review of NCRM. Specifically, we expect the work that will be undertaken under the theme of 'complex data with linked and time-dependent structures' to be of use to a broad range of non-academic user groups who are attempting to find suitable ways for managing and analysing increasingly large and complex data structures. For instance, the findings of WP1 and WP2 are likely to be of interest to data collection agencies and survey practitioners seeking to maximise fieldwork efficiency and to mitigate nonresponse bias, while WP3 will benefit data collectors and researchers working at the interface of the medical and social sciences. We also expect the findings from our research into the 'pedgagogy of methodological learning' to be of practical use to organisations such as the ESRC, the Nuffield Foundation and Wellcome Trust, who have an interest in building methodological capacity, particularly (though not entirely) in the area of quantitative and statistical methods.

Second, the Centre's commissioned programme of methodological research (which will commence in 2015) will be of additional benefit to non-academic users across all sectors. The more flexible nature of the commissioning process in NCRM's next stage will enable this work programme to respond strategically to emerging need and to reflect the concerns of these non-academic user groups. We will involve key non-academic stakeholders and gate-keepers representing the different sectors in our consultations, assessments and reviews to ensure that the commissioned projects produce outcomes which are of clear benefit to them. Non-academic users will also occupy strategically important roles on the Centre's new Advisory Committees.

Third, the substantial programme of face-to-face short courses and seminars that the Centre will run on an annual basis will enable non-academic researchers to attend high quality training at affordable rates and in accessible locations throughout the UK. NCRM's existing assessments of training needs show that non-academic researchers have particular difficulty in securing time to attend face-to-face training events. We therefore anticipate the Centre's expanded online training resources to be of particular benefit to these groups.

Finally, our research and training programmes will have a positive impact on the general public, albeit indirectly and in the longer term, through the delivery of higher quality research evidence to inform public policy.

Publications


10 25 50
publication icon
Brunton-Smith I (2017) Detecting and understanding interviewer effects on survey data by using a cross-classified mixed effects location-scale model in Journal of the Royal Statistical Society: Series A (Statistics in Society)


publication icon
Cernat A (2016) Estimation of Mode Effects in the Health and Retirement Study Using Measurement Models in Journal of Survey Statistics and Methodology

publication icon
Chandola T (2017) Retirement and Socioeconomic Differences in Diurnal Cortisol: Longitudinal Evidence From a Cohort of British Civil Servants. in The journals of gerontology. Series B, Psychological sciences and social sciences





publication icon
De Waal T (2017) Calibrated Hot Deck Imputation for Numerical Data Under Edit Restrictions in Journal of Survey Statistics and Methodology


 
Description NCRM has undertaken a strategic review of methodological research needs in UK social science. This informed the ESRC's research strategy in research methods for the period 2016-2019. We also undertook an assessment of methodological training need which has informed the Centre's training strategy and fed into ESRC's Doctoral Training strategy.

In addition to strategic reviews and assessments, the NCRM research programme comprises 7 inter-related Work Packages. The majority have been running for a little over one year, so are still at a relatively early stage. Nonetheless, a number of findings have already been produced.

We have undertaken methodological research in the area of survey methodology which has developed a new method for studying the effects of interviewers on data quality. As part of this research we have developed new metrics for interviewer performance assessment and are in discussions with a leading data collection agency about how they might implement the new methods to monitor and improve interviewer performance. Research has also been undertaken into cost-effectiveness and accuracy of different data collection for surveys. Surveys are immensely important for government, academic, commercial and voluntary sector organisations and this research will help to improve the way this type of research is done and to also reduce the substantial cost of data collection. The research has been studying interviewer calling patterns and developing new ways of monitoring performance on large scale interviewer surveys. We have been working with a private sector partner to ensure that the work has relevance to actual practice.

The UK has been at the cutting edge of recent developments in qualitative longitudinal research (QLR). The goal of QLR is to enable the exploration of complex processes, context and trajectories of social change and continuity. NCRM research is extending and developing secondary analytic practice in working with complex longitudinal data drawing on material from the ESRC Timescapes study. In our work so far, we have developed of analytic strategy using an archaeological metaphor to capture the layered 'scaling down' process of working across multiple sets of qualitative longitudinal data and their waves of data collection. This has involved 1. 'Surface survey' (mapping the scope and nature of the data) 2. A geophysical survey (corpus analysis) 3. Shovel test pits (case analysis across and within the data sets). We are now moving onto a 4th stage - Deep excavations. Using the outcomes of our 'surveys' and 'test pits', this has involved a narrowing of our substantive focus to 'arguments' and the development of an associated keyword list. To facilitate our 'geophysical surveys' (see above) we have utilised a range of computerised forms of analysis, including Nvivo, Leximancer and 'R'. To support our work we have completed training in using the CAQDAS package Leximancer, uploading of data set and initial analyses.

Data Stewardship Organisations (DSOs) are charged with producing useful data products whilst minimising the risk of disclosure of personal information. Work in this area has traditionally focused on the disclosure risk associated with census and survey data products in the context of other information that an 'intruder' might use to identify persons and households. NCRM research addresses how such disclosure risks must be reconsidered in the rapidly changing linked data environment, in which extensive geographically and temporally referenced data about persons, addresses and households can be combined in analytically powerful ways. Work to date has shown, inter alia, that the classical Fellegi-Sunter linkage approach could be extended to cases where there are no key variables. Linkage was performed using Bayes factors generated from a full probability model fitted from labelled training data. However, the method was only shown to be effective (but much more effective than existing approaches) for 1 to 1 matching scenarios with low numbers of record pairs.

A further strand of NCRM research addresses the problem that provision of courses in advanced social science research methods is unlikely to ensure that capacity is developed in areas of strategic need without attention to the quality of the pedagogy associated with such courses. Hence our aim is to provide an evidence base in which the call for 'more training' can be translated into knowing the kinds of training that are suited to the particular challenge of advanced methodological learning. We have carried out research on the pedagogy of methodological learning, involving learners and teachers of research methods to understand ways of learning and whether and how these are effective. We find that teachers of methods are developing pedagogic content knowledge that is specific to advanced method teaching. This is characterised by active learning that engages learners in the methods at hand, Learning by Doing to give learners experience of research practice, and reflexivity to help learners understand the different ways in which research problems can be engaged with. Methods teachers teach both with and through data, and respond to the challenges of diverse learners and dynamic and difficult content.
Exploitation Route Our findings have already been used to inform policy for the ESRC. Additionally, our work in the area of survey methodology can be used by survey data collection agencies to monitor and improve interviewer performance and in the methods used by UK pollsters to carry out and report on opinion polls.
Sectors Communities and Social Services/Policy,Digital/Communication/Information Technologies (including Software),Education,Energy,Environment,Government, Democracy and Justice
URL http://www.ncrm.ac.uk/research/
 
Description 9th International Conference on Social Science Methodology 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact tbc
Year(s) Of Engagement Activity 2016
URL http://www2.le.ac.uk/departments/sociology/research/rc33-conference
 
Description @NCRMUK Twitter 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The @NCRMUK Twitter account has 7,753 followers in January 2017 and receives on average 1.8k impressions per day. The average engagement ranges from 0-40 Twitter users.
NCRM posts to Twitter automatically every time a new course is online. The NCRM Communication Manager also posts highlights and special advertisements to Twitter 3-5 times a week. We also ask our stakeholders to retweet our posts or tweet for us.
Year(s) Of Engagement Activity 2015,2016,2017
URL https://twitter.com/NCRMUK?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor
 
Description Autumn School on the use of biomarkers in social science research 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact . Aug 31st PM Introduction: Tarani Chandola (NCRM, Manchester) and Meena Kumari (ISER, Essex)

. Aug 31st PM Session 1: Measuring and analysing social processes, biological processes and their interactions (Noriko Cable, Jessica Abell, Meena Kumari, Scott Montgomery, Mel Bartley and David Blane: ICLS, UCL https://www.ucl.ac.uk/icls)

The session explores methodological issues around the measurement and analysis of social exposures, biological processes and their interactions in longitudinal survey datasets. Topics covered include: the biological significance of 'standard' biomarkers that are available in many survey datasets, how they are measured and incorporated in analysis, and how such biomarkers are theoretically and conceptually linked to social processes measured in surveys.

. Sept 1st AM Session 2: Developing social-biological models (Noriko Cable, Jessica Abell, Meena Kumari, Scott Montgomery, Mel Bartley and David Blane: ICLS, UCL https://www.ucl.ac.uk/icls)

The focus of this session was on supporting the autumn school participants to develop their own models regarding social-biological interactions and processes. The participants worked in groups, supported by input from researchers from ICLS, presented their social-biological models to the class, and received feedback.

. Sept 1st PM Session 3: Using 'Omics Data in the social sciences (Adam Stevens, Kris Mekli, Neil Pendleton and Nicholas Rattray, fRAILL Manchesterhttp://www.micra.manchester.ac.uk/research/fraill/)

This session focused on the "new" biomarker data that are increasingly available in longitudinal surveys, with presentations from members of the "Frailty, Resilience And Inequality in Later Life research team" (fRAILL) at Manchester. These was introductory sessions on Genomics (Adam Stevens and Kris Mekli), Epigenetics (Neil Pendleton), and Metabolomics (Nik Rattray) with particular consideration of how such data can be incorporated in social science analyses. These introductory talks were followed by an open discussion on the challenges of analysing 'omics data in the social sciences.

. Sept 1st Evening Keynote speaker: Professor Scott Montgomery, Orebro University http://www.oru.se/English/Employee/scott_montgomery/

. Sept 2nd AM Session 4: Statistical challenges of measuring longitudinal and lifecourse biomarker data (Natalie Shlomo, Joe Sakshaug, Alex Cernat, NCRM Manchester http://www.ncrm.ac.uk/research/WP3/wp3.php)

This session featured initial findings from the NCRM research project on missing biomarker data in longitudinal surveys. There will be a description of the different patterns of missing biomarker data and comparison with other types of missing data in surveys (Alex Cernat). This will be followed by an introduction to methods for compensating for missing data under different missing data mechanism assumptions and methods for compensating for missing data (Joe Sakshaug and Natalie Shlomo). The session included a computer based practical session.
Year(s) Of Engagement Activity 2016
 
Description ESRC PGR Conference 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact We promoted NCRM activities to postgraduate students - from courses, online resources, our Autumn School and networking.
Year(s) Of Engagement Activity 2016
URL http://www.nwdtc.ac.uk/conference-2016/
 
Description ESRC Research Methods Festival 2016 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Work Package 1
• Sturgis, P. (2016) Fieldwork effort, response rate and the distribution of survey outcomes: a multi-level meta-analysis.
• Durrant, G.B. (2016) Assessing risks of nonresponse bias during survey data collection: evidence from the UK Census nonresponse link study.
• Maslovskaya, O., Durrant, G.B., Smith, P.W.F. (2016) Predicting response and call sequence length using paradata in a longitudinal survey.

Work Package 2
. Davidson, E. and Weller, S. (2016) Working across multiple qualitative longitudinal studies: lessons from a feasibility study looking at care and intimacy: Strategies for Working with Combined Qualitative Longitudinal Data Sets session.

Work Package 3
. Tampubolon, G. (2016) Inflammation, allostatic load in cognitive ageing and mental health.
. Cernat, A. (2016) Missing data in bio-social research: issues, practice and recommendations.

Work Package 4
. Smith, D. and Elliot, M. J. (2016) "How to de-blend a data soup" presented at the NCRM RMF 2016 Session - The New Data Environment: Opportunities and Challenges.
. Robard,s J.; Martin, D. and Gale, C. "Engineering geoprivacy using automated zone design" presented at the NCRM RMF 2016 Session - The New Data Environment: Opportunities and Challenges.

Work Package 5
. Collins, D. (2016) Research methods pedagogy in the digital era. Learning Social Science Research Methods: Let's Talk Pedagogy Session.
Year(s) Of Engagement Activity 2016
URL http://www.ncrm.ac.uk/RMF2016/home.php
 
Description Innovation Forum: BIG QUALIDATA - Tackling Analysis of Very Large Volumes of Qualitative Data in Social Science Research 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact BIG QUALIDATA: Tackling Analysis of Very Large Volumes of Qualitative Data in Social Science Research
9 May 2016, University of Edinburgh (with audiences at University of Manchester and University of Southampton via video-link)
Chaired by Lynn Jamieson with guest presenters Professor Ken Benoit (LSE), Dr Elena Zaitseva (Liverpool John Moores University) and Professor Wendy Olsen (Manchester University)

This was also a useful experiment in utilising technology to link audiences in different locations.
Year(s) Of Engagement Activity 2016
 
Description Media coverage in connection with content from the 2016 Research Methods Festival 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Research undertaken by the NCRM and the 2016 NCRM Research Methods Festival were the subject of or mentioned in the following media coverage:

Politics.co.uk - The real reason the pollsters got the general election so wrong
The Conversation - Can you trust the EU referendum polls?
Guardian - Scientists aren't superheroes - failure is a valid result
Social Care Guardian - How can care homes ensure a good death for residents?
LSE Impact Blog
- Visualising data in 3D: Handling complexity through visceral and tactile experiences of data
- Modelling engagement: using theatre based workshops to explore citizenship and research participation
Year(s) Of Engagement Activity 2016
 
Description Media coverage in connection with the British Polling Council / Market Research Society Polling Inquiry, Chaired by Professor Patrick Sturgis 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Professor Patrick Sturgis chaired the British Polling Council / Market Research Society Inquiry into the 2015 British General Election Opinion Polls.

The first press release communicated the preliminary results and was picked up by more than 250 UK and international media outlets including local printed and online publications, TV and radio (e.g. Liverpool Echo, Jersey Evening Post, Sky News Sunrise, Wave 105, Bloomberg News Online and Reuters UK) and national media (BBC News Online, BBC Radio 4, BBC Radio 5 Live, The Guardian, The Times and The Independent). The second press release communicated the final results and the publication of the final report and was picked up by similar media organizations but in much smaller numbers (about 20 outlets).
Year(s) Of Engagement Activity 2016
URL http://eprints.ncrm.ac.uk/3789/
 
Description NCRM Explore App 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact In March 2017, NCRM launched the NCRM Explore app which allows users to receive personalised research methods' content which best matches their research interests. It includes all up to date research methods' publications, videos, podcasts and training courses in a searchable and customisable interface. It also contains the latest tweets from NCRM. This app is for iPhones, iPads and Android phones and tablets.
Year(s) Of Engagement Activity 2017
URL https://www.ncrm.ac.uk/app/
 
Description NCRM Facebook 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The NCRM Facebook account was set up in the summer of 2015. We now have 512 followers and a reach between 1- 2.6k. The post popular likes in 2016 were pictures from the Research Methods Festival at the University of Bath, followed by anything about big data, funding opportunities and polls.
Year(s) Of Engagement Activity 2015,2016,2017
URL https://en-gb.facebook.com/ncrmuk/
 
Description NCRM Innovation Forum: Crowd-sourcing the Future? 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Citizen Social Science Methods, Data and Challenges

Digital data is now generated in huge volumes by many citizens and is documenting peoples lives in detail. Moreover citizens are increasingly generating data as part of a sense of their own need to respond and report including during humanitarian crises. These approaches link to debates about civic engagement and the observant digital citizen. Digital citizens are becoming the eyes and ears of the world and the citizen response is becoming part of the tools to tackle social problems and political challenges previously left to the state. There are new opportunities to address intractable social issues but also increasing numbers of voices and truth claims.

In this interactive one-day conference we will explore the future of citizen science and citizen social science methods. We will examine the different types of applications, methods, the data and the challenges posed including in terms of the debates about expertise, divisions of labour, different ways of seeing, data quality, questions about what might still be going undocumented and the ethical issues raised. We will also consider the ways in which more people can be encouraged to participate as citizen scientists.

The conference will include talks by leading academics and practitioners involved in using citizen science and citizen social science methods. There will be opportunities to try out citizen science tools and to contribute to the debates.

Outline Schedule

11am - Introductions and citizen science project demonstrations.

11.15am - Methods, Data, Ethics - Professor Chris Lintott (Zooniverse/University of Oxford) and Professor Muki Haklay (University College London).

12.15pm - Practitioner Short Talks 1 - Methods, Data and Ethics. Including: Ben Rich (BBC), Hilary Geoghegan (Reading), Will Dixon (Manchester), K. Purdam (Manchester).

1pm - Lunch and more project demonstrations.

1.30pm - Practitioner Short Talks 2 - Methods, Data and Ethics. Including: Erinma Ochu (Salford), Monika Buscher (Lancaster), Alex Albert (Manchester), Liz Richardson (Manchester).

2.30pm - More Participation? - Redesigning Research and the Policy Interface - Professor Muki Haklay (University College London).

3pm - 3.45pm Round Table Discussion - Ground Up Data and Crowd Sourced Applications.
Year(s) Of Engagement Activity 2016
 
Description NCRM Innovation Forum: Streaming into the future: An agenda setting workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact As the data world increases in its complexity with gathering pace, data that is linked through time will become an increasing focus for the social and behavioural sciences. These data come in many different forms and granularities from the traditional longitudinal data provided by the UK's world envied cohort studies through user data generated social media data to tracking data generated by mobiles and live streams from sensors.

This workshop will bring together social and data scientists who work with these data with the aim exchanging ideas and develop the agenda for the social and behavioural sciences. In plenary we will see presentations on cutting edge methods and new research questions made possible by these data and in break out groups we will draw out the key themes and future methodological research agenda.

The intended output of the workshop is a position paper for submission to a top journal. All workshop attendees will be invited to co-author that paper, which is why we are inviting applications to attend this event. Applications are made via the registration link below.

The workshop will run from 13:00 on 19th January to 14:00 on the 20th January. There will be a workshop dinner on the evening of the 19th.

Confirmed speakers are:

David de Roure, Oxford University
Mark Birkin, Leeds University
Jonathon Bright, Oxford University
Idris Eckley, Lancaster University
Mark Elliot, University of Manchester
Niklas Loynes, University of Manchester
Suzy Moat, Warwick University
Michelle Morris, Leeds University
Alex Singleton, Liverpool University
Year(s) Of Engagement Activity 2017
 
Description NCRM LinkedIn 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact We now have 1,218 followers on LinkedIn which is an increase of just under 200 compared to February 2016.
Year(s) Of Engagement Activity 2015,2016,2017
URL https://www.linkedin.com/company/esrc-national-centre-for-research-methods
 
Description NCRM YouTube 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact We now have 333 subscribers (68 in February 2016) and a total of 521 videos in the NCRM playlist. The videos available have been viewed 26,563 in 2016 (likes 194, dislikes 5, comments 16, shares 1,188)

The most popular videos in 2016 were:
- What is the regression discontinuity approach? (Mike Brewer): 2,463 views, 17 likes
- What is sensory ethnography? (Sarah Pink): 2,171 views, 8 likes
- Structural equation modeling: what is it and what can we use it for (Patrick Sturgis): 2,092 views, 20 likes
- Crimes against data (Andrew Gelman): 2,081 views, 29 likes
- What are qualitative research ethics? (Rose Wiles): 1,661 views, 8 likes
Year(s) Of Engagement Activity 2016,2017
URL https://www.youtube.com/user/NCRMUK/videos
 
Description NCRM online learning resources 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact During the reporting period, NCRM commissioned a new series of online learning resources. These six resources complement other already existing NCRM online resources such as the NCRM podcast series, NCRM working paper series, NCRM videos, the ReStore repository and a collection of external online resources. Existing material will be expanded using practical material (e.g. computer workshops and exercises), or interactive material. Details and a link to each of the six new online resources are available via the link below www.ncrm.ac.uk/resources/online
Year(s) Of Engagement Activity 2015,2016,2017
URL http://www.ncrm.ac.uk/resources/online/
 
Description NCRM website 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact NCRM's web presence is spread across several services. We monitor these through Google Analytics, AwStats and also through a combination of our own registration statistics and server log files. In 2016, we recorded 163,002 visits to the NCRM website which is almost 30,000 more than in 2015. In this reporting period, we have also increased the page views (509,200 from 443,730 in 2015). A total of 112,914 users visited the NCRM website - a breakdown by demographics is as follows:
59.67% UK; 13.79% US; 2.11% India, Russia 1.87%, Australia 1.43%, Germany 1.35%, Canada 1.25%, less than 1% Netherlands, Ireland, Italy.
Year(s) Of Engagement Activity 2015,2016,2017
URL https://www.ncrm.ac.uk/
 
Description Research Event - Maintaining high response rates - is it worth the effort? (London) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Durrant, G.B., Moore, J., and Smith, P.W.F. (2016) Assessing dataset representativeness during survey data collection - Evidence from the 2011 Census Non-Response Link Study, session on Maintaining high response rates - is it worth the effort?

Demand for places at this research methods seminar was high and it was fully booked (125 attendees). The event successfully attracted many attendees from the target audience (survey practice) and 78% of attendees were non-academics.
Year(s) Of Engagement Activity 2016
URL http://eprints.ncrm.ac.uk/3786/1/Maintaining%20high%20response%20rates%20%E2%80%93%20is%20it%20worth...
 
Description Research Event: Opinion Polling in the EU Referendum: Challenges and Lessons 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Research Event on 8th December 2016: Opinion Polling in the EU Referendum: Challenges and Lessons
Organised jointly by NCRM, British Polling Council and Market Research Society and held at the Royal Statistical Society, London.

The referendum on Britain's membership of the European Union presented the opinion polls with a formidable challenge. Support for Remain and Leave crossed party lines. Rather than a debate between left and right, opinions reflected a division between social liberals and social conservatives. Even if the polls had overcome the difficulties that had beset them in 2015 - and that appeared to be still a work in progress - there was no guarantee that methods that had been honed to estimate party support in a general election would work effectively in this seemingly very different environment.

As a result, the campaign was marked by a lively debate about polling methodology, and significant methodological experimentation and adaptation by polling companies. In the event the final polls correctly indicated that the result would be close, but for the most part, incorrectly indicated that Remain would be the most likely winner.

With speakers from the polling companies and members of the BPC/MRS inquiry into the performance of the polls in the 2015 election, this seminar features presentations of how the polling companies set about their task and independent evaluations of the methodology that they used. Its aim is to identify the key lessons to be learned from the referendum for the future of opinion polling.

• John Curtice: Polling in the EU Referendum: an overview
• Ben Page: The challenges of polling by phone in the EU Referendum
• Adam Drummond: The challenges of polling via the internet in the EU Referendum
• Patrick Sturgis: Sampling and mode of interview
• Stephen Fisher: Treatment of don't knows and turnout weighting
• Will Jennings: The effect of methodological adjustments during the campaign
Year(s) Of Engagement Activity 2014,2016
URL https://www.ncrm.ac.uk/polling/eu.php
 
Description Research Methods Seminar: Approaches to analysing qualitative data: Archaeology as a metaphor for method 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact "How can we 'dig down' and get an analytic grip when working with large and complex bodies of qualitative data? The metaphor of archaeology enables qualitative analysts to think about what lies 'underneath' the corpus of material being analysed, working extensively and intensively to identify and excavate meaning."

In this seminar, researchers working with different bodies of qualitative materials discussed how they approached their analysis, from a range of methodological perspectives.

The speakers were:

Professor Emeritus Clive Seale (Brunel University)
An archaeological approach working with keyword analysis of a large corpus of qualitative data

Dr. Emma Davidson (University of Edinburgh)
A layered archaeological approach to analysis across multiple sets of qualitative longitudinal data

Professor Maria Tamboukou (University of East London)
Archaeology of knowledge and working in the archives

32 people attended the seminar which was generally very well received. Of the 22 participants who responded to an evaluation questionnaire, 86% reported that the seminar had influenced their approach and the techniques discussed would be useful in their work.
Year(s) Of Engagement Activity 2016
URL http://bigqlr.ncrm.ac.uk/
 
Description Research Methods Seminar: Teaching Research Methods: Building on Research to Develop Expertise 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact NCRM invited teachers of advanced social science research methods to discuss findings from our new pedagogic research through workshops, networking, panel discussion and keynotes.

PROGRAMME

11.00 Welcome and Speakers

• John MacInnes, Professor of Sociology, Associate Dean Quantitative Methods, University of Edinburgh.
John has played a key role as Strategic Advisor to the ESRC on QM Training and leads the QM Pedagogy research in the National Centre for Research Methods. He will be talking about teaching quantitative methods in the face of statistics anxiety.

• Melanie Nind, Professor of Education, University of Southampton, Co-Editor International Journal of Research & Method in Education.
As a Co-Director within NCRM Melanie is working to enhance the ways in which methods training can be enhanced by being research-informed. She leads the NCRM Pedagogy of Methodological Learning study and will be talking about the pedagogic knowledge methods teachers/trainers have and use.

• Richard Rogers, Professor of New Media & Digital Culture, University of Amsterdam.
Richard is Director of the Govcom.org Foundation (Amsterdam) and the Digital Methods Initiative. He had been an expert contributor to the NCRM pedagogical research and will speak on the topic of training and capacity building and digital methods.

12.15 Workshops designed to create a learning community collaborating pooling expertise to make best use of research evidence alongside their professional knowledge.

Workshop 1: Responding to the statistical anxiety of methods learners and teachers
Workshop 2: Identifying key features of a good pedagogic resource for methods teachers & learners

13.15 Lunch

14.00 Workshops

Workshop 3: Making constructive use of an emergent typology of methods pedagogy
Workshop 4: Using digital technology to enhance the teaching of research methods

15.00 Panel including John MacInnes, Richard Rogers, Sarah Lewthwaite (Research Fellow, NCRM), Gabi Durrant (Deputy Director, NCRM), Louise Corti (Associate Director, UK Data Service) and Howard Davis (Co-Director, WISERD).
The panel will be answering your questions and focusing on the changing landscape of research methods and methods training, teaching and learning.

15.45 Close
Year(s) Of Engagement Activity 2016
URL http://pedagogy.ncrm.ac.uk/
 
Description The Working Across Qualitative Longitudinal Studies: A Feasibility Study Looking at Care and Intimacy website 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact The Working Across Qualitative Longitudinal Studies: A Feasibility Study Looking at Care and Intimacy website has been visited circa 4,000 times (at end February 2017) and has 142 subscribers. The website features, amongst other content, a series of team blogs and guest blogs:

Research team blog # 6: Getting out of the swamp
Research team blog # 5: Time in Timescapes
Research team blog # 4: Approaches to Analysing Qualitative Data: Archaeology as a Metaphor for Method, 18th October 2016
Research team blog # 3: Case Histories in Qualitative Longitudinal Research, 6th & 7th October 2016
Research team blog # 2: NCRM Research Festival
Research team blog # 1: Getting started

Guest blog # 8: Dr Sarah Wilson: Using qualitative secondary analysis as a tool of critical reflexivity
Guest blog # 7, Dr Gregor Wiedemann: Computer-assisted text analysis beyond words
Guest blog #6, Nick Emmel: Revisiting yesterday's data today
Guest blog #5, Sue Bellass: The challenges of multiple perspectival QL analysis
Guest blog #4, Libby Bishop: Data from the past and for the future - Qualitative longitudinal data available at the UK Data Service
Guest blog #3, Prof Rachel Thomson: Case histories in QLR
Guest blog #2: Dr Fiona Shirani: Visual approaches in QLR
Guest blog #1, Dr Anna Tarrant: Reflections from the Men, Poverty and Lifetimes of Care study
Year(s) Of Engagement Activity 2015,2016,2017
URL http://bigqlr.ncrm.ac.uk/