Methodology

The quantitative research carried out as part of this project included five surveys run online with a UK sample of adults aged 18 and over. Public First is a member of the British Polling Council (BPC) and a company partner of the Market Research Society (MRS) and abides by their rules. 

Survey design

Survey questionnaires went through several rounds of iteration. Quantitative opinion data is always subject to a number of possible biases, most importantly a tendency for participants to agree with statements or “satisfice”. To counteract this, we design the survey to vary the types of questions shown, and incorporate binary choice scales and multi-select questions as well as agree-disagree likert-style questions.

Reaching the sample

Participants were recruited through online proprietary panel providers, where individuals register their interest in taking part in online research in return for incentives. By their nature, online samples exclude those with low levels of digital access, however they enable strong control over the demographic make-up of the final sample. The sample is targeted to be representative of the UK public, along interlocked age and gender, regional and socio-economic grade lines. All surveys are subject to some level of recruitment bias, where those who are interested in taking part may represent a more “engaged” group of people than if the selection was truly random. By reaching the sample through online panels, incentives can be tailored to meet specific groups to encourage the participation of those who are less likely to volunteer.

The survey also features attentional checks for inattentive responding, both in the form of trap questions where participants are instructed to select a specific answer option in order to proceed, and open-response questions which are retrospectively assessed for “nonsense” responding. All surveys also feature “reCAPTCHA” checks for bots, to prevent surveys being flooded by scripted responses, with a second check for such problems during analysis of any open responses.

Analysis

Our section – Download the Data – provides access to the full data tables, broken down by demographic groups. In general, sample sizes of less than 100 are considered too small for statistical analysis, and are avoided throughout our analysis. For a sample of over 8,000 the margin of error is +/- 1%, meaning we can be 95% confident that the real proportion lies within 1% of the reported statistic at the top level.

When samples are split, either through different message testing, or crossbreaks, this margin of error will increase. As a rough guide, the margin of error for different samples sizes are:

• • 4,000 sample size: 2%
  • 2,000 sample size: 2%
  • 1,000 sample size: 3%
  • 500 sample size: 4%
  • 100 sample size: 10%

After the General Election was called for July 2024, CaSE ran a second, larger poll, to complete its public opinion research to support the R&D sector. This poll explored:

• Aspirational R&D breakthroughs and attitudes to when, and in which countries, they might be achieved
• Priorities for R&D and R&D-related actions that the next Government could take
• Political leaders as messengers about R&D investment linked to economic growth and jobs
• Changing attitudes to local actions, R&D investment and R&D’s perceived benefits
• Attitudes to UK universities and international students and researchers

Public First polled a total of 6,111 UK adults. This included 2,011 on 29 March – 2 April and 4,100 from 7-9 June 2024. Public First worked with Electoral Calculus to produce a Multi-Level Regression with Poststratification (MRP) analysis of these results, modelling the expected attitudes at a constituency level. Using the demographic information provided by the participants, multi-level regression models are fit to assess the relationship between demographics and views on R&D in 13 questions. This enables us to make predictions as to how individuals who fill specific demographic categories (e.g. male, 18-24, white British…) would respond to each question. These predictions are then “post-stratified” – using known proportions of people in these demographic categories in each Westminster constituency so that we can predict the likely outcome of asking the question in that local area.

This methodology means that hyper-local context or factors cannot be taken into account. For example, a controversy surrounding a specific local candidate for Parliament would not be picked up by this process.

Please note that due to the size of sample in Northern Ireland and the unique political correlates, models do not include projections for these constituencies.

The qualitative research carried out as part of this project consisted of focus groups. This involves participants, selected to meet certain demographic or attitudinal criteria, participating in a facilitated discussion led by a moderator. 

These groups were held online. Since the pandemic, much qualitative research which would have traditionally been conducted in-person has migrated online. This offers increased accessibility for participants who no longer have to travel for the group, and allows participation to more easily fit around other commitments. We have also found it encourages more participants to express their views, as participants tend to take turns to speak more than they would in a physical setting. The challenges of an online setting are mainly infrequent technical difficulties, accessibility for those who are digitally excluded, and the risk of distractions within the home.

Focus groups are useful for understanding – in people’s own words – why they hold the views they do. However, the resulting data necessarily comes from a small sample size, so it is primarily conveyed via quotes rather than statistics. We avoid drawing conclusions about the “proportion” who hold views on the basis of qualitative work, and instead focus on exploring broad themes.

Recruitment

Recruitment is typically done on a geographical basis, focussing on a specific area such as a city, parliamentary constituency or broad region. This research involved groups in:

• Greater Manchester
• Wimbledon
• Mansfield
• Leeds
• Derby
• Blackpool
• Teesside
• Dundee
• North Wales

With these recruitment areas we aimed to cover a range of urban and rural areas, across the whole country, and areas where the R&D sector had differing levels of visibility (such as areas with Universities, R&D businesses, or neither).

Discussion Guides and running groups

Moderators who facilitate the focus groups follow a discussion guide, which establishes the key questions to be answered throughout the group, with ample space for discussion to deviate if new topics or discussion points arise.

For copies of the discussion guides used in our groups, please contact Rebecca Hill.

The role of the moderator is to ensure:

• Everyone in the group is given opportunity to express their views
• The conversation is not derailed by arguments between participants, or by individual participants dominating the conversation.
• The conversation remains on topic. This is particularly important when the discussion covers broad or abstract areas like R&D.
• That, where possible, relevant points made by participants are explored in more detail by prompting with follow-up questions. 

Our groups in Phase 1 made use of headlines about R&D to understand how participants felt, and spoke about, R&D carried out in different areas. In Phase 4, the majority of the groups made use of hypothetical research projects which were designed to indirectly represent key variables in R&D that might influence opinions (such as timelag, location) to understand how participants traded-off these variables

During Phase 4, we adjusted our discussion guides on the basis of each group. One of the main ways we adjusted the discussion guides was to shift from a discussion about R&D in the abstract towards discussing specific research project proposals. In early groups, participants would quickly latch onto any tangible examples of R&D which were given, and discuss these rather than the abstract benefits or costs of “investing in research”. 

Analysis

Analysis of qualitative data is normally carried out through detailed readings of the transcripts, identifying broad themes and language used by the participants, and looking for patterns in how participants respond to the specific questions. In our analysis we have aimed to demonstrate both where the broad findings of the quantitative research have been supported by qualitative data and where it has been challenged. We have aimed to present any quotations from the qualitative work with as much context as possible, to ensure these can be reliably interpreted.

The quantitative research carried out as part of this project included five surveys run online with a UK sample of adults aged 18 and over. Public First is a member of the British Polling Council (BPC) and a company partner of the Market Research Society (MRS) and abides by their rules. 

Survey design

Survey questionnaires went through several rounds of iteration. A key consideration for the survey design was the level of information which the survey respondent had at any specific point in the survey – for example, whether they had been given an official definition of terms such as “R&D”. Given the intention for the results of this work to be easy for other researchers to use, we made the decision to keep the order of key sections of the survey identical between respondents, so that it would be easy to interpret the level of information and “priming” which participants have before answering each question.

Quantitative opinion data is always subject to a number of possible biases, most importantly a tendency for participants to agree with statements or “satisfice”. To counteract this, we design the survey to vary the types of questions shown, and incorporate binary choice scales and multi-select questions as well as agree-disagree likert-style questions.

Our Phase 3 survey also had the objective of producing an attitudinal segmentation of the public, based on their views about R&D investment. We aimed to keep the questions for this segmentation focussed on fundamental drivers and motivations, such as people’s propensity to worry about the future, or a desire to take risks, rather than on specific political issues which R&D could play a role in. While we knew the latter would likely offer interesting splits, and as our analysis showed political issues were important to support for R&D, we felt there were existing political segmentations which would prove more direct in this regard (such as the More in Common political segmentation, which our Phase 3 sample included).

In Phase 3, the broad order of the survey was as follows:

Reaching the sample

Participants were recruited through online proprietary panel providers, where individuals register their interest in taking part in online research in return for incentives. By their nature, online samples exclude those with low levels of digital access, however they enable strong control over the demographic make-up of the final sample. The sample is targeted to be representative of the UK public, along interlocked age and gender, regional and socio-economic grade lines. All surveys are subject to some level of recruitment bias, where those who are interested in taking part may represent a more “engaged” group of people than if the selection was truly random. By reaching the sample through online panels, incentives can be tailored to meet specific groups to encourage the participation of those who are less likely to volunteer.

The survey also features attentional checks for inattentive responding, both in the form of trap questions where participants are instructed to select a specific answer option in order to proceed, and open-response questions which are retrospectively assessed for “nonsense” responding. All surveys also feature “reCAPTCHA” checks for bots, to prevent surveys being flooded by scripted responses, with a second check for such problems during analysis of any open responses.

Analysis

Our section – Download the Data – provides access to the full data tables for Phases 2 and 3, broken down by demographic groups. In general, sample sizes of less than 100 are considered too small for statistical analysis, and are avoided throughout our analysis. For a sample of over 8,000 the margin of error is +/- 1%, meaning we can be 95% confident that the real proportion lies within 1% of the reported statistic at the top level.

When samples are split, either through different message testing, or crossbreaks, this margin of error will increase. As a rough guide, the margin of error for different samples sizes are:

• • 4,000 sample size: 2%
  • 2,000 sample size: 2%
  • 1,000 sample size: 3%
  • 500 sample size: 4%
  • 100 sample size: 10%

Segmentation Analysis

See our Segmentation section for the results of our segmentation analysis.

The attitudinal segmentation carried out on the larger sample as part of this work was produced through a combination of factor analysis and K-proto clustering, which allows for the combination of continuous and categorical data. A number of segmentations were produced throughout the analysis, varying in: the number of “clusters” or “segments”, the technique used (which included K-means with and without prior factor analysis, Hierarchical Clustering), the questions involved in the segmentation and how the answer options were grouped together (e.g. whether “Strongly Agree” and “Agree” responses were considered identical).

The final segmentation framework was decided through qualitative analysis of the different segment outputs. There were some segments (namely the Ideologically Aligned, Ideologically Conflicted and Issue Driven) which were relatively consistently found across methodological approaches. The segmentation which split the remaining cluster of people into Future Focussed and Present Focussed was chosen because we felt it to be useful for understanding how different issues and focuses could appeal to this group who were typically inclined to be supportive R&D, but spanned political and ideological outlooks. An alternative approach could have involved participation in different R&D-related activities as part of the segmentation, although this was ultimately rejected as it mainly split out those who gave to charity, which we felt was not a granular enough distinction.

Attitudinal segmentations like this are by their nature noisy, with segments often including participants who do not fit the qualitative interpretation applied to each segment. Further, given that segmentations tend to be a way of identifying latent information in the data, it is always possible that future research which explores the same segments in more detail uncovers a novel way of understanding a group which is a better fit for the data. The outputs of this segmentation are intended to demonstrate one way in which the data could be interpreted and used, alongside more traditional demographic splits.

This survey set the baseline for our work and informed the design of the main quantitative phase.

• Understanding and awareness, including of R&D funders and actors
  • Example question: “As far as you are aware, what does R&D stand for?”, followed by a list of terms

• International competition, UK’s place in the world, including on R&D
  • Example question: “Do you agree or disagree: Britain is falling behind other countries on R&D”

• Benefits of R&D, including in addressing certain issues
  • Example question: “In general, do you think R&D has a role to play in addressing the… Cost of living”

• Attitudes to potential and realistic arguments for and against R&D

• Messengers: Four messages tested, each against four messengers

• Attitudes to R&D terminology, to understand views at survey end

• Demographic questions – some of which formed the basis for ensuring representative recruitment in line with the adult population of the UK.
• Preliminary ideological questions – these were used for the attitudinal segmentation, and explored issue such as the prioritisation of long-term problems, global issues, and risk-taking.
• Split-sample testing of different R&D terminology – this generated unprompted data on the words or phrases that participants felt best described the topic of R&D.
• Further ideological questions and awareness testing – these fed into the attitudinal segmentation, which came after a definition of R&D was provided to all participants. This included things like association of R&D with the person’s local area, belief that R&D benefits some more than others, and level of interest in new discoveries and inventions.
• Split-sample abstract message-testing – these questions examined how messages which focussed purely on “R&D” in general performed among the respondents. This involved each participant seeing one argument for increased investment in R&D, one for reduced investment, and then six of twelve messages for increased investment in R&D which were paired with opposite framings, with each one of a pair shown to half the sample (such as a frame focussed on the future, and a frame focussed on the past).
• Split-sample issues-based message testing – these questions placed an issue (e.g. healthcare, environment) at the centre, and tested the idea of pursuing that goal via R&D or non-R&D means. Issues were randomised between respondents, to avoid respondents comparing between issues and prioritising one over another.
• Split sample testing of messengers – these questions tested 4 separate messages and 8 messengers for each (with the exception of the political message, which had 4 messengers but 2 messages, one of which was “Science Superpower”). Participants were presented with a headline attributed to the messenger, and asked questions about their response to the headline.
• Messengers in general, and Britain as a “world leader” – these questions explored which messengers were considered trustworthy or biassed when talking about R&D investment, and gauged how much it mattered to people if the UK fell behind other countries. For the latter section, participants were split with half of the sample seeing the context that the UK currently invests less than France and Germany in R&D.

This polling was carried out ahead of launch to re-test key questions, draw in latest examples of Government R&D narratives, and test visual and written messaging in more detail.

• One poll carried out a deeper exploration of R&D messages, including messaging formed around ‘slogans’, sentences and paragraphs.
• The other poll explored reactions to political messaging on R&D, along with what visual elements worked well, tested in pairs, for instance using either bold or subtle imagery

This phase sought to further understand attitudes towards different terminologies to support the R&D sector’s advocacy. Topics included:

• Terminology, focusing on “Research & Development” vs “Research & Innovation”
  • Example question: “Looking at the following groups of people, do you think it makes more sense for them to talk about “Research & Development” or “Research & Innovation”, or is there not much difference?”

• Timeframes, exploring how attitudes to time lags differed depending on the subject
  • Example question: “Imagine the Government was planning to invest money into a new program of research [to achieve specific goal], which is expected to have results within [X] years. Which of the following comes closest to your view?”

• Political priorities, exploring public perspectives of importance of R&D to parties
  • Example question: “In your view how much of a priority should the political parties in the UK be putting in the following areas?”, followed by a list of topics, such as ‘funding research generally in the UK’ and ‘helping the NHS’

• Support for local research, and reasons for support or opposition
  • Example question: “Which of the following, if any, do you think of advantages to your local area of having more Research & Innovation carried out in your area?”

• Attitudes to investment, education, skills and R&D messengers