Methodology

The qualitative research carried out as part of this project consisted of focus groups. This involves participants, selected to meet certain demographic or attitudinal criteria, participating in a facilitated discussion led by a moderator. 

These groups were held online. Since the pandemic, much qualitative research which would have traditionally been conducted in-person has migrated online. This offers increased accessibility for participants who no longer have to travel for the group, and allows participation to more easily fit around other commitments. We have also found it encourages more participants to express their views, as participants tend to take turns to speak more than they would in a physical setting. The challenges of an online setting are mainly infrequent technical difficulties, accessibility for those who are digitally excluded, and the risk of distractions within the home.

Focus groups are useful for understanding – in people’s own words – why they hold the views they do. However, the resulting data necessarily comes from a small sample size, so it is primarily conveyed via quotes rather than statistics. We avoid drawing conclusions about the “proportion” who hold views on the basis of qualitative work, and instead focus on exploring broad themes.

Recruitment

Recruitment is typically done on a geographical basis, focussing on a specific area such as a city, parliamentary constituency or broad region. This research involved groups in:

• Greater Manchester
• Wimbledon
• Mansfield
• Leeds
• Derby
• Blackpool
• Teesside
• Dundee
• North Wales

With these recruitment areas we aimed to cover a range of urban and rural areas, across the whole country, and areas where the R&D sector had differing levels of visibility (such as areas with Universities, R&D businesses, or neither).

Discussion Guides and running groups

Moderators who facilitate the focus groups follow a discussion guide, which establishes the key questions to be answered throughout the group, with ample space for discussion to deviate if new topics or discussion points arise.

For copies of the discussion guides used in our groups, please contact Rebecca Hill.

The role of the moderator is to ensure:

• Everyone in the group is given opportunity to express their views
• The conversation is not derailed by arguments between participants, or by individual participants dominating the conversation.
• The conversation remains on topic. This is particularly important when the discussion covers broad or abstract areas like R&D.
• That, where possible, relevant points made by participants are explored in more detail by prompting with follow-up questions. 

Our groups in Phase 1 made use of headlines about R&D to understand how participants felt, and spoke about, R&D carried out in different areas. In Phase 4, the majority of the groups made use of hypothetical research projects which were designed to indirectly represent key variables in R&D that might influence opinions (such as timelag, location) to understand how participants traded-off these variables

During Phase 4, we adjusted our discussion guides on the basis of each group. One of the main ways we adjusted the discussion guides was to shift from a discussion about R&D in the abstract towards discussing specific research project proposals. In early groups, participants would quickly latch onto any tangible examples of R&D which were given, and discuss these rather than the abstract benefits or costs of “investing in research”. 

Analysis

Analysis of qualitative data is normally carried out through detailed readings of the transcripts, identifying broad themes and language used by the participants, and looking for patterns in how participants respond to the specific questions. In our analysis we have aimed to demonstrate both where the broad findings of the quantitative research have been supported by qualitative data and where it has been challenged. We have aimed to present any quotations from the qualitative work with as much context as possible, to ensure these can be reliably interpreted.

The quantitative research carried out as part of this project included two surveys run online. Both were run with a UK sample of adults aged 18 and over. Public First is a member of the British Polling Council (BPC) and a company partner of the Market Research Society (MRS) and abides by their rules. 

Survey design

Survey questionnaires went through several rounds of iteration. A key consideration for the survey design was the level of information which the survey respondent had at any specific point in the survey – for example, whether they had been given an official definition of terms such as “R&D”. Given the intention for the results of this work to be easy for other researchers to use, we made the decision to keep the order of key sections of the survey identical between respondents, so that it would be easy to interpret the level of information and “priming” which participants have before answering each question.

Quantitative opinion data is always subject to a number of possible biases, most importantly a tendency for participants to agree with statements or “satisfice”. To counteract this, we design the survey to vary the types of questions shown, and incorporate binary choice scales and multi-select questions as well as agree-disagree likert-style questions.

Our Phase 3 survey also had the objective of producing an attitudinal segmentation of the public, based on their views about R&D investment. We aimed to keep the questions for this segmentation focussed on fundamental drivers and motivations, such as people’s propensity to worry about the future, or a desire to take risks, rather than on specific political issues which R&D could play a role in. While we knew the latter would likely offer interesting splits, and as our analysis showed political issues were important to support for R&D, we felt there were existing political segmentations which would prove more direct in this regard (such as the More in Common political segmentation, which our Phase 3 sample included).

In Phase 3, the broad order of the survey was as follows:

• Demographic questions – some of which formed the basis for ensuring representative recruitment in line with the adult population of the UK.
• Preliminary ideological questions – these were used for the attitudinal segmentation, and explored issue such as the prioritisation of long-term problems, global issues, and risk-taking.
• Split-sample testing of different R&D terminology – this generated unprompted data on the words or phrases that participants felt best described the topic of R&D.
• Further ideological questions and awareness testing – these fed into the attitudinal segmentation, which came after a definition of R&D was provided to all participants. This included things like association of R&D with the person’s local area, belief that R&D benefits some more than others, and level of interest in new discoveries and inventions.
• Split-sample abstract message-testing – these questions examined how messages which focussed purely on “R&D” in general performed among the respondents. This involved each participant seeing one argument for increased investment in R&D, one for reduced investment, and then six of twelve messages for increased investment in R&D which were paired with opposite framings, with each one of a pair shown to half the sample (such as a frame focussed on the future, and a frame focussed on the past).
• Split-sample issues-based message testing – these questions placed an issue (e.g. healthcare, environment) at the centre, and tested the idea of pursuing that goal via R&D or non-R&D means. Issues were randomised between respondents, to avoid respondents comparing between issues and prioritising one over another.
• Split sample testing of messengers – these questions tested 4 separate messages and 8 messengers for each (with the exception of the political message, which had 4 messengers but 2 messages, one of which was “Science Superpower”). Participants were presented with a headline attributed to the messenger, and asked questions about their response to the headline.
• Messengers in general, and Britain as a “world leader” – these questions explored which messengers were considered trustworthy or biassed when talking about R&D investment, and gauged how much it mattered to people if the UK fell behind other countries. For the latter section, participants were split with half of the sample seeing the context that the UK currently invests less than France and Germany in R&D.

Reaching the sample

Participants were recruited through online proprietary panel providers, where individuals register their interest in taking part in online research in return for incentives. By their nature, online samples exclude those with low levels of digital access, however they enable strong control over the demographic make-up of the final sample. The sample is targeted to be representative of the UK public, along interlocked age and gender, regional and socio-economic grade lines. All surveys are subject to some level of recruitment bias, where those who are interested in taking part may represent a more “engaged” group of people than if the selection was truly random. By reaching the sample through online panels, incentives can be tailored to meet specific groups to encourage the participation of those who are less likely to volunteer.

The survey also features attentional checks for inattentive responding, both in the form of trap questions where participants are instructed to select a specific answer option in order to proceed, and open-response questions which are retrospectively assessed for “nonsense” responding. All surveys also feature “reCAPTCHA” checks for bots, to prevent surveys being flooded by scripted responses, with a second check for such problems during analysis of any open responses.

Analysis

Our section – Download the Data – provides access to the full data tables for Phases 2 and 3, broken down by demographic groups. In general, sample sizes of less than 100 are considered too small for statistical analysis, and are avoided throughout our analysis. For a sample of over 8,000 the margin of error is +/- 1%, meaning we can be 95% confident that the real proportion lies within 1% of the reported statistic at the top level.

When samples are split, either through different message testing, or crossbreaks, this margin of error will increase. As a rough guide, the margin of error for different samples sizes are:

• • 4,000 sample size: 2%
  • 2,000 sample size: 2%
  • 1,000 sample size: 3%
  • 500 sample size: 4%
  • 100 sample size: 10%

Segmentation Analysis

See our Segmentation section for the results of our segmentation analysis.

The attitudinal segmentation carried out on the larger sample as part of this work was produced through a combination of factor analysis and K-proto clustering, which allows for the combination of continuous and categorical data. A number of segmentations were produced throughout the analysis, varying in: the number of “clusters” or “segments”, the technique used (which included K-means with and without prior factor analysis, Hierarchical Clustering), the questions involved in the segmentation and how the answer options were grouped together (e.g. whether “Strongly Agree” and “Agree” responses were considered identical).

The final segmentation framework was decided through qualitative analysis of the different segment outputs. There were some segments (namely the Ideologically Aligned, Ideologically Conflicted and Issue Driven) which were relatively consistently found across methodological approaches. The segmentation which split the remaining cluster of people into Future Focussed and Present Focussed was chosen because we felt it to be useful for understanding how different issues and focuses could appeal to this group who were typically inclined to be supportive R&D, but spanned political and ideological outlooks. An alternative approach could have involved participation in different R&D-related activities as part of the segmentation, although this was ultimately rejected as it mainly split out those who gave to charity, which we felt was not a granular enough distinction.

Attitudinal segmentations like this are by their nature noisy, with segments often including participants who do not fit the qualitative interpretation applied to each segment. Further, given that segmentations tend to be a way of identifying latent information in the data, it is always possible that future research which explores the same segments in more detail uncovers a novel way of understanding a group which is a better fit for the data. The outputs of this segmentation are intended to demonstrate one way in which the data could be interpreted and used, alongside more traditional demographic splits.