Introduction

Evidence-based policy, or more generally the need to consider existing knowledge, has become a mantra in policymaking over the last few decades. What is interesting is that this process originates directly from policymakers, probably due to the worldwide turn in public administration and public policy towards greater efficiency and effectiveness. This turn has been termed ‘new public management’ and began approximately three decades ago. The rising emphasis and institutionalisation of this mantra is particularly an Anglo-Saxon story that started under the initiative of the UK’s Blair government in 1997. At the time, the problem of systematic knowledge utilisation in public policy finally found political relevance, although it had already intermittently represented an issue over the entire last century. This role has always been relevant for grand theorists (Merton, 1949), and it has been particularly central in public policy since the 1970s (Kelman, 1968; Lindblom & Cohen, 1979; Weiss, 1979), with a focus not only on the possible utilisation of social sciences but also, generally speaking, on making knowledge usable for policymakers. However, thanks to the governmental turn, this issue has finally become a real component of policymaking. This has given rise to extensive literature about what the role of knowledge in policymaking is and when, whether and how it works. This literature has been expanded, especially in health and environmental policy, for which the role of knowledge appears simpler to objectify. However, it has extended to cover many other policy sectors. This expansion has brought an incredible variety of contributions, both theoretical and empirical, in which it is very easy to become lost.

This paper attempts to manage this variety through a metareview that, in contrast to existing reviews mostly focused on single policy sectors or single, specific issues, is applied to a large range of disciplinary fields according to a novel multicriteria framework of analysis.

This variety of contributions shows that there is an epistemological divide between scholars who assume that knowledge is external to policymaking and can thus be directly applied to it and scholars who consider the use of knowledge to be endogenous to policymaking and thus dependent on its characteristics and dynamics.

Highlighting this divide, this paper reviews the broad literature on the use of knowledge in policymaking through a framework that distinguishes between different types of knowledge utilisation (evidence-based policy, knowledge diffusion, data-driven policy and policy advice) analysed on the basis of the goals of studies (whether they are more or less empirically or theoretically oriented), the policy field of application and the analytical content explored. From this analytical treatment, a fine-grained map of the literature on the use of knowledge in the policy process is offered and discussed.

Furthermore, this descriptive result is accompanied by an analytical effort to map and analyse the literature that considers the use of knowledge as a ‘subjective’ phenomenon and thus treats its practical application as driven by contextual factors and by the interests and ideas of policy actors.

Based on such a large set of multidisciplinary publications, this focus on barriers and facilitators of knowledge in policymaking helps fill an analytical gap in the literature (where this kind of issue is either consistently addressed by one-sector reviews or anecdotal illustrations or completely neglected due to analytical attention to how knowledge can be used for better policies) and thus examines the actual evidence about them. Finally, the emerging empirical evidence on how some structural components of policymaking serve as barriers or facilitators of knowledge in policymaking more than others is connected to various research streams in public policy and public administration that include knowledge in their theoretical frameworks, although without an explicit connection to the large literature on the policy use of knowledge. Integrating these streams of research would be very useful to better understand the mechanisms and processes through which knowledge is treated in policymaking.

In Sect. 2, the characteristics and gaps in the literature will be presented, and the needs and goals of this metareview will be illustrated. Section 3 presents the research design, methodology and analytical dimensions adopted for conducting the metaanalysis. Section 4 presents and analyses the results that are discussed in Sect. 5. In the concluding section, some ideas for further research are proposed.

The unfinished world of evidence in policymaking: Why this metareview is needed and helpful

The epistemological divide on the role of knowledge in/for policymaking

The fertile research on the role of knowledge and evidence in policymaking of the last three decades clearly resonates with the literature on knowledge utilisation of the 1970s. Scholars such as Carol Weiss (1979), Charles Lindblom and David Cohen (1979) and Martin Rein (Rein, 1976) have already offered a detailed treatment of the possible roles of knowledge in policymaking and thus of the potential barriers to directly influencing policies. Thus, every kind of assessment and analysis of the role of scientific evidence in policymaking should start by being aware that knowledge cannot directly impact policymaking but can have different roles within its scope. Among these roles, direct impact is rare, generally occurring only for specific problems. Carol Weiss (1979) has identified seven ways in which knowledge (of social sciences, although this can be extended to the hard and natural sciences) can influence policies: knowledge-driven (when policies are treated as a research process); problem-solving (when specific scientific evidence is applied to an existing problem); interactive (when scientific research is one of the informative resources of policymakers); political (when evidence is used to certify predecided policy preferences); tactical (when evidence is used as a tool of political discourse against critics or to show that something should be done without a direct decision); enlightenment (when evidence provides decision-makers with ways of making sense of a complex world); and social intellectual enterprise (when social science research ‘responds to the currents of thought, the fads and fancies, of the period’ (p.430)).

The subsequent literature has been developed according to these lines, from both theoretical and empirical points of view. However, the responses to the research questions related to the involvement of knowledge in policymaking are always differentiated according to the theoretical approach adopted, which means that every research question will be answered in a different way according to how policy is defined and understood (Cairney, 2016; Parkhurst, 2017). This means that there is a persistent watershed in analysing this topic according to the adopted perspective with respect to the logic of how knowledge and evidence work, which depends on the analytical picture of how policymaking works. This implies the presence of a persistent divide: on the one hand, there are those who think that policymaking is a rational-optimal process that proceeds, necessarily, through linear logic between problems and solutions; on the other hand, there are those who are aware that there is no linearity in policymaking and that problems and solutions depend on the interaction of various political, ideational and psychological factors. This watershed is intrinsically epistemological as it relates to different conceptions from the viewpoint to be taken when considering reality. As such, it divides scholars into two general groups (Head, 2016): the champions of the prominence of knowledge who assume an objective role of knowledge (Chalmers, 2005) and the champions of the adaptation of scientific knowledge to the context because there is always a subjective interpretation of existing knowledge (Neylan, 2008).

This epistemological divide has been labelled in different ways: rationalists vs. constructivists (Newman, 2017) or supporters of comprehensive rationality vs. those who assume a bounded rationality perspective (Cairney, 2016). Overall, this never-ending divide exists between two scholarly approaches. The first approach is adopted by those who assume that the investigated phenomenon (in this case, how policy works) is objective, i.e. it exists apart from personal ideas and thoughts, and thus must be developed according to specific laws that must be discovered. The second approach is adopted by those who assume a subjectivist perspective that considers policymaking to be a complex activity whose developments depend on continuous and iterative processes of problem framing and social construction in which policy actors seek to pursue their interests/ideas.

What characterises the differences between these two groups of scholars is not only the different epistemological conceptions of how policies work but also, consequently, ideas on how to produce knowledge for and in policymaking.

In fact, for the first group (the champions of the objective perspective, and thus of the prominence of science), what matters is to develop rigorous and reliable methodological tools and evaluation techniques capable of showing that well-constructed knowledge truly works in terms of indicating solutions for policy problems (Boruch & Rui, 2008; Haskins & Margolis, 2015; Mosteller & Boruch, 2004; Nussle & Orszag, 2014). Scholars who hold this perspective assume not only that knowledge is external to the policy process but also that knowledge should/must be produced specifically for better policymaking.

The second group (the champions of the subjective perspective) assumes that knowledge cannot have a direct impact on policymaking due to political dynamics, varieties of societal values and differences between socioeconomic interests, which structurally shape policymaking (Lindblom, 1979; Majone, 1989; Radin, 2006; Weiss, 1999). According to the subjectivist epistemological perspective, this stream of literature underlines that concepts such as knowledge and evidence cannot be objective because they necessarily depend on the characteristics of the involved actors and decisional context (Adams, 2004; Maddison, 2012). Consequently, in the reality of policymaking, there is not significant room for a kind of hierarchy among types of knowledge, and so there cannot be a supremacy of the scientific type with respect to the other existing types (such as personal experience, lay knowledge, public consultations and the opinions of bureaucrats). Thus, the subjective perspective focuses on the role of knowledge in policymaking.

This divide can surely appear rough and simplistic.Footnote 1 However, it shows the two pivotal, different epistemological positions through which the role of knowledge is conceptualised: the objective perspective programmatically theorises knowledge for policymaking, while the subjectivist perspective considers knowledge in policymaking. In this article, we adopt the latter perspective when focusing on barriers and facilitators.

Why this metareview is needed and helpful

The use of knowledge in policymaking has been the object of an enormous number of studies that have increased over the last three decades. This time period experienced an increased focus on the role of ideas, expert advice, evidence and methods of knowledge utilisation (Christensen, 2021). There has been increasing empirical attention on these topics that has followed the rise of the evidence-based movement in medicine (Parkhurst, 2017) and has spread through other policy fields, such as education (Davies, 1999; Slavin, 2008) and criminal justice (Welsh & Farrington, 2001).

The emphasis on the role of knowledge has pushed it towards institutionalisation, not only in the public debate but also in scholarly research. It has become a kind of pervasive mantra that has also contributed to creating significant critical opposition to certain meanings. Thus, the field is overcrowded with contributions from different disciplines. Various attempts have been made to order this multifaceted world. Various review articles and metasystematic reviews address this effort, thanks to which it is possible to understand the state of the art, the eventual scientific accumulation and the possible analytical or empirical gaps.

These analyses have been conducted using various methodological tools, from expert surveys to ethnography and from interviews to systematic analyses (French, 2019). This extensive effort of investigation often offers a modest contribution to a definitive comprehension of the various dimensions of the policy use of knowledge (Nutley et al., 2007). This observation, however, should not cast this topic as intractable; reasonably, it makes sense to proceed to analyse it systematically to attempt to solve the puzzle (Cairney, 2016; Petticrew & Roberts, 2008).

In fact, various dimensions of the puzzle are still underinvestigated, depending either on the types of criteria adopted for the reviews or on the fact that this type of research is usually focused on a single policy area. Furthermore, from a comparative public policy perspective, these reviews seem to be particularly adopted in health policy, environmental and climate policies,Footnote 2 criminal policy and education policy, while they are less developed in other policy fields. Thus, there is a need for a larger review that is capable of mapping the intersectoral distribution of studies focused on analysing the role and characteristics of knowledge and evidence in public policy. This kind of broader review would also be useful to better understand the content of the literature. In fact, what is clearly lacking is a more detailed analysis of both the content (more or less theoretical or more or less empirical) and research questions adopted by the literature on the relationship between knowledge and public policies.

Above all, a large intersectoral review could provide a more detailed and consistent view of the central problem of the literature on the policy use of knowledge, which is represented by the factors that may favour (facilitators) or disfavour (barriers) the constructive inclusion of knowledge and evidence in policymaking. What is known about this dimension is extensive, but it requires more in-depth treatment. In fact, we already know that barriers and facilitators vary according to epistemological positions: as summarised by the most reliable reviews (Cairney, 2016; French, 2019; Innvaer et al., 2002; Oliver et al., 2014; Parkhurst, 2017), the “objectivists”, i.e., those who focus on the role of knowledge for policymaking, emphasise factors such as the cultural differences between knowledge producers and policymakers, and thus they focus on the problem of knowledge translation; the “subjectivists” focus instead on the complexity of policymaking and thus on the fact that every policy can be characterised by contested definitions of the problem at stake, political and ideological polarisation, differences in interests and various types of biases.

Furthermore, the actual knowledge with respect to barriers and facilitators is based, above all, on the contributions of specific policy fields, where different types of reviews of the role of knowledge and evidence have been conducted. For instance, we can find many of these reviews in health policy (Morgan, 2010; Oliver et al., 2014; Parkhurst et al., 2018), environmental policy (Miljand, 2020), education policy (Tight, 2021; Zawacki-Richter et al., 2020) and criminal policy (Petrosino et al., 2018).

However, in previous, highly cited metareviews, the main barriers include an absence of personal contact between researchers and policymakers, a lack of clear or relevant research evidence, a lack of timeliness, relevance or opportunity, mutual mistrust between scientists and policymakers, a lack of policymakers’ research skills or awareness and power, and budget struggles (Cairney, 2016; Innvaer et al., 2002; Oliver et al., 2014). Regarding facilitators, previous reviews suggest that clarity of evidence, clear recommendations and good relationships between scientists and policymakers are the main facilitating factors (Contandriopoulos et al., 2010; Innvaer et al., 2002; Oliver et al., 2014). Thus, overall, previous systematic reviews have mostly adopted an objectivist perspective, focusing on dimensions that are in one way or another related to the prominence of science and on the type of relationship between science and politics, while many characteristics of the policy dynamics that are relevant in driving and shaping the possible role of knowledge in policymaking have been underestimated.

What is also missing is a general map of the dimensions of the epistemological divide when analysing the use of knowledge in public policy.

Hence, there is a clear need for a better mapping as well as a reordering of the literature on knowledge in public policy. Moreover, in light of the distinctions between the objective/subjective epistemological perspective and knowledge for/knowledge in policymaking, a better understanding must be obtained regarding how the epistemological divide works in the related literature. Such work should focus on the evidence emerging from the literature that adopts a subjective perspective to clarify the implications of this perspective in terms of emergent barriers/facilitators.

This paper will try to do this according to the following:

  1. i.

    a multisectoral perspective of analysis;

  2. ii.

    a specific typification of knowledge based on different dimensions of its empirical focus;

  3. iii.

    a novel classification of the literature on knowledge and public policy according to the analytical goals pursued;

  4. iv.

    and, finally, a more fine-grained and deeply (subjectivist) policy-driven categorisation of barriers and facilitators, from which it is possible to shed light on the complex context in which knowledge is treated in policymaking.

From this multilevel analytical framework, a more inclusive map of the role of knowledge in public policy is offered, leading to a more detailed exploration of the different streams of literature that have focused on facilitators and barriers that can be encountered throughout the path to knowledge involvement in policymaking. Furthermore, given the distinction between objective and subjective epistemological approaches, specific attention will be devoted to the stream of literature based on the latter, which focuses on the characteristics of the policy process. Thus, the paper does not aspire in any way to articulate what is feasible for making knowledge and evidence work in policymaking; rather, the main aim of the paper is to order the field to unveil potential patterns of scholarly treatment of the analysed topic.

To clarify this point, exactly because we embrace a subjectivist perspective on public policy, we are well aware that barriers and facilitators cannot be considered outside their context and thus that, in every way in which they are identified, they cannot be understood as working in isolation because they are embedded in the characteristics of the related policy dynamics (Wellstead et al. 2018; Bach Mortersen and Verboom 2020). At the same time, it is important not only to obtain better knowledge about the difference in the variety of barriers and facilitators emerging from studies based on the two epistemological perspectives as defined above but also to reason about the nature of the evidence offered by the subjectivist perspective and what it can teach us in terms of further and improved analysis.

Research design and methodology

Data selection at a glance: a PRISMA-S approach

The sources to be reviewed were selected according to several criteria in line with the principles of the PRISMA-S approach, an updated and extended version of the PRISMA approach that is particularly suitable for use by authors of all disciplines (Rethlefsen et al., 2021), as detailed in Fig. 1. The approach that guides this process is based on the assumption that selecting literature, similar to selecting any other kind of data for scientific research, is a sampling process. As such, it aims first to identify the population and then to clarify the criteria for the inclusion or exclusion of sources (Yorks, 2008).

Fig. 1
figure 1

Source: Own elaboration

Source selection process.

Full size image

More specifically, the data selection procedure started with a keyword search that was performed on the Scopus and Web of Science online databases by using the search words that are shown in Table 1. They were applied to a large number of subject areas,Footnote 3 with a timeframe set between 1990 and 2020. The subject areas and categories of the medical field were excluded not only because the main goal of the paper was specifically to map the situation in the other fields but also as a result of the specific role held by metareviews in the medical profession.Footnote 4 The search criteria and results were reviewed and validated by both authors as well as by an external reviewer.

Table 1 Keyword search, conducted between May and August 2020
Full size table

The two datasets obtained through this keyword search were then merged, and any resulting duplicates were removed by combining manual deduplication with the use of Zotero software. At this point, all the sources without abstracts were also removed, and the remaining 4,743 records were screened according to their relevance to the subject of study. This first screening led to the removal of 3,358 records because they did not address the relationship between evidence and policy.

As a result, 1,385 relevant studies (all papers from journals) were identified as fully eligible for the goal of this review.

The analysis of the sources: conceptualisation of categories and classification of contents

The 1,385 selected papers were classified according to four criteria: the research design of the source, the empirical application, the policy sector, and the analytical goal.

The research design focuses on whether the source is a theoretical proposal, a literature review or an empirical analysis. Specifically, theoretical proposals are important because they advance knowledge in a scientific discipline and elaborate questions that can usefully guide empirical research while at the same time providing illuminative insights that inform professional practice (Van de Ven, 1989). Literature reviews enable us to achieve some sort of scale economy and leverage different types of knowledge and approaches as enshrined in different pieces of literature as well as to identify gaps and criticism in previous research for the benefit of both research and practice (Borrego et al., 2014; Sleep & Clark, 1999), especially when dealing with highly interdisciplinary subject matter such as the use of knowledge in policymaking. Finally, empirical analyses are particularly suitable for understanding phenomena in their context and therefore building context-dependent knowledge (Andersen & Kragh, 2010; Flyvbjerg, 2006; Ridder, 2017).

The empirical application of sources focuses on the roles that knowledge can play in policymaking. Assuming this perspective, by reflecting on the literature and its possible classifications, four categories were conceptualised:

  1. 1.

    data driven, which groups papers whose main focus is on the direct role that data can have in leading policy decisions (Lansky et al., 2007; van Veenstra & Kotterink, 2017);

  2. 2.

    knowledge dissemination, which groups papers whose focus is explicitly on the processes and strategies through which knowledge is diffused (Kingston, 2012);

  3. 3.

    policy advice, which groups publications that are focused on the science-policy interface, how the scientific and policy communities work together to reach decisions and how policymakers obtain the information they need (Engels, 2005);

  4. 4.

    evidence-based, which groups all papers that analyse the role of evidence in decision-making processes for any step in the policy cycle (Claes et al., 2015; Clark, 2012).

The third criterion is the classification of the selected sources into policy sectors to appreciate variations across different policy areas. As shown in the previous section, the literature about the role of knowledge and evidence in policymaking is generally sectoral and dominated by literature reviews in specific sectors. Hence, it is important to cover a wide array of policy sectors and identify the values assumed by our key analytical categories (barriers to and facilitators of knowledge involvement in policymaking), not only in the well-researched sectors. We have thus identified 25 categories (see Table 3).

The type of analytical goal pursued is directly linked to the adopted distinction between the subjective perspective (knowledge in policymaking) and the objective perspective (knowledge for policymaking). Thus, in conceptualising this distinction, we have tried to distinguish between papers focused on grasping, understanding and explaining the role of knowledge in the complex and multidimensional context of policymaking and other papers that can be characterised either by prescriptive content (telling policymakers the extent to which they should use knowledge) or by the consideration of tools and models through which knowledge is or should be processed to make it usable for policymaking. Thus, the examined works were classified into three categories. The first two belong to the objective stream and are conceptualised as follows:

  • prescriptive, which includes works that adopt a normative perspective with regard to the use of knowledge for policymaking as either something to be pursued or avoided in line with the findings of previous literature reviews that highlight the existence of different schools of thought with variegated normative views (Cairney, 2016; French, 2019; Newman, 2017);

  • tool-oriented, which examines practical instruments and practices (such as the development and use of scientific evidence based on randomised control trials or RCTs) that allow knowledge for policymaking to concretise in either its evidence generation or evidence use dimension. To clarify the greater theoretical or empirical focus, this category has been divided into two subcategories:

    • techniques/model analysis including works analysing the empirical adoption of specific techniques to generate and use knowledge for policymaking;

    • techniques/model proposal, including papers that make specific proposals about which models or techniques should be adopted to generate and use knowledge for policymaking.

The third category includes those items that belong to the subjectivist stream and have been conceptualised as process-oriented because they focus on policy actors and their relationships when engaging with evidence, in line with a basic definition of policy processes as interactions that develop between different actors and involve public policy over time (Weible, 2018). To clarify whether a more theoretical or empirical focus is adopted, this category has been divided into two subcategories:

  • policy process analysis (which groups papers that present empirical studies about the role of knowledge in policy dynamics);

  • analytical framework proposal (which groups papers that propose theoretical frameworks to analyse the role of knowledge in the policy process).

Finally, we proceeded to screen our final sample by focusing on the dimensions of barriers and facilitators. The first step was to categorise the content of the final sample to identify works that (1) presented problematic aspects related to the feasibility of adoption of knowledge; (2) identified or elaborated on strategies and techniques used to solve such problems; and (3) combined the identification of barriers with the proposal of solutions. The application of this criterion resulted in the selection of 604 pieces.

Thus, we finally restricted our focus to barriers and facilitators with specific regard to policy process-oriented studies and hence to those that belong to the subjective stream of research (224 papers). This choice is coherent with our epistemological position and the final aim of this paper, which is to explore the literature that considers barriers and facilitators in context and is thus based on a subjective perspective of the role of knowledge in policymaking.

The analysis of barriers and facilitators of the 224 selected papers required further classificatory effort. To do this, we first analysed the most relevant categorisations adopted in similar reviews, and we found them partially unsatisfactory from a policy perspective. For example, Mitton et al. (2007) distinguish barriers and facilitators according to four general categories (individual, organisational, related to communication, related to time or timing); Oliver et al. (2014) propose six general categories (contact and collaboration, organisation and resources, research and researcher characteristics, policymaker characteristics, policy characteristics, other); and Cairney (2016), by drawing from the related literature, extracts three general barriers in health and environmental policies (demand and supply driven barriers and facilitators, timing and opportunity, policymaker research skills). We found these classifications insufficiently precise to grasp certain relevant policy, political and organisational dimensions that intrinsically characterise policymaking and can thus be considered the real drivers of the use of knowledge in policymaking.

Overall, adopting a subjective perspective means that barriers and facilitators are not ‘objective’ because there are no neutral elements in policymaking and, thus, the same element could be a barrier or a facilitator according to the context or the specific theoretical framework. Therefore, we opted to focus on the necessary components of policymaking that are synthesised in six general categories that can be considered the constitutive blocks of policy dynamics: policy capacities, values/ideology/beliefs, actors’ relations, interests, institutional and processual arrangements and types of evidence. From a subjective perspective, exactly because these six categories are constitutive elements of policymaking, they can be either facilitators or barriers to the use of knowledge in policymaking.

Policy capacities group individual, organisational and state abilities with respect to understanding and adopting the use of knowledge and evidence; values/ideology/beliefs is a general category that includes all the characteristics of policymaking that relate to the emotional and cognitive dimensions through which policies are framed and thus that necessarily matter when knowledge and evidence should be defined, discussed and eventually adopted; actors’ relations include all the ways through which policy actors (not only decision-makers and scientists) interact with each other in the treatment of knowledge in policymaking; interests involve barriers and facilitators that directly refer to the characteristics and the dynamics of the interest-based behaviour of policy actors; and types of evidence group the various dimensions through which knowledge can be perceived, treated and constructed.

These six general components of policymaking were operationalised by identifying the most relevant ways through which they can constrain or facilitate knowledge in policymaking, according to both an analysis of the cited resources and a first reading of the coded dataset of the 224 papers under investigation. Table 2 presents the full picture of the selected classificatory system through which the content of the papers was coded.

Table 2 Classification of the six main components of policymaking as barriers to and facilitators of the use of knowledge in policymaking
Full size table
Table 3 Distribution of empirical foci across policy sectors
Full size table

The selection process involved an accurate reading of abstracts by both authors (when an abstract was not clear enough, we proceeded to a quick look at the content of the whole article). The intercoder reliability is very high (approximately 85%). The more controversial cases were resolved through a measured discussion of the divergences. The last step of the process (focused on 224 papers) involved a full reading of the articles by the two authors to extract information regarding barriers and facilitators.

Findings

Exploring the quantity and quality of literature on the use of knowledge

The corpus of literature collected shows great variation across our chosen descriptive and analytical dimensions. First, a slight majority (53%) of the research designs are empirical analyses, followed by theoretical proposals and, finally, literature reviews, as shown in Fig. 2. This means that most of the scholars concerned with the use of knowledge in public policymaking have explored this issue empirically by examining local realities, comparatively or not, and drawing lessons from their analyses. However, it should be underlined that approximately 35% of the papers focus on theoretical proposals. This shows a potential over-theorisation in the field.Footnote 5

Fig. 2
figure 2

Source: Own elaboration

Distribution of research designs over the total sample.

Full size image

Figure 3 presents the distribution of the empirical focus of the studies. Most of them are included in the evidence-based category. Fewer studies focus on the actors and processes of policy advising, followed by studies interested in the dissemination of knowledge from those who produce it and, finally, articles that focus on data-driven processes.

Fig. 3
figure 3

Source: Own elaboration

Distribution of empirical focus over the total sample.

Full size image

Table 3 presents the distribution of studies among policy sectors according to their empirical focus. The first group of studies focuses on the environmental sector, while studies on healthcare policy rank third. Furthermore, it must be noted that the last 18 policy sectors (from Development) cover only 19% of the total sample. This confirms that there is a deep, asymmetric concentration of studies on the role of knowledge in a few policy sectors.

Regarding the distribution with respect to the empirical focus, overall, the “evidence-based” focus prevails in all policy sectors, although the other three foci appear to have some relevance in the unspecified category—consisting of papers in which there is no reference to a specific policy sector—and in welfare, health, education, justice, drug, agriculture, gender and tobacco. However, it is also interesting to look at the variation over the three decades that fall within the scope of this metareview. The analysis of this variation shows a significant increase in the overall interest in knowledge involvement in policymaking, which can inter alia be explained in light of a more general growth in the scientific publication production, as well as of a greater concern for the role of knowledge in policymaking (see Appendix I, Tables B1, B2, and B3 and the comments therein).

Table 4 shows instead that just under half of the papers examined in this metaview (46%) focus on the analysis of policy processes from a theoretical or empirical point of view. As shown in Appendix I (Tables D1, D2, and D3), moreover, this distribution has not particularly changed over time, with the only exception being the prescriptive papers, whose percentage has more than halved relative to the first analysed time period.

Table 4 Distribution of analytical goals across empirical focus
Full size table
Table 5 Distribution of papers presenting barriers and/or facilitators across analytical goals
Full size table

Table 4 shows how indeed more than half of the selected literature is devoted to proposing prescriptions (telling policymakers the extent to which they should use knowledge) or, on the technical side of the use of knowledge, to analysing or suggesting tools and models through which knowledge is or should be processed to make it usable. Overall, this part of the literature is interesting from a technical point of view because it offers a broad perspective of (1) how to perform and use systematic reviews to make actual knowledge usable for policymakers; (2) how to use RCTs or design ex-ante assessments or modelling frameworks against policy uncertainty; and (3) evaluative systems through which policymakers can forecast or monitor the effectiveness of their decisions. These are all interesting dimensions in relation to knowledge utilisation that could be useful for policy scholars, especially for those interested in a prescriptive policy design perspective. However, they do not deal directly with the practical problem of the use of knowledge, which concerns whether and how it can have a key role in policymaking and which conditions can favour or disfavour this role. Thus, there is much to learn from the literature, but its prevalence in quantitative terms also shows that many scholars in different disciplinary fields are focused more on how to produce potentially usable knowledge than on understanding the conditions under which the existing knowledge is used in policymaking. The difference between these two goals is that the former mainly concerns the qualities inherent in the produced knowledge, while the latter mainly concerns the components of policymaking and how they interact with the knowledge input.

Table 5 presents the distribution of barriers and facilitators among the five analytical goal categories. Most of the facilitators are identified as an outcome of analyses that focus on the technical features of knowledge generation and use, followed by those provided in relation to the analyses of policy processes. Moreover, it is interesting that both policy process analyses and analytical framework proposals, which will be the focus of our in-depth examination of single barriers and facilitators in Sect. 4.2, display a trend inversion compared to the general predominance of facilitators: when analysing policy processes or proposing frameworks to that aim, barriers prevail. These data already suggest that most problematic dimensions are inherent in policy processes.

Barriers and facilitators of knowledge “in policymaking”

Finally, this metareview selected 224 papers devoted either to analysing policymaking or to presenting a theoretical policy framework aimed at grasping, from a policy perspective, the role of knowledge and evidence in policymaking (the full list is presented in Appendix II). These papers were coded according to 28 constraining empirical dimensions and 31 facilitating empirical dimensions through which the six components of policymaking can act as barriers or facilitators (see Table 2).

Before presenting and commenting on the empirical dimensions emerging from the analysis, some clarifications are needed. First, it must be underlined that what is presented in the following tables simply represents the frequency with which the six fundamental components of policymaking (and the related empirical dimensions) recur in the analysed sample, according to the conceptualisation framing of the theoretical papers and empirical evidence coming from the analytical papers. Thus, the data simply show how the specialised literature has analysed the use of knowledge from a subjectivist perspective, and thus, they should be understood as the output of the specific theoretical approach adopted in both the theoretical and analytical papers. These data should thus not be considered objective evidence regarding barriers and facilitators but rather an illustration of how the theoretically driven and empirically oriented literature treats the relevant components of policymaking. This means that frequency cannot be considered an indicator of relevance. This is particularly true if we consider that, for example, among the 183 empirical cases, 172 are qualitative case studies. At the same time, these data support some thoughts and comments about the different propensities of the six constitutive elements of policymaking to facilitate and constrain contextualised knowledge in policymaking.

Table 6 presents the recurrence of the six general components of policymaking as barriers and facilitators.

Table 6 Recurrence of six types of policymaking components as barriers and facilitators in the 224 selected papers
Full size table

The data show that values/ideologies/beliefs are the components of policymaking that emerge in the examined literature as having the greatest propensity to be obstructive (one-third of occurrences), followed by actors’ relations, policy capacities and the type of evidence. While the relevance of ideas and cognitive biases to policymaking is a highly considered matter in public policy (Beland & Cox, 2011; Campbell, 2002), the fact that this category is the most signalled barrier is noteworthy and reflects the subjective and constructivist perspective of the use of knowledge in policymaking. It is also interesting that actors’ relations and policy capacities are considered more problematic than the type of evidence; this clearly means that the problems related to the type of evidence are relevant but less so than expected, contrary to what is assumed, for example, by the objectivist perspective that considers the quality of evidence to be a fundamental characteristic for its usage in public policy. In contrast, here, it emerges that, when considering the characteristics of policymaking, evidence depends on how it is processed by actors, by their ideas and values and by their capacities.

It is interesting that both interests and institutional and processual arrangements are not as appreciated as barriers. Perhaps interests might have been underestimated (due to their close relationship to values/ideology/beliefs), but their low score in the analysed sample, together with institutional arrangements, raises the question of whether the role of knowledge and evidence depends less on institutions and self-interest and more on capacities, relationships and ideas.

Shifting to facilitators, we find that there is no symmetry between them and barriers. In fact, the prevailing category (44%) is represented by factors related to actors’ relationships, followed by policy capacities and values/ideology/beliefs. According to our sample, then, both from an empirical analysis of policymaking and from proposals of theoretical frameworks, it is significantly more probable that the ideational dimension works as a constraint rather than a resource for the role of knowledge and evidence in public policy. Furthermore, it is relevant to emphasise that the type of evidence is not a successful facilitator: it appears to be, in relative terms, more a barrier than a facilitator. Table 6 shows that the main aspects signalled as crucial to increasing the use of knowledge in policymaking are those related to improving actors’ relations and the policy capacities involved in policymaking.

It must be underlined that, also seen from a diachronic perspective, it emerges that values, ideologies and beliefs have been considered to be barriers since the first decade of the considered time span, although equated over time by actors’ relations and, to a lesser extent, policy capacities. Moreover, the policy capacity dimension gradually emerges as a facilitator over the examined timeframe (see Tables E1, E2, and E3 in Appendix I).

It is possible to obtain a more fine-grained view of this evidence by focusing on the occurrence of single factors that are the empirical ways through which the six general categories work in the reality of policymaking (see Table F in Appendix I). Our analysis discloses that in a context of high variety, there is a significant concentration of the operational dimensions of both barriers and facilitators. Analytical capacity is ranked first both among the barriers and the facilitators. Regarding the barriers, three of the first seven factors belong to the values/ideology/beliefs category and thus proved to be construed as challenging constraints in the reviewed literature. This also shows that the role of evidence and knowledge might be significantly bordered when issues are treated ideologically and become a matter of political conflict, divided values and ideational bias. Interestingly, among the first seven factors, there is one (nature of scientific inquiry) that belongs to the type of evidence category. On the facilitators side, four of the first seven factors belong to the actors’ relationships category, which is therefore conceptualised in the reviewed literature as the most promising or manipulable in improving the role of knowledge in policymaking. Moreover, the role of the individual capable of exercising the role of leader or broker does not appear to be conceptualised as a significant facilitator, which is surprising given the large body of literature on leadership.

The data on barriers and facilitators can also be analysed in relation to different policy sectors (Table 7).

Table 7 Top two* policymaking components as barriers and facilitators across policy sectors
Full size table

The values/ideology/beliefs category ranks first as a barrier in most policy sectors and is absent in only a few sectors. At the same time, there are other policy fields with few selected papers (such as regulation, drug policy and sports policy) in which the prevalent barrier is ideational. Among the facilitators, there is no general pattern of prevalence. Policy capacities are ranked first in education and justice, while actors’ relationships are ranked first in climate and environment. It is interesting that a few sectors, such as energy, fisheries, gender, sports and transport, do not have any suggested facilitators, which may, however, be due to the low number of papers analysed.

Discussion

The data presented above are rich and could be discussed from different points of view. Here, we focus on those that are more relevant from a public policy perspective.

First, the field of knowledge utilisation is elusive and slippery, including in metareviews. It is relevant to emphasise that the potential results depend on the research questions and thus on the keyword search. We used keywords that would allow a wide-mesh sieve. Thus, we could include relevant items that could have been excluded with more specific targeting. In this way, it was possible to build a detailed map of how knowledge utilisation is treated in various streams of literature and thus to determine how much of this literature must be considered of partial, if not marginal, interest for policy scholars.

The different focus adopted by this metareview—based on the distinction between objective perspective (knowledge for policymaking) and subjective perspective (knowledge in policymaking) and the consequent choice to privilege the latter for deeper analysis—shows different results in terms of barriers and facilitators compared to those of previous studies as presented in Sect. 2.2, mostly belonging to the objective perspective and thus emphasising those factors that constrain or facilitate the use of knowledge (considered external to the policy process): clarity of evidence and proper channels for knowledge translation.

By adopting a subjective perspective, we have assumed that barriers and facilitators are not objective elements in policymaking but that the constitutive components of policymaking could constrain and facilitate the use of knowledge according to the context (and thus also to the researchers’ theoretical approach).

Being aware of these assumptions, it should be noted that, according to the picture emerging from the papers belonging to the subjective perspective, values/ideology/beliefs, problems regarding the relationships between actors and (low) policy capacities emerge as pivotal factors constraining the proper use of knowledge (see Table 6). With regard to the first mentioned barrier (values/ideology/beliefs), it was noted, for instance, that “[i]ndividuals and groups with contrasting belief systems interpret and use evidence differently—not just because they are actively pursuing particular policy results, but because of how existing beliefs, motivations, and values structure the cognitive processes through which evidence is understood and applied” (Parkhurst, 2016: 384). In terms of actors’ relations, some of the reviewed literature has detected difficulties in science communication to nonscientists. For instance, it has been pointed out that “[t]he rising number of involved actors in communication on the science-policy interface raises new challenges when speaking about quality: scholarly information has to be understood by politicians and the general public (who) do not necessarily engage in scholarly activities themselves” (Sokolovska et al., 2019: 5). The difficulties in actors’ relations, as emerged from our metareview, are twofold and mainly caused by the differences between the academic and policymaking communities, as “academics have difficulty communicating research in an applicable manner, and policymakers, in turn, tend to focus on operational motivations” (Wilkinson et al., 2017: 88). Finally, in terms of policy capacity, our reviewed literature shows that its low level is a key constraint to improving evidence-based policymaking, with an example of this being found in the short-term focus of the actors involved in policy analysis (Howlett, 2009).

Thus, according to our reviewed literature, ideational factors and a lack of trained administrators of appropriate organisations seem to matter more than the type of evidence available. The skills of policymakers are important, but they should be included in the more general capacities (individual, organisational, systemic) that characterise a specific policy setting, the government and its bureaucracy. Furthermore, when relations between actors are analysed, there is an emphasis not only on the interaction between scientists and policymakers but also on the relevance of other stakeholders, such as the community, specific groups of citizens and vested interests. For instance, in the case of water governance in the city of Ahmedabad, one of our reviewed items identified insufficient stakeholder involvement as a factor contributing to explaining the weak linkage between research and policy and, conversely, stakeholder involvement broadening as a factor that might facilitate “a long-term joint knowledge production process” (Aartsen et al., 2018: 2455).

Finally, we find that financial issues do not stand out in the selected sample. This indicates that the problem of the impact of knowledge utilisation is not related to the existence of sufficient evidence (whose lack would justify the consideration of funding as a barrier) but to other factors.

Regarding facilitators, this metareview clearly provides a very different picture than the literature based on other reviews, which are very often based on an objective perspective (cf. Section 2). On the one hand, this work reinforces in a detailed way those perspectives that call for better governance of evidence (Parkhurst, 2017). In fact, what emerges is not only that in the concrete dynamics of policymaking, the triggers to improve the role of knowledge involve factors related to policy capacities and actors’ relations but also that some specific factors could matter more than others, such as analytical capacities, the direct involvement of citizens and local stakeholders and coproduction. The importance of analytical capacities in policymaking is supported by scholars who state that the policymaking process needs and produces knowledge and that during this process, some (analytical) questions must be answered, such as “Which aspects of the problem should be considered more important? What information is relevant and should be used? Who are the principal stakeholders? Who else is affected by the situation and the possible policies? Which resources are allocated, where how and when? What matters in terms of potential consequences?” (De Marchi et al., 2016: 23). To answer these questions, policymakers need analytical capacities. These capacities are also key elements of co-production practices, which are, in turn, functional to the use of knowledge in policymaking. Our reviewed literature also outlines other elements needed for these practices to work well. Along with the development of capacity and social capital, these elements include the active participation of knowledge users (e.g. policy implementation actors such as clinicians) in the creation of knowledge, the equality of researchers and policy actors in the conduct and application of research, the reciprocity of benefits between researchers and policy actors, and the setup of infrastructures that incentivise coproduction relations (Heaton et al., 2015: 1489). Finally, some of the literature we reviewed argues that public (citizens’) participation should be reconsidered, not only to solicit valuable information from key stakeholders but also to leverage participation itself as empirical evidence. This reviewed literature identifies practical rules for conducting meetings with citizens, such as recording them to avoid losing important information, as well as the benefits of reconsidering public engagement as research, which might stem from improving science communication to broad audiences and demonstrating to citizens that policy decisions are made in consideration of their input (Hall et al., 2016). Again, however, such efforts to employ public engagement as evidence require adequate analytical capacities.

A significant element emerging from the analysis of barriers and facilitators is the asymmetric role of ideas/values and ideology: this necessary, and inescapable, component of policymaking acts more as a barrier than as a facilitator. It must be noted that values and beliefs can be only partially manipulated, as the Advocacy Coalition Framework convincingly underlines (Sabatier, 1993; Weible & Jenkins-Smith, 2016). Thus, this asymmetry could be justified exactly because values and ideas are less changeable or manipulable in policy dynamics than other dimensions such as actors’ relationships and policy capacities. However, it could be clearly observed that the main elements of policymaking do not act in isolation and thus that this asymmetry could depend on the theoretical/analytical perspective of the authors but also on the specific policy fields analysed or on the high level of politicisation of the treated case.

Thus, this asymmetric role of ideas/values and ideology cannot be taken for granted, but the reviewed literature gives this general signal that is reinforced by other evidence coming from the analysis. In fact, for example, from a focused analysis of our final sample of 224 papers, it seems that ideational maps that hamper the use of knowledge in policymaking—for instance, international assessment results in the field of education—could be overcome by means of enhanced policy capacity, such as in the form of an improvement in strategic leadership (De Lisle et al. 2014). At the same time, and more representatively in terms of the literature reviewed, divergences in values, priorities and beliefs can be surmounted by means of enhanced actor relations. For instance, the pragmatic “forma mentis” of many policymakers in the healthcare sector, who tend to look mainly at the knowledge that they find immediately usable—which can represent an obstacle to the absorption of knowledge that they do not perceive as such—can be overcome by means of improved relationships between actors, where researchers proactively learn more about policymakers’ priorities and seek to develop mutually beneficial relations with them (Wye et al., 2015). The same happens when the priorities, ideas and/or values and knowledge systems of policymakers and researchers are not the same, as is sometimes the case in the field of environmental protection. This ideational difference is indeed surmountable, according to the reviewed literature, through preemptive behaviour on the part of researchers who take action to know in advance what these priorities are (Karam-Gemael et al., 2018); by means of procedures and practices that unlike the search for consensus that is the prevalent orientation in some fora (e.g. the Intergovernmental Science–Policy Platform on Biodiversity and Ecosystem Services—IPBES) are able to accommodate pluralism, contestation and distinct perspectives and knowledge systems (Díaz-Reviriego et al., 2019); by means of collaborative, resource-intensive co-learning arrangements between policymakers and scientists that are properly organised in terms of governance (Morgan, 2017); or even by attempting at co-production, which some of the reviewed literature presents as an important practical arrangement of the relationships between actors—although not as a panacea (Sylvester & Brooks, 2020).

This evidence invites us to take seriously the hypothesis that knowledge production should be appropriate and localised (Parkhurst, 2017) and that policy capacities could also be a pivotal resource for any kind of policy process (Wu et al., 2018). Overall, the present metareview on this point indicates that to increase the likelihood of a more effective role of knowledge in policymaking, inclusive policymaking processes should be introduced, bureaucrats should be better trained and the organisation’s capacity to collect and analyse data should be increased. Again, this evidence may depend on the characteristics of the case studies analysed by the reviewed literature and thus should not be taken for granted. However, these findings indicate a path for further analysis and in-depth empirical research.

The third relevant point that emerges from our metareview is that three constitutive elements of policymaking (values/ideology/beliefs, actors’ relations and policy capacities) prevail as barriers and/or facilitators, while the other three categories (including the factors considered more relevant by the other sectoral reviews as summarised in Sect. 2.2) are residual or relevant in only a few policy fields. The limited number of papers found and scrutinised on these aspects indicates the small number of interested scholars working on them. The question is thus raised regarding why interests, institutional and processual arrangements and types of evidence emerge as less relevant in favouring or constraining the use of knowledge in policymaking. While the low relevance of the type of evidence in policy dynamics can be related to the fact that, according to a subjective perspective, it has a high chance of being an intrinsically weak component of policymaking due to the intensity and strength of the others, it is less easy to justify the low relevance of the other two components. Obviously, the usual disclaimer can be applied, i.e. the specific subjective perspective of the reviewed works: for example, it could be the case that the authors have not clearly distinguished the ideas and interests in their research or have focused more on the institutionalised actors’ interactions and less on the institutional arrangements in which these interactions happen. This could also depend on similar mistakes in coding the data. However, the evidence emerging in the review significantly indicates this difference among the six components of policymaking, and this is something that should be addressed by further research. If corroborated by further empirical investigation, this difference could represent a significant “discovery” with respect to the determinants of the use of knowledge in policymaking.

The fourth relevant point is why there are so many sectors in which there is insufficient analytical attention to the role of knowledge in policymaking. We counted 18 policy sectors that are clearly underrepresented in the sample, while constant attention was given over time to sectors such as environmental and health policy. Sectors such as labour, social work, public administration reform, transport, energy, higher education, urban development and migration emerge as uninteresting to scholars who are interested in the policy use of knowledge. This may be due to the keywords we used or the fact that the role of knowledge is less relevant in policy fields where it is more difficult to objectify it, which seems to be the case for health, the environment and climate, where the bulk of knowledge is apparently more homogeneous and internally more shared by scientists. Thus, this underrepresentation may be a consequence of the more disputable nature of the issues at stake and of the different theoretical approaches and empirical evidence coming from scholars. However, it must be emphasised that many scholars of public policy are more interested in the dynamics of policymaking, whereas the different types of knowledge utilisation are usually treated as one of the possible factors influencing policymaking. Among the various literature streams in public policy that pay attention to the role of knowledge in policymaking, the following can be noted:

  • The stream focusing on the ideational dimension of policymaking, thanks to which it is possible to grasp whether and how ideas can be constraints or facilitators (Swinkels, 2020);

  • The recent Policy Design perspective, which underlines how the technical capacities of government can make a difference in terms of the quality of policy design and analyses the factors that can constrain or facilitate it (Capano & Mukherjee, 2020; Howlett, 2019);

  • The Policy Capacity literature, which focuses on defining the different capacities that are necessary to produce good policies (Mukherjee et al., 2021);

  • The Advocacy Coalition Framework, which emphasises that there are certain conditions under which deep beliefs can be overcome by evidence-based knowledge and that experts are not neutral (Jenkins-Smith et al., 2018);

  • The stream that focuses on co-production and collaborative governance, which is fundamental to understanding how the involvement of stakeholders is essential for good governance and thus for an appropriate role of knowledge in policymaking (Ansell & Gash, 2008; Poocharoen & Ting, 2015).

The richness of these approaches in public policy, therefore, could justify the fact that so many policy fields lack consistent literature on knowledge utilisation: these approaches consider knowledge as an element of their designed policy setting, even if the role of knowledge is not directly enlightened as pivotal.

At the same time, this richness calls for a higher integration with studies specifically devoted to analysing knowledge utilisation in policymaking. In fact, on the one hand, these studies, such as the public policy streams of research listed above, share the common subjective perspective that assumes the complexity of policymaking as an inescapable feature and consequently that knowledge is not a phenomenon that can be considered external to the policy dynamics themselves. On the other hand, this metareview clearly shows that the three fundamental components of policymaking (values/ideology/beliefs, policy capacities, actors’ relations) that emerge as more relevant in constraining and/or facilitating knowledge in policymaking are exactly those on which there is a flourishing public policy literature. Greater integration would help better clarify the drivers and activated mechanisms through which this set of factors can constrain or facilitate a better use of knowledge and thus find and potentially design possible solutions to make knowledge more relevant in policymaking. In addition, overall, due to the emerging relevance of policy capacities and actors’ relations, this greater integration would help demonstrate ways to better design actors’ relations and improve policy capacities to eventually make the role of knowledge in policymaking more effective.

Conclusions

This paper has presented a broad, multidisciplinary metareview of knowledge utilisation in public policy. Through novel conceptualisation and a precise process of case selection, this work has offered a detailed map of the empirical and theoretical content and sectoral distribution of studies focused on different types of knowledge utilisation (data-driven processes, knowledge diffusion, evidence-based policy and policy advice). Furthermore, by considering the epistemological divide between the objective and the subjective approaches to the use of knowledge in public policy, it has deepened the analysis of the literature that adopts the subjective approach by abandoning the conceptualisation of barriers and facilitators as objective factors in the policy process but instead considering them as empirical shapes that can characterise, according to context and policy dynamics, the most relevant components of policymaking.

By following this research design, the results necessarily differ from those of previous metareviews (mostly belonging to the objectivist perspective) and indicate that three types of policymaking components could be more relevant than others in constraining or facilitating the use of knowledge in policymaking: values/ideology/beliefs, actors’ relationships and policy capacities. These three categories are also relevant for various streams of public policy. This calls for more integrated research that should be committed to explaining how and when these factors work as barriers and facilitators and, consequently, to better enlightening the processes through which knowledge is more or less included in a relevant way in policymaking.