Bibliometrics Higher education Science policy
Research impact now!
September 21, 2017
0
, , , , ,

In my previous post, I laid out some history of research assessment and measurement, all so that in this post I could explore research impact assessment, which was a major topic of discussion at the 2017 Science & Technology Indicators (STI) conference in Paris. In this post, I’ll summarize the major lines of discussion I encountered at STI, use the history from the last post as a basis for diagnosing those underlying challenges, and perhaps even hint at some avenues to resolve these tensions.

(more…)

Data mining Open access Science policy
Data mining: Open access policies and outcomes
September 20, 2017
0
, , , , ,

During our data mining project for the European Commission, one of the case studies we undertook to test our framework to guide data mining for policy research explored open access (OA) publications in the European context. Specifically, the question we aimed to tackle was whether institutional OA policies have an effect on the share of an institution’s papers available in OA, and if so, to what degree. An answer to this question would provide actionable advice for institutions that are looking to increase the availability of their research. Here’s what we found.

(more…)

Bibliometrics Higher education Science policy
A short history of research impact
September 14, 2017
2
, , , , , ,

During the 2017 Science & Technology Indicators (STI) conference in Paris, a number of discussions touched on impact assessment, which has been a topic of growing interest within the research community. That researchers are increasingly aware of impact, impact pathways and impact assessments comes as no great shock, given that the research policy community is increasingly focusing on impact as the basis for funding decisions. The discussions at STI raised some substantive concerns with the current trajectory of discussions about research impact. In this post, I’ll lay out some relevant history (as I understand it) that contextualizes current discussions about impact. In the next installment, I’ll summarize those points from STI 2017 that stood out to me as the most insightful (and provocative), drawing on the history laid out here in order to explore what I think these comments reflect about the underlying research system.

(more…)

Data mining Science policy
Data mining: Exploring the connection between innovation, growth and prosperity
September 13, 2017
0
, , , , , ,

In the most recent post in our ongoing data mining blog series, we explored the effect on innovation of research collaboration across disciplinary and sectoral boundaries. That topic was worth exploring because beliefs that such collaborations are effective levers to promote innovation are foundational to many policy choices, and there is scant evidence available to determine whether these levers work or not (and how powerful they are). The present post will take that line of exploration one step further: we usually promote innovation as a way to drive social and/or economic prosperity, creating “jobs and growth,” often with some qualification about these developments being “inclusive,” “smart,” or “sustainable,” or helping out “the middle class.” Such approaches have been particularly emphasized since the Financial Crisis a decade ago. The purpose of this blog post—and the case study on which it is based—is to explore the relationship between innovation and growth, especially for small and fast-growing firms.

(more…)

Data mining Science policy
Data mining: Cross-boundary collaboration and innovation
September 6, 2017
0
, , , , , , ,

In our data mining project for the European Commission, two of the six case studies treated levers for promoting innovation, and we’ll start to tease those apart here. In brief, collaboration across disciplinary and sectoral boundaries is believed to promote innovation, while innovation in turn is believed to support broader economic and social prosperity. Even though these premises are often used as the foundations on which to build policies for science and the economy, there is surprisingly little evidence about whether they’re true—no matter how intuitively appealing they may be, these premises are still quite speculative. These are important gaps in our knowledge base, and we’ll look here to start filling in the evidence around levers of innovation.

(more…)

Data mining Science policy
Budgets, timelines and expectations: The influence of project novelty
August 30, 2017
0

In the previous post in our ongoing series on data mining for policy, I touched on the importance of organizational context for successful management of these projects. In today’s installment, we dig into novelty and the pressure it exerts on the budgets, timelines and expectations of the project. Novelty is an interesting beast. Alongside the excitement and exhilaration of the new comes the fear caused by venturing down unknown roads, as well as the frustration that comes from hitting a few dead ends you can’t avoid because you don’t already know the way. Below the fold, I’ll discuss how to manage novelty effectively in data mining projects for policy.

(more…)

Data mining Science policy
Data mining: Organizational context matters
August 23, 2017
0
, , , , ,

The previous posts in this series on data mining to inform policy have covered our initial technical framework and two of its further developments. In this post, I present some of the project management lessons we learned over the course of the data mining project, which are largely drawn from the two less successful case studies that we carried out. In particular, I’ll be looking at the impact of organizational context on the conduct of data mining projects and, ultimately, on their chances of success.

(more…)

Data mining Science policy
Data mining: The value of a scoping phase
August 16, 2017
0
, , , , ,

In previous posts in our data mining series, we laid out our initial technical framework for guiding data mining projects, then supplemented that with plug-ins to facilitate its use for R&I policy research specifically. These plug-ins helped to overcome the challenge of applying a generic framework to a specific thematic area. However, there was another major challenge that we identified in using data mining for policy research, and this difficulty prompted another revision: the reorientation of the framework to include a scoping phase. This week’s post explores that challenge and how the reorientation helps to solve it. Take a look!

(more…)

Data mining Science policy
Data mining: Technical framework plug-ins for the R&I context
August 9, 2017
0
, , , , ,

In my previous post, I outlined the initial technical framework developed by Science-Metrix in the course of the data mining project for the European Commission documented in this blog series. This initial data mining framework—strongly inspired by existing frameworks—provided a solid foundation on which to build. However, to support data mining in a policy context in particular, and research & innovation policy specifically, further development was needed. This post covers some of the more novel work we undertook in this project, bringing new suggestions into the data mining space. There were two main drivers for these further developments; the generality of the framework is treated here, and its feedback loops will be covered next week.

(more…)

Data mining Science policy
Data mining: The root of a technical framework
August 2, 2017
0
, , , , , ,

Continuing on in our series of posts on data mining for policymaking, this post presents the initial technical framework developed by Science-Metrix to guide the conduct of data mining projects in a government context (with some shout-outs to other contexts as well). This seven-step framework formed the basis of our case studies, and effectively lays out the steps through which data mining projects progress from stem to stern. It’s a great introduction for those who have never participated in data mining. For more experienced practitioners, the framework provides the basis for understanding the main challenges to data mining in a policy context, as well as the recommendations we put forward to address those challenges and get more value out of the process.

(more…)