2021 Northwestern

Abstract

Evaluation in science, whether per review, performance metrics, or networks of peer-to-peer judgements, determines who stays, who leaves, and what work is rewarded in science. Ideally, a researcher should be evaluated based on their true merit, but evaluation is subjective, and deeply embedded in a particular social, political, and cultural context. In my research, I explore what social and contextual factors relate to evaluation in science. I conduct four studies examining different areas of evaluation. The first study, focusing on peer review, examines the extent to which gender and national bias persist in journal peer review at eLife. The next studies investigate how bias and contextual factors shape two very different kinds of performance metrics: the first examines how demographic factors such as gender and ethnicity relate to student evaluations of teachers, whereas the latter explores how the use of disagreement citations differs across fields, and how this might necessitate more nuanced understandings of citation-based metrics. Finally, I turn towards evaluative networks; specifically, I focus on how networks of mobility, which are both driven by evaluation, and can drive a researcher’s future success, are not equally available to everyone. I use a neural embedding technique to encode millions of researcher’s individual trajectories into a vector-space representation which, when interrogated, reveals the importance of geography, language, culture, prestige, and more, in structuring global mobility, and thus the options available to individual researchers. Together, these studies paint a picture of how social and contextual factors shape evaluation, and therefore success, in science. Changes to policies and practices are necessary to remedy these issues and promote holistic evaluation, and to create a more equitable, open, and effective system of science.

Date
Jan 8, 2021 11:47 AM — 11:47 AM
Event
CSSI Research Seminar
Avatar
Dakota Murray
Doctoral Candidate in Informatics

I study the social factors shaping science and scientific careers