Note: if you’re a regular reader of New Things of the Sun and have a few minutes, please fill out this short survey. Thanks!
It’s long been assumed that the best sorts of innovation happen when smart people work in an environment where spontaneous face-to-face interaction is the norm. Importantly, if that’s true, it implies the widespread transition to more remote work - where spontaneous face-to-face interaction is not possible - poses a threat to innovation. I’ve written about these ideas a lot before, but this week, I want to look at a case study for a sector that:
engages in frontier knowledge work
has strong incentives to adopt practices that produce better outcomes
has been well studied
has increasingly moved to a model of remote collaboration
I am talking, of course, about academia.
Remote Collaboration in Academia
Academic research is a sector where knowledge workers try to innovate - the whole game is trying to push the knowledge frontier outward. Whether or not the system could work better, it certainly does seem to work, generating new and useful knowledge pretty much every day. It’s also a system that’s highly competitive, where thousands of individuals compete with each other for jobs and space in journals.
And yet, despite strong incentives to use any possible edge to generate new and better research, academics are increasingly forgoing the option to work with their local colleagues.
Agrawal, McHale, and Oettl (2015) is a study of the changing nature of collaboration in evolutionary biology. They find the the number of distinct institutions represented on evolutionary biology papers has steadily increased from 1.4 to 2.4 over 1980-2005, while the average distance between coauthors on papers has risen from 350 to 550 miles over the same period.
Evolutionary biology isn’t some weird outlier. Freeman, Ganguli, and Murciano-Goroff (2015) look at publications in particle and field physics, nanoscience and nanotechnology, and biotechnology and applied microbiology over 1990-2010 and calculate the share of articles by authors who reside in the same city (not necessarily the same university). The share of these colocated articles dropped from over 50% in 1990 to 40% by 2000, where it stayed until 2010 or so. The majority of papers in these fields are written by teams of coauthors who are not local.
Lastly, in Clancy (2020) I pulled data from the National Science Foundation’s science and engineering indicators on the share of US journal articles in Scopus with authors belonging to more than one institution, or more than one country. The results are pretty consistent with Freeman, Ganguli, and Murciano-Goroff (2015): the share of publication done at different universities rose through the early 2000s and then had a decade long “pause” in the trend towards more remote collaboration during the 2000s. Since then though, the trend has been back up, with 76% of papers coauthored by people at different institutions, or just 24% all belonging to the same institution. Maybe we are worried that different institutions might sometimes be close enough to enable frequent face-to-face interactions though? But if you look at international collaborations, the trend is the same.
All told, the share of papers produced by academic teams primarily engaged in remote collaboration - rather than primarily local face-to-face interaction - has been on the rise for decades, and now accounts for the majority of papers.
Are we sure face-to-face matters anymore?
A set of related papers can help us see why this change is underway.
Probably the strongest evidence for benefits from being able to collaborate locally is Agrawal, McHale, and Oettl (2017), another paper studying evolutionary biology. In this one, the authors look at what happens to the research productivity of a department when they successfully recruit a star researcher to join the department (“stars” are defined to be researchers whose citation-weighted publication count is in the top 10%). In one exercise, they take the set of academics working in the department before it gets a star hire, and divides them into two groups: those who have previously cited the star’s work (“related” scientists) and those who haven’t. The productivity (annual citation-weighted publications) of scientists working on related stuff goes up 96% in the years after a star is hired!
Now, a potential objection to this paper is that star researchers don’t just get randomly allocated to different departments. It could be that departments that are already “on-the-up” are the ones that attract stars. Maybe, for example, a department has just secured a big endowment that lets them go on a hiring binge and provide better resources to everyone in the departments. In that case, we might falsely attribute higher research productivity that is actually due to more resources to the appearance of a star researcher. But if that was the case, we would expect to see research productivity for the department to rise for everyone around when a star is hired - and we don’t. Instead, scientists unrelated to the star see no impact from the star’s arrival. (Agrawal, Oettl, and McHale also try to answer this concern in a few other ways, but don’t find anything that alters their core conclusion).
So this result is a bit puzzling, given the trend towards more distributed academic teams. It also contradicts a few other studies. Let’s look at this studies and come back to this one.
Dubois, Rochet, and Schlenker (2014) looks at the same time period for the set of all mathematicians (in the world) who published at least two articles in a mainstream math journal between 1984 and 2006. Rather than look at the impact of an individual researcher on the productivity of a department though, it looks at the impact of a department on the productivity of an individual. By keeping track of the moves of individual mathematicians, they can see how the productivity of researchers changes when they move to top universities. If being able to have frequent face-to-face interactions with other great researchers is really important for producing good research, then we ought to see the research productivity of mathematicians jump up when they move to a department full of great researchers.
While it is true that top departments have more productive faculty, it turns out this is just because they hire more productive faculty. Once you take into account the productivity of a researcher (i.e., from before they get hired to a top department), the authors don’t really find any increase in productivity when these people move to top departments. It seems that the ability to have lots of face-to-face interactions with other top mathematicians is not doing much to raise their productivity (which was already high).
Ultimately, these are both observational studies, and so we might be a bit worried that something funny is going on to deliver these results. In an ideal experiment, we would randomly shuffle around researchers, looking at the impact on departments that randomly receive (or lose) a researcher to those that do not. We can’t do that, but two other papers identify some (grim) natural experiments where faculty in some department suddenly lose the opportunity to have frequent face-to-face interactions with a top researcher. In all cases, there doesn’t seem to be much value to it.
Waldinger (2012) looks at German physics, chemistry, and mathematics departments over 1925-1938. In 1933, there was an awful event that led to some departments losing colleagues and other departments not, for reasons quite unlikely to be correlated with factors driving academic productivity: the Nazi dismissal of Jewish and politically “undesirable” scientists. Like Agrawal, Oettl, and McHale, Waldinger looks at the research productivity of the faculty who were left behind. And, surprisingly, he finds no impact. The subsequent research trajectory of faculty in departments that lost local colleagues were no different than departments that didn’t lose anyone. (This isn’t to say there was no effect; as he documented in Waldinger 2016 - departments that dismissed faculty saw total output drop, and were never able to recover by hiring replacement faculty with the same research output).
Azoulay, Graff Zivin, and Wang (2010) are also interested in the impact of losing a colleague: they use sudden and unexpected deaths as a plausibly random shock to a department. Azoulay and coauthors identify 10,349 elite life scientists, as well as their coauthors. They then further identify a subsample of 112 eminent life scientists who died suddenly and unexpectedly between 1979 and 2003 before the age of 67, and who were still active researchers at the time of their death. It is the coauthors of these eminent scientists that they study. They match each scientist who experienced a sudden and unexpected death of an eminent coauthor to a similar scientist working with another eminent coauthor who did not die.
They find the unexpected death of a colleague reduces output by 5-8% relative to those who did not. Of this group, about 12% of coauthors were colocated with the deceased at the time of their death. But when Azoulay, Graff Zivin, and Wang look to see if this group is more adversely affected by the death than distant coauthors, they find just a small effect (in the opposite direction) that can’t be statistically distinguished from zero. Losing a coauthor hurts; but in terms of it’s impact on measurable research outputs (setting aside the real human cost), it doesn’t look like coauthors able to engage in frequent face-to-face interaction were any more affected than those who couldn’t.
So we have four studies now. One finds that evolutionary biologists benefit (a lot) when a talented colleague working on similar stuff joins the department. The other three find local access to a colleague, however, doesn’t seem to have much impact. And we actually know that long-distance collaboration is becoming the norm in evolutionary biology, precisely the field where another study documents strong benefits to a local star researcher. How can we square these results?
I think there are two things going on. First, I suspect proximity might be quite important for forming working relationships, but less important for maintaining them. Second, the importance of local colleagues is declining over time, and most of the results on evolutionary biology are driven by hires in an era when the effects of locality were stronger than they are today. Let’s start with the former.
Getting to know you
Suppose local face-to-face interactions are important for forming new relationships but less important for doing productive work once a relationship has been formed. In that case, we would expect hiring a new star to boost the output of local researchers working on related topics, because they could form a new collaborative partnership. And that’s basically what Agrawal, Oettl, and McHale (2017) find. Most of the increased output of evolutionary biologists when they hire a new star research seems to stem from publications they coauthored with the star.
Conversely, if the opportunity to collaborate disappears, everyone who would normally collaborate with them - near and far - would be equally adversely affected. That’s essentially what Waldinger (2012) and Azoulay, Graff Zivin, and Wang (2010) find. Waldinger notes that coauthorship across departments was already pretty common in Germany before 1933 because Germany had a well integrated national research community with frequent national conferences and mail correspondence about scientific research. Azoulay, Graff Zivin, and Wang’s study focuses on people who were already coauthors with the deceased - definitionally a group that has likely already cleared the threshold of “forming a relationship” - and no longer needs proximity to work together productively. So when the opportunity to collaborate disappears, due to one tragedy or another, they’re equally negatively impacted, whether they are in the department or not.
We’ve got some complementary evidence that colocation is important for forming academic working relationships. In a figure I’ve reproduced before, Freeman, Ganguli, and Murciano-Goroff illustrate how the academic collaborators in their study first met:
Look at how non-colocated and international collaborators first met: 90% of the time, it was during some kind of face-to-face interaction.
Change over time
In Agrawal, McHale, and Oettl (2017), most of the new hires of star researchers that they observe occur prior to 1995. That might be important, because we have a number of papers that suggest the benefits of local face-to-face interactions where stronger prior to the year 2000.
The most direct test of this is Kim, Morse, and Zingales (2009), which examines the research productivity of academics in economics and finance over 1970-2000. Like Dubois, Rochet, and Schlenker (2014), they’re going to try and identify the productivity bump a researcher gets from moving to a top department by tracking the moves of individual academics over their careers. In this case, research productivity is measured in terms of how many articles an economist/finance professor writes per year, adjusted for the quality of journal. They actually do find that moving to a top department helps researchers be more productive - in the 1970s. But the productivity bonus researchers got for moving to a top 25 university dropped in the 1980s, and had disappeared by the 1990s.
Two other recent papers zero in on the flow of knowledge across space, as measured by citations. Head, Li, and Minondo (2019) looks at the probability that there is a citation between two different mathematics papers. In the figure below, they plot how the elasticity of distance changes over time - roughly speaking, the more negative, the less likely a citation between math papers by authors who are far apart.
In the left figure, in orange, we see the distance penalty has dramatically shrunk between 1990 and 2010. That’s the main finding of interest for today - being close matters less and less, in terms of the probability two researchers know about each other’s work. (See this post for a bit more discussion of this paper). In the right figure, the authors break this out for citations between US papers (in orange) and citations between US and non-US countries. Surprisingly (to me), there has actually never been a strong penalty for distant papers in the USA, at least since the 1990s. But the probability a paper from abroad is cited used to be substantially affected by distance and no longer is.
Hellmanzik and Kuld (2020) perform a similar exercise for economics, but focused entirely on international citations. They begin with a gravity model, which is traditionally used to statistically model the flow of exports and imports between countries, but then adapt it to model the flow of citations between economics papers written by authors in different countries. After controlling for a variety of factors, they find papers in 1979-1988 were nearly 6 times as likely to cite papers from their own country as compared to international ones, but by 2009-2016 they were just 1.4 times as likely to papers from their own country. Measures of the “distance” penalty, wherein countries that are farther away are even less likely to be cited also declined over the same time period.
Hellmanzik and Kuld also introduce a new measure that helps account for why some countries are more likely to cite papers from each other (again, after adjusting for lots of other potentially important stuff like language): the number of hyperlinks from one country to another. As you might expect, they find countries that are more interconnected by the internet are more likely to cite each other’s work, and that this effect has grown stronger over time.
Taken together, I read this as providing a different but complementary line of evidence that local interaction is not as important as it used to be for doing frontier knowledge creation.
The Upsides of Remote Collaboration
So I think proximity was (and perhaps still is) important for forming collaborative working relationships, but these relationships remain productive even when academics are far away from each other. At least, this is increasingly the case.
But at the same time, I think there’s another factor that increasingly pushes academics to collaborate remotely. That is the ever-rising set of knowledge needed to push the frontier, which academics are responding to by drilling into ever narrower specialties and forming teams of specialists to tackle research projects. When academics specialize more and more, it becomes ever less likely that the specialty you need to complete a project happens to reside in the same department.
We’ve got a few pieces of evidence in support of this view.
First, when Freeman, Ganguli, and Murciano-Goroff (2015) survey coauthors about “what they bring” to a collaboration, the most common response (80%) for non-colocated authors was “unique knowledge, expertise, or capabilities”, beating out responses like access to data, funding, or specialized equipment, none of which cleared 60%.
Azoulay, Graff Zivin, and Wang (2010) provide some similar evidence. They perform a series of exercises to try and sniff out what kinds of factors exacerbate the decrease in research productivity that comes when an eminent coauthor passes away. For example, they look to see if the negative impact of a coauthor’s death is stronger when the coauthor has plausibly better access to NIH grant funding. It isn’t. They look to see if it’s stronger when the coauthor has better potential to network in the field. Again, it isn’t.
Lastly, they look to see if the negative effect of a coauthors death is stronger when the coauthor works on more similar topics. This time, they find a robust correlation. That again suggests these collaborative partnerships are formed as a way to acquire the knowledge necessary to tackle hard problems, rather than as a way to share grant funding or curry favor in the scientific hierarchy.
Agrawal, McHale, and Oettl (2017) also get some results that I think also underscore how specialized science is becoming. Recall this is the paper that found evolutionary biologists benefit a lot if they hire a new star researcher, but only if they’ve previously cited their work. It turns out, this isn’t very common: just 9% of the faculty in a typical department are related to the star in this way. The other 91% of scientists in the field of evolutionary biology apparently work on subfields too dissimilar to benefit from the opportunity to collaborate with a star in evolutionary biology.
Agrawal, Oettl, and McHale (2015) also find some additional suggestive evidence consistent with this story. They find that, over time, stars have been producing a growing share of all citation-weighted work. They argue this is because stars build larger pools of potential collaborators (for example, by training more doctoral students) which becomes increasingly important as it gets more important to find partners with just the right specialty. A bigger pool to draw on also lets stars be more selective. Their paper provides evidence that stars do indeed produce papers with a larger set of unique coauthors, and that they have grown more selective over time.
Two cheers for dispersed teams!
In sum then, my story is basically this.
it’s not actually that hard to collaborate productively at a distance in academia, at least once you’ve gotten to know someone.
innovation requires ever more collaboration among specialists as knowledge accumulates.
Over time, falling travel and communication costs have increasingly favored building those teams by turning to remote colleagues with the right specialization.
I think these trends are a harbinger for things to come for the non-academic world (to some extent it’s already happening). Innovation by distributed teams will probably become more and more important as access to just the right specialty becomes ever more important and falling communication costs make it ever easier to do this well. To the extent innovation via dispersed teams has been shown to work pretty well in academia, that’s a reason not to be too concerned about it in other domains.
But there are caveats.
First, the nature of academic work might be particularly well-suited to innovation by distributed teams, as compared to other forms non-academic innovation. Projects are discrete, and finite, and many forms of academic projects have a modular character - one person can do the experiment, another can analyze the data, and so on. They also ultimately hinge on the creation of a digital artifact - the paper - rather than a physical prototype (although in many cases there is a lot of work that happens in the physical world on the way to a paper). Still, it might be that innovation in hardware, for example, doesn’t work nearly so well with a distributed team.
Second, while I think it’s quite feasible to productively work together at a distance, I suspect that academia still relies heavily on face-to-face interaction for one crucial step in the innovation process: meeting new people. If we’re thinking about applying the lessons of academia to non-academic examples of innovation by distributed teams, much hinges on whether colleagues at fully remote organizations get to know each other as well as they would if they were all in the office. As best as I can tell, this is certainly possible, but some deliberate planning is often necessary.
Besides the departmental work-unit, academia has evolved a rich set of mechanisms to circulate researchers and give them opportunities to form relationships with people who are unlikely to be colocated colleagues for the majority of a career. On the longer side, these include training - often in one place for a PhD and another for a postdoc. But they also include the seminar circuit and frequent academic conferences. Collectively, these temporary nexuses account for another 40-50% of non-colocated collaborative relationships in Freeman, Ganguli, and Murciano-Goroff’s survey. Non-academic sectors that hope to be as effective as academia at innovation with distributed teams may need similar distant networking opportunities, whether those take the form of apprenticeships, conferences, online social networks, etc.
If you liked this post, you might also like:
Share this post