Remote Work and the Future of Innovation

Why I'm an Optimist

Note: This is the second “argument” essay on New Things Under the Sun. These longer pieces make bigger arguments, drawing on the narrower “claims” articles normally published. Like the rest of New Things Under the Sun, it will be updated as the state of the academic literature evolves; you can read the latest version here.

What happens to innovation if we let knowledge workers live and work far from their peers?

For hundreds of years, cities and in-person collaboration have been associated with innovation. But the forced experiment with remote work in 2020 has proven many jobs can be performed pretty well without stepping into the office. This has led many companies to adopt a more permanent remote work footing. But the long-run effects on innovation are unclear.

My tentative assessment is that we’ll be ok. In this essay, I’ll map out my reasons for optimism, but also point out what I think are the big unknowns. To begin, let’s try to figure out why innovation and cities have traditionally been associated with each other.

What are cities doing for innovation?

Cities have long been associated with innovation: they generate an outsized share of patents (relative to population), scientific papers, and economic activity (Balland et al. 2019). Why?

An innovative city packs lots of scientists, innovators, and entrepreneurs into close physical proximity. This facilitates innovation in several ways.

Learning

To begin, knowledge spillovers play an enormous role in innovation. Knowledge spillovers are what we call it when someone benefits from research conducted by someone else. As discussed in the article “Knowledge spillovers are a big deal”, measured in a variety of ways, something like half of the value of R&D efforts accrue to projects other than the one for which research was originally conducted. For a couple of reasons, cities have traditionally had significant advantages for helping scientists and inventors learn about ideas developed by other people.

• Physical repositories of codifiable knowledge are primarily in cities. As discussed in the article “Free knowledge and innovation”, the presence of these repositories (in the form of public libraries and patent libraries) has a measurable positive impact on innovation.

• Codifiable and non-codifiable knowledge also resides in the heads of people. The article “Importing knowledge” looks at the migration of people with specialized knowledge to argue being around people with expertise helps that expertise spread into new minds.

• Lastly, the article “Urban social infrastructure and innovation” looks at some suggestive evidence that cities with more places for people to socially mix leads to more sharing of ideas from residents of cities.

For all three reasons, it’s traditionally been easier to learn about new ideas and new research by living in a city, especially a city packed with other experts. It’s not just that you can meet up with the relevant expert on a topic and ask them technical questions as the need arises. Perhaps much more important is that a lot of our most important learning comes from our social network, and close physical proximity with lots of people helps us build really high quality networks (from an innovation perspective).

The article “Adjacent knowledge is useful” looks at some studies that support the idea that the most important knowledge spillovers come from fields that are merely adjacent to our own, rather than being identical to it (or super far away). It can be tough to learn about this knowledge by searching for information at the library, or asking a technical expert about them, because you might not even know there is relevant information out there to be searching for. Casual acquaintances and coworkers can be a bridge to this kind of adjacent knowledge as they chat about their work in informal and open-ended contexts. And the article “Why proximity matters: who you know” surveys some studies that indicate physical colocation seems to provide serious advantages to forming relationships with people working in different fields.

Let’s lump all this together and call it the learning advantage of cities.

Team Assembly

Second, innovation is no longer the preserve of lone geniuses. The article “Are ideas getting harder to find because of the burden of knowledge” presents a variety of evidence that bigger and bigger teams are increasingly the norm in innovation, while “Highly cited innovation takes a team” looks at evidence that teams usually produce more highly cited work than individuals. As discussed at length in “Are ideas getting harder to find because of the burden of knowledge”, the rise of teams is almost surely driven by the need to apply more and more knowledge to contemporary problems. This is because it’s usually easier to bring all that knowledge to bear on a problem by hiring a team of specialists than to find a generalist with sufficient depth of knowledge in many different fields.

The importance of teams leads to another advantage for cities. One of the studies cited in “Highly cited innovation takes a team” finds the performance of an innovative team is driven to a greater extent by the “worst” team member, rather than the best. Specifically, the citations received by an innovative team’s output (patent or paper), is more correlated with the typical citations received by the person on the team who gets the least citations on their own work, rather than the citations received by the person on the team who gets the most citations. By expanding the pool of potential collaborators, and facilitating the formation of a rich social network to help inventors identify the talent of potential collaborators, big cities facilitate the formation of really good teams with no weak links. Let’s call this second advantage of cities the team assembly advantage.

Group Execution

Third, and closely related, if teams are needed to innovate, and if it’s easier to work together as a team by being physically together, then teammates that live close enough to each other to meet up regularly have an advantage over those that don’t. Hence, one of the advantages of cities. Let’s call this the group execution advantage of cities. We’ll have more to say on this later.

Market Size

Fourth, the large number of people who reside in a city can be customers as well as teammates. Large markets do seem to encourage more innovation. The article “Medicine and the limits of market-driven innovation”, reviews evidence from the medical sector that shows there’s a pretty clear link between drug development and the size of the market for a drug. Of course, ultimately many businesses hope to sell to the world at large, so that the size of the local market isn’t so relevant. Even still, cities can provide a large enough market to test out early stage ideas before rolling them out to wider markets. We’ll call this the market size advantage of cities.

Value formation

Lastly, we have a lot of evidence that the decision to be an entrepreneur (or inventor?) is strongly influenced by your social network. In short, if lots of your friends and acquaintances are inventors or entrepreneurs, you are more likely to become one yourself. The articles “Entrepreneurship is contagious” and “The ‘idea’ of being an entrepreneur” survey some of the literature on this, establishing that:

• Entrepreneurs tend to be found in social clusters

• Quasi-random exposure to entrepreneurial peers is associated with becoming an entrepreneur

• This effect is stronger when peers occupy a similar social position

• The effect may be weaker once the “idea” has been planted

Since you’re more likely to form friendships with people who are geographically close to you, you’re more likely to know an innovator if you live in an innovative city. That, in turn, might transform people who would otherwise follow a conventional career path into innovators themselves. Let’s call this the value formation advantage of cities.

Put these five advantages together - learning, team assembly, group execution, market size, and value formation - and you’ve got a potent innovation generating machine.

The Fading Advantages of Cities

One could point to additional advantages of cities. For example, it may be easier to raise funding for a new venture in cities because funders want to be able to monitor their investments and this biases them towards local investments. Or funders may draw on a social network to identify promising new founders and ideas. But, at least in this case, those are variations on the big five advantages I laid out already; except in this case, the business that is benefiting from learning and group execution is a venture capital firm.

Once we lay out the reasons why cities have traditionally been associated with innovation, we can see the reasons are not laws of nature but just rules of thumb whose utility is contingent on an underlying social and technological context. And that context has changed over the last few decades. Let’s take a second look at these five advantages in light of this fact.

Learning revisited

Knowledge now circulates relatively frictionlessly at much larger distances than in the past. For example, there are various ways to measure the importance of local knowledge in patents - what they choose to cite, what kinds of combinations of technology they embody, and what words they include. As documented in the article “Increasingly distant knowledge spillovers”, all three of these indicate the localization of knowledge is fading. The article “An example of successful innovation by distributed teams: academia” also documents a similar line of evidence in academic work.

Why is this happening? Obviously a big part of this is the internet. The article “The internet and access to distant ideas” reviews a few studies that look at the impact of the early roll-out of the internet. Internet access appears to have enabled more collaboration between geographically distant firms, and more citation of each geographically distant work by colleagues. Or consider the article “Free knowledge and innovation”, which was discussed earlier as a justification for why cities have typically enjoyed a learning advantage - they were the places were physical libraries were. But the era of the physical library is over. The same article also shows how, today, wikipedia exerts a detectable influence on the direction of innovation, and how the local advantage of having a patent depository library disappeared once searchable online databases came into being.

What about the tacit knowledge that has historically resided primarily in people’s heads? Well, digital technology - think videos, simulations, interactive software, and augmented reality - can also expand precisely what kinds of knowledge are codifiable (Burja 2019, Whitaker 2021). Moreover, even if tacit knowledge largely resides in the heads of experts, the internet (and travel) makes those heads accessible even if you don’t live nearby. The article “Planes, trains, automobiles, and innovation” looks at a few studies that document improved transport links facilitate more collaboration between geographically distant places. More generally, as discussed in “Why proximity matters: who you know”, studies of collaboration between distant coauthors and citations of distant ideas suggests that while cities seem important for helping people form these connections, once they are formed the benefits of those relationships - in terms of learning - seem to be quite robust to distance.

But what about making those connections in the first place? We’ll come back to that.

Team Assembly Revisited

Cities facilitate team assembly by bringing together a large pool of potential teammates. However, in a world where a team can be drawn from anywhere on the planet with an internet connection, cities no longer hold the same advantage they once did.

One of the studies discussed in “The internet and access to distant ideas” finds some support for this idea. While US counties with the most patents over 1990-1995 also had the fastest growth of patents over 2000-2005, this effect was reduced by greater internet access. Internet access let regions that would normally have been left behind keep up; moreover, this effect was largely driven by an increase in long-distance collaborations, i.e., greater use of long-distance teams.

And there is reason to think the advantages offered by access to a global workforce will grow. The rising burden of knowledge discussed above is likely to create demand for larger teams composed of individuals with ever more specialized skills. If teams need to draw on ever more niche sets of skills, the gap between what you can get in a city versus a global market will widen.

To see why, consider a numerical example. Suppose the skills of workers are drawn from a standard normal distribution. On average, if you pick a worker at random, you would expect their skill level to be the average value of this distribution, i.e., zero. But if you can hire from a pool of workers, then the bigger the pool, the better you can expect the top candidate will be. If you have 10 workers, it can be shown that you would expect this “top” worker to have a skill level of 1.5. And if you can go up to 100 workers, that jumps to 2.5. And if you can go up to 1,000 workers, you would expect the top worker to have a skill level of 3.2.

Now suppose, that 10% of the global workforce is available in your local supercity. As illustrated in the table below, as the skill set gets increasingly specialized, the gap between the best in the local labor market and the best in the world gets larger and larger.

That implies the advantage of access to a non-local workforce will rise if specialization rises.

Group Execution Revisited

Once a team is assembled, it needs to execute. However, the internet, and more broadly, the entire digital communication ecosytem has significantly eroded the advantages of being together for executing group projects. A litany of digital tools have sprung up to facilitate long-distance knowledge work: Zoom, email, slack, Github, dropbox, and so on. The maturity of these technologies are a chief reason why remote work is likely to stick - on a project-by-project basis, they are more than sufficient to get the job done (Clancy 2020 pgs 3-8, Clancy 2021). Of course, gaps remain but these can potentially be closed by infrequent face-to-face meetings, enabled by low-cost travel (again, see the article “Planes, trains, automobiles, and innovation”). And note that, even if remote collaboration is less productive than colocated collaboration, if the extra time per day needed to complete a project remotely is less than the time of daily commutes, remote collaboration remains preferred by workers on net.

Market Size Revisited

It remains true that access to large markets probably facilitates innovation, and that early access to large markets can probably help innovative firms with their initial launches. But a growing share of goods are now purchased online and delivered via increasingly robust long-distance transportation networks. For many goods, it is now possible to access global markets - or at least national ones - from the start.

From Evans (2021)

Value Formation Revisited

A difficult to measure effect of remote work could be that dispersing remote workers across the country (or world) makes the distinctive culture of innovation and entrepreneurship in some cities disappear as the density of innovators in a place falls below some threshold. If the presence of this kind of culture led more people to consider becoming entrepreneurs, then its disappearance could mean fewer people to try their hands at doing something new. But there are two reasons to be cautious about jumping to this conclusion.

First, as discussed in “The ‘idea’ of being an entrepreneur” there is some evidence that once the idea of entrepreneurship is planted, further reinforcement isn’t particularly useful. For example, the children of entrepreneurs are not more likely to become entrepreneurs if they are socially exposed to entrepreneurs in other walks of life, while the children of non-entrepreneurs are. If you only need to be have the entrepreneurial seed planted once, that could imply we would have more entrepreneurship at the national level by scattering our “entrepreneurial seeds” more widely, since we would thereby expose more people to the idea (and fewer people multiple times).

Second, while I am not aware of any studies on the topic as they relate to entrepreneurship and innovation, it certainly seems possible to transmit other kinds of values online. Currently, the most salient examples are negative - think ISIS, incel culture, and QAnon. But it may be that, quietly and under our nose, values related to innovation and entrepreneurship are also flourishing online.

But do we see this in the data?

So for each of the five advantages of the city, I’ve argued there is reason to believe these advantages are eroding. But there is an easy retort to all this: if the advantages of the city are waning, shouldn’t we have observed a decrease in the importance of the city for innovation?

And we haven’t. Andrews and Whalley (2021) track the spatial concentration of patenting in the USA over the very long run, relative to population density. Indeed, there has been a massive increase in the concentration of patent-holders since the 1990s. In other words, in the USA more innovative activity (as measured by patents) is occurring in a smaller geographic region, precisely during the era when I’m arguing the advantages of doing so have been waning.

From Andrews and Whalley (2021)

What’s going on? The following is more speculative than the previous sections, but I think it’s important to try and offer a response to this fact, so bear with me. Over time, I hope I’ll be able to ground this section in more claim articles written on this website.

My view is basically as follows. To begin, the computer revolution spawned a bunch of innovative new companies related to digital technology. Importantly, over this time period, highly innovative sectors seem to be associated with increasing market concentration (Autor et al. 2020). In other words, sectors where a lot of innovation is happening seem to be characterized by winner-take-most dynamics: winning firms get a disproportionate share of the total industry profit. That has contributed to a rise in the geographic concentration of patenting in a few ways.

First, in a more concentrated market, more innovation is performed by fewer larger firms, concentrating patenting activity in the geographic footprint of these superfirms. Second, because the gains to being best are disproportionately high in innovative sectors, innovative firms highly value even minor competitive “edges.” Even if it’s true that today cities offer no advantage for innovation, this is a comparatively recent phenomenonundefined that postdates the founding of today’s major innovative firms. Accordingly, even if the benefits (from an innovation perspective) were smaller than in the past, lots of these companies still chose to pack into the same few cities, since the value of winning increased. And because the entry of new firms has been on a long-term decline, the legacy of the past looms large today.

Moreover, clustering begets clustering. That’s because most of the advantages of the city increase when the density of talent and expertise in a city rises. In the extreme, when the global talent pool all happens to resides in the same city, firms in that city don’t gain anything by seeking remote workers from outside it. And indeed, Andrews and Whalley (2021) show the remarkable rise in the geographic concentration of patenting is to a large extent driven by just a few superstar cities, particularly in California.

Lastly, to some degree, I think the extent of geographic clustering has been due to incorrect beliefs about the importance of clustering. While these beliefs were probably appropriate in the 1990s and maybe the 2000s, I suspect firms overrated the advantages of clustering for innovation, especially since the 2010s when the internet was becoming mature. The mass adoption of remote work following the forced experiment with it during the covid-19 pandemic shows us it is, in fact, possible for lots of firms to have mistaken beliefs about how well distributed work works.

Meanwhile, under the surface, we do see other evidence consistent with the waning advantage of cities. For example, here is the share of US patents whose inventors list addresses that are more than 100 and 500 kilometers from each other. Remote collaboration in invention had been increasingly normal for decades, despite the decisions of firms to cluster.

Clancy (2020)

We see a similar phenomenon in academic collaboration, a sector we turn to next.

A glimpse of the future?

The article, “An example of successful innovation by distributed teams: academia” illustrates a lot of the themes discussed above, as well as offering a potential case study for a future with more distributed work. The article documents that collaboration between academic coauthors who are geographically distant has been increasing and is now quite normal in academia, and that academics who move to a top department do not normally experience a significant bump in their research productivity. In other words, people whose job it is to generate novel ideas don’t seem to get much out of the ability to work in close physical proximity with each other. This is notable, because academia is highly decentralized and competitive: if close contact between researchers was important for producing good work, academics would have strong incentives to do it.

Viewing this through the lens of the five advantages of cities then, academia illustrates how an industry might get along fine without clustering. Geographic proximity doesn’t seem so important for learning anymore in academia; the knowledge used to produce new research circulates at a wider and wider distance, especially if there are social ties between people. Neither does proximity seem that important for team formation, with academics instead opting to select from a global pool of potential collaborators. Proximity also doesn’t seem that important for group execution among academics; a combination of carving up the work, digital collaboration tools, and occasional travel for in-person visits suffices. Neither does local market size seem to matter much, with academics instead disseminating their work via journals, the internet and long distance social networks. (It’s not clear to me how to think about value formation, but at least academia has no trouble convincing new people to sign on, despite being distributed)

Academia is also potentially interesting because it has two characteristics that might become more common in other sectors in the future. First, the burden of knowledge problem is quite severe in academia. Academic fields have become extremely niche and specialized. “An example of successful innovation by distributed teams: academia” argues this is one reason for the rise of teams in academia; it has become too unlikely a person with just the right sub specialty needed for a project will reside in the same department. That’s consistent with my earlier argument that as the burden of knowledge problem becomes more severe, there are increasing benefits to accessing the global pool of potential collaborators.

Second, because academic research is bundled with teaching, it is unusually geographically disaggregated for an innovative industry. Unlike the private sector, where the Bay area can become a focal point for all ambitious tech workers, in the current university system its not really possible for agglomerations of research talent to grow without bound. As soon as they outstrip the population of undergraduate students, hiring of faculty slows (and this anti-clustering effect might be exacerbated by the reluctance of elite universities to expand their undergraduate enrolment). The fact that academia gets along ok despite being so geographically diffuse suggests that a more diffuse workforce in other industries may not pose much of a problem for innovation.

What we don’t know

But I think academia also exposes the big unknown in this pivot to remote work: how do social connections form in the absence of proximity? As discussed in “An example of successful innovation by distributed teams: academia”, most academics appear to form their initial collaborative partnerships with some kind of face-to-face interaction. In fully 65% of non-local collaborations, initial contact was made while collaborators were colleagues or had an advisor-advisee relationship. In other words, these relationships were formed under conditions of sustained proximity. Some of the studies discussed in “Why proximity matters: who you know” also find that proximity helps people form initial contacts (which can then be productively maintained at distance). As noted above, proximity seems especially good at helping people meet who don’t otherwise work in the same field, and so might not find out about each other otherwise.

But it doesn’t follow that a lack of proximity will necessarily lead to static (or eroding) social networks. The fact is, until quite recently, proximity was basically the only way to meet (I guess excepting pen pals?). The growth of the internet has led to the development of new online social spaces. Think twitter, metaverse, gather, (World of Warcraft?) and even just improved search tools that let us discover other people’s work (who we can then cold-email). We know next to nothing about how well these spaces work to connect geographically distant people and existing work using data that predates the current era isn’t that informative.

Most people perceive that the quality of distanced social interaction, whether over zoom or twitter, are a poor substitute for the real thing. Fair enough. But that might be less true of a generation that grew up natively living with these tools, instead of immigrating to digital domains later in life. Moreover, while the quality of interactions may be lower, the quantity is surely higher. And in a future where specialization becomes increasingly important, sorting through a larger volume of potential contacts to find a good match might be more important than meeting fewer people but having a better experience on each one.

Even supposing physical meetings are necessary to form social ties, we also have little knowledge about how best to organize work to form ties. Is it really necessary for people to come into the office 5 days a week in order for colleagues to build durable relationships? Or will an annual work retreat plus a company slack channel do just as well? Or maybe we need to enforce 15 minutes of (virtual) coffee with someone new every day?

Whether by design or not, it’s notable that academia - which I’ve argued might be a harbinger of a more remote world to come - has several social institutions that encourage the formation of social ties among geographically distant colleagues. The article “Academic conferences and collaboration” looks at just one of these institutions - the academic conference - and finds a lot of reason to believe these help connect geographically distant peers. But academia has many other complementary institutions. There is the tradition of going elsewhere for work, instead of staying where you did your PhD (and possibly doing a postdoc somewhere else too). Seminars regularly rotate visitors through departments. And sabatticals allow people to temporarily reside somewhere new at a later stage in their career. It may turn out that other industries that come to rely more heavily on a distributed workforce benefit from similar institutions.

Lastly, it’s also possible that social networks, as a carrier of information, are less important today than in the past. I’ve already mentioned how new digital mediums probably allows for the improved transmission of tacit knowledge; but there isn’t much academic work on this I’m aware of. It’s also possible that social networks consisting of people who are geographically nearby (but not otherwise in the same field) aren’t so important for exposing us to ideas and knowledge that is outside our normal routine anymore. We’ve got a proliferation of high quality podcasts, youtube channels, and explainer journalism. This is another area where we know little.

Ultimately, I’m optimistic. For one, the entire knowledge workforce is not going to go 100% remote right away. Once the worst of the pandemic is behind us, it looks likely that a hybrid arrangement will prevail across many sectors for the immediate future. I suspect that will suffice to allow us to retain the benefits of proximity in forming social ties, while also giving us a persistent prod to figure out exactly how much (or how little) we need to come into the office. Meanwhile, the variation in work arrangements, a greater appetite for experimentation with online alternatives, and the intense interest of academic social scientists should mean we learn a lot about what works and what doesn’t over the next five to ten years. My hope is that we end up more innovative than ever as innovation networks becomes increasingly less parochial, while simultaneously enjoying greater freedom to live in the places that nourish our lives outside of work.