Age and the Nature of Innovation
Age and the Nature of Innovation
New ideas vs deep expertise; conceptual vs experimental innovation
Like the rest of New Things Under the Sun, this article will be updated as the state of the academic literature evolves; you can read the latest version here.
You can listen to this post above, or via most podcast apps here. The previous post also now has a podcast available here.
Are there some kinds of discoveries that are easier to make when young, and some that are easier to make when older?
Obviously yes. At a minimum, innovations that take a very long time basically have to be done by older innovators. So what kinds of innovations might take a long time to complete? Perhaps those that draw on deep wells of specialized knowledge that take a long time to accumulate. Or perhaps those that require grinding away at a question for years and decades, obsessively seeking the answers to riddles invisible to outsiders.
What about innovations that are easier when young? Well, we can at least say they shouldn’t be the kinds of innovations that take a long time to achieve. That means discoveries that can be made with years, not decades, of study. But what kinds of innovations that don’t take long study to make are still sitting around, like unclaimed $20 bills on the sidewalk?
One obvious kind of unclaimed innovation is the kind that relies on ideas that have only been very recently discovered. If people learn about very new ideas during their initial training (for example, for a PhD), then we might expect young scientists to disproproportionately make discoveries relying on frontier knowledge. At the same time, we might look for signs that older scientists build on older ideas, but perhaps from a place of deeper expertise. Indeed, we have some evidence this is the case.
Age, Frontier Ideas, and Deepening Expertise
Let’s start with Yu et al. (2022), a study of about 7mn biomedical research articles published between 1980 and 2009. Yu and coauthors do not know the age of the scientists who write these articles, but as a proxy they look at the time elapsed since their first publication.
Below are several figures, drawn from data in their paper, on what goes into an academic paper at various stages of a research career. In the left column, we have two measures drawn from the text of paper titles and abstracts. Each of these identifies the “concepts” used in a paper’s title/abstract: these are defined to be the one, two-, and three-word strings of text that lie between punctuation and non-informative words. The right columns relies on data from the citations made by an article. In each case, Yu and coauthors separately estimate the impact of the age of the first and last author.1 Moreover, these are the effects that remain after controlling for various other factors, including what a particular scientist does on average (in economics jargon, they include author fixed effects). Together, they generally tell a story of age being associated with an increasing reliance on a narrower set of older ideas.
Let’s start in the top left corner - this is the number of concepts that appear in a title or abstract which are both younger than five years and go on to be frequently used in other papers. Measured this way, early career scientists are more likely to use recent and important new ideas. Moving to the top-right figure, we can instead look at the diversity of cited references. We might expect this to rise over a career, as scientists build a larger and larger knowledge base. But in fact, the trend is the opposite for first authors, and mixed at best for last authors. At best, the tendency to expand the disciplinary breadth of references as we accumulate more knowledge is offset by rising disciplinary specialization.
Turning to the bottom row, on the left we have the average age of the concepts used in a title and abstract (here “age” is the number of years that have elapsed since the concepts were first mentioned in any paper), and on the right the average age of the cited references (that is, the number of years that have elapsed since the citation was published). All measures march up and to the right, indicating a reliance on older ideas as scientists age.
This is not a phenomenon peculiar to the life sciences. Cui, Wu, and Evans (2022) compute some similar metrics for a wider range of fields than Yu and coauthors, focusing their attention on scientists with successful careers lasting at least twenty years and once again proxying scientist age by the time elapsed since their first paper was published. On the right, we again have the average age of cited references; these also rise alongside scientist age.
On the left, we have a measure based on the keywords the Microsoft Academic Graph assigns to papers (of which there are more than 50,000). Between two subsequent years, Cui and coauthors calculate the share of keywords assigned to a scientist’s papers which recur in the next year. As scientists age, their papers increasingly get assigned the same keywords from year to year (though note the overall effect size is pretty small), suggesting deeper engagement with a consistent set of ideas.
Lastly, we can look outside of science to invention. Kalyani (2022) processes the text of patents to identify technical terminology and then looks for patents that have a larger than usual share of technical phrases (think “machine learning” or “neural network”) that are not previously mentioned in patents filed in the preceding five years. When a patent has twice as many of these new technical phrases as the average for its technology type, he calls it a creative patent. He goes on to show these “creative” patents are much more correlated with various metrics of genuine innovation (see the patent section of Innovation (mostly) gets harder for more discussion).
Kalyani does not have data on the age of inventors, but he does show that repeat inventors produce increasingly less creative patents as time goes by.
This figure shows, on average, an inventor’s first patent has about 25% more new technical phrases than average, their second has only 5% more, and the third patent has about the same number of new technical phrases as average. Subsequent patents fall below average. This is consistent with a story where older inventors increasingly rely on older ideas.
As discussed in more detail in the post Age and the Impact of Innovations, over the first 20 years of a scientists career, the impact of a scientist’s best work is pretty stable: citations to the top cited paper published over some multi-year timeframe is pretty consistent. The above suggests that might conceal some changes happening under the hood though. At the outset, perhaps a scientist’s work derives its impact through engagement with the cutting edge. Later, scientists narrow their focus and impact arises from deeper expertise in a more tightly defined domain.
Conceptual and Experimental Innovation
So far we’ve seen some evidence that scientific discoveries and inventions are more likely to draw on recent ideas when the innovator is young, and an older, narrower set of ideas (plus deeper expertise?) when the innovator is older. I suspect that’s because young scientists hack their way to the knowledge frontier during their training period. As scientists begin active research in earnest, they certainly invest in keeping up with the research frontier, but it’s hard to do this as well as someone who is in full-on training mode. Over a 20-40 year career, the average age of concepts used and cited goes up by a lot less than 20-40 years; but it does go up (actually, it’s pretty amazing the average age of concepts used only goes up 2 years in Yu et al. 2022).
I argued at the outset we might expect this. The young cannot be expected to make discoveries that require a very long time to bring about. But among the set of ideas that don’t take a long time to bring about, they need to focus on innovations that have not already been discovered. One way to do that is to draw on the newest ideas. But this might not be the only way.
The economist David Galenson has long studied innovation in the arts, and argues it is useful to think of innovative art as emerging primarily from two approaches. The first approach is "experimental." This is an iterative feedback driven process with only vaguely defined goals. You try something, almost at random, you stand back and evaluate, and then you try again. The second approach is “conceptual.” It entails a carefully planned approach that seeks to communicate or embody a specific preconceived idea. Then the project is executed and emerges more or less in its completed form.
Both require a mastery of the existing craft, but the experimental approach takes a lot longer. Essentially, it relies on evolutionary processes (with artificial rather than natural selection). It's advantage is that it can take us places we can't envision in advance. But, since it takes so long to walk the wandering path to novelty, Galenson argues that in the arts, experimental innovators tend to be old masters.
Conceptual approaches can, in principle, be achieved at any point in a lifecycle, but Galenson argues there are forces that ossify our thinking and make conceptual innovation harder to pull off at old ages. For one, making a conceptual jump seems to require trusting into a radically simplified schema (complicated schema are too hard to plan out in advance) from which you can extrapolate into the unknown. But as time goes on, we add detail and temper our initial simplifications, adding caveats, carveouts and extensions. We no longer trust the simple models to leap into the unknown. Perhaps for these reasons, conceptual innovators tend to be young geniuses. Perhaps it also gets harder to take a fresh look at a domain as we are socialized into accepting various assumptions and ways of thinking about a given domain.
Galenson’s theory was devised to explain artistic innovation, but we can see similar ideas in play among scientists.
Conceptual and experimental innovation among Nobel laureates
We can begin with social scientists. Galenson and Weinberg (2019) uses 31 Nobel-prize winning economists as their dataset. Their goal is to identify years when these economists made their most impactful work, and then to score that score that work on how conceptual or experimental it is. Then, they can see if more conceptual innovators tend to make their largest impacts when young, and more experimental innovators make their contributions when older. (Note, for this and the rest of the post, by “experimental” I mean experimental in Galenson’s sense, not in the sense that the inventors work with experiments)
For each Nobel laureate, Galenson and Weinberg identify the years in which their work received an unusually high number of citations (two standard deviations above their average), and then they go about characterizing the work as conceptual or experimental. To do that, they look at a sample of pages of each Nobel laureates work and score the sample on various criteria they associate with conceptual and experimental innovation. For example, the more pages with equations or formal proofs, the more conceptual they classify the innovation. That’s because they think of conceptual innovators as working deductively, deriving their findings from a priori logic. Meanwhile, the more pages referencing specific places, time periods, industries or commodities, the more experimental the work. Here the argument is experimental innovators are assumed to work inductively, often generalizing from data and facts that have been accumulated over a lifetime. At the end of this exercise, they can rate economists on a spectrum from very experimental to very conceptual.
Whatever you want to call it, their index certainly differentiates two very different kinds of economists. Famous economic historians like Douglas North and Robert Fogel, who are steeped in the concrete and real, end up on one extreme, and abstract theorists like Kenneth Arrow and Gerard Debreu end up on the other side. Robert Solow, who did a combination of empirical and theoretical work, ends up in the middle. When they statistically estimate the age at which economists achieve unusually high impact, they find the extreme Arrow-Debreu type conceptual innovators are most likely to make their biggest contributions at age 35, while the extreme North-Fogel type experimental innovators are most likely to make their biggest contributions at age 56. There is also a small tendency for the research of economists to get more experimental and less conceptual as they age, but this effect is rather small. To a first approximation, it’s truer that people have talent for research that is some mix of experimental or conceptual, and this underlying type predicts the age they are likely to make their mark.
That’s from a sample of just 31, but Jones and Weinberg (2011), and Jones, Reedy, and Weinberg (2014) performs a related analysis for winners of the Nobel prize in physics, chemistry, and medicine. Whereas the economics Nobel is closer in spirit to a lifetime achievement award, the Nobel in these other fields is for a specific discovery. Jones and Weinberg don’t classify these discoveries as “experimental” or “conceptual”, but they do break the winners down into theoretical and empirical discoveries. This is not an exact match with Galenson’s ideas, but theoretical innovation seems more closely related to conceptual innovation. Across all three fields, they find theoretical discoveries tend to be made at an age that is four years younger than empiricists.
Young and Disruptive?
Nobel laureates in general is a bigger sample than merely economics Nobel laureates, but it’s still a pretty small sample, though worth paying attention to if we think only the biggest scientific breakthroughs count for much. But we have some hints that this notion that conceptual innovation favors the young is a broader phenomenon.
One line of evidence comes from a measure of how disruptive is a given scientific paper or invention, based on how often a paper/patent is cited on it’s own, versus in conjunction with the papers/patents it cites. The intuition of this disruption measure is that when a paper or patent is disruptive, it renders older work obsolete and hence older work is no longer cited by future scientists and inventors working in the same area.
But while this disruption index was not designed to measure whether a discovery is conceptual or experimental, in the sense meant by Galenson, I think it seems related. We can imagine conceptual work is more likely to show up as more disruptive in this index, since it doesn’t arise from a deep engagement with earlier work; it’s a new fully formed thing, reasoned through in advance. Maybe for this kind of highly conceptual work, people don’t feel it’s important to cite the work’s own references, as they aren’t particularly germane.
In contrast, the typical experimental discovery might well be just one more incremental step in a direction of ever increasing novelty. It’s not a break with the past, and perhaps earlier work continues to be cited. Indeed, Galenson argues in the arts conceptually innovative artists tend to have their acclaim more concentrated on a small number of breakthrough pieces, while experimentally innovative artists have acclaim spread over many more pieces.
All that throat clearing is to say we have quite good evidence that papers and patents get less disruptive as scientists and inventors age. The figure below on the left comes from scientific papers across all fields, from Cui, Wu, and Evans (2022). The figure on the right corresponds to patents, and comes from Kaltenberg, Jaffe, and Lachman (2021). (For more on these papers, see Age and the Impact of Innovations).
Across both cases, older innovators produce less “disruptive” work (that is, their work is increasingly likely to be cited in conjunction with its own references).
A Time for All Things
The story up to now is that some innovations take a long time to achieve, possibly because they draw on deep wells of specialized knowledge, possibly because they require lots of iterative exploration; possibly both. We would expect this kind of work to be more prevalent among older scientists and inventors, and that seems to be the case, as measured by the novelty of phrases in their work, the keywords assigned to their work, the age of their cited references, and the tendency of later inventors to cite their work in conjunction with earlier antecedents. When this kind of work results in highly impactful work that results in a Nobel prize, the prize-winner is likely to have done the work when they were older.
Younger innovators have this door closed to them, in terms of impact. Their success is instead more likely to be founded on recent ideas, where low-hanging fruit are still there to be plucked. At least, that’s the impression one gets from the novelty of phrases in their work and the age of their citations. Alternatively, their work might draw on conceptual breakthroughs that can emerge from thinking through the implications of a simplified model - that might account for their tendency to score much higher on disruption indices and for Nobel prize winning work of this type to be achieved at a younger age.
But why can’t the paths preferred by the young also remain open to older innovators? Maybe younger innovators are forced to rely on the newest ideas and conceptual innovation to have a big impact, and older innovators simply add to this set of options the option to innovate with deeper expertise about older ideas and experimental innovation. Such a dynamic would also generate the flavor of result we’ve discussed: on average, younger innovators would use newer ideas and more conceptual innovation, but only because they are using exclusively those approaches and older innovators use those approaches plus other options open only to older innovators.
I think this is part of the story. Cui, Wu, and Evans compute the distribution of the age of references, relative to the year of first publication for scientists with different career lengths, and find scientists are most likely to cite work published right before their own first publication. But the longer a career, the wider the distribution of reference ages; new work is cited nearly as much as old work.
On the conceptual versus experimental divide though, this doesn’t seem to be the case that older innovators do both and younger innovators are exclusively conceptual, though are evidence here gets thinner. If we look at Nobel laureates, it simply isn’t the case that conceptual breakthroughs are widely dispersed across the young and old, while experimental breakthroughs are clustered among the elderly. Instead, it seems that Nobel prize winning conceptual/theoretical breakthroughs really are clustered among the young, and experimental ones among the old (it would be interesting to see the distributions of disruption index scores across age ranges, since we have much larger samples there, but neither paper reports it). Perhaps, as Galenson argues, there really is something about age that inhibits conceptual breakthroughs, whether it’s neurological or an outcome of accumulating facts that force us to build nuanced mental models? If that’s the case, perhaps the rising burden of knowledge - which raises the age at which innovators seem to get innovating - could begin to make major conceptual innovations increasingly rare. Indeed, maybe that’s a reason science seems to be getting harder!
Lastly, let’s return to the question of what kinds of innovations are sitting there, undiscovered, even though they do not necessarily rely on a very long period of study. I can think of one more category of innovation that could qualify, namely, innovations that depend on seeing old ideas in a new light. As we’ve discussed, perhaps that favors the young, who do not come with preconceptions and assumptions. But I would also bet on outsiders, who develop their assumptions and models of the world in another context. In a sense, such people are “young” to a new research domain, even if they are not biologically young. Indeed, older innovators might have an advantage here, since they may be able to connect ideas that are new to them to a pre-existing knowledge base. But only if they put themselves in the position to encounter ideas that are new to them, if not new to the world.
Thanks for reading! As always, if you want to chat about this post or innovation in generally, let’s grab a virtual coffee. Send me an email at matt.clancy@openphilanthropy.org and we’ll put something in the calendar.
These authorship positions signify important information in biomedicine. Typically the first author is junior and designs/executes the experiment, analyzes the data, and writes the manuscript; the last author tends to be the principal investigator of the lab where the first author works and supervises/guides the whole process.