A big-data approach to history could help save the future
In its first issue of 2010, the scientific journal Nature looked forward to a dazzling decade of progress. By 2020, experimental devices connected to the internet would deduce our search queries by directly monitoring our brain signals. Crops would exist that doubled their biomass in three hours. Humanity would be well on the way to ending its dependency on fossil fuels.
A few weeks later, a letter in the same journal cast a shadow over this bright future. It warned that all these advances could be derailed by mounting political instability, which was due to peak in the US and western Europe around 2020. Human societies go through predictable periods of growth, the letter explained, during which the population increases and prosperity rises. Then come equally predictable periods of decline. These “secular cycles” last two or three centuries and culminate in widespread unrest – from worker uprisings to revolution.
In recent decades, the letter went on, a number of worrying social indicators – such as wealth inequality and public debt – had started to climb in western nations, indicating that these societies were approaching a period of upheaval. The letter-writer would go on to predict that the turmoil in the US in 2020 would be less severe than the American civil war, but worse than the violence of the late 1960s and early 70s, when the murder rate spiked, civil rights and anti-Vietnam war protests intensified and domestic terrorists carried out thousands of bombings across the country.
The author of this stark warning was not a historian, but a biologist. For the first few decades of his career, Peter Turchin had used sophisticated maths to show how the interactions of predators and prey produce oscillations in animal populations in the wild. He had published in the journals Nature and Science and become respected in his field, but by the late 1990s he had answered all the ecological questions that interested him. He found himself drawn to history instead: could the rise and fall of human societies also be captured by a handful of variables and some differential equations?
Turchin set out to determine whether history, like physics, follows certain laws. In 2003, he published a book called Historical Dynamics, in which he discerned secular cycles in France and Russia from their origins to the end of the 18th century. That same year, he founded a new field of academic study, called cliodynamics, which seeks to discover the underlying reasons for these historical patterns, and to model them using mathematics, the way one might model changes to the planet’s climate. Seven years later, he started the field’s first official journal and co-founded a database of historical and archaeological information, which now contains data on more than 450 historical societies. The database can be used to compare societies across large stretches of time and space, as well as to make predictions about coming political instability. In 2017, Turchin founded a working group of historians, semioticians, physicists and others to help anticipate the future of human societies based on historical evidence.
Turchin’s approach to history, which uses software to find patterns in massive amounts of historical data, has only become possible recently, thanks to the growth in cheap computing power and the development of large historical datasets. This “big data” approach is now becoming increasingly popular in historical disciplines. Tim Kohler, an archaeologist at Washington State University, believes we are living through “the glory days” of his field, because scholars can pool their research findings with unprecedented ease and extract real knowledge from them. In the future, Turchin believes, historical theories will be tested against large databases, and the ones that do not fit – many of them long-cherished – will be discarded. Our understanding of the past will converge on something approaching an objective truth. [Continue reading…]