Big Data in History

Patrick Manning, Big Data in History, London 2013

The author, Patrick Manning, is Professor of World History at the University of Pittsburgh. He pursues a big goal: As a Director of the Center for Historical Information and Analysis (CHIA) he wants to develop and build up a world wide historical archive, thinking that time has come to create a coherent record of human social change. He compares his project to those in climate modeling and genetic databases. The small book is an introduction into the project of the CHIA, that exists since 2007.

The global dataset on human societal activities should include four to five centuries. It will provide a new comprehensive documentation of the past and us such a basis for planning global policies for the future. Currently available historical information has to be searched dispersed in separate archives. Big Data in history will digitize growing portions of it, link scattered records by place, time and topic and create a global picture of the various changes in human society. The initial stage of the project focuses on economy, politics, health and climate. Later il will address ideas, culture and values.

Professor Manning’s challenge is huge: there are big quantities of complex data to be collected and processed. Historical data in small files need to be digitized, documented and transformed to become parallel to other datasets before they can be analyzed. But the value will be great: Historical key valuables will enable us to learn about global processes of growth, cycles and interactions, Manning believes.

Bild Big Data in HistoryHe defines archive as “world-historical data resource”, refering to a “whole system of repository, documentation and analysis” (see Figure). Level 1 will contain all crowdsourcing ingest and pre-processing of data. In Level 2 each dataset undergoes “harmonization”, “a set of processes that enables datasets to be linked to others”. The key result will be a consistent set of metadata that will “contribute to the broader goal of approximating a world-historical ontology”. Level 3 focuses on the aggregation and on datamining, Level 4 on the formal analysis of data. Here a very exciting task will be done: Combining data and social theories in order “to estimate missing values in historical data” (see chapter six to this). Here, Manning thinks about early times and areas where data are scarce, as in Africa for instance. Level 5 will help to visualize data.

Manning’s project is based on the principle that a global analysis needs global data. But this will work only if the “obstacle of misconceptualizing history” is overcome, he says: “Too often, history is presented simply as narrative of distant ancestors, with little attention to the logic of historical change”. Such weakness in historical theory is compounded by “weakness in conceptualization of the world”: Some leave out whole continents, notably Africa, other overestimate the isolation of whole regions. Manning pleads for an interplay among scales and regions. He illustrates this with historical variables as lifespan, migration, textiles, silver, empires, social inequality and epidemic.

Manning’s project advocates “the notion of global patterns in human society”. His model for that is the theory of systems that are conceived as “collections of interacting components combining to a larger whole”. A world wide historical archive will require the unification of social science analysis and advanced techniques for the simulation of data. All the other technique necessary for the implementation is already available, Manning says. He describes it in the chapters four to six.

Mannings project is very interesting. He transfers parts of the concept of Viktor Mayer-Schönberger and Kenneth Cukier on Big Data and of the We think Movement of Charles Leadbeater into the sphere of historical and social sciences. There are, obviously, some open questions to such a huge project: First, Manning probably underestimates the necessity to select data before collecting and processing it – a challenge that is well known to existing archives. Second, the simulation of missing data would be quite a revolutionary method in history – something to be backed thoroughly by theory. Third: Manning’s concept runs totally opposite to decentration, the principle mode of working in the digital era. And, four: It will be interesting to confront his arguments with those of Hans Ulrich Gumbrecht, the big skeptic on the global scale and The Big Now (or the enlargement of the presence). Finally, a remark on the science of history: Historians normally work with more sophisticated questions and issues as presented by Manning.

Guido Koller

Senior Historian, Swiss Federal Archives, CH-3003 Berne, Switzerland

More Posts - Website

Follow Me:

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.