Averting the Digital Dark Age

This project seeks to identify the cultural, intellectual, and theoretical factors during the early 1990s which contributed to the momentous decision to pursue ambitious and widespread web archiving. Secondly, it traces the technical development of new web archiving systems from the “digital dark age” worries of 1995 to the sophisticated systems developed in 1996, particularly at the Internet Archive. Finally, it will assess the accessibility, effectiveness, and impact of these new web archiving systems using the attacks of September 11th, 2001 – a major historical event accompanied by intense online activity – as a case study.

Full Description

On 6 August 1991, Tim Berners-Lee posted a description of the World Wide Web onto the Internet newsgroup alt.hypertext, as well as instructions for running a web browser. Those with an Internet connection could now create websites and link to others. This ultimately started the process of creating the Web that has so profoundly shaped our world today. While initial adoption was slow, reflecting low rates of global Internet access, within two years the Web was growing. By 1995, pundits and others began worrying about the fate of all of the information that people were putting on the Web. Unlike the conversation around backups or preserving physical storage, this conversation centered on the impact that this would have on the human cultural record. Was a “digital dark age” upon us, where electronic information would simply be lost?

In 1996, web archiving began in San Francisco with the Internet Archive, in Sweden with the National Library of Sweden’s Kulturaw3 project, in Australia with the National Library of Australia’s Preserving and Accessing Networked Documentary Resources of Australia project, and a year later, with the University of North Texas’s CyberCemetary project. By 2001, the Wayback Machine was launched, granting users seamless browsing of archived websites dating back to 1996. Today, web archiving is a routine part of library and archive operations, with national libraries, universities, and individuals alike engaging in the distributed, collaborative, process of preserving websites and other digital objects for future use. In short: 1996 and 1997 saw a dramatic transformation in terms of how we preserve information. The “digital dark age” lasted only between the launch of the Web in 1991 and the beginning of these programs five years later in 1996.

This project seeks to understand this digital preservation moment through three objectives. First, it seeks to identify the cultural, intellectual, and theoretical factors during the early 1990s which contributed to the momentous decision to pursue ambitious and widespread web archiving. Secondly, it traces the technical development (and geographical dispersion across three continents (North America, Europe, and Australia) of new web archiving systems from the “digital dark age” worries of 1995 to the sophisticated systems developed in 1996, particularly at the Internet Archive. Finally, it will assess the accessibility, effectiveness, and impact of these new web archiving systems using the attacks of September 11th, 2001 – a major historical event accompanied by intense online activity – as a case study.

The time for this project is now. Confronted with a medium shift, governments and other organizations implemented policies to preserve information for future use. As we are today in the midst of a similar shift – towards transitory communications on Microsoft Teams, Slack, and other systems as well as wholesale move towards the “platform” Web (Facebook, Twitter, etc.) – having an in-depth historical understanding of how we have responded to similar challenges is invaluable. This will help ensure that our record is stewarded for the next generation.