The Programming Historian

Depending on your vantage point, we have a looming opportunity – or a looming problem. Historical digital sources have reached a scale where they defy conventional analysis and now call out for computational analysis. The Internet Archive alone has 2.9 million texts, there are 2.6 million pages of historical newspapers archived at the Chronicling America site of the US Library of Congress, the McCord Museum at McGill University has over 80,000 historical photographs, and Google Books has now digitized fifteen million books out of their total goal of 130 million. Archives are increasingly committed to preserving cultural heritage materials in digital, rather than more traditional analog, forms. This is perhaps best exemplified in Canada by digitization priorities at Library and Archives Canada. The amount of accessible digital information continues to grow daily, making digital humanities projects increasingly feasible, and for that matter, necessary.

In this post, I will do two things. Firstly, I will give a sense of how much information is out there, and make the case for why Canadian historians need to start thinking about it. Secondly, I will introduce readers to the Programming Historian, a wonderful resources that at least puts you on the right track to a programming frame of mind.

TMI?

Too much information? (Photo of FEMA Publications Warehouse, WikiMedia Commons - http://bit.ly/zjmlYc

Information overload is not new. People have long worried about the impact of too much information. In the 16th century, the German priest Martin Luther decried that the “multitude of books [were] a great evil,” in the 19th century Edgar Allan Poe bemoaned that “[t]he enormous multiplication of books in every branch of knowledge is one of the greatest evils of this age,” and as recently as 1970, American historian Lewis Mumford lamented that “the overproduction of books will bring about a state of intellectual enervation and depletion hardly to be distinguished from massive ignorance.” The rise of born-digital sources must thus be seen in this continuous context of hand wringing around the expansion and rise of information.

Despite the frustrations of microfilm for today’s historians, as well as the pitfalls of separating the wheat from the chaff amongst rising numbers of modern sources, historians have undoubtedly benefitted from these technical developments. This is perhaps disproportionately for those engaged in social and cultural pursuits. Historians will profit meaningfully from born-digital sources. These, however, do present added – albeit surmountable – challenges due to their scope and production processes. Sources do not always have attributable or reliable authorship, are often undated, but in aggregate can give a sense of the zeitgeist of a time.

Library of Congress (Photo from WikiMedia Commons - http://bit.ly/ArU8YZ)

Storage price is falling. For example, James Gleick [in his book, The Information: A History, a Theory, a Flood] estimates that the Library of Congress collection is around 10TB (although the LOC itself claims around 200TB). These would previously have been unimaginable figures; I can now pick up 10TB of data storage for under a thousand dollars. Born-digital collections are larger, of course: the LOC’s digital collection is 254TB, larger than their print holdings, and the Internet Archive now has 3 Petabytes (PB) of information, growing at 12TB/month! In Canada, LAC has about 4TB of federal government web information and 7TB in its own internet archive. Information is also being preserved through programs such as the Roy Rosenzweig Center for History and New Media’s September 11th Digital Archive, the Hurricane Digital Memory Bank (focusing on Hurricanes Katrina and Rita), and, as of writing, the #Occupy archive. Online content is curated and preserved en masse: photographs, news reports, blog posts, and now tweets. These complement more traditional efforts at collecting and preserving oral histories and personal recollections, which are then geo-tagged, transcribed, and placed online.

What can we do about this conventional and especially born-digital deluge? There are no simple answers, but historians must begin to conceptualize new additions to their traditional research and pedagogical toolkits.

Where to Start: Programming

By the end of the Programming Historian, you'll have a basic know-how of Python and will be able to tackle projects requiring textual analysis.

One important thing we can do with this deluge of information is learn how to interact with digital information on a mass scale. Luckily, we have a tremendous resource available to us: The Programming Historian, by William Turkel and Alan MacEachern, hosted on the Network in Canadian History & Environment (NiCHE) site. Why might you want to open up this free, open-access website book?

• If you were to try to deal with born-digital sources in a traditional manner, you would spend A LOT of time flicking through websites. Much of it hasn’t been curated, and realistically, you could not read every blog comment published on a given day in Canada, navigate the tweets, or so forth. For this, you will need computational analysis.
• The same holds true for the conventional array of information discussed above: if you want to use 2.6 million newspaper pages to their full potential, there must be a way to “distant read” it.
• Digital history is ‘hot.’ The American Historical Association, meeting right now, is full of panels and twitter has been afire with the field. Even if you do not necessarily see yourself using programming languages, it behooves you to be able to understand it.
• And, most importantly, it isn’t that hard, and it doesn’t take that much time. You could move through the whole guide in a weekend, or – better yet – break it into small chunks, spending 20-30 minutes here and there.
• Finally, I believe we’ll also have to equip the next generation of historians, as I’ve written about elsewhere on ActiveHistory.ca.

The Programming Historian is very straight forward, but by the end of it, you’ll be able to do the following:

• In an automated, systematic fashion, you will be able to take a website and extract all of the words from it for further analysis.
• Establish word frequency, similar to what a Wordle word cloud displays (the possibile utility of this is discussed elsewhere on this site). Indeed, you will be able to make your very own tag clouds!
• Move beyond word frequency to see the keyword-in-context – i.e. you see that the word ‘aboriginal’ appears a hundred times in a given site, so why not see where it has appeared. This enables you to move very quickly to the relevant information.
• Download and harvest information automatically. Say you find a large collection of a hundred websites. Rather than clicking repeatedly through each to download the information, a simple script can do it for you!

Conclusion (and a proviso about why we don’t all have to be programmers!)

Don’t be afraid. It’s New Years, so why not make it your resolution as a historian to figure out some of these very basic steps. It could make you a better historian, or in any case, will equip you to figure out what’s going on. In any case, it’s an additional tool in one’s toolkit. Unlike earlier social science histories of counting with computers in the 1970s (which did revolutionize areas of historical inquiry), it is important to remember that we can use broad analysis to find issues, but then move dynamically down into context.

That all said, historians will not all have to become programmers. Just as not all historians need a firm grasp of Geographical Information Systems (GIS), or a developed understanding of the methodological implications of community-based oral history, or in-depth engagement with cutting edge demographic models, not all historians have to approach their trade from a computational perspective. Nor should they. Computational history – to use only a few examples – does not replace close reading, traditional archival inquiry, or going into communities to uncover notions of collective memory or trauma. Indeed, computational historians will play a facilitative role and provide a broader reading context; yet there will still be historians, collecting relevant primary and secondary sources, analyzing and contextualizing them, situating them in convincing narratives or explanatory frameworks, and disseminating their findings to wider audiences.

Surveying all of cosmic history using ChronoZoom: you can't even see human history up there in the upper right corner.

Historians aren’t always the best at crossing the hall to the sociologists across the way, let alone the astronomers, physicians, or geologists across campus. Scientists who study the Big Bang, however, are engaged in history – just a (very) different kind. Similarly, those who study the very long-term geographical forces that have shaped Earth, those who study evolutionary processes across flora and fauna, even those who study broader, galactic or universal phenomena, are often seen as very distinct from historians.

Big History, a new and emerging field, seeks to bridge these very real but also occasionally artificial disciplinary boundaries. It can be hard, however, to really establish how we can go forward and what a Big History approach might look like in real, deliverable terms (Bill Gates and David Christian have a great project also looking at how to teach these concepts to classrooms). Look no further: ChronoZoom, from the University of California-Berkeley’s Department of Earth and Planetary Science, has a working model that gives us a sense of what this might look like.

What is ChronoZoom? Just as historians cannot reasonably access millions of books without heavy computational aids, humans have difficulty even conceiving of the scope of ‘big’ human history that covers billions of years. ChronoZoom, “a tool to aid the comprehension of time relationships between events, trends, and themes,” aids in this. The most developed big history project yet in existence, ChronoZoom ties together extant online resources (scholarly articles, photographs, audio-visual media, etc.) by placing them along a constant timeline, stretching along a 13.7 billion year continuum from the Big Bang until the present day. While this would be initially overwhelming, its use of Microsoft’s Seadragon Deep Zoom technology – a smooth means of transitioning around an extremely large and high-resolution file without overwhelming the computer (similar to how one navigates Google Maps) – will allow a user to swap between an overview of natural history, a snapshot of human history, to a more focused overview of major events in the 19th century. Historians use time as a primary frame of analysis, and ChronoZoom represents the first major search engine project to recognize it as a constant base. Imagine using this in a classroom, to give students a sense of how long ago events truly were. Or, just sitting at home, using it almost as a “total perspective vertex” as Douglas Adams humorously foresaw in his Hitchhiker’s Guide to the Galaxy.

ChronoZoom after literally zooming into all of human history (3500BC onwards)

ChronoZoom isn’t fully fleshed out yet, but a very good working demo let’s us see some of the potential. Please fire it up yourself here (you’ll need to download Microsoft Silverlight for it to fully work). At first, it looks like a static poster (pictured above): sophisticated, to be sure, but nothing too exceptional. In the lower left hand of the poster we have the Big Bang, and, in the upper right, we have human history. However, you can’t see any human history at this point – it’s too small. But if you click on the ‘human history’ bar at the top, we begin to rapidly and fluidly zoom into the upper right hand of the corner. Cosmic history recedes into the foreground, Earth & Life history as well, pre-history appears and disappears.. and then we see human history. This process really needs to be played out for yourself: watching billions of years fly by, receding, our ‘human history’ a mere dot in this billions of years.

December 2000: on the very same chart that contained all of Cosmic History!

And from here, we can zoom in even more: to the 2nd Millenium A.D. for example, to the 20th Century alone, to its 10th Decade, to the year 2000, until finally, in the very upper-right corner of the map, we see New Year’s Eve of the year 2000 (and the beginning of our 3rd Millenium). This is a fantastic way to describe, deliver, and allow people to dynamically manipulate an otherwise inconceivable amount of historical information covering an absurd amount of time.

It seems weird to be describing, in text, such a dynamic website. What are you waiting for? Get out there and check it out! What do you think about it? I’d love to hear in the comments below.

If Big History fascinates you, please watch this TED video by David Christian on Big History. I guarantee you’ll find it entertaining and provocative.

The network of links stemming from ianmilligan.ca (activehistory.ca alone was too big!). This gives you a visual sense of the power behind hyperlinked information!

We will need to make dramatic changes to history undergraduate curriculums by aggressively implementing digital literacy programmes. This will benefit both our students and the historical profession.

Why? Let’s imagine how a future historian will tackle the question of what everyday life was in September 2011 – today. She will have a tremendous array of sources at her fingertips: the standard newspaper and media reports and oral interviews that we use today, but also a ton of added sources that would help give a sense of the flavour of daily life. Two hundred million tweets are sent every day. Hundreds of thousands of blog posts. Incredible arrays of commentary, YouTube videos, online comments, viewership and readership numbers will all hopefully be available to this historian.

But how will she read it all? Realistically, nobody is ever going to be able to get through all the tweets for even just one day: let alone categorize, analyze, and meaningfully interact with it. She’ll need to use digital tools. We are at a crossroads. This sort of history won’t be the be all and end all of future historical research, but I believe that somebody is going to do this sort of social history. Let’s make sure our future students are ready for it!

Using Mathematica, I have been able to track the rise and fall of the terms "war" (red) and "peace" (blue) across a comprehensive Top-40 Lyrics Database.

We need to begin thinking about how we are going to train historians of the future, today. Somebody is going to do this work. They are probably sitting in high school or elementary school classrooms today. When they show up at the university, let’s make sure that we’re ready to train them to write the history of today.. tomorrow. This is not simply for historians who fashion themselves as social scientists, as opposed to those who see themselves as pure humanists. It’s about deploying a tool which can provide information through which we can drape our stories, our interviews, our human anecdotes, etc.

Historians need to begin thinking about digital literacy and writing programs that will help access these sources. What’s going on right now? Tons. I have previously discussed one of the biggest current projects, the Culturomics project and their accessible Google n-gram viewer. You can see the rise and fall of a word or phrase (an ngram) and see how much it has been used across several centuries. It’s an incredible project, albeit not without some caution needed in how it is approached. There are also several digital history projects ongoing, some of which has garnered considerable attention (such as the Criminal Intent project in the New York Times).

This is just a hint of what’s to come. We need to be able to populate these future projects with even more historians. Which means thinking about how to train them today. Training these people by graduate school is simply too late, however. We need to begin training undergraduates in their first year. Indeed, as a recent study carried out at the University of Rochester indicates, students won’t adopt new technologies by the time they get to graduate work – the risks are too high. Let’s get them as undergrads.

What could a digital literacy programme look like for the next generation of historians, so that they’re ready to begin thinking and tackling these issues?

• We’ll need a firm grasp of the historiographic context of this shift – i.e. the old school quantitative historians, who crunched the census of Hamilton for example, or poured considerable time and effort into understanding demographic shift.
• Basic digital tools: What is cloud computing? How can we secure and backup our data?
• How can we digitally organize conventional sources? I’ve discussed this before in my post on Zotero.
• Grasping the SHEER SCOPE of large digital depositories. It’s one thing to say that Google Books has fifteen million books. It’s another to really grasp this. And to further realize what a drop in the bucket that is compared to other repositories of automated data being collected every day.
• Basic programming? The ‘Programming Historian‘ is a great start. What most of us will have to do won’t be so complicated and we need to be able to do it ourselves. While well-funded projects may be able to raise the funds to recruit teams of programmers to join them, or others may form collaborative and interdisciplinary work-teams, many historians will not be able to do so. They should be self-sufficient in this regard, at least for more simple and routine data mining exercises.

Our students should be able to come out of undergraduate history programs and be truly equipped for our knowledge economy and for the future demands of the profession. This will help teaching, research, and labour market outcomes. Information is increasingly being generated by the internet, written on the internet, and being consumed by internet users. People need to be able to create it, interact with it, in a fluent, comfortable manner.

What do you think? Should historians make this shift? Or are there disadvantages that I’m overlooking in my enthusiasm for this field of research? I’d love to hear from you all, especially as I begin my next project (a digital history of postwar English-Canadian youth).

Grow, Canada, Grow!

Wolfram Alpha likes to help you out. By simple typing ‘history’ into the answer engine, you’ll get a page demonstrating all the cool things you can do for historical topics. This is a reliable source of simple information for anybody wanting to quickly access basic facts. As noted by Léon Robichaud in the comments, however, however, the data on Canadian Prime Ministers is faulty – Meighen was the 9th Prime Minister but had the 3rd shortest term; similarly, 1st Prime Minister returns Harper! For example, if you type ’3rd Prime Minister of Canada’ into the engine, you’ll see it parsed as ‘Canada’ ‘Prime Minister’ ’3rd.’ And then you would [erroneously] learn that the third Prime Minister of Canada was Arthur Meighen, he governed for 2 months and 27 days, he was born in Toronto 137 years ago and died 51 years ago at the age of 86.

What’s going on with the Prime Minister section? Turns out that Wolfram is parsing the information in a peculiar way! It’s interpreting the list of prime ministers by term-length. You can see this by pulling up the list (press ‘more‘ on sequence to see Alpha’s take on it). There you have the correct sequence, but the data parses it by focusing on the LENGTH field as opposed to the sequence you have. This appears to be a unique manifestation, as opposed to the American list of presidents.

This is perhaps a more valuable lesson than the original post entailed! In any case, I’ll report this to Wolfram Alpha as it’s clearly not returning the data that we want. That way they can refine their results better. There is a ‘feedback’ button on the bottom of every page.

Say that you’ve always wondered what your ‘third cousin’ really is: a simple search demonstrates a genealogical chart! Population information abounds in the database: you can learn the population growth percentages of Canada between 1945 and 1960, or the population circa 1911 (7.21 million people, the 33rd largest country then in the World, and probably weighing approximately 504 462 metric tons). You can also access basic biographic (Harriet Tubman) or comparative biographic information (Stephen Harper and Pierre Trudeau), information on historical events (D-Day), or even defunct empires (Austro-Hungary).

The showdown everybody has been waiting for: the name Ian vs. the name Edith!

For contemporary information, you can do a search such as ‘Income Inequality of Canada vs United States‘ and quickly learn out Gini indexes, income share held by the various fifths of the population. How frequent does your own name appear? I learn that ‘Ian’ is the 72nd most popular name in the United States, and it has been steadily increasingly in popularity since 1960 or so. It’s a young name. Edith, on the other hand, is pretty unpopular these days – although it was pretty popular back in the day. Indeed, about 2% of people in their late 80s are named ‘Edith’ (as opposed to less than half a percentage under 10 these days). You can even do showdowns: “Ian vs. Edith” to see the relative ranges [my apologies to any of our readers named Edith - quite frankly, we both pale in comparison to the most popular US given names!]. Or simpler, everyday things: exchange rates, both historically and today (right now $2500 Canadian gets you $2538.38 US).

You can have a lot of fun with this information. For each result, you can see what sources were used to generate the information. If anything, it’s a snapshot into the future: refined, intelligent knowledge acquisition.

What do you think? Toy or tool? Would you encourage your undergraduates to check it out and play with it? How trustworthy is the information? Are you just thrilled to have a simple way to figure out how you’re related to your grandmother’s aunt?

[Text in red was modified and added from the original post]

Google Books even has the full text of LIFE magazine!

Organizations, activists, and laypeople are trying to put the sum of all printed knowledge on the internet. They’re facing copyright issues, ethical and moral debates, but it’s marching on nonetheless. Why should we have to travel to archival repositories, especially if they’re in an already convenient form like microfilm? Shouldn’t everybody have access to information, not just the select few who happen to have institutional affiliations? When it comes to access to information, we should be on an even playing field. Lay people interested in history, undergraduates, cash-strapped professional researchers, and all can benefit from several internet resources that put an incredible amount of information at your finger tips.

In this post, I’ll introduce people to the Internet Archive, the Haithi Trust, and Google Books. I hope to show you that there are incredible numbers of primary sources, digitized books, internet snapshots, among other things, out there. From an 1888 report on the Knights of Labor by a Canadian Legislative Committee, to the music video for the “first rap single ever released in Canada,” to American prohibition speeches, they’re all out there – free, accessible, and often downloadable.

The Internet Archive Webpage

What’s the Internet Archive? In a word, it’s incredible. It has collections of videos (530k+), live music (over 94k concerts), audio (over 914k recordings), and texts (2.8m+). It also has 150 BILLION old internet pages. For Canadians, it has a large collection of digitized microfilms from Library and Archives Canada (the above Knights of Labor report, for example), as well as radio clips, community videos, concerts, and others. An activist project, everything on the Internet Archive is free of charge. It continues to grow. If you use firefox, you can also put an Internet Archive extension into your browser.

The WayBackMachine – part of the Internet Archive – is an invaluable resource as well, both for researching and nostalgia. What did cbc.ca look like in the past? You can go all the way back to 1996 on a variety of dates to see what news issues appeared on the front page, how the layout of the page has changed, and maybe even do a systematic study of how it appeared. It’s the newspaper archive of the internet. It’s also nostalgic… look at this York University webpage from 1997!

François-Xavier Garneau's History of Canada - a free book, fully downloadable on Google Books!

Google Books, which is increasingly popping up in all of our web searches, has over fifteen million books scanned – and has the jaw-dropping goal of digitizing every single unique book existing in the world by 2020. They’ve run up against publishers and institutions. For copyright reasons, many books can only provide snippets – a percentage to give you a sense of the book, maybe a quotation or two, but you still have to buy access to it. Books in the public domain (approx. 2 million) are fully accessible and you can even download it as a very high quality PDF! Outside of the United States – like in Canada – many books have murky copyright status to Google so we need to go to the Internet Archive where public domain works are. There have been concerns raised, however.

To do a full text search of Google Books, make sure you select the ‘Full View’ tab on the left column of your search. For example, ‘Canada History’ will bring up 464,000 full text works! You can download these, and read it on your computer or even your e-reader! For example, say you want to read François-Xavier Garneau’s History of Canada (1866). Click here (PDF link), download it, and you’re good to go. And that’s just one small snippet.

Finally, Haithi Trust is a partnership of several major university libraries to digitize material (some by themselves, others from Google Books who copied their books – they demanded them back, apparently). They have almost nine million digitized volumes, of which 2.4 million are in the public domain. For Canadians, they have a decent collection. Search using the ‘only full view’ option and you can see a vast array of primary and secondary sources.

I see these resources as goods, although we need to continue to consider the intellectual rights of creators (especially those who don’t have state support for universities) as well as the importance to understand that it’s not all on there… yet.

Unigram comparisons for 'nationalize' and 'privatize'

Tracking the rise and fall of ideas throughout fifteen million books would have been impossible. Until now, thanks to the Google Books Ngram Viewer. Much like my previous post on Wordle tried to illustrate, we need to make sense of large quantities of information in order to do ‘big history’ and provide a context into which we can write our smaller studies. They’re also awesome for teaching or just playing around with and having (shock) fun with history.

On the chart at above right, we see a Google Ngram for two phrases: ‘nationalize’ in blue, ‘privatize’ in red. Does it surprise you? The idea of “privatize”ing is almost unheard of until the 1970s, and really picks up stream by the late 1980s and peaks in the 1990s. Conversely, nationalize slowly trends upwards until the 1970s, and then declines. This might not be surprising, but it’s an example. In this post, I’ll tell you what an ngram is, show some cool pictures, and hopefully drive you to have some fun with this.

Bigram "youth delinquency" (1900-2000)

What is an ngram? A dictionary defines it as “a sequence of variable characters that stands for a word or string of words in a corpus.” Blagh. What it really is are looking for PATTERNS within a broad ton of materials. If we’re looking for “nationalize,” we’re looking for a unigram. If there are two words in a pattern, it is a “bigram,” three is a “trigram,” and as they get bigger we just call them n-grams. At left is an example of a bigram: “youth delinquency.” Here, we can see an idea that really isn’t discussed until the early 1920s, skyrockets during the Second World War (soldiers were away, and people feared that youngsters were deprived of role models, etc.) and stays fairly high thereafter. Results are normalized against the numbers of books published, and are percentages rather than sheer appearances.

I’ve used this in my own teaching. You can play with the “youth delinquency” n-gram and ask students to postulate why there were certain heights and lows, based on their own reading of articles. It forces them to think a bit outside the bounds but also to come up with questions of their own.

The Rise (and fall?) of the unigram 'Canada'

The goal is to find something that maybe YOU CAN’T EXPLAIN YOURSELF – and then come up with something to look into. Let me know in the comments if you can come up with something cool!

Here are some options you can play with:

1. Change the “corpora” that you’re searching, from American English and British English to books written in French, Hebrew, German, Spanish, Russian, English fiction, etc. This can be easily done with the drop down box above the graph.

2. Smoothing: the examples that I’ve set are using smoothings of three. Basically, if you don’t smooth, you see massive spikes – especially in the early years (google claims that before the 19th century only about 500,000 books were published). Here is the technical explanation: (and also accessed with a drop down box)

Often trends become more apparent when data is viewed as a moving average. A smoothing of 1 means that the data shown for 1950 will be an average of the raw count for 1950 plus 1 value on either wide: (“count for 1949″ + “count for 1950″ + “count for 1951″), divided by 3. So a smoothing of 10 means that 21 values will be averaged: 10 on either side, plus the target value in the center of them.

3. Play with dates: you can begin in 1500 and go all the way to 2000. If you want to be more specific, play with the date ranges and see a more specific one.

One thing to look out for is the “medial s” if you’re dealing with pre-1800 sources. As William Turkel explained in a visit to York, a certain four letter expletive turns up to a dramatic degree in the pre-1800 period – it then becomes “suck” from that period onwards. You can look at this chart (warning, explicit language) for a vivid demonstration of what Google and Turkel are explaining.

Cities of Canada

I’ll leave you with one biggie: a chart looking at the unigrams of Toronto, Montreal, Vancouver, Calgary, Edmonton, Saskatoon, Halifax, and Winnipeg. Here we can see how Toronto really takes off after the early 1960s, reflecting its growth and current economic position. Whether this is a good thing or bad thing I’ll leave to you.

The trigram "League of Nations" against the bigram "United Nations" (1914-2000)

Pretty neat stuff, eh?

Source shoutout

Jean-Baptiste Michel*, Yuan Kui Shen, Aviva Presser Aiden, Adrian Veres, Matthew K. Gray, William Brockman, The Google Books Team, Joseph P. Pickett, Dale Hoiberg, Dan Clancy, Peter Norvig, Jon Orwant, Steven Pinker, Martin A. Nowak, and Erez Lieberman Aiden*. Quantitative Analysis of Culture Using Millions of Digitized Books. Science (Published online ahead of print: 12/16/2010)

The 1933 CCF Regina Manifesto (all images produced by Wordle.net)

With politicians out on the hustings, what better time than to go through the old political speeches, manifestos, and platforms. Using Wordle, we can throw them up and look at word clouds. They’re not just pretty, but they can let us see the evolution of political thought and what words were capturing Canadians. They also let us see what things remain the same: most Throne Speeches over the last 15 years are nearly identical, stressing ‘government,’ ‘Canada,’ ‘Canadians, ‘etc. But we can see discontinuities: the 1933 Regina Manifesto, for example, contrasted with contemporary NDP promises and platforms (‘family’ and ‘home’). Reading all the documents might be preferable, but this is quick (it takes a minute to produce the picture at left) and has great possibilities for dealing with large quantities of information.

A proviso: this isn’t stunning analysis. It’s more just a demonstration of how we could take textual information, throw it up into a visual format, and then see changes over time. The examples I use might seem straightforward to some of our readers. But you could put these down in front of people who aren’t familiar with the story, and see how the CCF of 1933 is demonstrably different than the NDP of today.

What is a word cloud? Basically, it goes through a text, counts how often particular words appear, scrubs out ‘stop words’ (which are common words like ‘the’), and then throws them up on the screen. If the most common word was ‘Canada,’ it would appear both larger and bolder. If the least common word was ‘socialism,’ it either wouldn’t appear at all or in tiny, almost imperceptible text. The clouds here were created using Wordle.net, although you could make your own with a simple programming script. Wordle is more accessible, however, and much prettier.

The 1933 Cooperative Commonwealth Federation’s Regina Manifesto word cloud captures some element of the spirit. MUST is the most important word, capturing the urgency which its drafters saw the need to profoundly change Canada’s economic system. Others: PUBLIC, SYSTEM, ECONOMIC, POWER. Canada appears to a lesser degree, and also some words you wouldn’t see in a contemporary stump speech: capitalist, socialized, even ownership and workers sneaking in there.

The 1970 'Manifesto for an Independent Socialist Canada' (Waffle Manifesto)

If you were to then throw up the 1970 Waffle Manifesto, you’d see ‘Canadian,’ ‘Canada,’ ‘American,’ capturing the national aspects of it. But still socialist, socialism, capitalism, struggle, capturing the radical critique of the present order.

The NDP Platform (c. early April 2011 before it's been fleshed out)

It gets trickier when you want to compare it to recent NDP platforms, as manifesto has faded in favour of sound bits and short less precise promises. But if we put the 2011 NDP platform in, we see different emphases: family, families, make, home, affordable. Gone are the imperatives, the ‘musts,’ the ‘workers,’ the ‘capitalist’, other such analysis. If you’re teaching or trying to understand the shift in political discussion based on these textual materials, why not convert it into visual form for some quick and easy analysis? In about 5 minutes, you can make a point to an audience (many of whom like to actually see stuff, like me, instead of just read it).

Here are some other examples: the Students for a Democratic Society Port Huron Statement (1962 and the Liberal Red Book (1993).

NDP Platform from April 10th (post updated to add)

1962 Port Huron Statement by Students for a Democratic Society

The 1993 'Little Red Book' for the Liberal Party

Many people use Twitter for personal social/professional pursuits: finding links, having communication with a broad audience, self-promoting your blog on making history relevant (“follow us,” we cry). But you can use twitter in the classroom to create a sense of community, facilitate communication out of class, and hopefully open students’ eyes to the enormity of the world and the role that digital communication plays in ongoing events. As a long-term skeptic about the utility of twitter – and somebody who continues to avoid Facebook – I hope to reach the digital skeptic here.

When I first heard of Twitter in mid-to-late 2006, it sounded inane. 140 characters seemed restrictive for text (SMS) messaging, let alone as a means to communicate over the internet. We have e-mail, I probably snidely dismissed, and then went back to predicting the eminent end of Facebook. It wasn’t really until 2009 that I realized I had been wrong.

It was at the Great Lakes THATCamp, where co-editor Jim Clifford and I went to Michigan Sate University, that I saw the power of an ad-hoc network. By adding the hashtag #THATCamp, your message went to everybody following that particular tag. You didn’t have to be friends, you didn’t have to know each other, but you could carry on conversations throughout the Conference. Have you ever sat in one panel at a large conference and wondered what was going on in the next room over? Whether there were opportunities for cross-talk that were simply gone because the programme committee decided a certain way? Wished that you knew where people were congregating for an ad-hoc debate on historical materialism, or social history, or whatever? During the G20 in Toronto, following the #g20 hashtag quickly let you realize what was going on, where protests were, where police were, and painted a disturbing picture of quickly eroding civil liberties. And, of course, the recent events in the Middle East and North Africa continue to demonstrate the organizational powers of twitter.

But enough of that. How could an instructor use Twitter? This is me thinking out loud here – something that I hope to try next year as a TA.

Firstly, you have to get students to sign on at the start of the class (day one). Put it on the syllabus. Demo how it works on the first day of class. Like everything that you do involving digital humanities, we have to assume that digital literacy levels are very low. Take them to the twitter.com homepage, show them how to make an account (maybe make your official course account at this time), and demonstrate a few trending issues. Say you’re running a class on sexuality. Why not see what the current debate on #circumcision is? Could academics play a role in informing debate?

This is probably the most crucial point. As Sean Kheraj pointed out to me via Twitter, there just aren’t enough people bought into Twitter.

Secondly, encourage your students to use it during class. In a large lecture hall format, if a student thinks something is cool or has an interesting point, it may not be feasible to interrupt the flow of a lecture for every comment. At schools like York University, I also suspect there’s a culture of passivity in the lecture hall. So get them to put comments up, attached with a hashtag, say #HIST1000 [or whatever your course code is].

Imagine how gratifying it would be to see: “Wow, just learned a neat point in my #HIST1000 class about X,” questioning an issue, or even asking for clarification from peers. The problem is, if only one or two or even ten students have bought in, this would be a boring conversation. Hence the importance of that introductory lesson on day one of the course.

Most importantly, however, it would continue the conversation outside of class. At York University, a commuter campus located on the periphery of Toronto, I’m continually shocked by the sheer level of student alienation. Many students note that they have difficulty making friends, as they rush from class to class, worry about making bus connections, and face such large classrooms that peer interaction is so limited.

So imagine: asking for lecture notes if somebody couldn’t make it? Having difficulty finding a reading? Forming ad-hoc study groups? Complaining about the reading? Even complaining about the instructor! Sure, we have Moodle and the occasional course blogs. But those quite frankly, aren’t conducive to off-the-cuff communication. They’re coldly, resistant to customization, and really only provide a venue for a course forum.

Finally, as Sean mentioned in a tweet, it might help students follow your research. Let them see what you do, get insight into the daily life of a researcher, the questions asked and considered. And even if you just reach one or two students, what a difference that might make!

This is part of the ongoing ‘step-by-step‘ series which aims to guide users through online research tools and teaching aids. For Monday, stay tuned to a discussion about Twitter in the classroom.

In this post, I’ll explain to students how to install Zotero on their home computers. As a teaching assistant, I’ve found this to be the most useful technological skill that I’ve taught undergraduates – many have confirmed this by noting how they now use it. The explicit inspiration for this comes from William Turkel’s ‘Going Digital in Two Hours,‘ a fantastic workshop that he ran for York University’s Graduate Programme in History last year. Kudos to him!

Why Zotero? In short, it will properly format footnotes/citations (critical if you’re taking courses amongst several disciplines) and keep a research database in the ‘cloud’ (i.e. you can log in on any computer and it’s all there). For graduate students and faculty working on large documents, it can also streamline referencing and make sure that you have perfect footnotes.

My philosophy here is to assume that students are starting from ground zero: don’t assume they use Firefox, don’t assume that they use MS Word – you need to sell them on every aspect if they’re going to find time in their often busy lives.

STEP ONE: Download and Install Firefox Why Firefox? Firefox is both free and open source (which means that anybody can use or alter the program’s code). You will probably find it quicker than Internet Explorer. Most importantly, however, you can download extensions. These improve your browser, and can make it a specialized research tool and bibliography manager. You could spend literally hundreds on these tools commercially, but here you can get them for free.

BROWSE here and click on the ‘Download Firefox’ button. It will automatically detect your operating system. It should be under 20MB.

STEP TWO: Install Zotero and Integrate it with your Word Processor In my opinion, this is the coolest thing you will find in Firefox. I wish I had known about this earlier. Again, it will do your footnotes for you, properly, with short-forms and ibid., and automatically reformat them if you add footnotes in between.

BROWSE to http://www.zotero.org/ , click on ‘download.’ When you have to reboot your browser, do so. You will now have zotero running at the bottom. When you go to a website about a book (either through York library or Amazon.ca), you will now see a little book icon in your web browser address line.

Click it, and you’ve added a book to your Zotero database. This is a good way to collect a list of books, journal articles, etc. You can actually add notes about them, and begin building a comprehensive research database.

Most handily, though, you can now link this into Word or OpenOffice

CLOSE Word or OpenOffice. BROWSE here. Download your plug-in. This will be done by clicking on either ‘Install the Word for Windows Plugin,’ or the ‘Install the Word for Mac Plugin.’ Mac users will have to download an additional file, by clicking on ‘install PythonExt from zotero.org.’

If you are confused, please watch this video.

Now you will have an INSERT CITATION BUTTON or MENU, which you can click to add a footnote. It will bring up a list of your books and articles. You click on this work, and it will generate a citation. The first time you use it, it should ask you what citation style you want. Most historians use ‘Chicago Style.’

Note in the screen above, that you can type in the page number! You may need to rejig the first citation of any book slightly, but subsequent ones will show up in either proper short form or as ibid if they directly follow the previous one.

It will also automatically generate a bibliography.

This is useful as you can use Zotero as your central notekeeping database. It also saves to the ‘CLOUD,’ or the internet. This means that you can use Zotero on multiple computers and your data should remain the same if you are logged in.

Do laptops have a place in the lecture hall? An ongoing debate has raged over whether they do. At York University, the on-campus newspaper Excalibur noted that many history professors were opposed to the use of laptops in lecture halls, a discussion continued by a large departmental meeting in early January. This has not been an isolated discussion, however. In this post, I hope to provide some background to the debate by noting some of the other sites of discussion, and then break it down into what I see as the two main issues: civility and the role of the professor, as well as learning and the role of technology. Keeping this in mind, I believe that we need to reframe our teaching approach as adult education and adapt to the use of technology. Certainly, there will be times when we should close our laptops and enter into discussion, but this does not need to lead to a blanket ban of technology in our classrooms.

As wireless internet became widespread across university campuses during the early years of the millennia, different approaches appeared. Some universities, like Acadia, had already incorporated technology into almost all of their teaching through the Acadia Advantage programme (begun in 1996). Others, however, began banning these tools as distractions (as a 2003 article from the University of Saskatchewan indicates).

Lately, however, it seems to be reappearing as a serious issue. A debate played out on the GlobeCampus blog, inspired by a London Free Press article about debates at the University of Western Ontario. In University Affairs, Fred Donnelly argued that laptops heralded a return to pre-industrial notions of multitasking – factory masters imposed strict single-tasking work regimes – and should not be flatly dismissed. This earned a derisive response from Todd Pettigrew at Macleans.ca On Campus, snorting that “while professors pat themselves on the back for being in touch and progressive, for creating a dynamic new learning environment, they are really creating an environment of increased contempt for learning and study.”

This post is focused on lecture-format instruction, rather than group work or one-on-one interaction. In this case, I feel that one of the main issues is the idea of civility and the role of the professor. Who controls the agenda in the classroom? Do we have a moral obligation towards our students – making them into ‘book-centric’ agents, strengthening their attention spans? Crucially, however, do they need to ‘mind their manners’? Do they owe it to the instructor to provide their full and undivided attention? What degree of control do course instructors and teaching assistants have over their students?

Certainly, instructors have the obligation to ensure an amendable learning environment for all. Students cannot loudly speak during lectures, as this disrupts their peers’ ability to learn. Similarly, they should not watch live-stream videos: be it NHL games, TV shows, movies, YouTube videos, etc, nor can they play video games, manage their online farms, or other such active, moving image-heavy activities. These colourful interventions draw the eye towards them and are a flashpoint for intra-student conflict. If they’re ensuring an amenable learning environment for all, professors also need to allow students with learning disabilities access to laptop computers (and the solution of banning all but those with registered learning disabilities strikes me as discriminatory and uncomfortable).

However, unobtrusively checking one’s e-mail, catching up on a quick update of how badly the Leafs are losing, or thumbing through a few BlackBerry messages is arguably different. They are less visibly stimulating, more passive (although a valid point can be made that online advertisements are becoming increasingly irritating), and I feel that they are not unduly disruptive to peers. Indeed, if some peers are distracted by this, that’s part of an issue they’ll have to deal with as they move through academic, corporate or non-profit worlds. I understand that this may be a point of contention – and I am, to some degree, making a value judgment that privileges certain kinds of activities over others. In my mind, in any case, the debate shifts from intra-student to the dynamics of student-professor respect. Do professors deserve respect from their positions alone, or does this need to be earned? Do they have an educational role beyond that of subject matter expert: are they teaching students to be ‘good, respectful citizens’ – beyond the content of their history course?

A moral responsibility towards students has long been a factor in the modern university, harkening back to the heady days of ‘in loco parentis’ (in place of the parent): women subject to curfews, same-sex dormitories, severe restrictions over reproductive rights, control over course selection, prohibiting off-campus activities and an array of political and intellectual speech. Much of this was dismantled in the 1960s through student protest and the active engagement of student governments, although small relics live on here and there. I think the laptop debate contains a moral dimension.

Yet, to me, the key point is that we are providing adult education. Students may seem young at the age of 18, but they’re now able to vote, be deployed overseas with the Forces, sign contracts and all of the trappings of legal adulthood. They are making a tremendous financial and personal sacrifice to attend university, an increasingly expensive proposition with increasingly dwindling outcomes. To support this, they’re working hours upon hours at part-time jobs, balancing interpersonal relationships, romances, family obligations, among others. A liberal arts education is about more than cost-benefit analyses, but we need to accept that students are autonomous agents with the ability to manage their own time. If they’re not disturbing other students in the class, why not let them use their laptops? Leadership and respect are transformative and earned processes.

Secondly, do laptops enhance or detract from the overall educational experience? They can draw students out of the classroom, lead to some disengagement, and are indisputable sources of some personal distraction (just as newspapers, doodling, daydreaming, editing papers for other courses, or concocting elaborate lecture hall related games). Yet for others, they let them delve deeper into a point raised in lecture, run down primary documents, take exquisitely organized notes, double-check moments when the professor might seem uncertain, and other tasks. Lest you raise an incredulous eyebrow, every single example provided here occurred in the last semester of the course I’m TAing at York. Laptops can also transform the isolated island of the lecture hall into a significant node in a broader network of knowledge. A recent study in a Calgary high school also demonstrated that “computer technology can dramatically improve the way students learn — but only if coupled with engaged teaching.”

We need to further incorporate computers into our teaching and learning processes. A survey of my undergraduate class demonstrated that all had access to a computer. Not all will have the same capabilities, however, and we do need to watch out for creating a ‘classed’ student experience where only some students can afford the latest and greatest technologies. As one student observed during our recent departmental meeting, they’re now all encouraged to buy laptops when entering the program at considerable financial cost. They now have a tool at their hand that can potentially:

• provide unparalleled access to information
• provide unparalleled opportunities for widespread co-operation: why not launching course wikis (everybody can dynamically edit and maybe provide ever-developing and dynamic course notes), or course twitters (allowing a counter-discussion which could both challenge and accentuate the course lectures) – what a way to transcend the arbitrary barriers between course sections or even courses across departments or the university?
• allow us to challenge the implicit strictures imposed by pen/paper – taking notes and organizing material in their own way, rather than the one way provided by the instructor
• transcend learning disabilities to some degree, provided students can afford individual laptop PCs
• and more – we need to begin seriously thinking about opportunities and the structures that could make them realizable.

In the course that I teach with, we’ve launched a course twitter feed. We hope that during the lectures, students will tweet questions and reactions, creating a conversation that can exist politely alongside the master narrative provided by the course director. Then, at the mid-class break, a teaching assistant will summarize the online reaction and use that as a launchpad for a quick discussion or issues to be further developed in tutorial. Who knows if it will be successful, but we need to try. Many of our students know more about this stuff than we do, and we can learn from them.