Wolfram|Alpha API and its Historical Potential

Thinking about a comment by Jonathan Goodwin on my last post about extracting weather data for specified dates in a historical document, I looked into the Wolfram|Alpha API. Luckily, it’s pretty powerful and you could then replicate some of these findings within your own programming language or environment. For those of you who haven’t used an API, I want to just give you a quick sense of how you can call this information yourself. This is a very basic introduction (i.e. letting you know the API exists), but I remember my first tinkering with APIs so hopefully this can help.

It could also be used to jump-start work into the other things I’ve been doing with MathematicaRead more

Extracting Weather Data for All Dates in a Document: Adding Context to a Document

The weather in Ottawa for Wednesday, November 25th, 1981 (date extracted from Trudeau-Lévesque correspondence). What if we could get this data for every date in a document?

Building on my theme of using Wolfram|Alpha to figure out things about the past, I wondered if I could write a program that could take a document or secondary source, extract all of the dates, and then let you know about that day’s weather. I envision this as eventually being a reader that pops up weather data as you’re reading something, giving you some added context about the date being mentioned. It would have to be automated, though, otherwise you might not want to use it!

Luckily, Mathematica has date recognition functions built in. As a proof of concept, I decided to test it out on the Wikipedia page for Stephen Harper (Canada’s current Prime Minister). We’ll try it out on primary documents below. Some jigging is required depending on the date format, but I could easily write a function that could get all dates. Read more

Topic Modelling in the Lyrics Database, Part Three: Talking to Wolfram Alpha

(and here my series continues… I’m blogging through August mainly to keep the work going when it can be so easy to sneak away, and this is more of an internal diary than anything else!)

Importing economic data into Mathematica – it really is this easy…

Mathematica is made by the same company, Wolfram Research, that brings us Wolfram Alpha – the computational knowledge engine that powers parts of Siri, as well as being an overall fun resource to use as historians, tinkerers, or well, anybody (I’ve written about it before). As a diversion, I thought I would start comparing economic data to the topics that I am finding through MALLET.

Using free-form input, let’s get annual figures for unemployment in the US, 1964-1989.

With that done, we can then manipulate our data – getting them into comparable datasets – and begin to run correlations. Let’s see if we can find correlations in topic occurrences against the unemployment rate… Read more