Humanities Digital Workshop

This project employs the techniques of probabilistic topic modeling to test a set of longstanding assumptions about the periodization of German literary history. Scholars have applied a fairly consistent set of period designations to categorize German literature written during the span of roughly one hundred years between 1750 and 1850: “Enlightenment and Sensibility,” “Storm and Stress,” “Weimar Classicism,” “Romanticism,” “Biedermeier,” “Young Germany,” and “Realism.” Applying the MALLET topic modeling toolkit to a data set of 154 novels written between 1731 and 1864, the project has been evaluating whether these novels do in fact cluster together in ways that supports the scholarly consensus, or whether there might be hidden thematic structures in these works that point to new ways of thinking about their “proximity” to one another. This analysis has the potential to shed light on a range of research questions related to the literary history, especially with regard to understanding those features of texts that might cause us to classify them together. Is similarity, for example, best grasped in terms of similar themes (as a topic modeling approach would suggest), or should distinctions of style and structure play an equally significant role? To what extent do variables such as genre (the novel) or gender (male versus female authorship) generate patterns of similarity that challenge traditional ways of thinking about classification? A final component of the project involves an attempt to find compelling ways to represent visually the idea of a proximity among texts as measured across multiple variables. Network diagrams would seem to offer a particularly promising model for such visualizations.

Tagger Database

For the purposes of this project, we're examining and quantifying changes in the German novel by examining, categorizing and quantifying changes in the nouns in the novels. In effect, we're saying that, if German novels change as a result of changes in Germany's material culture, then we should see a change in the categories of nouns.

We understand a noun's "category" to be its place in a taxonomy of terms, similar to the taxonomy found in the hyponym-hypernym relationships in a formal lexicon like Wordnet (for more about Wordnet, see the article about it in Wikipedia). For example, a novel might use the noun "chair". "Chair" is a type of "Furniture", which is a type of "Artifact", which is a type of "Concrete Object."

To support this kind of categorization of nouns, we implemented a database and web front-end using our standard combination of MySQL and Django. We added an additional set of functions to permit users to quickly assign categories to nouns.

Our first version of the database included just two novels, which Professor Erlin and and a couple of students categorized manually. We pulled results from the database, and Professor Erlin presented his findings at a conference in the fall of 2009.

Our next version of the database includes 114 novels. And, since it's obviously time-consuming to manually categorize so much data, we're exploring ways of automatically categorizing nouns.

Natural Language Processing

To support Material Culture in the 19th Century German Novel, the HDW is developing a number of new capabilities:

• Natural Language Processing, including tokenization, part of speech tagging, stemming and lemmatization.
• The management of large-scale databases. The latest version of this database holds > 37 million tokens.
• The use of standard linguistic taxonomies. We've started using Wordnet, mostly as a way of getting used to handling large taxonomies, and of evaluating the usefulness of taxonomy driven methods against smaller sets of documents in English/

We recently acquired a copy of Germanet, an equivalent of Wordnet in German, and have just started examining it.