2 Rubric and Assessment

• Coursework: 60%
• Capstone Exercise: 20%

• Community: 20%

• All work indicated in each section of the workbook under ‘What you need to do each week’ has to be completed.

• All work has to be completed to a satisfactory level, per the criteria below.

By the end of this course, you should be able to do the following:

1. Understand how to use the idea of ‘the productive fail’ to use computing power effectively as an historian;
2. Understand, plan, and employ concepts of computational reproducibility in the service of history
3. Employ web tools to build history in public, collegially
4. Weigh the evidence appropriately, understanding how it was created, in order to communicate the compelling story
5. Understand the ways digital tools change us as we use them, to create compelling history that is self-reflexive

These objectives map against the criteria for your coursework and capstone exercise grades like this:

Absent	Criteria	Present
	1. Productive ‘Fail’ documented
	2. Reproducibility fully enabled
	3. Collegiality amply demonstrated
	4. Evidence carefully and fully engaged with, analyzed
	5. Self Reflection present in all work	.

Over the course of the term, you will produce different pieces of work that speak to these different criteria:

Criterion 1: online fail logs to keep track of what you actually do at your computer

Criterion 2: online notebooks that tell the story of what you did, and why, and what you did about it when things didn’t work

Criterion 3: web annotations of each others’ logs/notebooks, and of the readings, and responding to others’ annotations

Criterion 4: storytelling and engagement with the data, its limitations and its strengths

Criterion 5: your growth as a historian over this course.

2.1 Weekly work

Each week, you will complete (with certain caveats, as detailed in the workbook)

1. exercises that are appropriate to your tech level. How do you know if it was appropriate? If the exercise was easy, then you do the next one. You push yourself until you get to the point where you are stumped. You will keep a fail log and an online notebook for the exercises. The point here is not to stump you, but to push you from your current comfort level.

2. collaborative reading assignments, which you will demonstrate that you have done via collaborative web annotations of either the fail logs, notebooks, or assigned readings. Exercises are organized by modules, and the sequence of modules reflects the sequence of doing digital history.

nb there must be evidence from each module that you have fully engaged with the exercises and the collaborative reading. How to demonstrate this evidence will be covered below. A checklist of what should be completed by when is provided below. Work must be completed in a timely manner. Follow, and keep to, the schedule.

2.1.1 More about the fail log (process notes) and notebook (blog posts)

You will keep the fail log in a repository on Github that details the work you do for each module. What can go into this? Everything and anything. Files you create. Notes you make to yourself on how to use a particular tool. Random thoughts as they occur. Here is an example for you to refer to.

I call it a ‘fail log’ because more often than not, things don’t work at first. We learn by trying, by failing productively, as it were. In essence, there are many ways to craft digital history; you need to leave yourself breadcrumbs so that you can return to a project after a hiatus and know what it was you were doing/thinking at the time. Caleb McDaniel calls this open notebook history and you should read that post of his before going any further. FYI, here’s my own repository for my research: http://github.com/shawngraham.

You should also keep a narrative of your research that connects the dots and explains the thinking behind what you were trying to do (and what you documented in your fail log); think of this as your lab notebook. A blogging platform is a good way of doing that. FYI, here’s mine: Electric Archaeology; I’ve tried another variation on the model here. There is no one correct way to do this; you will have to work to find a way that feels authentic to you.

The modules in the workbook contain readings, and exercises aimed at various levels of comfort with things digital. That is to say, if you’ve never done anything with your computer other than updating Facebook, there are exercises designed to introduce you to computing; if you are doing a computer science minor on the side, there are exercises designed to challenge you too. The idea is that you work through the exercises until you get stumped. You detail where you were at and how you got stumped, in your narrative in your notebook. You ask each other for help, encourage each other, and discuss the exercises, in our Slack space.

I am not interested so much in your final product as I am interested in your progress. The assessment of the fail log and the notebook is not on whether you end up with a ‘right’ or ‘wrong’ piece, but rather, how you have grown and challenged yourself. Should you be someone for whom the exercises hold no challenge, then I challenge you to write the next exercise yourself. I’m always learning: I would be disappointed if I learned nothing from all of you. Again, the challenge of these modules: you must push yourself beyond your comfort level. That is what I am looking to assess. If I see that you are only trying the minimum level when it is apparent that you could be pushing further, that will be deemed unsatisfactory.

2.1.2 More about the collaborative readings

Digital history is not done in isolation. In contrast to other courses you may have done, I expect you to collaborate, to help each other out, to problem solve, to draw each other’s attention to the important points. In the first instance, annotating the readings and workbook make this experience more like a study group. In the second, I’m quite content that you should help each other out with the exercises. Really?! I hear you exclaim. Yes, but: if you had help working through an exercise (for instance), you would make a public note of the help you needed, who helped you out, and how. How will you do this? You could make a post in your notebook about how you helped someone out - or how they helped you out. You could read someone else’s notebook about their work and perhaps that sparks a thought or idea in you. You could then write about that, and link back to your peer’s original work.

This collaborative community building can also happen in our private Slack space. Maybe you organize folks into a reading group. Maybe you set up a group video-chat session. All these things create a cohesive group, and I will be watching for evidence of this.

Unacknowledged help or unacknowledged collaboration will be grounds for determining unsatisfactory work.

2.2 Capstone Exercise

The capstone exercise will have two parts:

1. an analysis (a work of digital history) of the provided digital dataset using at least two or more techniques/tools that you learned in the exercises. This analysis has to be available on the web, with your data and methods fully documented such that someone else could undertake the analysis for themselves.
2. a ‘paradata’ essay (or video, or other digital form) that reflects on your growth as a historian over the term with reference to the analysis in 1.

nb the capstone exercise may be submitted at any time up to midnight on the very last day of the course. Since the work will live on the web, you submit by sending me an email with ‘Capstone Exercise HIST3814o - submitted’ as the subject. I will make an archival copy of that project the following day. More information on the capstone exercise will be provided below.

There is no midterm. There is no final exam.

It is OK if your capstone exercise doesn’t come together in the way you initially expected. I recognize that sometimes, a project fails to come together. I believe in ‘productive failure’, where we learn from what works and what doesn’t work. I want to explicitly signal to you that it is safe to try something here that might not get to where you want it to go - and that’s ok. As long as you discuss honestly and without flinching what worked, what didn’t, why that was, and what you need to know or do moving forward, you stand a very good chance indeed of meeting the requirements of this class.

As an example, let me share with you the post-mortem of my very first digital history project I did when I joined Carleton, ‘How I lost the crowd’, a project that imploded - and how I salvaged worth from it.

2.3 Real Names Policy

You do not need to use your real name or identity on any public-facing work that you do in this course. Nor do you need to explain to me that you wish to use a pseudonym. It is sufficient that you send an email to me with the following message:

‘I would like to use the following username in all public-facing work: xxxxxxxx’

…where xxxxx is the name you have selected. For safety’s sake, if you decide to use a pseudonym, do not use one that you have used on any other website or social media platform.

2.4 Submitting evidence

At the end of each module, fill in the report form and provide the links to:

1. appropriate entries in your fail log
2. appropriate links in your notebook
3. appropriate links to annotations.

The first exercises in the workbook will walk you through setting up your notebook and fail log.

Because of the compressed schedule of a summer course, work that is submitted late will be considered not completed.

2.5 Once more, but LOUDER:

Students should contact me if they are unable to complete coursework due to illness or other exigency. Otherwise: late work will not be graded.

2.6 Schedule and due dates

This course runs from May 6 to June 18, 2019 Weekly reports on the previous week’s work due by Midnight, Monday. EG, you do the start up work during the week of May 6 - 12, and submit the work to me on the 13th. So:

• May 13: Getting Up and Running. Complete ‘Getting Started’ in the Workbook.
• May 20: Module 1 Open Access Research (May 20th is a holiday; work can come in until 9 am May 21st)
• May 27: Module 2 Finding Data
• June 3: Module 3 Fixing Data
• June 10: Module 4 analysis
• June 18 (Tuesday): Capstone Exercise

2.6.1 Overview of work to be done/submitted

• Getting Started: Setup, Initial blog post, Annotations
  
• Module 1: Blog post re readings & blog post re exercises, Fail logs, Annotations
• Module 2: Blog post re readings & blog post re exercises, Fail logs, Annotations
• Module 3: Blog post re readings & blog post re exercises, Fail logs, Annotations
• Module 4: Blog post re readings & blog post exercises, Fail logs, Annotations
• Capstone Exercise: Personal Reflection, Project repo, Project itself

2.7 Workbook structure

The workbook may be found at workbook.craftingdigitalhistory.ca. The modules in the course are built around the progressive steps of working with big data:

• Module 1 Principles of open access research and your digital identity
• Module 2 Finding Data
• Module 3 ‘Wrangling’ Data, or getting it into useable shape
• Module 4 Analyzing data, or matching the appropriate tool to the question
• Capstone Exercise- Communicating the Findings, visualizing (graphing, writing, plotting, mapping) data patterns, or telling the compelling story

These modules are built around the following ideas:

• Identify and define the limitations of useful sources of historical data online
• Compare and employ appropriate tools to clean and manipulate this data with a critical eye to how the tools themselves are theory-laden
• Analyze data using various tools with an awareness of the tendency of tools to push towards various historiographic or epistemic perspectives (ie, the ‘procedural rhetorics’ of various tools)
• Visualize meaningful patterns in the data to write ‘good history’ across multiple platforms, with critical evaluation of the limitations
• Model best practices in open access data management as mandated by SSHRC and other research agencies
• Develop an online scholarly voice to contribute data and reflection to the wider digital history community

2.8 Calculating grades

If you have demonstrated to me that you have satisfactorily met the criteria detailed above in the rubric (which map to the learning objectives for this course), you will receive an A. If you demonstrate that you have satisfied four of the five criteria, then you will receive a B. Three of the five: a C. Two of the five: a D. One of the five: an F.

Note that ‘satisfactorily’ meeting the criteria means completing all of the work that I have set for you. BUT, note also that you have the power to determine how much work that actually is, provided that you can justify why you haven’t pushed further. That is, I will look at your fail logs and your notebooks to determine whether or not the module exercises were challenging for you. It is thus in your best interest to keep careful and complete fail logs and notebooks as you do both the modules AND your capstone exercise. I can only grade where the evidence exists.

In exceptional cases, where the work completed goes above and beyond, an A+ may be possible.

We all come to digital work with differing degrees of digital literacy. By framing assessment as ‘satisfactory’ I recognize that each of you are different, and I assess you against your own starting point and your own growth.

You will receive feedback and be in communication with me throughout the course; the feedback is meant to help you grow as a scholar.

The capstone exercise is assessed against the same criteria as the coursework. Community participation is assessed through your substantive and generous collaboration via the annotations and in the Zulip space.

2.8.1 The process of grading coursework

Here are the instructions I give to my TAs on how we grade.

We are looking to see, in each week, that the students have

• done all the required work as listed in the workbook
• done the work in a substantive manner

Each week, we leave comments on posts, and we reply to annotations. By Wednesday we need to provide written feedback via email, explaining what is satisfactory and nudging where things can be improved, and where we also link to where we have responded to the student.

Our feedback each week should encourage dialogue with the student, and be framed within the criteria 1 - 5 below. Keep an eye out for evidence that the students are collaborating, and that they are responding in a meaningful way to each other’s work.

ALL COURSE WORK HAS TO BE COMPLETED, AND ALL WORK HAS TO BE SATISFACTORY, TO GET AN A

We can nudge students to complete things, when we note that they are missing. We can also be human, and adjust accordingly, in special cases.

We do not require students to ‘prove’ illness, or other issues. We will take them at their word.

What constitutes ‘satisfactory’ work? Satisfactory work demonstrates that the student:

1 Understands how to use the idea of ‘the productive fail’ to use computing power effectively as an historian.

2 Understands, plans, and employs concepts of computational reproducibility in the service of history.

3 Employs web tools to build history in public, collegially.

4 Weighs the evidence appropriately, understanding how it was created, in order to communicate the compelling story.

5 Understands the ways digital tools change us as we use them, to create compelling history that is self-reflexive.

Not every one of these criteria apply to each week or module; in aggregate, at the end, all must be present in the body of work that the students have produced over the course.

In the tracking spreadsheet, you simply tick off whether or not the student has done the work, entering a 1 or 0 in the column as appropriate.

Final grade for coursework

Missing module work reduces the grade.

Work that is completed, but is done in a perfunctory manner, similarly reduces the grade (as it isn’t satisfactory).

Work that is above and beyond the call of duty can be graded upwards.