About ten years ago, I made one of the best work-related decisions I’ve ever made: Switching from Windows (and SPPS) to linux (and R). Not having to deal with the endless load times of SPSS or the inevitable Windows slowdowns that make you think, after six months of use, that you need a new computer—plus all the unfixable bugs, unexplainable crashes, and generally strange behavior—has done more for my blood pressure than any medication or change in diet ever could. And linux support for many different programs has improved so that finding linux versions of things you need is kind of like finding vegetarian food in the Midwest: Although I still go into restaurants expecting that the vegetarian option will just be a salad where they leave off the ham, most of the time you can find something that is actually pretty good. This has made my life better.

Me in 2009. With Windows.

Me in 2019. With linux.

But it is true that for some things, there are simply no linux alternatives. Tax software, for instance. And until recently, I was stuck using my Windows virtual machine so I could run the software that I used to create multiple-choice exams for my large Intro Psych class (software provided by the textbook that I used to use before switching to the excellent, and free-for-students Noba Project text).

Because using a Windows virtual machine is also pretty frustrating, this year I searched for an alternative, assuming that someone must have made a package for R that could do this. This led me to R/Exams.

R/Exams

R/Exams is an R package that can be used to create paper-based and on-line exams. It can select items from a pool and randomize the order of those questions for presentation to students. It can also format them so that you can use the questions in on-line course management systems. It’s a really nice package that does a lot of things well. However, I found the default settings to be less than ideal for the types of exams I give. For instance, many of the default settings assume that each respondent will receive a unique exam form and will respond to that exam with an answer sheet created by the R/Exams package. But I want to take advantage of my university’s scoring office, which means that I need a small number of forms with corresponding scoring keys that can be used by the scoring office. The documentation about how to change these settings is not always clear (though the package authors are very responsive at the support forum); and changing the output often requires editing LaTeX templates. Because I think that many people might want to use R/Exams the same way that I do, I thought it might be helpful to write about the changes I needed to make when using R/Exams in my class. I am certainly not an expert on the intricacies of the package (or on LaTeX), so I’d love to hear suggestions about better ways to do this in the comments.

R/Exams has a lot of functionality that I don’t really need. The main purpose that I have for it is to create multiple pdf versions of the same set of questions for my very large Intro Psych class. So I primarily rely on a function that does exactly that. That function is exams2pdf(). Once you’ve created a set of questions (each in its own file, either in Rmarkdown or Sweave format; see here), you can construct an exam like this:

### Load package
library(exams)

### Create an exam, in this case from the example items that come with the package
myexam <- list(
  "tstat2.Rnw",
  "ttest.Rnw",
  "relfreq.Rnw",
  "anova.Rnw",
  "cholesky.Rnw"
)

### Print the exam to pdf
examPdf <- exams2pdf(myexam, n = 2, name = "exam")

This will create two pdf files (labeled “exam1.pdf” and “exam2.pdf”) each with the five questions listed above. However, the first thing you’ll notice is that these two versions have the exact same order of questions, which is not what I expected. What this command does do nicely is actually change the numbers in the math-based problems; so the questions and answers are actually different for different students (no cheating!). This is a great feature, but one that I won’t use, given the types of questions I use in Intro Psych. To get Exams to randomize the order of the questions, you kind of have to trick it, using an option that is meant to randomly select a subset of questions from a broader pool. The code below does just that (notice the added c() around the items, and the nsamp option in the call to exams2pdf):

### Create an exam in a way that order can be randomized
myexam <- list(c(
  "tstat2.Rnw",
  "ttest.Rnw",
  "relfreq.Rnw",
  "anova.Rnw",
  "cholesky.Rnw"
))

### Print the exam to pdf
examPdf <- exams2pdf(myexam, nsamp = 5, n = 2, name = "exam")

Now if you look at the files that were created, you will see that the order is randomized. By the way, if you wanted to randomly select a certain number of questions from a larger pool of items, you could change the nsamp option to some number less than the total number you included in the list statement. So the following would get you two different exams, made up of three questions selected from the five that you listed in myexam.

### Randomly select 3 items and print the exam to pdf
examPdf <- exams2pdf(myexam, nsamp = 3, n = 2, name = "exam")

Note that if you use this option to select from a larger pool, the questions that are included on each exam will be different; I’m not sure how to randomly select the same set of questions from a pool and then randomize that set across two different forms.

So this is a good first step, but it still was not what I needed. First (and most importantly), you’ll see that the pdf that is printed actually has the solutions alongside the questions. Not good if these are the forms you actually want students to use when taking the exams! In addition, I want to be able to change the formatting to use smaller margins, different font, and to not have the title “Problem” before each individual question (if I’m carrying 500 exams across campus, I want these questions to fit on as few pages as possible).

The many, many boxes of exams I have carried across campus.

I think that exams2pdf() was actually designed to print a version of the exam that included solutions for students to use after they completed the exam. R/Exams also has a function called exams2nops() that does much of what exams2pdf() does, but without solutions. You can try it by running the following code:

examNops <- exams2nops(myexam, nsamp = 3, n = 2, name = "exam")

Now you will get something that might be closer to what you want, as the solutions aren’t printed. But you also get an answer sheet that students can use to actually record their responses (which I don’t need, given that I am using the university’s scantrons). In addition, I would actually like to be able to have this function print out a list of answers that I can give to my TAs to use when creating the scoring key for our scoring office.

Modifying R/Exams Defaults

So to get exactly what I wanted, I needed to modify the template that is used by R/Exams when creating this output. This template needs to be in the “tex” directory of the package directory (which will differ depending your system). You can find my modified template file here.

I started with the provided “exam.tex” template and then made a couple changes (most of which were suggested on the support forums, some of which I figured out on my own¹). First, I changed the font-size to 10pt in the opening line:

\documentclass[10pt]{article}

In addition, I used the geometry package to change margins:

\usepackage[left=.5in, right=.5in, top=.5in, bottom=.75in]{geometry}

Third, I changed the answerlist environment to prevent answer lists from spanning multiple pages:

\newenvironment{answerlist}{\renewcommand{\labelenumi}{(\alph{enumi})}\begin{samepage}\begin{enumerate}[topsep=0pt,itemsep=-1ex,partopsep=1ex,parsep=1ex]}{\end{enumerate}\end{samepage}}

Next, I wanted to use the answer sheet as a scoring sheet (i.e., something that shows a list of correct answers). So I changed this line:

\newcommand{\extext}[1]{\phantom{\large #1}}

to this one:

\newcommand{\extext}[1]{\textbf{\large #1}}

This now shows the correct answer for each question.

In addition, at the end of the template, I changed the order of the “exercises” (all the individual questions) relative to the “questionnaire” (which is the answer sheet, or as I am using it, the list of correct answers). This prints the answers at the end of the pdf, and I can just remove those pages when sending the exam to the printer. So now in my template, the line

%% \exinput{questionnaire}

comes right before the end of the document. I also made some modifications to the title page. I actually don’t use the title page (I replace it with one I’ve used for other exams), but I could probably just modify this one to make it work for my purposes.

So now the final command that I run to get the exams, formatted the way I want them is:

finalExam <- exams2pdf(myexam, n = 2, nsamp = 5, name = "finalExam", template="myexam2")

This produces two formatted pdf files, each with the same questions in different orders, and each with a list of answers appended to the end (careful not to copy those last pages in the version you distribute to students). Add the type of title page you want, and they’re ready for the printer!

An R/Exams Workflow

Switching to R/Exams (from “cut and paste” or from using some other software) does require some additional changes to your workflow. Each exam question is stored in its own .Rnw or .Rmd file (I use .Rmd), so you need to rewrite your questions in this format. Personally, I organize sets of questions in separate directories by content area, and then select the questions that I want and add them to the list in “myexam” from the code above. Taking old exam questions and putting them into Rmarkdown format takes a little bit of time, but I think in the long run it will be worth it. Now I don’t have to rely on proprietary (and clunky, and soon-to-be-unusuable) software to create nicely formatted exams from an existing pool of questions.

There are a couple of things I’d love to change, but I don’t have the expertise that is required. First, I’d like to have the answer lists formatted as two columns when all the responses are less than a half a line long and as one column otherwise; but I don’t know enough about LaTeX to do that (and nobody could answer my question on twitter). Second, I’d like to figure out how to randomly select questions from a pool and to create multiple versions with different orders but that use these same questions. I can’t figure out how to get R/Exams to do both. It would also be pretty amazing to develop a converter that could take questions from course management software and output them to individual Rmd files; but that is certainly beyond my abilities. So if anyone has solutions to those issues, let me know!