Introduction

Using the App

There are two ways we recommend using swirlify to write lessons. One way is to use the shiny course authoring app we’ve created which can be launched using the swirlify() function like so:

# If you're creating a new course, or if you're switching from one lesson in a 
# course to another lesson in the same course.
swirlify("My Lesson", "My Course")

# If your current lesson is set you can just use:
swirlify()

The app contains a text editor and a series of text boxes that will help you write each question. You can choose what kind of question you want to write from a drop-down menu. Later in this document we’ll describe how these different questions work in swirl.

Using a Text Editor and `wq_` functions

You can also use a text editor and an R console to write questions in a lesson. Type W + Q + Tab in the R console in order to see all of the wq_ functions available (“wq” stands for “write question”). Executing any one of those functions will add a questions template to the current lesson. You can then fill in this question template in your text editor. Since you will be using a text editor and the R console together, we highly recommend using RStudio for writing your lesson.

General `lesson.yaml` Structure

Lessons are sequences of questions that have the following general structure:

- Class: [type of question]
  Key1: [value1]
  Key2: [value2]

- Class: [type of question]
  Key1: [value1]
  Key2: [value2]
...

The example above shows the high-level structure for two questions. Each question is demarcated with a hyphen. Every question starts with a Class that specifies that question’s behavior inside of swirl. What follows the class is a set of key-value pairs that will be used to render the question when a student is using swirl.

Types of Questions

The Meta Question

The first question in every lesson.yaml is always the meta question which contains general information about the course. Below is an example of the meta question:

- Class: meta
  Course: My Course
  Lesson: My Lesson
  Author: Dr. Jennifer Bryan
  Type: Standard
  Organization: The University of British Columbia
  Version: 2.5

The meta question will not be displayed to a student. The only fields you should modify are Author and Organization fields.

Message Questions

Message questions display a string of text in the R console for the student to read. Once the student presses enter, swirl will move on to the next question.

Add a message question using wq_message().

Here’s an example message question:

- Class: text
  Output: Welcome to my first swirl course!

The student will see the following in the R console:

| Welcome to my first swirl course!

...

Command Questions

Command questions prompt the student to type an expression into the R console.

The CorrectAnswer is entered into the console if the student uses the skip() function.
The Hint is displayed to the student if they don’t get the question right.
The AnswerTests determine whether or not the student answered the question correctly. See the answer testing section for more information.

Add a message question using wq_command().

Here’s an example command question:

- Class: cmd_question
  Output: Add 2 and 2 together using the addition operator.
  CorrectAnswer: 2 + 2
  AnswerTests: omnitest(correctExpr='2 + 2')
  Hint: Just type 2 + 2.

The student will see the following in the R console:

| Add 2 and 2 together using the addition operator.

>

Multiple Choice Questions

Multiple choice questions present a selection of options to the student. These options are presented in a different order every time the question is seen.

The AnswerChoices should be a semicolon separated string of choices that the student will have to choose from.

Add a message question using wq_multiple().

Here’s an example multiple choice question:

- Class: mult_question
  Output: What is the capital of Canada?
  AnswerChoices: Toronto;Montreal;Ottawa;Vancouver
  CorrectAnswer: Ottawa
  AnswerTests: omnitest(correctVal='Ottawa')
  Hint: This city contains the Rideau Canal.

The student will see the following in the R console:

| What is the capital of Canada?

1: Toronto
2: Montreal
3: Ottawa
4: Vancouver

Figure Questions

Figure questions are designed to show graphics to the student.

Figure is an R script located in the lesson folder that will draw the figure.
FigureType must be either new or add.
- new specifies that a new figure is being drawn.
- add specifies that more features are being added to a figure that already has been drawn, for example if you were adding a line or a legend to a plot that had been drawn in a preceding figure question.

Add a message question using wq_figure().

Here’s an example figure question:

- Class: figure
  Output: Look at this figure!
  Figure: draw.R
  FigureType: new

The student will see the following in the R console:

| Look at this figure!

...

The student will also see the figure in the appropriate graphics device.

Video/URL Questions

Video/URL questions give students the choice to open a URL in their web browser.

VideoLink is the URL that will be opened in the student’s web browser.

Add a message question using wq_video().

Here’s an example video/URL question:

- Class: video
  Output: Do you want to go to Google?
  VideoLink: https://www.google.com/

The student will see the following in the R console:

| Do you want to go to Google?

Yes or No?

Numerical Questions

Numerical questions ask the student to type an exact number into the R console.

Add a message question using wq_numerical().

Here’s an example numerical question:

- Class: exact_question
  Output: How many of the Rings of Power were forged by the elven-smiths of
    Eregion?
  CorrectAnswer: 19
  AnswerTests: omnitest(correctVal = 19)
  Hint: Three Rings for the Elven-kings under the sky, Seven for the Dwarf-lords
  in their halls of stone, Nine for Mortal Men doomed to die...

The student will see the following in the R console:

| How many of the Rings of Power were forged by the elven-smiths of Eregion?

>

Text Questions

Text questions ask the student to type an phrase into the R console.

Add a message question using wq_text().

Here’s an example text question:

- Class: text_question
  Output: What is the name of the programming language invented by 
    John Chambers?
  CorrectAnswer: 'S'
  AnswerTests: omnitest(correctVal = 'S')
  Hint: What comes after R in the alphabet?

The student will see the following in the R console:

| What is the name of the programming language invented by John Chambers?

ANSWER:

Script Questions

Script questions might be the hardest questions to write, however the payoff in a student’s understanding of how R works is proportional. Script questions require that you write a custom answer test in order to evaluate the correctness of a script that a student has written. Writing custom answer tests is covered thoroughly in the answer testing section.

Script is an R script that will be opened once the student reaches this question. You should include this script in a subdirectory of the lesson folder called “scripts”. You should also include in the “scripts directory” a version of this script that passes the answer test. The name of the file for the correct version of the script shoud end in -correct.R. So if the name of the script that the student will need to edit is script.R there should be a corresponding script-correct.R.

Add a message question using wq_script().

Here’s an example script question:

- Class: script
  Output: Write a function that calculates the nth fibonacci number.
  AnswerTests: test_fib()
  Hint: You could write this function recursively!
  Script: fib.R

The student will see the following in the R console:

| Write a function that calculates the nth fibonacci number.

>

Here’s an example fib.R:

# Write a function that returns the nth fibonacci number. Think about
# what we just reviewed with regard to writing recurisive functions.

fib <- function(n){
  # Write your code here.
}

Here’s an example fib-correct.R:

# Write a function that returns the nth fibonacci number. Think about
# what we just reviewed with regard to writing recurisive functions.

fib <- function(n){
  if(n == 0 || n == 1){
    return(n)
  } else {
    return(fib(n-2) + fib(n-1))
  }
}

Here’s an example customTests.R which includes test_fib():

test_fib <- function() {
  try({
    func <- get('fib', globalenv())
    t1 <- identical(func(1), 1)
    t2 <- identical(func(20), 6765)
    t3 <- identical(sapply(1:12, func), c(1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144))
    ok <- all(t1, t2, t3)
  }, silent = TRUE)
  exists('ok') && isTRUE(ok)
}

Answer Testing

Answer testing determines whether or not a student’s response to a question is correct. Ultimately an answer test is a function that returns TRUE if the student’s input should be considered correct or FALSE if it should be considered incorrect. Swirl includes some answer tests that we believe cover 80% of the types of questions that swirl course authors want to write. For the remaining 20% course authors can write their own answer tests. If you have questions about writing your own answer tests please do not hesitiate to get in touch with us by emailing info@swirlstats.com.

Answer tests should be specified after the AnswerTests: key in several different kinds of questions. You can specify multiple answer tests for a question by separating each test by a semicolon. Each answer test must return TRUE in order for the the student’s answer to be considered correct. Here’s an example of a question that uses two answer tests:

- Class: cmd_question
  Output: Assign the mean all the numbers from 1 to 10 to a variable called x.
  CorrectAnswer: x <- mean(1:10)
  AnswerTests: var_is_a('numeric', 'x');omnitest(correctExpr = 'x <- mean(1:10)')
  Hint: Just type x <- mean(1:10).

Built-in Answer Tests

omnitest

The omnitest() function is the workhorse of answer testing. It’s very possible to write an entire swirl lesson using only omnitest(). Omnitest can test for a student answering with a correct expression, a correct value, or both!

Testing for an expression

The question below can be answered only by entering plot(1:10) into the R console.

- Class: cmd_question
  Output: Plot the integers from 1 to 10.
  CorrectAnswer: plot(1:10)
  AnswerTests: omnitest(correctExpr = 'plot(1:10)')
  Hint: Try using the plot() function.

Testing for a value

The question below can be answered by entering any expression that evaluates to 4. For example: 4, 2 + 2, 2^2, sqrt(16), etc.

- Class: cmd_question
  Output: Enter an expression that evaluates to the number 4.
  CorrectAnswer: 4
  AnswerTests: omnitest(correctVal = 4)
  Hint: Any expression will work as long as it evalualtes to 4!

Testing for an expression or a value

The question below will issue a warning to the student if the expression they enter evaluates to the value specified by correctVal but they use an expression that doesn’t match correctExpr.

- Class: cmd_question
  Output: What's the result of adding 2 and 2?
  CorrectAnswer: 2 + 2
  AnswerTests: omnitest(correctExpr='2 + 2', correctVal = 4)
  Hint: Try using the addition operator.

For more information about omnitest() see ?omnitest.

any_of_exprs

The any_of_exprs() function allows you to test whether a student used one out of several expressions.

Here’s an example question using any_of_exprs():

- Class: cmd_question
  Output: Enter an expression that creates a vector of all of the integers from
    1 to 10.
  CorrectAnswer: 1:10
  AnswerTests: any_of_exprs('1:10', 'seq(1, 10, 1)')
  Hint: Try using the colon operator.

expr_creates_var

The expr_creates_var() function explicitly protects the value of a variable from being changed erroneously. Imagine a student is asked to assign 10 to the variable x in one question. Then in the next question the student is asked to assign 2*x to x, but mistakenly assigns 3*x to x. In the case where AUTO_DETECT_NEWVAR has been set to FALSE in customTests.R the expr_creates_var() function will prevent the variable x from being assigned the wrong value.

Here’s a series example questions using expr_creates_var():

- Class: cmd_question
  Output: Assign 10 to the variable x.
  CorrectAnswer: x <- 10
  AnswerTests: omnitest(correctExpr='x <- 10')
  Hint: Just type x <- 10.

- Class: cmd_question
  Output: Assign x*2 to the variable x.
  CorrectAnswer: x <- x*2
  AnswerTests: omnitest(correctExpr='x <- x*2');expr_creates_var('x')
  Hint: Just type x <- x*2.

expr_uses_func

The expr_uses_func() function tests whether a student used a particular function when answering a question.

Here’s an example question using expr_uses_func():

- Class: cmd_question
  Output: Use the mean() function to find the mean of any vector.
  CorrectAnswer: mean(1:10)
  AnswerTests: expr_uses_func('mean')
  Hint: Use the mean function with any numeric vector as an argument.

Any expression that uses mean() will satisfy the question above.

val_matches

The val_matches() function is used exclusively with text questions in order to test whether a response matches a regular expression.

Here’s an example question using val_matches:

- Class: text_question
  Output: What is the capital of Chile?
  CorrectAnswer: Santiago
  AnswerTests: val_matches('[S|s]antiago')
  Hint: Ryhmes with Zandiago.

For more details about all of the built in answer tests, check out the documentation in swirl with ?AnswerTests. We also recommend browsing the many answer tests that are used in current swirl courses, which you can find through the Swirl Course Network.

Custom Tests

If you’re unable to find a built-in test for your question, you can write your own test. All of your custom tests should be written inside of customTests.R.

Answer Testing API

We’ve created a set of functions that form an API for swirl’s internal state. Many of these functions are useful when writing custom tests. You can find these functions here.

Custom Tests for Script Questions

Remember: an answer test is just an R function that returns TRUE or FALSE. The answer test function should return TRUE if the student’s answer is correct or FALSE if it is incorrect.

One of the most common reasons for writing a custom test is to evaluate the correctness of a script that a student has written as part of a script question. Below is an exmaple of a custom test that is meant to test whether a student has implemented a function that mimics the behavior of mean().

test_func <- function() {
  # Most of this test is wrapped within `try()` so that any error in the
  # student's implementation of `my_mean` doesn't interrupt swirl.
  try({
    # The `get` function retrieves the student's definition of `my_mean` and
    # assigns it to the variable `func`.
    func <- get('my_mean', globalenv())
    
    # The behavior of `func` is then tested by comparing it to the behavior of
    # `mean`.
    t1 <- identical(func(9), mean(9))
    t2 <- identical(func(1:10), mean(1:10))
    t3 <- identical(func(c(-5, -2, 4, 10)), mean(c(-5, -2, 4, 10)))
    ok <- all(t1, t2, t3)
  }, silent = TRUE)
  
  # This value is returned at the result of the answer test.
  exists('ok') && isTRUE(ok)
}

The custom test above is then used in a question in lesson.yaml:

- Class: script
  Output: Make sure to save your script before you type submit().
  AnswerTests: test_func()
  Hint: "Use the sum() function to find the sum of all the numbers in the vector. Use
    the length() function to find the size of the vector."
  Script: my_mean.R

Below is an example my_mean.R which should be in the scripts directory within the lesson directory:

# You're free to implement the function my_mean however you want, as long as it
# returns the average of all of the numbers in `my_vector`.
#
# Hint #1: sum() returns the sum of a vector.
#   Ex: sum(c(1, 2, 3)) evaluates to 6
#
# Hint #2: length() returns the size of a vector.
#   Ex: length(c(1, 2, 3)) evaluates to 3
#
# Hint #3: The mean of all the numbers in a vector is equal to the sum of all of
#          the numbers in the vector divided by the size of the vector.
#
# Note for those of you feeling super clever: Please do not use the mean()
# function while writing this function. We're trying to teach you something 
# here!
#
# Be sure to save this script and type submit() in the console after you make 
# your changes.

my_mean <- function(my_vector) {
  # Write your code here!
  # Remember: the last expression evaluated will be returned! 
}

Below is the corresponding my_mean-correct.R which should also be in the scripts directory within the lesson directory:


# You're free to implement the function my_mean however you want, as long as it
# returns the average of all of the numbers in `my_vector`.
#
# Hint #1: sum() returns the sum of a vector.
#   Ex: sum(c(1, 2, 3)) evaluates to 6
#
# Hint #2: length() returns the size of a vector.
#   Ex: length(c(1, 2, 3)) evaluates to 3
#
# Hint #3: The mean of all the numbers in a vector is equal to the sum of all of
#          the numbers in the vector divided by the size of the vector.
#
# Note for those of you feeling super clever: Please do not use the mean()
# function while writing this function. We're trying to teach you something 
# here!
#
# Be sure to save this script and type submit() in the console after you make 
# your changes.

my_mean <- function(my_vector) {
  # Write your code here!
  # Remember: the last expression evaluated will be returned! 
  sum(my_vector)/length(my_vector)
}

Writing Lessons

Introduction

Using the App

Using a Text Editor and `wq_` functions

General `lesson.yaml` Structure

Types of Questions

The Meta Question

Message Questions

Command Questions

Multiple Choice Questions

Figure Questions

Video/URL Questions

Numerical Questions

Text Questions

Script Questions

Answer Testing

Built-in Answer Tests

omnitest

Testing for an expression

Testing for a value

Testing for an expression or a value

any_of_exprs

expr_creates_var

expr_uses_func

val_matches

More

Custom Tests

Answer Testing API

Custom Tests for Script Questions

Writing Lessons

Introduction

Using the App

Using a Text Editor and wq_ functions

General lesson.yaml Structure

Types of Questions

The Meta Question

Message Questions

Command Questions

Multiple Choice Questions

Figure Questions

Video/URL Questions

Numerical Questions

Text Questions

Script Questions

Answer Testing

Built-in Answer Tests

omnitest

Testing for an expression

Testing for a value

Testing for an expression or a value

any_of_exprs

expr_creates_var

expr_uses_func

val_matches

More

Custom Tests

Answer Testing API

Custom Tests for Script Questions

Using a Text Editor and `wq_` functions

General `lesson.yaml` Structure