Supplemental Tabs

The “Errors and Feedback” tab displays different types of content, depending on the problem type for which the learner’s solution was run and the outcome of their attempt.

Standard block-based feedback compares the learner’s solution to the correct Parsons solution (see this section for how Parsons solutions are chosen if the problem contains multiple solutions). It tells learners the number of:

• Correct blocks present (a.k.a. non-distractor blocks)

• Distractor blocks present

• Correctly ordered blocks

• Correctly indented blocks

• Correctly filled-in blocks (if working on the Faded Parsons problem type)

Custom block-based feedback provides summary statistics about a learner’s block-based solution that contains 1 or more custom blocks. These items include:

• A description of how different their solution was from the expected solution

• A comparison of how many blocks their solution used, versus the expected solution

• Counts of how many original blocks matched the expected order, indentation, and faded text (if applicable)

Across all problem types, for each unit test related to the problem, learners see a separate feedback item. This feedback contains:

• Test name and description

• Test code (what the test looked like when it ran)

• Expected output

• Their code’s output

If a learner’s submission results in a runtime error or some other standard error, the error appears with the following information:

• Error name

• Line number on which the error occurred

• The code that caused the error

• A full traceback for the error (hidden behind an accordion button by default)

If the learner prints or logs anything out in their solution, that information will appear on the “Output” tab.

The “Tests” tab contains a list of all unit tests created for this problem, displayed in a read-only code editor.

The “Other Solutions” tab is currently under development. The intent is to display solutions created by other learners, aggregated in meaningful ways (e.g., “most common solution”, “shortest solution”, etc). Content on this tab would only be available once the learner made at least one successful attempt.