Errors and Feedback
The “Errors and Feedback” tab displays different types of content, depending on the problem type for which the learner’s solution was run and the outcome of their attempt.
Standard block-based feedback compares the learner’s solution to the correct Parsons solution (see this section for how Parsons solutions are chosen if the problem contains multiple solutions). It tells learners the number of:
Correct blocks present (a.k.a. non-distractor blocks)
Distractor blocks present
Correctly ordered blocks
Correctly indented blocks
Correctly filled-in blocks (if working on the Faded Parsons problem type)
Custom block-based feedback provides summary statistics about a learner’s block-based solution that contains 1 or more custom blocks. These items include:
A description of how different their solution was from the expected solution
A comparison of how many blocks their solution used, versus the expected solution
Counts of how many original blocks matched the expected order, indentation, and faded text (if applicable)
Across all problem types, for each unit test related to the problem, learners see a separate feedback item. This feedback contains:
Test name and description
Test code (what the test looked like when it ran)
Their code’s output
If a learner’s submission results in a runtime error or some other standard error, the error appears with the following information:
Line number on which the error occurred
The code that caused the error
A full traceback for the error (hidden behind an accordion button by default)
If the learner prints or logs anything out in their solution, that information will appear on the “Output” tab.
The “Tests” tab contains a list of all unit tests created for this problem, displayed in a read-only code editor.
The “Other Solutions” tab is currently under development. The intent is to display solutions created by other learners, aggregated in meaningful ways (e.g., “most common solution”, “shortest solution”, etc). Content on this tab would only be available once the learner made at least one successful attempt.