Attempt Tracking

Codespec tracks the learner’s problem solving activity at two levels: snapshots and attempts. Snapshots are the more fine-grained details of what the problem interface looked like at an exact point in time (e.g., what action the learner took to trigger the snapshot, which block they were moving, what the layout was at the time, etc.).

Attempts are more high-level. They store the common information that a group of snapshots shares. For example, if a learner is working on the Parsons problem type for a given problem variant and they make a series of block-related moves (adding, removing, reordering, indenting, etc), all of those snapshots are for the same problem variant, problem type, Parsons problem, and Parsons solution. Hence, they are grouped into the same attempt.

Attempts also record the overall start and end time for the snapshots they contain. Depending on what actions the learner takes, an attempt’s start and end time values can result it being in a few different states: unstarted, unfinished, completed, and submitted. Unstarted attempts have no start or end time yet (e.g., when a learner switches to a given problem type tab and hasn’t taken any further action yet). Unfinished attempts have a start time (because the learner took one of the possible attempt-starting actions), but no end time (e.g., they’re still solving the problem). Completed attempts have both start and end times. When a learner performs any of the attempt-ending actions, Codespec ends their attempt with the current time. Submitted attempts have start and end times, and contain submission snapshots, which may be correct or incorrect.

Learners trigger snapshots by taking specific actions. The following ones will automatically start an attempt if it is currently unstarted:

• ADDED (added a solution block)

• COPIED (copied a block)

• CHANGED_FADED_TEXT (modified faded text on starter or solution side)

• COMBINED_BLOCKS (used combine-blocks help) (N/A right now)

• CREATED_BLOCK (created a custom block)

• EDITED_BLOCK (edited an existing custom block)

• ELIMINATED_DISTRACTOR (eliminated a distractor block)

• FIXED_BLOCK (modified fix code block)

• INDENTED (indented solution block)

• OUTDENTED (outdented solution block)

• REMOVED (removed solution block)

• REORDERED (reordered starter or solution block)

• RESET (reset the problem)

• REQUESTED_MISSING_BLOCK (requested add-block help)

• REQUESTED_INDENTATION (requested indentation help)

• REQUESTED_ORDER (requested order help)

• REQUESTED_INCORRECT_BLOCK_REMOVAL (requested incorrect block removal help)

• TOGGLED_PSEUDOCODE (requested pseudocode help)

Similar to actions that start attempts, learners also take actions that end unfinished attempts:

• CHANGED_LAYOUT (switched between single and double column layout)

• CHANGED_PROBLEM_TYPE (switched to other problem type)

• CHANGED_VARIANT (switched to new problem variant)

In addition to snapshot actions, an attempt can also start when learner starts typing on the fix or write code tab, if it hasn’t already. Similarly, it can end when:

• The learner clicks the “Run” button (the attempt ends immediately, before/while Judge0 is evaluating the code)

• The learner clicks the “back” button in the Snapshot History

• The learner navigates to a different tab in the browser

• The learner clicks an internal- or external-bound link on the page

• The learner has not triggered any new snapshots for five minutes

What happens when a learner clicks the “Run” button multiple times without making any changes?

If a learner repeatedly clicks “run” without making any changes to their code, all of those subsequent submission snapshots are still associated with the same attempt as the original submission.

Is it possible for an attempt to have an end value, but not start value?

No. If an attempt is unstarted when an attempt ends, the attempt’s created­_at value gets saved as the start time. This means that ultimately attempts should always have a start and end value.