Pilot evaluation toolkit

A practical way to reflect on a pilot. It is not a validated instrument, it is not for grading, certifying or ranking learners, and it is not evidence that SkillRise caused any change.

1. Before the first session (discussion)

Spoken or written, all optional. Record themes, not individuals.

When a workplace instruction is unclear, what do people usually do?
What makes it hard to tell someone about a mistake at work?
What would you say if a coworker asked you to do something you thought was unsafe?

2. After the last session (same prompts, repeated)

Compare the group's themes, not individual answers.

Ask the three questions above again. What new considerations, options or wording appear?
Which situation from the sessions comes to mind first, and why?

3. Unfamiliar follow-up situation (transfer)

Read aloud. It is deliberately unlike any SkillRise workplace.

You've just started at a hardware store. On a busy Saturday your supervisor tells you to “sort out the paint desk” and leaves for a meeting. A customer is waiting for a color match, the mixing machine shows an error, and a coworker says she always just restarts it, though there's a sign saying to call maintenance. What's going on, what would you do first, and what wouldn't you do?

4. Observation prompts for facilitators

Note what you see or hear. Don't score or rank anyone.

Identifying pressures: Does the learner name more than one pressure (customer, time, coworker, safety)?
Explaining decisions: Do they give a reason connected to the situation, not just an answer?
Recognizing boundaries: Do they spot a line that shouldn't be crossed (here, the machine sign) and say how they know?
Considering alternatives: Do they mention another option and what it would protect or risk?

5. Optional learner feedback (relevance and usability)

Anonymous on paper, or verbal. Skippable.

How close did the workplaces feel to work you know or expect? (not close / somewhat / very)
Was anything hard to read, follow or use? What?
What one thing would you change?

6. Later follow-up (2–6 weeks after)

Only if you are still in contact through your normal program. Voluntary.

Have you used anything from the sessions in real life, such as a question, a way of explaining or a check? If you'd like, describe it without names.
Was there a situation where you wished you'd had something different to try?

Keep these four things separate

Engagement: People started, finished and talked about the workdays. Shown by attendance, the device-local completion counts and discussion. It does not show learning.
Confidence: What people say about how ready they feel (for example, the optional before/after ratings in The Morning Rush). Self-report, influenced by mood and setting.
Observed practice: What a facilitator sees or hears in the room, using the observation prompts. It describes behavior in a fictional situation, not at work.
Actual workplace outcomes: What happens in real jobs, such as hiring, retention or conduct. This toolkit does not measure these, and a short pilot cannot attribute them to SkillRise.

What the device-local results file contains

For every workday: when it started, when it reached the evidence card, the time taken, the optional pilot code and a random browser id.
Before/after confidence and relevance ratings only for The Morning Rush (community workdays), where those questions appear. Other workdays don't ask them.
Learners' reflections, evidence cards and My progress are never included.
Typed text (facilitator feedback, skills named in learners' own words) is left out unless you choose to include it. Don't type names, contact details or personal circumstances.