Designing Better Assessments that Change Behavior

Office Hours Recap: November 2025

The Challenge: Training programs succeed or fail based on behavior change, yet less than 40% of organizations measure it (source: LEADx 2024 Leadership Development Benchmark Report). When you can't see what changes on the job, training becomes compliance theater—something managers check off, not something that drives performance.

The Solution: Move beyond knowledge checks to measure actual behavior change. Scenario-based assessments and on-the-job skill demonstrations show whether someone can actually do the work, not just recall information. 74% of organizations using these performance assessments believe they effectively measure learning outcomes, compared to only 58% using traditional tests (ATD Research, 2024).

In our November Opus Office Hours, we sat down with Ambryn Alam, VP of People & Culture at Swingers, to explore how they've built assessment strategies that actually change behavior—not just check boxes.

What Makes Assessments So Challenging?

Before diving into solutions, we asked attendees about their biggest assessment challenges. The responses revealed three common pain points:

• Manager buy-in - "Managers see them as 'ticking a box'" rather than valuable development tools
• Confusion about methodology - Uncertainty about when to use digital knowledge checks versus in-person skill validation
• Assessments that don't predict job performance - Creating tests that look good on paper but don't translate to better work on the floor

💡These challenges reflect a fundamental issue: most assessments stop at knowledge verification when the real goal is behavior change.

Start with Your Goals, Not Your Assessment Format

Before you can build effective assessments, you need clarity on what you're trying to achieve. The most common mistake? Jumping straight to assessment building without defining what success looks like.

The framework is simple: Goals → Learning Objectives → Assessments

• Goals are your desired business outcomes ("I want to sell more table-side guacamole")
• Learning Objectives are the specific, measurable takeaways from training ("understand and apply basic upselling techniques")
• Assessments are how you measure whether those takeaways stuck ("scenario-based questions and skill validation with a manager")

Or put another way: I want my teams to do X → By completing training on Y → And we will measure early indicators of success with Z

This clarity is essential because it determines not just whether you assess, but what type of assessment you need.

Understanding Assessment Types: Knowledge vs. Skills

After defining your goals and learning objectives, you can match them to the right assessment approach. There are two distinct categories.

The key is matching assessment type to your training goals. As Ambryn explained: "You can put this in ChatGPT and have ChatGPT make up questions, but they're not necessarily going to be relevant or reinforce true understanding. It could be like, 'Tell me the ingredients in this cocktail.' Sure, bartenders need to know that. But if it's 'Describe what it tastes like,' or we have a process for how we handle certain employee relations stuff—here's the policy, here are the things you need to know, but then here's the scenario: how do you apply that policy?”

“Understanding the policy textbook versus what happens in the real world and how does it apply—it's not as black and white." - Ambryn Alam, VP of People & Culture at Swingers

Building Your Assessment Strategy: Three Levels

Effective assessment strategies layer three levels of evaluation, each serving a distinct purpose.

Level 1: Foundation

Purpose: Verify basic knowledge and comprehension

What it looks like: Multiple choice questions, true/false, short answer responses that confirm someone absorbed the core information

Example from Swingers: When launching their Halloween 2025 cocktail menu, Swingers started with a quiz testing bartenders' knowledge of ingredients, recipes, and allergen information. But they didn't stop there.

The quiz used open-ended questions rather than multiple choice. "We want to see that they can actually answer the questions and that they've absorbed the content in the course," Ambryn shared. This approach prevents the "clicking through" behavior common with traditional quizzes.

Level 2: Application

Purpose: Assess ability to apply knowledge in realistic scenarios

What it looks like: Scenario-based questions, case studies, and practical problem-solving that requires applying training concepts to actual situations

Example from Swingers: After the knowledge quiz, bartenders moved to skill validation. "There's the actual skill section where it's asking them, can they actually answer questions about the cocktails? Can they sell this cocktail? There might be a check-in where we say, 'Have them sell you their favorite cocktail.' Because that's what they're gonna have to do in real life. A guest is gonna come up to them and say, 'Oh, I'm really interested in this Halloween menu. What's your favorite?' They should be able to answer that."

This level combines two elements:

• Open-ended comprehension - Can they describe tastes, explain techniques, articulate the "why" behind procedures?
• In-person skill checks - Managers observe and evaluate actual performance, providing immediate feedback

As Ambryn noted: "For us, we've got two different elements to this. We've got a quiz, which as you guys can probably see, it's not a quiz with multiple choice or true false. It's open-ended because we want to see that they can actually answer the questions and that they've absorbed the content in the course. But then the second part of that is a check-in that we've set up where the manager—their manager—will actually grade their quiz."

Level 3: Behavior Change (and Beyond)

Purpose: Drive sustained behavior change and cultural alignment

What it looks like: Ongoing conversations, goal-setting, progress tracking, and relationship-building that extends beyond any single training event

This is where Swingers truly differentiates their approach through their "Coffee Chats" program.

💡Swingers' "Coffee Chats" represent Level 3 assessment in action. Each chat combines course content, self-assessment, and a manager check-in guided by the team member's responses. "It's really a series of guided conversations throughout their employment," Ambryn explained. "It never stops." The chats progress from Week 1 through monthly check-ins focused on goals and development, giving leadership visibility to plan proactively rather than react to turnover.→ Learn more about how Swingers built their Coffee Chat program

Real Talk: Opus Alone Won't Solve Your Problems, And That's OK

Throughout the conversation, a critical theme emerged: Opus is one piece of a larger program. "Any company could easily copy-paste the work that we've done—I'll give it to you," Ambryn noted. "But it doesn't work if that's not your actual approach. What we've done is we've layered this general approach into everything we do from how we recruit people, how we train them, all of our people practices."

The coffee chats work because they're supported by robust in-person training, manager coaching, consistent communication, and clear expectations from day one. The transparency itself is intentional—check-ins are visible to team members, creating accountability for both parties and teaching managers how to give meaningful feedback. "The questions are structured so they know what to talk about in the conversation," Ambryn explained. "We're trying to get them to build an actual rapport and relationship with their team, which has to be organic, but we're trying to give them a structure to approach that." As Barby Rivers, Director of Operations at Swingers added: "We set the bar high, but we treat people really good, and we want them to have an amazing experience with us."

Key Takeaways

1. Start with your goal, not your assessment format: Define what success looks like: I want my teams to do X → By completing training on Y → And we will measure early indicators of success with Z.

2. Match assessment type to what you're measuring:

• Testing knowledge? Use quizzes and scenario questions
• Testing skills? Add in-person validation with feedback
• Driving behavior change? Layer ongoing conversations and development tracking

3. Build assessor training into your process: Don't assume managers know how to give effective feedback. Structure your check-ins to teach the skill through the process itself.

4. Make transparency work for you: Two-way visibility in check-ins creates accountability and opens dialogue when understanding diverges.

5. Think in layers, not one-time events: Multi-level assessments see 34% better skill transfer. Foundation knowledge → Application in scenarios → Sustained behavior change through ongoing support.

6. The tool enables the program—it doesn't replace it: Apply the same assessment approach across all levels. Everyone deserves the same quality of development, whether they're a captain or a bathroom attendant.

💡For more Office Hours topics and recaps, find them here

‍