Bottom-Line Performance

Jennifer Bertram and I presented this session at the Society for Applied Learning Technologies conference yesterday. In the session, we started developing a rapid prototype for an audience member, based on his learning problem. As he described it, he has graduate students who don’t know what sources to look at when they start working on their first big research paper. (I remember being in exactly that boat in graduate school- some guidance would have been very helpful to me back then!)  We spent less than 20 minutes on the prototype in class, and I spent another 30 or so minutes finishing it up outside of class. As promised, here it is (ta da!)

rapid_prototype_salt_sample

One of the rules we use for rapid prototyping is to spend as little time as possible on the protptype. The idea is to get the functionality down so that

1. The SME / client / project manager can see how the course will function.
2. You know you’ve thought all the way through how it will work (and therefore have higher confidence that it will work in the final course.)

You’ll notice some things right away about this prototype. First of all, it’s VERY rough. The graphics are rudimentary, and the functionality is really easy to break. That’s all ok. Think of this as an electronic bar napkin - you’re just trying to communicate your ideas. That way everyone is on the same page when they decide whether the ideas are good or not.

I did this prototype in Powerpoint, because that’s the tool I’m most familiar with. I couldn’t take this directly into any e-learning development software. (This wouldn’t even work in articulate, because I used triggers to simulate a drag and drop, and triggers don’t translate into articulate). That’s ok. The SME could tell from this whether I’ve gotten a functionality that makes sense. Remember, this was our first idea for representing this particular learning problem - and the first idea is never the best…so this might not make a final course if we had more time to talk it through with the SMEs.

If you want to see the entire presentation, it’s available here salt_rapid_prototyping_030310

or through Slideshare.

P.S. If you were in the session - especially if you were the gentleman who played the role of SME for this particular exercise, please comment and let us know if the prototype looks the way you expected it to when you walked out of the session. And also, thanks for being a part of our great audience!

It’s going to be a little quieter than usual around the BLP offices this week, since several of our staff people are going to be presenting at the SALT conference. You know how conferences are, you get to learn about lots of new and different things.

With this as inspiration, we thought we’d take this opportunity to write about a wide variety of things this month. Our official topic will be “grab bag” – and our topics will vary much like a conference schedule – something for everyone.

If you have a topic you’d like to see us cover, feel free to post it in the comments, and we’ll try and cook up a post just for you.

Today I did something that I haven’t done in a really long time – I took a stroll around the neighborhood. I’m in the midwest United States so it hasn’t exactly been strolling weather. But today, there are signs of spring.  I can’t wait to get out in the yard, start the garden, and grill outside. It’s this time of year that we have a sense of renewal, we clean-out, and shape-up to usher in a new season. Besides, with the close of the Olympics I’m ready to put winter behind me and focus on fresh starts.

All of this got me thinking about what work processes should be dusted off, revamped, or cleaned-up? It seems like a good time to feed off the seasonal motivation to implement something new, both personally and for our organizations.

Earlier today, I listened to an interview with Daniel Pink, the author of Drive, about motivation. Daniel talked about how the ability to see progress is a key motivating factor for knowledge workers. One way Daniel tracks his own progress is that he gives himself sort-of personal performance review at the beginning of each month where he identifies what he wants to accomplish, learn, or improve on during the next 30 days. Then at the end of the month he has a pretty clear picture of his progress. I love this idea. I think we could apply it in the spirit of spring cleaning. It’s a simple way to make incremental change and evaluate progress over a short time period.

So, what’s on your mind to freshen up this spring? What organizational processes do you want to spiff-up, improve on, or change in the next 3-4 months?

A charming, eloquent, punctual, knowledge, available, eager SME is one of the working world’s greatest gifts. Occasionally, however, we see a mirage: an identified SME who can’t play that role. What can you do about it?

The Bigwig Mirage: an important stakeholder who is positioned as a SME, but does not have the requisite knowledge to fill the role.

How you know: the Mirage “started out in this role (20 years ago”); the Mirage controls the budget for the project.

What to do: get their input on the “big picture” of the project; provide regular status updates; include them in the review cycles, but not as the only or primary reviewer.

The Newbie Mirage: an intern or other person new to the organization who is positioned as a SME on the project because of available time, not knowledge.

How you know: the Mirage has limited or no tenure; the Mirage is identified as an intern.

What to do: find out if the Mirage has subject knowledge from other experience (a past job, for instance) that you can use; partner with the Mirage to develop questions and identify people within the organization who can answer them (benefit to being the new guy or an intern: easier to ask questions internally).

The Overworked Mirage: an actual subject matter expert who physically doesn’t have time to participate in the project.

How you know: You always get voicemail when you leave messages; when emails are returned, they are returned after business hours; the Mirage attends a meeting but is not willing to commit to timelines.

What to do: Obtain as much information as possible from other sources; identify what only they can provide, and at what key point in the project you need it; find out their preferred way to communicate and use it (list of email questions instead of a phone call, for instance).

The Saboteur Mirage: a person who wants the project to fail, for nefarious and irrelevant reasons, so gets involved to make sure it does not succeed.

How you know: You never get a decision made because of this person’s “issues” or “concerns”; decisions that were made get unmade; you hear interesting comments about this person’s involvement.

What to do: Make sure your stakeholders know what is impeding the deadlines; see if you can uncover the motivation for failure – is it something you can address?

The Unavailable Mirage: No SME at all.

How you know: you hear the question, “You can find this all on the Internet, right?”

What to do: Find it on the Internet . . . or anywhere else you can.

I love working with SMEs. (OK, most of the time I love working with SMEs. But really, I only love working with people in general most of the time.) Subject matter experts are great because they’re subject matter experts. I get to learn stuff from them. Sometimes, it’s cool stuff, and sometimes it’s just mildly interesting stuff, but’s always new to me. 

I try to make sure that SMEs don’t mind working with me. Because for all the SME jokes in our industry, they’ve got to have just as many about us crazy instructional designers. Afterall, we ask a lot of them. We want their expertise, but we want to have the right to change anything they tell us to make it more understandable. And we want to pull knowledge from them that has become so second nature they don’t even know they have it. Usually, we’re asking them to give us their time on top of everything else they have to do for thier job. When you think about it, we have to be kind of exhausting to work with.

So I want to make it as pain free as possible. For many of my projects, I spend a good chunk of time creating content worksheets or interview guides after I have a finished design. Leanne Batchelder wrote about our Mad Libs approach with content templates before.  That works great if I can’t meet with the SME face-to-face. If I can get face time, I usually use a six-step process:

1. Ask the SMEs to review the design before the meeting. I’ve encountered instances where my SMEs for a project were not the same group of people that helped me design the program. I want them to know what the goals are before we meet.
2. Go through the design document. I wrote it, edited it, read it a million times over, but I still need to go through it and pull out every question I have. This keeps me from forgetting to ask all of my questions. I don’t want to trap my SMEs for three hours in a meeting, then have to send 5 follow up emails because I fogot a key question in my meeting.
3. Create an interview guide or content worksheet. I have a background in journalism, so I prepare for every meeting like its an interview. I write questions like I will ask them, in the order I will ask them.  This is not always the order I intend to use the information in the course. 
4. Compare the interview guide against my source content. I want to fill in anything I can before I go into the meeting. Then, I can just have my SMEs confirm what I know or correct what I only thought I understood. It saves everyone time to have something to start from. At this step, I send a list to my SMEs of any documents I think they may already have, like standard operating procedures, forms, or plans.
5. Review the design one last time and print everything. Chances are, I still forgot something. So triple-check my work against my design. Then I print all of the source material and head off to my meeting. 
6. Send a summary of the meeting back to the SMEs. Often, SMEs need to provide me with additional information, documents, or forms. I try to send a list of anything they promised me to them within a few hours of our meeting. It helps me make sure I get all of the information I need and keeps them from having to remember one more thing.

So…anyone can create a course, right? But…very few people actually create GOOD courses that truly train people to do something.

If you are tasked with creating a course on a technical topic…and you have to rely on a subject matter expert (SME) to help you create this course, you need to get your SME to think in terms of outcomes as opposed to content.

Subject matter experts tend to want to share everything they know about a topic. It’s the course designer’s job to help them shape their thoughts into outputs rather than inputs. At BLP, we do this with a design meeting. Here’s a quick look at the main steps in our design meeting and the process we use to “shift” SMEs toward an outcome focus rather than an information focus.

Step 1: Verify required outcomes. This sounds amazingly simple, yet you wouldn’t believe how many times people say they need training and yet they cannot articulate a simple goal or outcome this training should achieve. In instructional design lingo, we call this a “course goal.” It’s a description of what learners will do on the job once training is complete. Here’s a few examples of course goals:

• When selling Product X, reps will position Product X against competition, communicate benefits, and successfully close at least 80% of sales.
• During customer phone calls or customer meetings, reps will appropriately communicate the “fit” of Product X for the customer’s particular situation.
• Patients will safely and confidently use Equipment X to do home treatment.

Often, SMEs who have been tapped to be content experts have thought very little about what OUTCOME they hope learners will have. They have focused instead on thinking about what content they want to deliver. We give them examples of possible course goals and then ask them questions to help them shift focus:

• When training is over, what does the employee need to be able to do on the job? (When you hear, “Well, they need to understand X, Y, and Z,” you simply re-direct them and say back. “But WHY do they need to understand X, Y, and Z? What will they DO with this information?)
• If this training is successful, what problem will you solve or prevent?
• If this training is successful, what improvement will occur?

Sometimes, SMEs simply cannot articulate a goal. If they can’t, try shifting to step 2 and seeing if you can distill the ultimate goal after seeing their responses to Step 2 questions.

Step 2: Define the learning objectives. Now - we don’t tell SMEs that we’re “defining learning objectives,” but it’s what we do. We ask these questions, and we have them write their responses to each one on individual Post-It Notes. Ideally, we’ll use different colored Post-Its for each “type” of question:

• What do learners need to know?
• What do learners need to do?
• What do learners need to believe/feel?
• What common mistakes do you want to prevent or what problem do you want to solve?

SMEs generally respond really well to this activity. They may not do a great job distinguishing between something people need to know (knowledge) versus something they need to do (skill), but they will brainstorm their thoughts and we can then organize them.

Step 3: Chunk everything. This is honestly what we do. Together with the SMEs we shift/move the Post-Its around into logical groups. Topic themes will emerge and natural hierarchies will form. We’ll put the Post-Its on flipcharts and name each chunk with a logical title.

Step 4: Identify required content. Now, the SMEs can start talking content. We will look at each objective we defined and discuss what content has to be included to support it. For example, if the objective is that the patient performs machine set-up tasks, then the content is clearly descriptions of the machine set-up tasks.

Step 5: Identify appropriate learning activities. This is something we sometimes do with SMEs..and sometimes without them. If an appropriate learning activity is a practice activity in answering customer questions, we will brainstorm with the SMEs what questions are typically asked, what common mistakes reps make in answering these questions, and what kinds of resources a rep typically has available. In other words, we use the SME to craft the structure of the learning activity and to help us populate it.

Step 6: Identify job aids. What tools can help the learner transfer the training to the job? Sometimes the answer is - nothing. Other times, SMEs will identify “job aids” they want the learner to have access to. Honestly, in our experience, this offers a nice outlet to the SMEs to spill out all the content they didn’t get to include in the body of the course. We call these “reference materials” and allow SMEs to include all the “extra” material that we helped them winnow out through Steps 1 - 5.

I did a podcast where I attempted to show a very abbreviated form of this design meeting process. You can check it out here (And, yes, I realize no one is going to nominate me for an Emmy or an Oscar for my scintillating performance in the podcast!):

It was just a few years ago that I was in the midst of kicking off a major succession plan across my organization. I can recall the presentations we delivered to the executive team to garner their support. We spoke of the coming talent shortage due to baby boomers retiring, the need to narrow skills gaps to prep people for leadership roles, and to provide cross training and developmental projects to build necessary skills. In these lean times I’ve sometimes wondered, were we wrong?

The answer is no.  It’s hard to imagine these gaps are real when the job market is tight. Despite gloomy unemployment numbers talent gaps are widening.  This month’s issue of T&D Magazine describes the widening talent gaps on multiple levels. The risk of retiring baby boomers and the lack of younger workers to replace them is just as true today as it was before the recession. Not only is there a lack of people in terms of numbers but companies are struggling to bridge leadership capabilities in the current workforce. In ASTD’s report “Bridging the Skills Gap” 40% of organizations ranked a lack in “bench strength in the company’s leadership ranks” as a leading factor as to why skills gaps exist in their organization. (1)

ASTD’s report also points to widening skill gaps as jobs are changing and educational attainment is lacking.  The changing nature of jobs has had particular impact in attaining workers with technology skills.  While this has some obvious impacts, some are more subtle. For example, another article in T&D this month “Don’t Delete the E-Messenger” cited a study from Watson Wyatt that demonstrates a “strong correlation between companies that have effective communication and those that meet their financial targets.  The companies with the most effective communications strategy are those that are looking to leverage social media.” The top three reasons companies cited for not implementing social media in the study are 1. limited staff or resources to implement social media, 2. limited knowledge of social media and 3.lack of IT support or technical capability. All of the reasons are due to a lack of knowledge and skill, interestingly legal reasons ranked in last place. (2)

So it seems learning and development and society as whole has quite a job in front of us as we prepare the workforce and ourselves as we strive to keep up with the pace of technology. It’s critical that organizations aren’t lulled into complacency by a high unemployment rate. Succession planning and other talent management strategies are just as necessary today as they were three years ago.
————
1.    Pat Galagan, Bridging the Skills Gap, T&D Magazine (February 2010)
2.    Aparna Nacherla, Don’t Delete the E-Messenger, T&D Magazine (February 2010)

I recently presented a case study at the central Indiana ASTD fall conference. The focus of the case study was on using e-learning templates to facilitate rapid design. But as part of the presentation, I talked about how we managed a large group of SMEs (over 40 total) on the project. The room became alive and there were so many questions and comments I had to halt the discussion. I had obviously hit a hot topic!

Working with SMEs can be a challenging part of the design and development of learning solutions - but it doesn’t have to be! Here are my tips for working successfully with subject matter experts:

1. Get everyone in the same room (physically or virtually) to kickoff the project. Even if the SMEs will be working on different phases or aspects of the project, it is best for them to hear the same message, and for them to raise questions or issues at this meeting for all to hear. I recently did this for a project and it became clear after about an hour in that the SMEs weren’t all on the same page regarding the job function, much less how to teach it. Uncovering this issue early allowed us to avoid massive rework down the road.

2. Give SMEs a clear job description with detailed authority parameters. One of the frustrations we often have with SMEs is that they either don’t do what we want them to, or they overstep what we thought their boundaries were. Either way, this problem can be mitigated by being very clear up front what the job is. And as learning professionals, we need to remember that the project isn’t the SME’s full time job - we have a responsibility to educate them! I recommend providing an actual written job description. The one I use is about two pages and includes:

• Course development phases and SME responsibilities at each phase (how much time should they set aside to review a design document? What kinds of edits do we expect at alpha vs beta versions of the course? etc.)
• Who’s responsible for scheduling meetings.
• Rules for deadline days (I meet my deadlines, you need to meet your deadlines. What time is “end of day”?, etc.)
• How should they share source content (collaboration space, email, etc.)? Who needs to receive source content?
• Team member roles and responsibilities.
• What’s already been decided by stakeholders that outrank the SMEs. If your stakeholder has already told you what the objectives have to be, be honest about that. Don’t let the SMEs work under the impression that they have more power than they actually do. It just leads to frustration for everyone down the road.

3. Make sure that SMEs understand the domino effect of their decisions. One of the biggest challenges is getting SMEs to meet deadlines to avoid project delays. The first thing we need to do as project managers is to ensure that the deadlines we set are actually realistic. And I mean realistic. Even if the SME says that he/she can meet a deadline, usually you can tell when that’s not 100% accurate. You might need to call shenanigans if you see any of these signs:

• Shifty eyes. If they can’t look you in the eye when they agree to a 24 hour review time, don’t hold your breath.
• Avoidance. When they see you in the hall, they immediately remember an important errand on the other side of the building.
• Meetings start by the SME telling you how they have become good friends with the cleaning crew during their evening work sessions.

Once you’ve determined that the deadlines are realistic, then have honest conversations when SMEs want to make changes and additions to the solution or change the timeframe. Be sure they understand what that means for other courses, the budget of the project, and the overall quality of the project. By nature, the SME often has a narrower focus than we do because they are the expert. Part of our job is to help them see the forest for the trees and consider all options before making a decision.

4. Keep SMEs updated. At BLP, we often use a weekly/biweekly status report to communicate how things are going with the project. It’s just an excel spreadsheet that lists the project phases, who’s involved, where things are, and what’s coming up next. For SMEs, the most powerful part of the report is the icons we use to indicate status. It’s a great visual way for them to see our perception of the project. We use icons to indicate:

• Things are great!
• SME hasn’t been responsive.
• Phase needs immediate attention.
• We have some concerns/cautions.

I hope these tips were helpful! What do you to make the devleopment process a happy one for both you and your SMEs?

I can take absolutely no credit for this. I have a printed copy taped up near my desk, and I think clearly shows the number one conflict between SMEs and IDs.  SMEs want to pass on all knowledge immediately because it is all important and necessary. IDs want to organize that into comfortable chunks for the learners.

To give credit, this is from one of my favorite blogs, Usable Learning.

Sorry about the quality, I scanned in my copy!

We've all had to sit through this course...

I have a colleague who once created a presentation called “Herding Cats: Working with SMEs.” Needless to say, her viewpoint on the value of SMEs was influenced by some negative experiences.

Cats have often been used to describe SMEs - independent and impossible to control...but still lovable.

Can subject matter experts (aka SMEs) make good developers? How do you manage them and keep them focused? Can you shift them from a content (input) focus to an outcome focus? How do you keep them from derailing your project by overloading you with content? If a SME doesn’t know anything about instructional design, how can you involve them in designing a learning solution? What about deadlines…how do you hold them accountable?

In our experience, which spans a lot of years, subject matter experts are critical to most of our projects’ successes. Conversely, they can also become the Achilles’ heel that hinders success or makes a project take far longer than it should to complete. How to you ensure the former scenario and prevent the latter one?

That’s what our February blog posts are about. Over the next four weeks, we’ll share our tips and tricks for maximizing the relationship with SMEs. Specifically, we plan to talk about:

• Managing expectations between the SME and the designer/developers and techniques for clarifying roles/responsibilities.
• Tools that can make it easier for SMEs to function as developers - and designers.
• Techniques that make it easier to hold SMEs accountable for delivering what they say they will deliver.
• How to speak the language of the SME rather than trying to teach the language of learning design to the SME.

We welcome your thoughts and ideas as well. If you’ve identified a great strategy or technique for partnering with SMEs, share it! If you have a question or a challenge, let us know that too and we’ll try to address it here.

Also look for a couple interesting interviews with SMEs. While we view them in a particular light, it’s always good to view the world from their stance as well.