Posts

Using Spaced Learning and Distributed Practice in Corporate Learning


Interested in spaced learning and distributed practice? Then download our free Primer on Spaced Repetition and Feedback Loops. This guide will teach you everything you need to know about these concepts so you can incorporate them in your own training.


Students are often told to study for a few minutes a day, every day, instead of cramming for a test. Musicians know that consistent, daily practice is the only way to learn a challenging piece. Research on the benefits of distributing our learning into small chunks has been around for a long time.

Formal Training a Small Amount of Overall Time

…So why do corporate trainers forget this fact when delivering eLearning and Instructor-led training? When did we start thinking of learning as a one-time event? If formal training is the only form of learning in your plan, and training is only happening a few times a year, L&D is not doing it’s job in supporting workplace performance. You must include opportunities for practice, reinforcement and reflection in your L&D mix.

Use Gamification to Space the Learning

In a recent article for Learning Solutions Magazine, Karl Kapp shared a case study on Pep Boys‘ approach to Retail Safety and Loss Prevention training. They used a gamification platform to deliver daily reinforcement of the monthly safety and loss prevention training through a fun, quiz-style game. Kapp describes the project in greater detail:

Associates answered quick, targeted questions related to risk, loss prevention, safety, and operational policies and procedures—standard questions in these areas. If they answered correctly, they played a slot-machine game titled “Quiz to Win” for a chance to win cash prizes. If they answered incorrectly, the system immediately presented a short training piece designed to specifically address the topic covered in the initial question. Questions repeated at various intervals until the associate demonstrated mastery of the topic. The entire process takes 30-90 seconds each day and associates do it either at the beginning of a shift or during downtime throughout the day.

Game-based learning and gamification have many different applications, but using a short game-like experience as a daily reinforcement activity is, frankly, an excellent idea. Pep Boys reported a voluntary participation rate of 95% and a 45% reduction in costly claim counts. I’m sure many trainers can’t claim a 95% participation rate for some mandatory training! By giving employees a fun and motivating experience they could complete in just a few minutes’ time every day, Pep Boys was able to ensure the training it delivers every month was properly reinforced. By making the reinforcement a fun, gamified experience, players were self-motivated to keep participating and reviewing the content.

Market Your Learning Internally

One of the best ways encourage distributed practice is to make employees want to come back and review. Offering some sort of external reward, being consistent and engaging in your reminders, and making the reminders memorable will all help.

Example: ExactTarget. Now a part of Salesforce, ExactTarget is a leader in the digital marketing space. It should come as no surprise that ExactTarget’s L&D function is best-in-class at marketing their training internally. We built a Knowledge Guru® game for ExactTarget to support the ramp-up for a new product launch, but it was ExactTarget’s reinforcement tactics and internal marketing efforts that really made the project a success.

Marketing Spaced Learning in an Organization

Example of an internal advertisement ExactTarget used to market their Knowledge Guru game.

ExactTarget furnished prizes for the top players and displayed advertisements on the LCD monitors all over their offices. They also sent consistent emails updating players on game progress and encouraged players to log back in and compete for prizes. This creates a culture of “fun” for the employees, but more importantly it encourages lots of distributed practice and reinforcement over a longer period of time than traditional methods.

Build spaced learning into your training design

Creating a separate reinforcement activity can be effective, and so can consistently marketing an activity and giving people an opportunity to return and review. But what about building opportunities for spaced learning and distributed practice right into the learning solution? That was our approach when designing Knowledge Guru. As this tutorial post explains, each training topic in the game requires players to climb three paths and deliver scrolls to the guru. The topic only has 5 – 10 total unique questions, but each of the three paths has a different iteration of the same question. By the time learners complete the paths, they have been exposed to the same information three different times.

Spaced learning and distributed practice in Knowledge Guru

Different iterations of the same question are placed on each path.

Guru Grab Bag Mode: The Guru Grab Bag mode in Knowledge Guru encourages players to return to the game and practice even after completing the game. Grab Bag is only unlocked when players complete all of the normal game mode topics. It’s easy for the L&D professional administering the game to encourage players to return to the game later to compete in the “Grab Bag Round,” where all questions in the game are mixed together and players try and see how big they can get their streak of correct responses. If a player plays Guru Grab Bag long enough, they will be exposed to all of the game questions again, which helps them log more distributed practice time.

Distributed Practice Helps the Bottom Line

When you’re trying to solve a performance challenge, you need to give learners the tools and opportunities to learn as quickly as possible. Failing to create a plan for sufficient reinforcement and distributed practice will only lead to increasing costs further down the line. Take some time up-front to plan the reinforcement and internal marketing of your training. If you can, build the distributed practice right into your learning design.

 

A Counterpoint to Ruth Clark’s “Why Games Don’t Teach”

In February’s Learning Solutions magazine, Dr. Ruth Clark started quite a buzz. She wrote an article with the provocative title, “Why Games Don’t Teach.” This was actually the second time she has written an article with the same title… the first appearing on the ASTD Learning & Development blog.

For those of us who are passionate about game-based learning (GBL), she caused a bit of an uproar. For those who are trying to figure out game-based learning, I fear she squelched them just with her headline.

Less Tell; More Games and Gamification

Games Teach; Poorly Designed Games Don’t

When you actually READ what Dr. Clark wrote, she isn’t saying that games are ineffective as learning tools., despite the provocative headline. She says that poorly designed games don’t teach… and not all games teach. I believe she wants to start a thoughtful dialogue about when games should be used – and when other learning solutions might be a better option. She also may want to stop the stampede of converting everything to a game-based learning approach, regardless of whether it’s the best tool.

Unfortunately, I think Dr. Clark’s article may do more harm than good because too many people won’t get past its title. I also think people may simply accept her statements about research without questioning them or digging deeper on their own. I respectfully disagree with Dr. Clark, and here’s a rather long summary of why.

It’s How You Interpret the Data

Clark’s argument for the lack of research focuses on one meta-study of numerous other studies on the efficacy of games in learning.  She says this study indicates there is no clear linkage to prove games are effective learning tools. Ironically, Dr. Karl Kapp, in his book, The Gamification of Learning and Instruction, uses this same meta-analysis to demonstrate evidence of the value of games. He has an entire chapter devoted to research and he offers a detailed explanation of what the meta-analysis says.

Dr. Kapp published his own rebuttal of Clark’s article in learning solutions magazine last week. In particular, the taxonomy he provides showing type of knowledge, instructional strategies and game elements is extremely valuable.

Dr. Richard Van Eck, an Associate Professor at the University of North Dakota and the graduate director of the Instructional Design and Technology graduate program, wrote a terrific article in 2006 titled “Digital Game-Based Learning. In his article he listed these studies as a very small sampling of the studies available showing efficacy of instructional games:

  • Meta-analysis of Simulation Games Effectiveness for Cognitive Learning (PhD dissertation), Indiana University, M. Szcurek, 1982 (link)
  • A Quantitative Review of Research on Instructional Simulation Gaming: A Twenty-Year Perspective, R.L. Van Sickle, in Theory and Research in Social Education, vol 14, no. 3 (1986) pp 245-264 (link)
  • “The Effectiveness of Games for Educational Purposes: A Review of Recent Research,” J.M. Randall, B.A. Morris, C.D. Wetzel, and B.V. Whitehill in Simulation and Gaming, vol 23, no. 3 (1992) pp 261-276 (link)

Dr. Van Eck does a nice job of stating what I THINK Dr. Clark intended. The emphasis is his:

“I believe we need to change our message. If we continue to preach only that games can be effective, we run the risk of creating the impression that all games are good for all learners and for all learning outcomes, which is categorically not the case. What we need now is 1) research explaining why games are engaging and effective and 2) practical guidelines for how, when, with whom and under what conditions games can be integrated into the learning process to maximize their learning potential.”

This is a much more accurate message than, “Why Games Don’t Teach.”  I also think Dr. Van Eck’s call for practical guidelines on when to use games is what Dr. Clark hoped to get across.

Dr. Van Eck also lists a slew of books and research articles published over the last few decades on game-based learning. These books have been published by the likes of Clark Aldrich, Jane McGonigal, James Paul Gee – people highly respected in the field of games and learning, each of whom has cited research within their books that support game-based learning.

Dr. Clark expressed concern over the lack of studies comparing a game-based approach to a non-game-based approach. I can share three that I think are good. These studies were conducted and shared by Dr. Richard Blunt in 2009. Blunt was the Director of Plans and Programs for the Department of Defense Advanced Distributed Learning Initiative. His area of expertise is game-and simulation design.

Blunt’s studies showed a significant correlation between the use of games and subsequent performance on a test. All three studies featured a game group and a control group that had traditional instruction. Both groups then took the same post-test on the material. This is from his abstract:

Three research studies were conducted at a national university to examine the differences in academic achievement among students who did and did not use video games in learning. Three different video games were added to approximately half the classes of freshmen Introduction to Business and Technology courses, third year Economics courses, and third year Management courses. Identical testing situations were used in all courses while data collected included game use, test scores, gender, ethnicity and age. ANOVA, chi-squared and t tests were used to test game use effectiveness. Students in classes using the game scored significantly higher means than classes that did not.

Read Dr. Blunt’s full paper here.

A less academic version was published in eLearn magazine in December 2009. It can be found here.

I’m certain Clark would point out some constraints in Dr. Blunt’s studies:

  1.  The games used were video games. She might argue that performance might have been different with a different game format.
  2.  Blunt’s results also showed that older students (those older than 41 years of age) did not have as significant of a performance difference.
  3. Clark might argue that these results indicate we need further research to determine which games are best suited for which age groups… or whether games are only useful for digital natives.

We can always use more research, but we have enough to take action

I am all in favor of ongoing research by those who can devote their time and attention to this research and share it out with those of us who are practitioners. However, I won’t disregard the plethora of research that I believe already exists to show games work… and to show that people require engagement and motivation to learn – something games can provide. I think a thoughtful instructional game designer can do reasonable audience analysis and formulate an action plan as to the best type of game for the target audience. We do this all the time at BLP – and we’ve had great (and measurable) success using games across age groups.

Replace traditional eLearning with games and gamification

Games are powerful teaching tools – and far more effective than the plethora of “click next to continue” solutions I see in the corporate world. A well-designed game lets people benefit from feedback mechanisms that offer continuous information on how well or poorly they are doing. It offers valuable context and it can give them  far more rehearsal and practice than other types of learning solutions – even “active” ones that Clark touts as equivalent to game-based learning.

I want to ask Dr. Clark, who has written some amazing books that help guide the design of learning and instruction, to focus more on encouraging people to explore games as a learning tool. I hope she can acknowledge that game-based learning has been proven effective in a variety of situations.  Dr. Clark is a thought leader; she has power to influence positive changes in learning design, getting people to think in terms of how they could create a game – as opposed to a more lecture-based or “tell” format. As Clark pointed out, people do need active learning, and well-designed games are highly active. I agree that a poorly designed game won’t work well – but neither will a poorly-designed group discussion, quiz, or role play.

My blog article, Game-Based Learning: Why Does It Work, summarizes the linkage I see between game elements and learning elements. My hope is that people look beyond provocative headlines on the value (or lack of it) for game-based learning. Instead, I hope practitioners will thoughtfully consider what’s required for learning to happen – and consider how game elements can support that learning process.

 

Doodlers Unite

I’m a doodler. Since junior high I have been drawing kites, little squares, flowers, and random swirls all over my paper. I used to think my doodling was an obsessive behavior. Over the years I actually worked hard to reign in my random scribbling. Based on a recent study, it seems I should regress and return to may doodling ways. British psychologist Jackie Andrade of the University of Plymouth conducted a study indicating that doodling helps focus your mind and your memory. (1)

The study had participants listen to a monotonous phone message about a party. Half the participants were instructed to color in squares and circles on a piece of paper while listening to the message. The other half of the participants had no task to perform. When asked to recall specifics about the message, the doodlers were able to recall 29% more information than the control group. (2)

In an interview with Newsweek, Andrade states that simple doodling may be indicative that some learners need a visual to help them concentrate. The simplicity of doodling doesn’t distract the mind in the same way that daydreaming or texting does. On the contrary, it can provide focus on the higher level task.

Maybe we should incorporate doodling into our ILTs? As a facilitator have you been concerned that doodlers are not paying attention? Are you a doodler?

(1) Dina Fine Maron, Doodle Zone, Newsweek.com (Feb. 2009)

(2) Doodling Primes a Wandering Mind, CNN.com (Feb. 2009)