HOW: CONTENT - Context-Free / Adaptive Learning vs Mandated Curriculum
Misha Eydman
I am starting this thread to foster a new discussion on various approaches to instructional design. This is form an earlier post by Diana Sharp:
Oh boy have I thought about this question:
While some of the content can be very structured and akin to how we learn today, for example a lesson in Algebra with assessments and clear progression; other types, like packages apps, which I think you were referring to as tools, could be a lot more self-contained. The issue then becomes, how do we motivate a child to limit their time on something like this, for example - song creating app, and allocating it to less engaging, but just as necessary study of, let's say reading? I can't say the solution is clear to me, but I am inclined to say that some kind of overall structure will be necessary, as opposed to completely context-free learning, in order to produce measurable results. With your background in instructional design, do you think this is something you can help us understand better or provide some direction?
You have no idea how this question once haunted me. The good news is that at least in regards to reading, I feel very strongly now about a particular way of answering it. The bad news is that the answer may need to be different across subject areas, though I suspect there will be similarities that perhaps we can figure out together with other subject-area experts.
Fair warning: This is going to be a long post, because it is such an important question, and I have thought about it so deeply over many years. That said, please know that I am also open to further discussion about it and alternative points of view, since I have learned that it is dangerous to assume that your most dearly held beliefs will never change.
I also think it might be helpful to know the context for my answer. I spent almost 15 years exploring and designing technology tools for early reading for in-school use. At that time, I mistakenly thought I knew a lot about creating really engaging reading tools. This was because the kids always were so extremely excited to work on our tools in school. I only realized later that this enthusiasm was due to the fact that our tools were, in fact, more engaging that most of the other things they might be made to do in class, but that was not a very difficult for thing for us to achieve. I failed to realize that we were successful only because the engagement bar set by other class activities was usually so low. I don’t fault the teachers here – at the time they were under a lot of pressure for skill gains and were required to provide a lot of scripted skill-based lessons that didn’t excite the kids much if at all. Had they been given more freedom to create engaging classroom activities, our activities might not have seemed so fun by such an overwhelming margin. And because the children spent time with our activities, and they were well-designed in a number of ways, they made skill gains in many areas (with the exception of the connected text skill that I mentioned in an earlier post.)
After that experience, I became involved in learning design team on a project for creating breakthrough technology for early readers…a project aimed at creating a consumer product for use outside of school. Suddenly I realized, with the additional wisdom gained by having my own young kids around that age at the time, that the types of activities we had always created for school use would never work. Sure, they fun to learn in class. But if a kid was deciding whether to do those types of activities in his free time, as opposed to all the other ways he could be choosing to spend his free time…there was no way that, at least once the novelty of the tech wore off, that a kid would think this was the most fun thing he could be doing at that moment. (I will interject here that I also realize my thoughts on this are based in American culture, for kids in families that have many toys, games, and other free time options that are not true of other cultures. So I don’t actually know how well they translate to global poverty contexts – that is something others on the team can weigh in on.)
I realized that in order for an out-of-school product to actually impact a child’s reading, we were going to have to find a way to make the activities fun at another quantum level – much more fun than what we had previously designed for in-school use. Because in the end:
• If the child doesn’t choose to do the activities, then no matter how educationally well designed they are, there will be no skill gains. Period. Motivation is king. Ignore this at your peril. You have to focus on the fun aspect as primary and then figure out how to make the activities build skills. For that, you have to trust the cognitive science on how those skills develop. I’ll get back to this regarding early reading soon.
I also realized something that had actually bothered me for many years, in regard to designing technology that used data from a child’s performance on educational activities:
· You can’t totally – or even mostly-- trust the data, especially in free-play activities while there is no parent of teacher watching. Why? Because kids are kids! Again, at least here in America, kids are often more interested in exploring the limits of the technology, seeing if they can thwart it into doing funny things, or maybe, say, just enjoying the funny sounds it makes when the computer tells them they answered wrong. Kids feel powerful when they don’t do what an adult – or a computer acting like an adult – asks them to do when they can get away with it. So if you think they are going to always answer with the answer they think is correct, and you will be able to ascertain their level of knowledge with any precision, you are setting yourself up for disappointment. That is not how kids play. And if kids of this age are going to be choosing to do something on their own, it has to feel like play. So the challenge is, how do you create something that feels like play and still promotes learning, once you give up the belief that you can test for their knowledge of where they are on the skills tree by asking them school-like questions in the way that adults ask them questions, and believe the data? More on this later. (Don’t worry, all is not lost.) Listen, I know that there are lots of products out there that purport to measure kids’ knowledge as they play the educational games, but I’m convinced a lot of the data is junk data, because kids – at least, American kids -- love answering things wrong just to see what happens when they play.
So, back to the central question: how to create activities to promote reading that would be so fun that kids would choose to do them versus their other free-time leisure choices? Long story short, I finally realized the answer has to lie in large part in exactly what Bodo has discovered with his children: personal interests. Kids do become passionate about things even at early ages – bugs, sports, dolphins, fairies, pirates, the list goes on and on. If you offer them reading to do related to those passions, they will choose to do it. They will demand to do it. I followed this principle with my own children as early readers and on to early adulthood – many trips to the library to get a huge volume of books on things they felt ownership of because they found them personally interesting – and for awhile my husband could never listen to music in the car when we were all together because the kids would say, “Dad, turn that music off, we’re trying to read!” It was awesome.
But that’s only part of the answer, and it doesn’t address some of the tech side. Very early on, kids have to learn about the letters and sounds, and the amount and level of text they can read is limited. A lot of the engagement of something they will want to read will also be dependent on having great pictures, because the text itself has to be limited in complexity and richness. So what makes the most sense for developing tech activities for these very early stages?
As part of that consumer product project, we worked with top-notch game designers. I learned a lot from them but will save some of that for another discussion. But the basic point is that the determination of how fun something is is going to depend both on the amount of effort something takes, and the payoff. I’ve written about that at some level here:
I also realized that games could be great for those early reader stages – because you can have text in games that is very short yet meaningful – “Go to the tree!” “Draw me a house!” “Touch your nose!”– and gives kids instant feedback. You can also have lots of letter-sound matching games. But…and here’s the key thing to keep in mind – if you try to design these games to “measure” skills in a traditional way, you won’t be as effective in getting kids to want to spend time with them and learn. Why? Because of the nature of play. Measurement means you are asking kids to give you one right answer. Play means you want to explore and try out lots of things and make things up yourself. So a “restaurant game” in which kids get “orders” to read, like “Make me two pizzas with cheese” or a “bakery game” in which you get orders like “I want a cake that is red” and then you try to assess their comprehension by whether or not they make the right thing is just never going to be as fun as a restaurant game or a bakery game where you get to make whatever you want in the kitchen and see how the customer reacts when you make a cake that has bugs and worms on it, when what he really wanted was a pretty cake with flowers, or what happens when you give a customer who wanted a small slice of pizza a giant pizza that is eight feet long. Here is something I wrote previously about it:
On the other hand – how much fun is it really to read something and always have to demonstrate comprehension? To complete an order? To follow instructions? To put things in sequence? To follow the steps to a goal that someone else has set? I worry that too much focus on being “sure” that kids are always demonstrating comprehension could threaten the fun of reading. Like the Heisenberg principle (that by measuring things you can change them in ways that are unknowable), the more consistently and scientifically we try to measure comprehension, the more we may unknowably affect the motivation for reading that will lead to it.
Of course, many fun games have clear goals that are pre-set: In Monopoly, you try to get all the money, in Clue you try to find the culprit, in the sledding game on Webkins (that for some reason is addictive) you try to get your bear down the mountain without crashing. And in all of these games we could devise ways for kids to read to succeed. So I am not at all anti-any-attempt to create situations where kids read and comprehend to succeed in a pre-set goal. But remember that in Monopoly, part of the fun is that you can choose to buy or not buy property; in Clue you can choose to go or not go to a certain room, and in the sledding bear game, you can choose to hop over an obstacle or make a turn or even make him wipe out. And unless we give kids those kinds of choices, I think we threaten the fun of the game, even though those choices by nature make it harder for us to really measure scientifically for sure if kids are really truly comprehending.
So where does this leave us? As I said, at least in early reading, the answer is to trust what we know about the developmental process. What we know is that kids need a threshold level of letter-sound knowledge (not complete mastery) to begin reading things they are interested in. And the more they read, the better they will get. It’s that simple. We need to offer them opportunities to build those initial letter sound skills and then draw them irresistibly into reading high volumes of stuff by offering them high-quality reading that is based on their interests. We also need to offer them easy help when they can’t read a word so they won’t get frustrated. How will the computer know when kids are ready for something harder? There are things we can do – for example, track how often kids are asking for help on words. If they do a lot of reading, they’ll become skilled readers. Period. The data shows this: in all of education research there’s only one finding I’ve seen that shows low-income kids scoring higher than high-income kids -- and that’s the finding that low-income kids who read more outside of school score higher than high-income kids who don’t. (Ask me if you want more on this.) I’ve also seen it my own life. I knew all the supposed skills that schools were teaching for reading, but how did I support my own kids at home? By focusing purely on making sure they had the opportunity to read high volumes of stuff they were interested in. But does this translate to reading scores on tests? Well, I can’t give you a scientific answer, but my daughter scored a perfect 800 on the reading SAT last year. I’ve seen the power of this approach for my children, and I don’t want to be in a situation of creating things for “other people’s children” that I would not have provided for my own children in the belief that it was the most effective thing to provide.
But Diana, you may ask, how does this translate to game design? I’ll give you an example. That consumer product project went on for three years and was eventually killed when a new executive came in who did not think the prime audience at the time was young kids. But we did successfully launch outside of that company one of the games, which won a number of industry laudates and awards. In that game, a character would ask a child to draw something (draw me a house!). The child had to read the request and could ask for help on any words. Then, the child could draw whatever crazy thing he wanted – a crazy house! And the software would figure out a way to animate the drawing so the character interacted with the drawing – for example went inside the house. There was no right answer, and that’s what made it fun! That’s what kept kids reading! They would read “I need a wheel” for a character next to a tractor and see a wheeless tractor on the farm and then they could draw the craziest wheel they could think of and the software would animate the drawing so what they drew acted like a wheel and the character rode the tractor around the farm. Peals of laughter. That’s just an example. The level of the character’s request (from simple short sentences to longer more complicated multi-sentence requests) ratcheted up based on amount of play and other things I think – can’t remember – like requests for word help. How did we know they were comprehending? Because the game would not be as fun if you didn’t know what the original request was. It was more fun to play the game if you knew what the character was asking than if you didn’t. That’s not a scientific measurement, but it worked in practice. Just an example – and will be glad to talk about this in more detail.
But Diana, you may ask, what about all that myriad of skills, especially comprehension and analysis skills, that cognitive psychologists have identified related to what a good reader can do? All those skill trees? Okay, I hate to break it to you, but while those MAY be helpful for assessment, there is a danger of killing engagement if you rely on them for activity design. And it’s partly the fault of my own profession. Back in the 1980’s, when I was trained, it was a heady time for cognitive psychologists. With new computers able to track to the millisecond the gazes and responses of adult readers, we began to get real evidence about what was going on inside their heads. At the same time, in classroom research with children, a raft of studies showed that good readers make inferences, question, and do all sorts of active strategizing to get meaning from text. Poor readers don’t.
And that’s where we made our mistake.
We reasoned, “Now that we know what good readers do, let’s invent ways to explicitly teach all those things to everyone.”
Study after study followed on teaching children reading strategies. The results? Enough statistically significant gains on comprehension tests to garner publication in a research journal and apply for the next grant. Were there enough gains to suggest we were going to truly erase the comprehension gap between good and poor readers? Honestly – no.
The reasoning itself wasn’t the problem. Our mistake as a field was in failing to pay enough attention to another approach: a developmental one. Before placing all our eggs in the explicit- instruction basket, we should have said, more often and more loudly, “Now that we know what good readers do, let’s also try to deeply understand how they got that way. Let’s find that out and then seriously try to provide that to everyone, starting as young as possible. Then maybe most kids won’t need all this newly-developed fluency, strategy, and comprehension instruction that wasn’t around for the good readers we originally studied.”
That is much harder work. It requires trusting what the reading DEVELOPMENT theory is, not the skill assessment theory. The reading development theory clearly shows that kids build reading skill by doing heavy volumes of reading of what they’re interested in. Prior to that they have to develop a threshold of phonics skills, and then you have to allow them to get word help as they need it when they start reading connected text. And give them tons of connected text to read that they will want to read. That’s the theory and it works. Designing skills-based computer activities beyond that is going to result in boring activities that kids will not voluntarily choose to do in their free time, so they won’t do them, they won’t read, and they won’t gain skills. At least, that is my current belief, but I am willing to be proven wrong by someone who has more creativity in this area than I have seen!
So for the x-prize, what’s our best strategy for getting measurable reading results on the EGRA? Design some fun engaging games to give kids a threshold level of letter-sound skills and word-reading. (We can also have some phonemic awareness games for rhyming etc for kids who just don’t seem to get the letter-sound games and need more support). You can even have a skill tree related to these early pre-reading skills :). Then as soon as possible, move them from reading words into reading connected text related to their interests. Start with fun games or song-making or communication-making activities in which they will read short sentences. As quickly and as much as possible, offer them more reading material like e-books, where they will be spending as much time reading as much as possible. You can even have types of structured-skill trees that tag reading material as far as its level of complexity and ratchet up what level of complex material kids get :). Supplement this type of activity with written communication opportunities that are also going to be very motivating for kids. The greater volume of what they read and write, the better they will do in terms of the measurable results on the EGRA. That’s what the reading development research says, and we need to trust it. If instead we offer them skills-based activities in which they are actually doing less reading than if they were just reading something they were interested it, and we try to measure those skills within the activities, we'll have more boring activities, they’ll do less actual reading, and the results on the EGRA will be worse.
I better stop here – I could go on, believe it or not, but I know no one is going to read a longer post, and I won’t blame those of you who stopped earlier. But I hope some of this helpful and as far as the domain of reading helps this project to be on the right track for the greatest success. I don’t know the best way to translate these ideas to math learning, etc., where there are more definite “skills” that have to be explicitly taught. I suspect the analog to “offer them as much reading opportunity as possible related to their interests” in the math field may relate to providing them with opportunities to build or construct or maybe even program things, where they practice those computational skills as much as possible. And that skills trees to track those skills may be more important. It’s a hard problem. But hope to be involved in helping with the solution. Glad to discuss/debate any of this.
Stu Holmes
https://youtu.be/idJIYbMVPRs
Here's a longer example of the app:
https://www.youtube.com/watch?v=ULjrvvZJHQI
Diana Sharp
F [fffff} OR [or or or or] G [gggggg] I [iiiiiiii] VE [vvvvvvv]
That would have showed kids playfully how those sounds make up the word "forgive" and also have them learn intuitively that in English, sometimes more than one letter is used for a particular single sound (part of the alphabetic principle for English). So ***sigh*** so close but they missed it pedagogically and could potentially cause some real confusion to kids about how letters and sounds work in words. Probably most kids wouldn't be harmed for life :) but the kids who are having trouble with letter-sounds and decoding and/or are dyslexic could get very confused. Just saying that these are subtleties of phonics-based and early-learning activity design that took me a long time to get as well, but I believe they are really important, especially for kids who don't have strong literacy backgrounds or support. It's a real feat to combine such great creativity and art and playfulness as exemplified here with the subtleties of good pedagogy and learning science but hoping this Dev4x project can really be great by doing both!
Diana Sharp
http://chicago.tinkeringschool.com/collected-thoughts-on-tinkering-and-education/
“The potential for engaged learning is inversely proportion to the know-ability of the outcome”
Gever Tulley
-When we present narrow learning objectives, we receive narrow results.
Kirill Kireyev
Misha Eydman
Diana Sharp
Best,
Diana
Natalie Denmeade
So for the x-prize, what’s our best strategy for getting measurable reading results on the EGRA?
2) (We can also have some phonemic awareness games for rhyming etc for kids who just don’t seem to get the letter-sound games and need more support). You can even have a skill tree related to these early pre-reading skills :).
3) Then as soon as possible, move them from reading words into reading connected text related to their interests. Start with fun games or song-making or communication-making activities in which they will read short sentences. As quickly and as much as possible, offer them more reading material like e-books, where they will be spending as much time reading as much as possible. You can even have types of structured-skill trees that tag reading material as far as its level of complexity and ratchet up what level of complex material kids get :).
4) Supplement this type of activity with written communication opportunities that are also going to be very motivating for kids. The greater volume of what they read and write, the better they will do in terms of the measurable results on the EGRA. That’s what the reading development research says, and we need to trust it. If instead we offer them skills-based activities in which they are actually doing less reading than if they were just reading something they were interested it, and we try to measure those skills within the activities, we'll have more boring activities, they’ll do less actual reading, and the results on the EGRA will be worse.