In-video quiz

[Daniel McKelvey]

I am looking for the ability to put a quiz inside a video.  Specifically, as the learning is watching the video, a quiz can appear anytime inside of the video.  So it's a powerful way to make the student truly comprehend the material.  If they don't answer the question correctly, the video many or many not continue until the right answer is selected.  Any suggestions here are greatly appreciated.

[Colin Fredericks]

Heh - just wrote that this week. :)

Check it out: 

https://edge.edx.org/courses/HarvardX/HXDEMO/2014/courseware/98e2362d5cfa4fee908ff8db31405795/6065c638713e4cbb92096c9663127d72/1

Hit "play" and you should see four different questions pop up in the first few minutes. There are controls underneath the video for resetting, skipping questions, and going back one question.

It would take some work to change things so that students would be unable to skip the questions, and much more work to make sure they answered correctly. You're welcome to use the code and change it to fit your needs.

[JAVID S]

Hi,i'm javid safi from iran,engineering of physics,university of Polytechnic(amir Kabir).myjob Telecommunications part of fiber optic&transmissions.

[Arash Dehghan]

Dear Javid

I am from Iran as well but i did not understand why did you send this post here???

[Piotr Mitros]

The way the platform was designed, you do this by having multiple video segments. So instead of a 30 minute video with 2 pop-up quizzes, you have it structured as 10 minute video/problem/10 minute video/problem/10 minute video. I designed the learning sequence in response to a number of UX issues with the early Coursera and Udacity/Know Labs platforms. 

To make this easy, there was even an option to have video start/stop times, so you take a 30 minute video, and include it 3 times, with start/stop at 0-10 minutes, then 10-20 minutes, and finally, 20-30 minutes. This was broken for a while; I have not tried this (I tend to shoot independent videos), so I'm not sure if it is working; if not, it'd be worth fixing. 

Piotr

On Friday, August 15, 2014 2:07:01 PM UTC-4, Daniel McKelvey wrote:

[Jane Manning]

Piotr: At Stanford, we've found the start/stop time feature to not be very robust for students (and last time I checked it had some UI issues wrt scrubber position - but I'm on vacation w v.slow internet so can't confirm that right now), so we advise instructors to chop the videos into pieces of the appropriate length, with problems in between.

But this is fairly brittle for the instructor - you can't easily decide to eg move the quiz to another spot in the video, or add an extra quiz (or remove one).

I'd be curious to hear about the UX issues you mentioned with the "in-video" style - can you say more about that?  The version of Colin's that I saw lacked indication on the scroll bar of where in the video the quizzes are, which is a UX issue, though seems like a motivation to build the functionality into the player, rather than giving up on the functionality altogether.  

Like Colin at Harvard (and others on this thread)  we get many requests for this feature.  I don't think instructors always know what's best for them, but in this case I can see why they're interested in this, so if there are some reasons why this feature isn't a good idea, I'd be interested to hear.

Jane

Sent from a phone.

[Piotr Mitros]

It's been a few years, so I'm not sure if I recall all of the details, but I'll give what I do recall off-hand. I'm not sure if these are so much counter to providing a different way of doing things, as they may serve as guideposts to how to design this well. The experience was a little painful in Know Labs 1.0 and in Coursera 1.0 (which copied Know Labs). MITx improved on it, and Udacity, aside from dropping the name "Know Labs," copied our UX. At the time, both MITx/early edX and Udacity were well ahead of Coursera in usability, in part, due to this. The UX in Coursera has improved in a number of ways, but this is still a place I find our platforms better. 

The core reasons are: 

* It's often handy to read or do assessments before watching videos (and not watch videos if you know everything), and in some cases, the reverse (e.g. I'll listen to videos when driving, and then do problems). Clicking on the little Coursera notch at the bottom is painful. 

* Long videos are pretty painful to navigate. The most common operation in a MOOC is to jump back by a few seconds or at most a few minutes (and in some cases, forward). With a 30 minute long video, this involves fine motor motion. With six 5 minute videos, this is pretty easy. 

* Conversely, chunks of the videos can have names (visible on mouse-over for the learning sequence), which is handy for finding the right one (if well-labeled and well-chunked, much more so than scrubbing through lecture content). 

* Dual navigation schemes are error-prone. If I had a quarter for every time I clicked "next" in Cousera after a quiz, instead of "continue", lost my spot, and had to go back, scrub around, find the next video marked as watched, etc. ....

* Modularity. There are few things as painful as having to rewatch the last 30 seconds of the video over-and-over to catch some part of a question you missed. In-video quizzes are limited in space, and generally not XBlocks, so limited in what you can put in. The edX design style lends itself to longer, more self-contained questions, which tend to be much more useful to students, both from a UX and a learning standpoint. 

* Likewise, the learning sequences lend themselves to richer content. As courses improve, increasingly I see richer things than simple multiple-choice questions in learning sequences, which really integrate transmission of knowledge with construction of knowledge. The in-video quizzes make a transition to this hard. 

* The last point is especially important -- when the questions mirror the videos too closely (e.g. Know Labs, where it literally overlapped the video), it is impossible to fix issues in questions without modifying the video, and vica-versa. It's good when there is some level of integration, but too much is too much. Likewise, individual video clips are easy to swap out. Even if the style changes, the transition is not too jarring. Continuous videos with questions lend themselves to less iterative improvement. 

My experience has been that most, although certainly not all, instructors who ask for this have not internalized the way this works in edX. Once we show the edX way, most tend to be happy with it. 

With regards to start/stop times, it worked well perhaps 2 years back. At some point, there were some issues with the scrubber which let you get beyond the beginning/end of the video. Then there were some issues where the scrubber showed the full length of the video, but started/stopped at marked points, which was super-confusing. I stopped following at that point. I recall this was one of the top priorities for one of our PMs, so it's possible she got it fixed. It'd be *very* worthwhile to fix if it is still broken (more so than making a new video player). Aside from instructor pain in chopping up videos, it saves students on slow connections a lot of waiting -- with the start/stop times, the browser could pre-cache video from the next segments. 

Other open issues are: 

* The somewhat painful interface in Studio for authoring learning sequences with many segments. 

* The somewhat slow JavaScript for going through learning sequences

* The inability to right-click/open-in-a-new-tab elements of a learning sequences. 

Piotr

[Jane Manning]

Piotr,

Thanks so much for this thoughtful response.  Some of these talking points will be really helpful to me when I talk with faculty who are asking for "in-video quizzes".  I'll address some of your specific comments in-line:

 

* It's often handy to read or do assessments before watching videos (and not watch videos if you know everything), and in some cases, the reverse (e.g. I'll listen to videos when driving, and then do problems). Clicking on the little Coursera notch at the bottom is painful. 

As someone who personally doesn't love watching videos, I couldn't agree with this more.  I think the original intention of the Coursera in-video quizzes was as strictly formative rather than as "assessments" - in fact I believe their original platform didn't allow counting them as part of a student's grade.  The first MOOC we ran on Class2Go (which supported in-video quizzes) had a clearly marked way to get at the in-video quiz content separately from the video.  I don't remember how many people used that option, but I think it's reasonable to see having only 1 navigation structure with a clear way to get at particular things as a big advantage.

 

* Long videos are pretty painful to navigate. The most common operation in a MOOC is to jump back by a few seconds or at most a few minutes (and in some cases, forward). With a 30 minute long video, this involves fine motor motion. With six 5 minute videos, this is pretty easy. 

That's a good point - the scrubber is more useful for shorter videos. (Though I guess if start/stop times in a single video are used rather than chopping, then this advantage goes away, depending on how the scrubber acts - it'd be ideal if it represented only the part of the video between the start and stop times.)

* Conversely, chunks of the videos can have names (visible on mouse-over for the learning sequence), which is handy for finding the right one (if well-labeled and well-chunked, much more so than scrubbing through lecture content). 

Yup, that's a good point, too.  Presumably even if you use start/stop times you can have names for the components with their particular start/stop times, so this advantage remains.

 

* Dual navigation schemes are error-prone. If I had a quarter for every time I clicked "next" in Cousera after a quiz, instead of "continue", lost my spot, and had to go back, scrub around, find the next video marked as watched, etc. ...

This seems like a fixable UI problem, but ok.

 

* Modularity. There are few things as painful as having to rewatch the last 30 seconds of the video over-and-over to catch some part of a question you missed. In-video quizzes are limited in space, and generally not XBlocks, so limited in what you can put in. The edX design style lends itself to longer, more self-contained questions, which tend to be much more useful to students, both from a UX and a learning standpoint. 

Yes, I think an advantage of edX over Coursera is the ability to have multiple types of things on one page, and being able to do this for a "quiz" between video sections is an advantage.

 

* Likewise, the learning sequences lend themselves to richer content. As courses improve, increasingly I see richer things than simple multiple-choice questions in learning sequences, which really integrate transmission of knowledge with construction of knowledge. The in-video quizzes make a transition to this hard. 

Same as above.

 

* The last point is especially important -- when the questions mirror the videos too closely (e.g. Know Labs, where it literally overlapped the video), it is impossible to fix issues in questions without modifying the video, and vica-versa. It's good when there is some level of integration, but too much is too much. Likewise, individual video clips are easy to swap out. Even if the style changes, the transition is not too jarring. Continuous videos with questions lend themselves to less iterative improvement.

I guess this is related to the "brittleness" I mentioned about wanting to be able to move a quiz to a different spot in a video.  If start/stop times worked reliably, you could have that flexibility just as you do with "in-video" quizzes on other platforms.

 

My experience has been that most, although certainly not all, instructors who ask for this have not internalized the way this works in edX. Once we show the edX way, most tend to be happy with it. 

I've found they've been willing to go with it, but because start-stop times haven't worked well for us, they've been frustrated at the amount of work to convert existing content by chopping videos (as well as associated caption files etc).

 

With regards to start/stop times, it worked well perhaps 2 years back. At some point, there were some issues with the scrubber which let you get beyond the beginning/end of the video. Then there were some issues where the scrubber showed the full length of the video, but started/stopped at marked points, which was super-confusing. I stopped following at that point. I recall this was one of the top priorities for one of our PMs, so it's possible she got it fixed. It'd be *very* worthwhile to fix if it is still broken (more so than making a new video player). Aside from instructor pain in chopping up videos, it saves students on slow connections a lot of waiting -- with the start/stop times, the browser could pre-cache video from the next segments. 

Pre-caching video with start/stop times sounds like a big advantage over chopped videos that I hadn't thought about.  In any event, it sounds like resolving the start/stop issue would be worthwhile.  I'm on vacation w/ slow internet (and I'm not working! ;-) so I can't verify the issues right now, but I'm pretty sure it hasn't worked well for us at Stanford.  Do you want to see whether you think it's working for you at the moment (including the scrubber bar acting correctly)?

 

Other open issues are: 

* The somewhat painful interface in Studio for authoring learning sequences with many segments. 

* The somewhat slow JavaScript for going through learning sequences

* The inability to right-click/open-in-a-new-tab elements of a learning sequences. 

Yup, all that is right, though these seem like somewhat separable issues from this.

Thanks again for writing this up.  If it turns out that the result of this discussion is that start/stop times in videos become usable, that'll be a big win.

Jane

 

[Yarko Tymciurak]

On Mon, Aug 18, 2014 at 7:27 PM, Jane Manning <ji...@stanford.edu> wrote:

Piotr,

Thanks so much for this thoughtful response.  Some of these talking points will be really helpful to me when I talk with faculty who are asking for "in-video quizzes".  I'll address some of your specific comments in-line:

Agreed - thank you, Piotr!

I won't comment in line, as I have a general, higher level observation:

It seems that the solution options being considered are from the wrong perspective, and Piotr's comments lean in the correct direction but not sufficiently far enough.

I am struggling with thinking about this, and have some gelling perceptions.

One:   moving quizzes around in a particular media is the wrong abstractions - they are two kinds of media/interactions forms.  Coupling one to the other is, I think, a core mistake.   It is that effect, not that design, which is desirable.  But if that effect is the goal, then the mode of learning / interacting ("reading" if you will) is missing, and either assumed, or forced to be primarily (centered on) video.  That is a coupling, and unfortunately narrow constraint.   There is also some perspective, consideration of the author in this (ease of creating, with an assumed solution of making video editing simpler, and thus slapping quizes, et.al., on top of that vehicle) - this is an important consideration, but the wrong center-of-perspective (which, I think, Piotr points out well).

For a reader of a traditional text, flipping back and forth to delve into a concept "not quite gotten" is an activity driven by the reader.  As in a traditional textbook, sidebar concept-check questions and end-of-section exercises (or quizes) are generally timed by the learner (sometimes to their detriment, when skipping - but I'm not about forcing an outcome in this discussion, rather setting up the appropriate model).

In the case of a multi-media "book" (reading / interaction), I would expect the pattern to follow that (for example) rather than watching the video with slides, and some bobble-head instructor in the corner (ala tech conference videos), I - the reader/learner - would want to "flip" thru the slides, and turn on (or mute) the video associated with them.   Part of that flipping would then, naturally, be the concept check at some point - which would only have media (video) associated with it if it were pertinent to that part of user-arrived-at content (i.e. part of the question of a quiz, for example).

This goes a bit further than Piotr's suggestion of "moving forward and updating things in the future" because it says the multimedia is not / should not be the center-point but rather a content delivery mechanism, not much differently treated than the page of a textbook (for example).   In this way, the autonomy and ordering of a quiz or question falls in naturally.

The additional richness of this way of looking at it is that this medium need not be linear, but can be comfortably tree structured, and dynamic:   side-roads to in-depth discussions or exercises about one topic (sometimes pointing to another online course, as a book would reference more than endnotes, but another textbook), as well as topically centered discussion spaces (not just all-course encompassing user forums, but local, in-context "around this section" discussion spaces - which could also be aggregated at the "bulk course discussion" area, entering which would place the learner/reader in full context of some specific discussion subjects - think of the richness!!!

To summarize, I'd take the lesson's Piotr has summarized, and leap-frog them even further, now.  It changes the architecture of the system in some useful (and I suggest simplifying) ways.

Regards,

Yarko

[Lyla Fischer]

Hi Jane, 

Thanks for the thoughtful response. I happen to be the PM that prioritized fixing the start/stop functionality within the video player. It seems like forever ago, but the ticket resolution date was only in early July, so there is a good chance that you were looking at the still-buggy functionality when you were last considering the start/stop functionality. 

If I remember correctly, the wonky behavior involved the video player ignoring start and stop times if the student interacted with the scrubber at all. Students can still use the scrubber to see any part of the video, including parts before the start time and after the end time, but the video will stop whenever it hits the designated end time. There is always a visual indicator on the scrubber that allows the student to see which segment of the video the author wanted the student to focus on. This way, if the student decides to explore or refresh memory from other aspects of the video, the student will not loose track of the primary educational activity in that module. 

I hope that helps!

-Lyla

[Jane Manning]

Thanks Lyla, we'll try it again and will report back!  

(but I'm out of town so wont do this immediately)

Jane

[Colin Fredericks]

I just updated mine with some bug fixes, so if you're using it you should download a new version of the zip file. (If you tried early and got a blank page, the link is now fixed.)

On the topic of whether or not this is a good thing - to me, the more tools I have available, the better. It's a great research question as to whether mid-video questions will be more or less useful, and in what ways - and now we can actually do that research! :)

[Paul-Olivier Dehaye]

I agree with Piotr and want to leap-frog with Yarko as well. 

Video should just be one content-delivery mechanism, and not the thread to which other activities (like a quiz) attach. 

Also, I find that Coursera's system actually adds complexity. What they call a lecture is a package of several items and constructed in two steps (enter quiz on top of video), compared to the relatively straightforward construction of learning sequences in edX (work on stuff where you want it). 

Paul

[Piotr Mitros]

Pardon super-long e-mail; feel free to skip to relevant bits. 

Jane:  

(Though I guess if start/stop times in a single video are used rather than chopping, then this advantage goes away, depending on how the scrubber acts - it'd be ideal if it represented only the part of the video between the start and stop times.)

... 

I'm on vacation w/ slow internet (and I'm not working! ;-) so I can't verify the issues right now, but I'm pretty sure it hasn't worked well for us at Stanford.  Do you want to see whether you think it's working for you at the moment (including the scrubber bar acting correctly)?

The feature was designed allow us to chop videos without needing to edit+chop. When it started showing things in the scrubber beyond the end was when I went back and re-chopped all of my videos. From Lyla's response, it appears the behavior is still buggy, at least by the original intent, design criteria, and ways it was used. 

Lyla:

Students can still use the scrubber to see any part of the video, including parts before the start time and after the end time, but the video will stop whenever it hits the designated end time.

Would you mind posting a bit more background here behind the decision? This is incredibly confusing behavior, and broke the content I was working with (and likely others were working with). When I saw this behavior, I just assumed it wasn't fully fixed yet. I just chopped up the videos by hand (the show had to go on, and I knew it was in the bug list). If this was requested by customers or we have evidence for the new behavior, it might be worth building in real start/stop times the way we had before. If this was not requested, it might be worth switching back to the designed behavior. 

Jane: 

Regarding formative vs. summative assessment, the point of formative assessment is to inform both the user and the instructor whether a given student knows something. This is precisely how I use them. When approaching most MOOC, I will typically: 

1) Try all of the assessments. This is a formative step. The goal is to discover what I know and what I do not know. The in-video quizzes are particular useful in this regard, as are any quizzes with infinite attempts. I find timed or limited-attempt quizzes kind of obnoxious, in part because they do not lend themselves to this. 

2) If I am successful, I will move on, generally without watching any videos or reading any text. I'll sometimes miss content (if the assessments sparsely cover the content), but that's okay. There's an effectively infinite amount of content out there, and I'll move on to higher-value stuff.  

3) If I am unsuccessful, I will go back to the videos or text. In most cases, I will try to fill in the gaps needed to do the assessments, but if it turns out the quizzes form sparse coverage over the content, or the content is especially interesting, I'll listen to the video (usually without video -- unless really required, this happens on a mobile device as I go to work). 

Unless they're horrifically designed (inane "were you watching" questions, asking about the color of someone's shirt or similar), graded or not, I find the in-video quizzes to be incredibly valuable precisely to tell me what I don't know. 

My experience with looking at student usage patterns is that patterns like mine are not uncommon (not necessarily identical, but with relatively limited video usage). Look at figure 5 of: 

  http://mitros.org/p/papers/who_does_what_acm.pdf

I've done a couple of deep dives, which suggested that other courses may have large numbers of students who earn certificates but do not watch videos -- but that was based on a relatively informal look at the data (e.g. looking at patterns for small numbers of random students and similar -- so where I may be wrong), so I'd take that as evidence rather than proof. Stanford may have more evidence here. There was a paper from you folks which grouped students, I believe, into four bins (something like people who did all the work, people who dropped part way, people who just watched videos, and people who never started). This was much more rigorous than my looks at the data, and I was surprised that there wasn't a bin for people who did more assessments than videos. 

Yarko/Paul-Olivier: 

I generally agree with you. Back when this was an MIT project, the pedagogy I originally designed the platform around was constructive learning, as defined by Miki Chi (note: this is *very* different from the more popular constructivist learning pedagogy). The gist of the particular way this was embodied in the platform is that students are guided through figuring things out for themselves. This pedagogy was largely in-line with what both of you are saying. In particular, it means: 

1) Videos are not the principal means of knowledge delivery. 

2) Assessments are integrated into the learning process. By virtue of how this is done, they are not the sort of thing you'd move around. 

If you'd like to learn more about the pedagogy of the platform, have a look at week 2 of: 

   https://edge.edx.org/courses/edX_UPEI/MPHYS/2014_Summer/about

(note: This is a course I'm currently working on, so most of things outside of week 2 are very much not done -- and even week 2 is very much in flux). 

For a broader view of where we were at the time, you can also look at: 

   https://docs.google.com/a/mitx.mit.edu/document/d/1SRSmUY6Q-zPCEY07_9dGfzaqAJyETqKuqhtYEVwcziU/edit#heading=h.tq8ihsu8oc0l

This second one is from the early days of MITx and edX, and gives my beliefs about what we know about how to design MOOCs at the time (it's a bit more broad than the specific pedagogy I designed around myself). I wanted to include materials like these in our training materials. I was in the minority, and edX took a strongly pedagogy-agnostic stance. Those documents were shared mostly with just my direct collaborators. 

Since designing the platform, I learned a few things: 

* The number and variety of good and valid ways to use the platform that I haven't thought of is immense. If you look at sometime like MIT 7.00x, or Berkeley CS188x, both are spectacular courses. Both were filmed in a classroom with a camera (although with extensive adaption of instruction so it could be processed to fit the modality). Both have a course design I would not have expected to work beforehand. 

* Going in, I thought that a video camera at the back of a classroom+a few multiple choice questions was not the right way to teach. Right now, while I still see that this while not an optimal way to teach, I see that it is still much better than nothing. A few months back, I took a rather esoteric course on Coursera (Sports and Building Aerodynamics). The pedagogical design of the course was not on par with something like MIT's intro CS course, but intro CS has an appeal to millions and a large course staff developing it. I don't know the appeal of S&BA, but I would guess perhaps thousands, and the course staff was much smaller. Naturally, the resources invested in this course were proportionally lower. However, simply having access to that content -- which otherwise I would not have learned -- creates a lot of value. 

* The optimal pedagogy is /incredibly/ discipline and level specific. When designing the platform, I thought the basis -- mastery learning, constructive learning, immediate feedback, some literature on motivation, and targeting different levels instructional scaffolding -- was pretty sound. Each time I go to a conference, there is a keynote from some educational researcher stating that MOOC platform designers need to be educators/not technologist, need to look at basic foundations of teaching and learning, "don't let the tail wag the dog," etc. The problem is that each of those educators has a different set of beliefs about what the basic foundations of teaching and learning. There are dozens if not hundred core things like:

-- Cognitive tutors (feedback, guidance, targeting specific misconceptions, modeling of learner knowledge)

-- Deliberate practice and skill trainers

-- Learning as a social process (which itself is at least a dozen completely different things -- from peer instruction a la Eric Mazur to group projects to discussions)

-- Spaced repetition

-- Affect, attitudes, and mindset interventions

-- Etc. 

That's even omitting the earful which videographers will give you about video best practices. We cannot do all of these in a given course -- even , and many do not apply to specific disciplines. Which of these is most important is a question of discipline, of opinion, of the affordances of the particular platform, etc. No instructor can implement all of the best practices even relevant to their discipline. Each platform and each course can practically do a few of these. I've seen fantastic courses which were indeed a camera at the back of a classroom with horrific video content where, e.g. fantastic assessments more than made up for the weak video component, the reverse (great videos, weak assessments), as well as assessments. 

The result of all of this is that I'd really like the platform to support a range of pedagogies, including things like videos from the back of a lecture hall with multiple choice questions. I don't want this to be done carelessly -- we should be thoughtful, and we shouldn't reinvent wheels -- but at the end of the day, in most cases, if people are thoughtful and want to do something, even if something is not how *I'd* do something, I do want to understand it and enable it. In this particular case, it's been my experience that most instructors who ask for this do not understand what the platform does, how, and why, but if there is a legitimate demand, we should find a way to meet it. It's also the case that I created the structure of the learning sequence in a rush as effectively a one-man project back in 2011. This was an improvement to Udacity and Coursera (which, themselves, were super-rushed). I'd be shocked if we couldn't do better today (if for no other reason, that even I can name quite a few ways to improve on it given e.g. months instead of weeks development time). 

Regarding video vs. text and flipping through things, have a look at: 

  https://github.com/pmitros/AnimationXBlock

The goal here is to combine the low cognitive load of video ("This resistor here pulls up on this node, turning on this transistor, causing the output to fall" with pointing requires much less working memory than "In figure 5, resistor R3 pull up on node N1, which turns on Q3, which in turn causes the output V4 to fall") with the self-pacing of text. This is pretty specific to some types of content (video is also great as a motivator and for several related reasons), but it gives much of the flip-through-a-book feel, but without the same cognitive load. 

Piotr

[Paul-Olivier Dehaye]

Thanks Piotr, very interesting super long email :)

Paul

Paul-Olivier Dehaye

SNF Assistant Professor of Mathematics

University of Zurich

http://user.math.uzh.ch/dehaye/contact_info.txt

[Lyla Fischer]

Lyla:

Students can still use the scrubber to see any part of the video, including parts before the start time and after the end time, but the video will stop whenever it hits the designated end time.

Would you mind posting a bit more background here behind the decision? This is incredibly confusing behavior, and broke the content I was working with (and likely others were working with). When I saw this behavior, I just assumed it wasn't fully fixed yet. I just chopped up the videos by hand (the show had to go on, and I knew it was in the bug list). If this was requested by customers or we have evidence for the new behavior, it might be worth building in real start/stop times the way we had before. If this was not requested, it might be worth switching back to the designed behavior. 

Can you clarify the behavior that you expected? I don't understand what use case you are trying to enable that is not currently enabled by the video player. Is there a need or use case that an author would want to restrict student access to unfocused areas of a video? 

Just to make sure that the functionality that is currently enabled is clear: It is currently the case that when a video starts playing, it will start playing from "start time" but the student still has the freedom to move wherever in the video the student wants to consume material. When the video hits the timestamp associated with "end time" then the video will end, regardless of what other aspects of the video the student had watched before that. If video hits either "end time" or the end of the video, if the student presses the play button, the video will start from "start time". Otherwise, it will start playing from wherever the student stopped watching. At any point in time, the student can place the scrubber back onto the visually indicated part of the video that the author specified the student should focus on. 

[Piotr Mitros]

Hi Lyla, 

An extreme use case would be something like Walter Lewin's 8.01x. In this case, when Walter asks a question to the class, the video stops. The stop feature in edX *prevents* students from seeing the next chunk of video until the student gives an answer. Walter has designed a pedagogy where this is essential to how he would like to run the course. I'm actually not a big fan of Walter's pedagogy for how I like to teach, but I would like to enable it in the platform. I would also prefer if Walter's videos were not chunked in a video editing program for many of the reasons mentioned above (instructor time in video editing programs, caching of video content, etc.). The really big one I'll add is management of subtitles. You need to cut up videos. This has about 4-8 hours learning curve to get ffmpeg parameters correct, but is then fast from the commandline. It's super-slow in the GUI video editing programs I've used -- they're not designed for it (open, cut video, export. Hit undo multiple times or re-open, cut again, being careful to make sure start/stop times exactly align). You need to upload all of the chunked videos to Youtube, and cut-and-paste a bunch of gibberish from Youtube into XML or Studio, being careful not to mess up the order. In XML, this is sort-of-doable, but managing long learning sequences in Studio is just painful. Then, you need to cut up and realign subtitles for each video, and again, upload them to edX. This last step, I have not found a way to do well, and it is the #1 reason I don't re-edit videos. Breaking videos client-side doesn't eliminate the subtitle pain, but it dramatically reduces it for many simple course edits. 

Aside from Walter, in the more general case, going on to the video beyond the narrative region for the element is simply confusing to students. The videos are chunked for a reason. We do provide a means for watching the whole video if students really want to -- in most cases, the learning sequence has the remaining segments, and otherwise, clicking the Youtube link takes you to the full video. We also have download buttons which grab the full video. However, an affordance like a scrubber makes it easy for students to do this accidentally. This often leads to nonsensical results. For example, if I'm converting an existing lecture to an on-line format, I want the overall experience to be integrated. It's not just a lecture with scattered MCQ "finger exercises" to test student understanding. It's the student actively playing a role in the learning process. In many cases, I won't play the whole thing -- I'll take a chunk of lecture out, where for example I work through a derivation on the board, and replace it with, for example, an assessment where the student does the derivation. If I do this, aside from wanting the student to do the work, I might take a slightly different approach to the derivation. A concise, cleverly elegant explanation is great for a lecture, but hard to figure out. An assessment is more likely to use a more conventional derivation. Including both -- without any indication that they're different -- is just confusing to a student. 

If I'm creating dedicated content for the medium, it typically has a very strong narrative which integrates knowledge transfer through video, text, assessments, and interactives. For example, I'll use a video to motivate a topic, but then, use text and assessments to present the topic. At that point, I'll use a video for a conclusion to ties things together. Providing an easy affordance to see the conclusion, without any indication that there's a discontinuity of narrative there, is just confusing. 

Add to that all the reasons related to navigation you saw above (e.g. being able to scrub through 5 minutes of video -- so 2 pixels per second -- vs. a 30 minute video -- so a third of a pixel per second). Add to that, at least in the version I saw, very confusing UX design. The segment of video which is to play is marked, but there is no indication to the student what that marking means. The first time I saw it, I thought that was the part of the video which was buffered. When it got to the end, it stopped. I thought the video download got stuck or stalled out. I hit the pause/play button. It went on playing, just as a stalled-out video often does. I had no indication or idea that I was doing anything wrong or I went outside of the bounds of the designed experience. Even if you do have a flat lecture with MCQ finger exercises, watching an entire video in learning sequence element #1, and then seeing it replay chunks of it in elements #3, #5, #7, etc. is bad user experience. 

The core UX principal here is that you want to provide easy, immediate, discoverable affordances for actions you want users to take. For actions you don't want the users to take, you want to make the affordances less central to the UX, and provide a clear indication to the user that they may be making an error. The sequences are designed to be just that -- sequential. We don't enforce that, but we clearly convey it in the UX. As it is, the scrubber, it gives a random way to go to a random, unrelated point in the sequence without any indication that the user is doing something wrong. 

Again, if this was requested by professors, I would suggest having both hard and soft start and stop times. Jane and I would use the hard ones, and whatever use case the current behavior covers would be covered by the soft ones (could you indicate these?). If this was not requested, I would consider bringing this back to the designed behavior. 

Piotr

[Lyla Fischer]

Hi Piotr, 

Awesome! Thanks for the detailed use case. I'll add it to the product requests so that our next investment in the video player can take them into consideration. 

I especially appreciate pointing out the difficulty of being able to click on the correct location within a video, when that video is very long and the region for the desired location is very small. That is a student use case where cut videos do seem to be better than start/stop videos, and for which some people might consider investing in cut videos. I'll also record the lack of clarity in the affordances in the scrubber. 

-Lyla

[Yarko Tymciurak]

Piotr -

Thanks much.  This is a very rich discussion, and much appreciated.

On Wed, Aug 20, 2014 at 8:39 AM, Piotr Mitros <pi...@mitros.org> wrote:

Pardon super-long e-mail; feel free to skip to relevant bits. 

Jane:  

(Though I guess if start/stop times in a single video are used rather than chopping, then this advantage goes away, depending on how the scrubber acts - it'd be ideal if it represented only the part of the video between the start and stop times.)

... 

I'm on vacation w/ slow internet (and I'm not working! ;-) so I can't verify the issues right now, but I'm pretty sure it hasn't worked well for us at Stanford.  Do you want to see whether you think it's working for you at the moment (including the scrubber bar acting correctly)?

The feature was designed allow us to chop videos without needing to edit+chop. When it started showing things in the scrubber beyond the end was when I went back and re-chopped all of my videos. From Lyla's response, it appears the behavior is still buggy, at least by the original intent, design criteria, and ways it was used. 

Lyla:

Students can still use the scrubber to see any part of the video, including parts before the start time and after the end time, but the video will stop whenever it hits the designated end time.

Would you mind posting a bit more background here behind the decision? This is incredibly confusing behavior, and broke the content I was working with (and likely others were working with). When I saw this behavior, I just assumed it wasn't fully fixed yet. I just chopped up the videos by hand (the show had to go on, and I knew it was in the bug list). If this was requested by customers or we have evidence for the new behavior, it might be worth building in real start/stop times the way we had before. If this was not requested, it might be worth switching back to the designed behavior. 

Jane: 

Regarding formative vs. summative assessment, the point of formative assessment is to inform both the user and the instructor whether a given student knows something. This is precisely how I use them. When approaching most MOOC, I will typically: 

1) Try all of the assessments. This is a formative step. The goal is to discover what I know and what I do not know. The in-video quizzes are particular useful in this regard, as are any quizzes with infinite attempts. I find timed or limited-attempt quizzes kind of obnoxious, in part because they do not lend themselves to this. 

2) If I am successful, I will move on, generally without watching any videos or reading any text. I'll sometimes miss content (if the assessments sparsely cover the content), but that's okay. There's an effectively infinite amount of content out there, and I'll move on to higher-value stuff.  

3) If I am unsuccessful, I will go back to the videos or text. In most cases, I will try to fill in the gaps needed to do the assessments, but if it turns out the quizzes form sparse coverage over the content, or the content is especially interesting, I'll listen to the video (usually without video -- unless really required, this happens on a mobile device as I go to work). 

I don't have a lot of course authoring experience, so I may be stating the obvious, or missing something existing (apologies in advance).

Given Piotr's prefered method of approaching a MOOC, and assuming the association (linking) of items rather than actual overlay or layering as a pre-action for output, this would suggest having:

• multiple "swim lanes" which form the course
• for example the following might be parallel swim-lanes
• video
• audio
• transcription
• quizes/questions
• text
• slides
• simulations
• exercises / projects
• grouping within a swim-lane
• a tangent, or deep dive might be a swim-lane which exists in only a small segment of the pipeline, if you want to think of it as a pipeline
• linking between swim-lane groups/blocks (or time-range)
• for example, a single multiple choice concept-checking question would associate to the video-range (start+end points);
• some ranges could have no association with another swim-lane at one area (i.e. discontinuous is ok)

Now, the student can have exposed for their navigation some (or all) swim-lanes.

In my case, for example,  I might prefer to "follow" flipping down the "slides" swimlane, and turn on automatically "exposing" (playing) the audio (not video) and quiz swim-lane with it.   I could transfer between the swim-lanes I "occupy" at certain transfer points.   For example, while taking quiz-1 (in quiz swim-lane), I might choose to stay there for a while, see how far ahead I can get, and at some point, switch back to slides (or perhaps video, if I desire).

Piotr would (given his stated preferences)  start in the quiz strem, and cross-over to other linked materials at different times.

At some level, this is probably not so different for authoring experience, with the addition of the concept of "swim-lanes", and perhaps some controls on which can be handed to student control (for example, - can't skip the final quiz on a section).

Additionally, an instructor might find it useful to view their material from a swim-lane layout, and add linkings (or sections in other lanes - such as a discussion lane in a particular context where discussion was desirable).

Of course, this still faces the problem of time-to-develop, but the structure and degrees of freedom it suggests - both for instructor/author, and student/reader - I imagine might be different.

This facilitates (I think) implementation of the scenario which Piotr layed of Walter Lewin's 8.01x, in his response just not to Lyla.

It also facilitates describing new types of swim-lanes.

Do xblocks implement this?  I think this is more typed, structured, at a different level.

Thanks for all the various levels of discussion - this is fascinating for me.

I will add, as I privately layed out at some length to Lyla at this year's PyCon - I envision the general MOOC/edx "course" paradigm expanding into a new hyper-rich ebook, a sort of textbook which never yellows, and carries with it a community connection.   This, then, also adds the importance of text (searchable), and relative de-emphasis on video for such an open-ended medium.   I look forward to this direction, despite all the current challenges that would present.

Regards,

Yarko

[Yarko Tymciurak]

... and sorry for the apparent forked levels of discussion here - there are really two separate threads.

[Piotr Mitros]

Yarko,

Now we're getting into a bit more nuanced and interesting questions. There are a few key questions before building something like this out, the #1 question is why you would want to have multiple lanes or modes of presentation. Two reasons you might *not* want them: 

1) Learning styles have been fairly thoroughly debunked. It's not the case that e.g. some people fundamentally learn better in a visual than auditory way, at least to the extent we can measure. 

http://steinhardtapps.es.its.nyu.edu/create/courses/2174/reading/Pashler_et_al_PSPI_9_3.pdf

2) Likewise, although there is substantial debate about this, the benefits of students self-navigating are, at the very least, overstated by constructivists: 

http://www.cogtech.usc.edu/publications/kirschner_Sweller_Clark.pdf

6.002x and MITx relied very substantially on student self-navigation, but that's because I didn't have time to build a full intelligent authoring system, and the course authors didn't have time to populate one. We built a lot of self-help resources, in the form of Askbot, tutorials, etc. My hypothesis -- although I have no evidence either way yet -- is that these kinds of self-regulated remediation resources can be almost as effective as a full ITS, and in some disciplines, more effective. 

A few reasons you might: 

- Different modes of consumption. I listen to MOOCs on the way to work (if you run Open edX, you can too: http://podcasts.edx.org/ecfs312x/ and https://github.com/pmitros/edxml-tools/blob/master/make_course_rss.py). This is a pure audio experience. If I'm at a computer, I don't want video at all (very few people do, as far as I can tell). I want the kind of constructive learning I showed above. Unless I already listened to the video, in which case, I'd like to just exercise what I learned. Likewise, I believe adult learners have _very_ different consumption patterns than residential students. 

- Different backgrounds. If we're learning physics, and I know calculus and you don't, but you have better study skills, we might want different lanes. 

- Accessibility. 

- i18n. 

Most of these are not well-suited to different swim lanes based on content structure and type (audio vs. video vs. problems), but perhaps something a little more semantic (such as learning objective tags), or perhaps something completely manual (we had a tabbed XModule in MITx, which let students select e.g. text vs. video presentation, but we never actually had time to build multiple modalities for content -- MOOCs are an insane amount of work with one modality -- so we never used it). 

Yarko: Regarding your vision, are you interested in building things in this direction, or is this abstract? My original vision for MITx, and Arjun's original vision at Berkeley's were to create a community platform similar to github for educational content. edX has a long ways to go, but there may be a few architectural things we can do early on which will smooth the path. Ross Strader and Stanford OLI will probably add learning objectives, which is one prerequisite, but there are a half-dozen others we could discuss, if you were interested in hacking. 

Piotr

[Yarko Tymciurak]

On Wed, Aug 20, 2014 at 3:38 PM, Piotr Mitros <pi...@mitros.org> wrote:

Yarko,

Now we're getting into a bit more nuanced and interesting questions. There are a few key questions before building something like this out, the #1 question is why you would want to have multiple lanes or modes of presentation. Two reasons you might *not* want them: 

Note:  I was speaking at one level about a structure, and the ability to either statically or dynamically.  Based on, lets say, performance you might have different in-depth or quick-summary channels setup for you - optionally or not).   My point was that the controls around the flow and linking between swim-lanes, and the swim-lanes and their view and setup can function most flexibly if decoupled.

1) Learning styles have been fairly thoroughly debunked. It's not the case that e.g. some people fundamentally learn better in a visual than auditory way, at least to the extent we can measure. 

http://steinhardtapps.es.its.nyu.edu/create/courses/2174/reading/Pashler_et_al_PSPI_9_3.pdf

2) Likewise, although there is substantial debate about this, the benefits of students self-navigating are, at the very least, overstated by constructivists: 

http://www.cogtech.usc.edu/publications/kirschner_Sweller_Clark.pdf

6.002x and MITx relied very substantially on student self-navigation, but that's because I didn't have time to build a full intelligent authoring system, and the course authors didn't have time to populate one. We built a lot of self-help resources, in the form of Askbot, tutorials, etc. My hypothesis -- although I have no evidence either way yet -- is that these kinds of self-regulated remediation resources can be almost as effective as a full ITS, and in some disciplines, more effective. 

A few reasons you might: 

- Different modes of consumption. I listen to MOOCs on the way to work (if you run Open edX, you can too: http://podcasts.edx.org/ecfs312x/ and https://github.com/pmitros/edxml-tools/blob/master/make_course_rss.py). This is a pure audio experience. If I'm at a computer, I don't want video at all (very few people do, as far as I can tell). I want the kind of constructive learning I showed above. Unless I already listened to the video, in which case, I'd like to just exercise what I learned. Likewise, I believe adult learners have _very_ different consumption patterns than residential students. 

Thanks for the links - I like this.

 

- Different backgrounds. If we're learning physics, and I know calculus and you don't, but you have better study skills, we might want different lanes. 

- Accessibility. 

- i18n. 

Most of these are not well-suited to different swim lanes based on content structure and type (audio vs. video vs. problems), but perhaps something a little more semantic (such as learning objective tags), or perhaps something completely manual (we had a tabbed XModule in MITx, which let students select e.g. text vs. video presentation, but we never actually had time to build multiple modalities for content -- MOOCs are an insane amount of work with one modality -- so we never used it). 

Yarko: Regarding your vision, are you interested in building things in this direction, or is this abstract?

In a real, pragmatic way I am immediately interested - not just from the learners vector, but from the author's vector also.

For example, I'm about to launch some non-university day-long workshops which I want to be backed by MOOC access, which will expand on the quicker workshop overview.

To not get all bogged down in the tech (i.e. to get the workshops going, but not let them go without offline, after expansion)  I'm starting with sphinx-doc based presentation (via hieroglyph plugin) to a runestone instance (which uses in-browser python and codelense - something I have no interest in, and will gut;  but which has simple multiple-choice or word answer plugins to sphinx).  

Some time early next year, I would start looking at and planning an integration or migration or midpoint strategy. I have some experimental playing I plan to do with ora.

For after workshop user experience, I will start with taking simple live video (no fancy processing) and extracted audio, and associate it with a slide (ergo, the "like reading a book" analogy), and include in-context discussions.  I think a one-man shop should be able to handle this.

I expect I'll be playing with this for a some years to come.

But there are a couple of points about this:

• the workshops will build on each other (2 years worth of topics at this point);
  
• discussion and interaction will guide revisions, other workshops, perhaps book-writing, ergo sphinx-doc type of form is attractive in that, and hieroglyph as inline link to fuller documentation, an outlining mechanism;
  
• the concept of leaving it as an (over time) increasingly rich e-book with community attached to topics is a natural consequence of this thinking. Unlike a course, it will have no particular stop time - it will be a reference, and set of active (running) tools.
  
• it occurs to me that this might be of interest (ongoing "book" style) for commercial use also (customer product training; employee training, etc.);

As you talked about driving and listening to audio, I am interested in mobile (i.e. public transit, commuter train in tunnel) - partitioning material into swim-lanes should also provide some path for managed, reasonable disconnected mobile clients.

My original vision for MITx, and Arjun's original vision at Berkeley's were to create a community platform similar to github for educational content. edX has a long ways to go, but there may be a few architectural things we can do early on which will smooth the path. Ross Strader and Stanford OLI will probably add learning objectives, which is one prerequisite, but there are a half-dozen others we could discuss, if you were interested in hacking.

I might be interesting in hacking (I will be doing my own).

I would be interested if someone had a paying position at this kind of level, where I would happily run out my remaining years before retirement (4+).

Thanks for all you've shared - very interesting.

Yarko.

[Yarko Tymciurak]

On Wed, Aug 20, 2014 at 4:22 PM, Yarko Tymciurak <yar...@gmail.com> wrote:

For example, I'm about to launch some non-university day-long workshops which I want to be backed by MOOC access, which will expand on the quicker workshop overview.

Sorry - this wasn't very clear:  I mean that workshop topics will have broader presentation, a way to dig deeper online.

 

[Piotr Mitros]

On Wednesday, August 20, 2014 5:23:02 PM UTC-4, Yarko Tymciurak wrote:

For after workshop user experience, I will start with taking simple live video (no fancy processing) and extracted audio, and associate it with a slide (ergo, the "like reading a book" analogy), and include in-context discussions.  I think a one-man shop should be able to handle this.

That sounds like a neat project. 

If you were to use edX for this, I would take the slides as a PDF, and convert them into a learning sequence. Here's a script I wrote to do this: 

  https://github.com/edx/pres2edxml

I would most likely use a Bluetooth headset to capture the audio to a cell phone. I probably wouldn't use it, but I might also capture video from the cell phone held up with a Gorillapod GripTight stand. I would, in v0, manually segment the audio, and manually add it to the learning sequence below the slides. Here's an audio player XBlock: 

  https://github.com/pmitros/AudioXBlock

The Audio XBlock was written back when XBlocks were super-prototype-stage, but I think it ought to still work (if not, it'll work with minimal changes). We have a few in-line discussion components. There's one in edX, and if you prefer other tools, you can embed other things (e.g. https://github.com/pmitros/DisqusXBlock, or Piazza through LTI). For what you're doing, the RecommenderXBlock might also be useful (https://github.com/pmitros/RecommenderXBlock)

The big chunk of the time would be spent on the chopping up of audio and aligning. What's really missing is a tool to do that automatically. I'm not sure what the best way to do this is. One possibility would be to build or extend a tool which controls slides from Android (something like https://play.google.com/store/apps/details?id=de.vrallev&hl=en) to also capture timing information. Another possibility would be to make a plug-in for OpenOffice to monitor the presentation, and again, capture timing. 

Regarding previous discussion, I was more asking about open source contributions than within edX, but edX is always hiring. I'm not sure whether the current openings would be a good fit for where your interests lie, but do shoot me a personal e-mail with resume+interests, and we'll follow up. 

Piotr

[Yarko Tymciurak]

Thanks, Piotr, for all the discussion, input, links, suggestions...

On Thu, Aug 21, 2014 at 5:19 AM, Piotr Mitros <pi...@mitros.org> wrote:

On Wednesday, August 20, 2014 5:23:02 PM UTC-4, Yarko Tymciurak wrote:

For after workshop user experience, I will start with taking simple live video (no fancy processing) and extracted audio, and associate it with a slide (ergo, the "like reading a book" analogy), and include in-context discussions.  I think a one-man shop should be able to handle this.

That sounds like a neat project. 

A little scary in the mid-term, but definitely interesting.

 

If you were to use edX for this, I would take the slides as a PDF, and convert them into a learning sequence. Here's a script I wrote to do this: 

  https://github.com/edx/pres2edxml

I knew of this, but only just looked at the code - clearly you get around a lot of libraries!

I would use edX for this, but for the resource / hosting costs of running my own instance (Lyla suggested I host others, handle whatever rough edges there might be w/ micro-sites - but I have only so many bullets in this six-shooter, so I have to be careful which I spend first:  workshop topic material first, definitely want in-slide concept checker during workshops, which will be ? 100 people is what we've decided for session cap at the moment - lets see if _anyone_ shows first!).   I'd like to have a server on-site with me, and see work out mobile connections and presenter feedback, so that's step one.

ORA and others I definitely have plans on what I want to try with them, how I want to play - so an eventual edx (as in nearer term rather than longer - within the next 1 to 1.5 yr?) is in the game, but I worry about increase resource demands of the instances.   I have some side-projects far removed from edx to invite some people to play on the topic of trying to pare this down (the lightest weight of these - just get to async python).

I would most likely use a Bluetooth headset to capture the audio to a cell phone. I probably wouldn't use it, but I might also capture video from the cell phone held up with a Gorillapod GripTight stand. I would, in v0, manually segment the audio, and manually add it to the learning sequence below the slides. Here's an audio player XBlock: 

  https://github.com/pmitros/AudioXBlock

The Audio XBlock was written back when XBlocks were super-prototype-stage, but I think it ought to still work (if not, it'll work with minimal changes). We have a few in-line discussion components. There's one in edX, and if you prefer other tools, you can embed other things (e.g. https://github.com/pmitros/DisqusXBlock, or Piazza through LTI). For what you're doing, the RecommenderXBlock might also be useful (https://github.com/pmitros/RecommenderXBlock)

Thanks - as a result of this thread, I did browse your repositories (which is why I asked about the "more than open source contribution" aspect).

I have a marketing person who is driving this - we have "borrowed" (friend) hand-held HD cameras (2 for staters) which we'll use, and regular lapel mics and PA system (and I think a digital recorder - certainly a tape one).   I immediately spotted the AudioXblock.   Disqus is a given for the first few sessions anyway (as we see how this is received, how many attend, etc.).   I will have to study the Recommender, only know from brief descriptions what it's role is.

The big chunk of the time would be spent on the chopping up of audio and aligning. What's really missing is a tool to do that automatically. I'm not sure what the best way to do this is. One possibility would be to build or extend a tool which controls slides from Android (something like https://play.google.com/store/apps/details?id=de.vrallev&hl=en) to also capture timing information. Another possibility would be to make a plug-in for OpenOffice to monitor the presentation, and again, capture timing.

Obviously, I am warned over and over to do _nothing_ from the tech end initially, just the workshop materials, but I have a strong vision and so in _some_ form it must be part of it, only as minimally as my initial resources and time allow.

Since I want _some_ on-site server, I'm thinking of recording to disk video and slides, then timing would be something possible at some point.   I've not seen remote presenter, but do have "Ultimate Mouse" wifi pointer for android.   I think, with an HTML5 presentation,  timing would be easiest to bind transition information from the browser instance into a listening server utility (I'm separately playing w/ mobile-first dart apps for deployable, sporadically disconnected apps - I can see doing this on-page as a utility;).  In any case, I appreciate the heads-up to keep my eye on collecting timing information.

From the mobil-first end, I keep thinking about adult learner, mobile access - I bet we could have some interesting discussions around that, architecture to enable, etc.

 

Regarding previous discussion, I was more asking about open source contributions than within edX, but edX is always hiring. I'm not sure whether the current openings would be a good fit for where your interests lie, but do shoot me a personal e-mail with resume+interests, and we'll follow up. 

I was responding to open source also - having done only a little edx consulting for commercial concerns, and constantly having not as much time and attention as I would like to give, this is a natural question.

The workshops are for an enjoyable retirement (hope to keep them up for a long time).

Will send off a personal note later.

Thanks so much - I've tagged this thread as my most interesting thread on edx to date. :-)

Regards,

Yarko

Piotr

[Jane Manning]

Awesome! Thanks for the detailed use case. I'll add it to the product requests so that our next investment in the video player can take them into consideration. 

I especially appreciate pointing out the difficulty of being able to click on the correct location within a video, when that video is very long and the region for the desired location is very small. That is a student use case where cut videos do seem to be better than start/stop videos, and for which some people might consider investing in cut videos. I'll also record the lack of clarity in the affordances in the scrubber. 

Thanks Lyla - do you have an expected timeframe in mind when these changes might be able to get into the player?  Based on this discussion, it sounds like:

(1) Everyone agrees that videos shouldn't necessarily be "primary", and many students don't even watch them, but the fact is that many courses use videos, and many instructors want to insert "quizzes" or other interactive exercises within videos.

(2) As Piotr described, there are some advantages to using the start/stop time feature rather than a javascript overlay on the video, especially in terms of students being able to access the exercises without watching the videos if they want to, and in being able to give meaningful names to different parts of the video.  

(3) As Piotr described and we've found at Stanford as well, working around this by cutting the video into pieces is labor-intensive and brittle (esp. wrt caption files).  But given the way start/stop is currently implemented wrt the scrubber, it may be the current best way to do this.

(4) It sounds like Harvard and Stanford aren't the only places getting requests to be able to more easily insert quizzes into videos.  With Piotr's arguments in hand, people can probably be convinced to use start/stop time rather than a js overlay, if that functionality works well wrt the scrubber etc.

I'm asking about the timeframe in part because "so that our next investment in the video player can take them into consideration" reads to me like "don't hold your breath", and I'm wondering whether given the broad interest and clear value here, if edx.org isn't able to prioritize this functionality, it might be a nice project to spec a bit and encourage someone in the open source community to make this change to the video player?  I don't know much about the video player's implementation, but I'm imagining that this change might not even be too big a project, and could be attractive for someone in the open source community to do if it had the support of edx.org. (If so, maybe Piotr could even be convinced to write a "story" about the necessary change, since it sounds like he's given a lot of thought to what the correct behavior is here.)

Jane  

[Yarko Tymciurak]

I agree - summarizing this discussion into a short, clear story, and have everyone review that for correctness in capturing concerns is probably a good next step.

 

Jane  

[Lyla Fischer]

Hi Jane - 

You're right. I wasn't particularly anticipating edx.org getting around to this any time soon, but I like the concept of using this particular feature as a test case for how to propose and push through changes in "the open source way". 

I've got two feature requests written at the moment (one for the scrubber size and granularity, and one for make-the-scrubber-not-look-like-a-buffer), but our feature request board isn't public at the moment. (A lot of the tickets need to be organized and cleaned before they can really be useful or appropriate to share). However, those two should be fine to share publicly, especially considering that they were generated publicly as well. 

I'll bring it up in the Product organization to see how we want to organize this, so that we can use this particular use case as a forerunner for even better community organization. It will probably involve having stakeholders like Piotr, Yarko, Paul-Oliver, and yourself affirming that your interests have either been captured or "I don't care any more". I'm very excited about this. Thanks, Piotr, for providing such a great example to nurture community development with!

-Lyla 

[Piotr Mitros]

Stories: 

https://docs.google.com/a/edx.org/document/d/1uQideyZHppDuEiwJKo5b4SYziSvwV0MgU9QLRQnJc5w/edit#

Thoughts? Takers? 

I added a second story to fix the pedagogical design of the scrolling subtitles, mostly for discussion, but it's something which I'd value seeing fixed. 

Piotr

[Piotr Mitros]

To everyone who requested access in the first 5 minutes: link should be public now. 

Piotr

[Jane Manning]

Piotr - I like this a lot - thanks for writing up!  I proposed in the doc a new paragraph of "Background" to motivate the change.

I'm hoping we can find someone interesting in building this.  I'd also be interested in Lyla's view on the "Conservative" vs the "Liberal" implementation.  My guess is that it might be easier to get the "Conservative" implementation merged, since then there won't be any arguments about users who expect different behavior.

(My preference, btw, would be for the 2nd story to be in a separate document for ease of reference to each story individually, but that's neither here nor there.)

Jane

[Piotr Mitros]

I added the paragraph you proposed, and gave you write access to the document. Regarding alternatives, I'm okay either way. 

I looked into usage statistics. Excluding edX demo/test/debug courses, start/stop times are used in five courses on edx.org, with 37, 12, 5, 2, and one use per course for each of those courses. That's a small enough set of users that we could reach out to them, and find out how important the old-style functionality is if someone were serious about building this and preferred liberal. 

Piotr

[Paco C]

Hi

I put this post on the forum last July, since we have already implemented a XBlock to use it in open edX

https://groups.google.com/forum/#!topic/edx-code/dOdtELtmmk8

Yo can get in from here

https://github.com/dsantin/PaellaInteractiveVideo

https://github.com/dsantin/xblockInteractiveVideo



El viernes, 15 de agosto de 2014 20:07:01 UTC+2, Daniel McKelvey escribió:

I am looking for the ability to put a quiz inside a video.  Specifically, as the learning is watching the video, a quiz can appear anytime inside of the video.  So it's a powerful way to make the student truly comprehend the material.  If they don't answer the question correctly, the video many or many not continue until the right answer is selected.  Any suggestions here are greatly appreciated.

[Holly]

Dear Piotr,

I am sorry, I am new to this and am trying to read quickly: but the in-video quiz is not availble yet in edX, is that right?

- Holly

On Tuesday, August 19, 2014 at 2:04:07 AM UTC+2, Piotr Mitros wrote:

It's been a few years, so I'm not sure if I recall all of the details, but I'll give what I do recall off-hand. I'm not sure if these are so much counter to providing a different way of doing things, as they may serve as guideposts to how to design this well. The experience was a little painful in Know Labs 1.0 and in Coursera 1.0 (which copied Know Labs). MITx improved on it, and Udacity, aside from dropping the name "Know Labs," copied our UX. At the time, both MITx/early edX and Udacity were well ahead of Coursera in usability, in part, due to this. The UX in Coursera has improved in a number of ways, but this is still a place I find our platforms better. 

The core reasons are: 

* It's often handy to read or do assessments before watching videos (and not watch videos if you know everything), and in some cases, the reverse (e.g. I'll listen to videos when driving, and then do problems). Clicking on the little Coursera notch at the bottom is painful. 

* Long videos are pretty painful to navigate. The most common operation in a MOOC is to jump back by a few seconds or at most a few minutes (and in some cases, forward). With a 30 minute long video, this involves fine motor motion. With six 5 minute videos, this is pretty easy. 

* Conversely, chunks of the videos can have names (visible on mouse-over for the learning sequence), which is handy for finding the right one (if well-labeled and well-chunked, much more so than scrubbing through lecture content). 

* Dual navigation schemes are error-prone. If I had a quarter for every time I clicked "next" in Cousera after a quiz, instead of "continue", lost my spot, and had to go back, scrub around, find the next video marked as watched, etc. ....

* Modularity. There are few things as painful as having to rewatch the last 30 seconds of the video over-and-over to catch some part of a question you missed. In-video quizzes are limited in space, and generally not XBlocks, so limited in what you can put in. The edX design style lends itself to longer, more self-contained questions, which tend to be much more useful to students, both from a UX and a learning standpoint. 

* Likewise, the learning sequences lend themselves to richer content. As courses improve, increasingly I see richer things than simple multiple-choice questions in learning sequences, which really integrate transmission of knowledge with construction of knowledge. The in-video quizzes make a transition to this hard. 

* The last point is especially important -- when the questions mirror the videos too closely (e.g. Know Labs, where it literally overlapped the video), it is impossible to fix issues in questions without modifying the video, and vica-versa. It's good when there is some level of integration, but too much is too much. Likewise, individual video clips are easy to swap out. Even if the style changes, the transition is not too jarring. Continuous videos with questions lend themselves to less iterative improvement. 

My experience has been that most, although certainly not all, instructors who ask for this have not internalized the way this works in edX. Once we show the edX way, most tend to be happy with it. 

With regards to start/stop times, it worked well perhaps 2 years back. At some point, there were some issues with the scrubber which let you get beyond the beginning/end of the video. Then there were some issues where the scrubber showed the full length of the video, but started/stopped at marked points, which was super-confusing. I stopped following at that point. I recall this was one of the top priorities for one of our PMs, so it's possible she got it fixed. It'd be *very* worthwhile to fix if it is still broken (more so than making a new video player). Aside from instructor pain in chopping up videos, it saves students on slow connections a lot of waiting -- with the start/stop times, the browser could pre-cache video from the next segments. 

Other open issues are: 

* The somewhat painful interface in Studio for authoring learning sequences with many segments. 

* The somewhat slow JavaScript for going through learning sequences

* The inability to right-click/open-in-a-new-tab elements of a learning sequences. 

Piotr

On Sunday, August 17, 2014 7:39:57 PM UTC-4, Jane Manning wrote:

Piotr: At Stanford, we've found the start/stop time feature to not be very robust for students (and last time I checked it had some UI issues wrt scrubber position - but I'm on vacation w v.slow internet so can't confirm that right now), so we advise instructors to chop the videos into pieces of the appropriate length, with problems in between.

But this is fairly brittle for the instructor - you can't easily decide to eg move the quiz to another spot in the video, or add an extra quiz (or remove one).

I'd be curious to hear about the UX issues you mentioned with the "in-video" style - can you say more about that?  The version of Colin's that I saw lacked indication on the scroll bar of where in the video the quizzes are, which is a UX issue, though seems like a motivation to build the functionality into the player, rather than giving up on the functionality altogether.  

Like Colin at Harvard (and others on this thread)  we get many requests for this feature.  I don't think instructors always know what's best for them, but in this case I can see why they're interested in this, so if there are some reasons why this feature isn't a good idea, I'd be interested to hear.

Jane

Sent from a phone.

On Aug 17, 2014, at 3:55 PM, Piotr Mitros <pi...@mitros.org> wrote:

The way the platform was designed, you do this by having multiple video segments. So instead of a 30 minute video with 2 pop-up quizzes, you have it structured as 10 minute video/problem/10 minute video/problem/10 minute video. I designed the learning sequence in response to a number of UX issues with the early Coursera and Udacity/Know Labs platforms. 

To make this easy, there was even an option to have video start/stop times, so you take a 30 minute video, and include it 3 times, with start/stop at 0-10 minutes, then 10-20 minutes, and finally, 20-30 minutes. This was broken for a while; I have not tried this (I tend to shoot independent videos), so I'm not sure if it is working; if not, it'd be worth fixing. 

Piotr

[Ned Batchelder]

Holly, that is true: we don't have in-video quizzes out of the box.  Maybe someone has built something like it?

--Ned.

--
You received this message because you are subscribed to the Google Groups "General Open edX discussion" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/ef0d5127-08a7-42ea-ab2e-fcc54bf06b8b%40googlegroups.com.

[John Lee]

I noticed there was a bit of discussion about the various design considerations of in-video quizzes... the AlgebraX and GeometryX courses currently on edx.org use these pretty much exclusively, and I'd be happy to share a little bit about what we've learned from running these two courses.

On Tuesday, August 19, 2014 at 2:04:07 AM UTC+2, Piotr Mitros wrote:

* It's often handy to read or do assessments before watching videos (and not watch videos if you know everything), and in some cases, the reverse (e.g. I'll listen to videos when driving, and then do problems). Clicking on the little Coursera notch at the bottom is painful. 

We found this to be really handy too. We need to make sure students are ready to see the next lesson, although we actually use the in-video questions to actually do this assessment. If their responses show that they don't understand a prerequisite topic, they're directed to a previous lesson instead of continuing with the current one.

 

* Long videos are pretty painful to navigate. The most common operation in a MOOC is to jump back by a few seconds or at most a few minutes (and in some cases, forward). With a 30 minute long video, this involves fine motor motion. With six 5 minute videos, this is pretty easy.  

* Conversely, chunks of the videos can have names (visible on mouse-over for the learning sequence), which is handy for finding the right one (if well-labeled and well-chunked, much more so than scrubbing through lecture content). 
* Dual navigation schemes are error-prone. If I had a quarter for every time I clicked "next" in Cousera after a quiz, instead of "continue", lost my spot, and had to go back, scrub around, find the next video marked as watched, etc. ....
* Modularity. There are few things as painful as having to rewatch the last 30 seconds of the video over-and-over to catch some part of a question you missed. In-video quizzes are limited in space, and generally not XBlocks, so limited in what you can put in. The edX design style lends itself to longer, more self-contained questions, which tend to be much more useful to students, both from a UX and a learning standpoint. 

We tried taking this idea to the extreme, and found it actually worked well for us -- rather than even six 5-minute videos, we found that we could do some interesting things with twenty 30-second videos. We haven't seen any complaints regarding users wanting to rewind any further than the 30-second-ish segment. We did make it possible for people to go to previous videos in the same "lesson" and re-watch them, but our logs indicate that the vast majority of people don't navigate backward to previous videos when given the ability to just play one of the really-short videos from the start, probably because they had to answer a question to show that they understood everything so far. We treat each of these questions as a decision point, and it affects which video(s) to show next.

 

* Likewise, the learning sequences lend themselves to richer content. As courses improve, increasingly I see richer things than simple multiple-choice questions in learning sequences, which really integrate transmission of knowledge with construction of knowledge. The in-video quizzes make a transition to this hard. 

This is the part where we found this approach most useful -- many students were surprised (the first time, at least) when they typed in a wrong answer, and were then directed to a short video explaining why that specific answer was wrong, and the entire time it looks like they're just watching one long video. As course developers this was especially important to us, since adding a 30-second video clip to a set of videos, and then hooking them up programmatically, is way cheaper for us to produce than modifying a 5-minute video clip and making the software jump around it.

 

* The last point is especially important -- when the questions mirror the videos too closely (e.g. Know Labs, where it literally overlapped the video), it is impossible to fix issues in questions without modifying the video, and vica-versa. It's good when there is some level of integration, but too much is too much. Likewise, individual video clips are easy to swap out. Even if the style changes, the transition is not too jarring. Continuous videos with questions lend themselves to less iterative improvement. 

At the tiny (~30 sec clips) scale we were doing this at, we found this not to be too much of a problem. In a lot of cases, we're not just swapping out videos or adding them, but we're also modifying the decision logic around them in such a way that maintains the seamless-video look, but adds branches where needed.

By and large, most of the learners seem to like this way of going through the content, but there are a couple tricky bits:

- Since there are lots of possible paths through the lesson depending on how the user answered the question, there's no great way to produce a single downloadable video or a PDF of slides that works for everyone. Perhaps a personalized download (which only shows the path that they took for this session) could do it, but for now we have to require our users to be online to access the content.

- In a similar vein, lots of users like to see the length of a video before deciding whether to jump into it, but with our approach there can be huge variance in the amount of time it takes to work through it (especially if they're answering questions along the way).

The courses are still running so we are still learning new things all the time, but I'm happy to discuss the approach in more detail if there's any interest.

To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/CAGtJPNOCi0t7QN2zf3zTxfxB-%2Bvr27%3Dcm4g0JpkVqAEpUuERJg%40mail.gmail.com.

[Holly]

Thank-you, Ned, and thank-you John!

[Piotr Mitros]

On Wednesday, March 18, 2015 at 4:19:00 PM UTC-4, John Lee wrote:

We tried taking this idea to the extreme, and found it actually worked well for us -- rather than even six 5-minute videos, we found that we could do some interesting things with twenty 30-second videos.

Thank you! This is excellent. I now have a new examplar for good sequence design. I've been trying to push a model based on very short elements with students continuously interacting since creating the platform. The examplar I used before was the Pythagoras sequence in a little course I created for edX-internal use. 

We haven't seen any complaints regarding users wanting to rewind any further than the 30-second-ish segment.

We've found that the threshold for users to actually complain about something is surprisingly high. Even with tens of thousands of users, we'd found a number of basic usability issues that users simply did not bring up; they tend to come up in user studies, or working with learners. With SchoolYourself, the overall experience is great, but I actually find the UX interface a little annoying, although not for reasons of quizzes overlaid on videos:

1. I can't skip ahead if I understand something. Especially as a self-paced learner in a MOOC, often coming in with some prerequisite knowledge, I'd like to be able to do just the quizzes in many cases. I'm often either just reviewing material I know, or learning gaps I don't.
2. The progress bar looks like a scrubber, but clicking on it or sliding it doesn't work. In contrast, the scrubber looks like the volume control in many video players (e.g. compare to Amazon Instant Video on iPad).
3. Embedded in edX, the close button is almost invisible on a large screen. It's associated with the full screen, rather than with the lightbox modal. It took me a minute or two to figure out how to close it on my 4k display.
4. I can't plan my time. There's no good way to see what/how much stuff there is to do, or the structure of the lesson, even coarsely.
5. Related to this, from a motivation standpoint, it's nice to be able to see progress.

If the sequences were fixed content, I'd much prefer a well-designed learning sequence style interface (although with autoadvance -- something we desperately need -- not with clicking a poorly designed "next" button as we currently have). Given adaptive content, as you mentioned, the problem of clearly showing and allowing navigation through a sequence is a good bit harder. 

SY does resolve some of the more common frustrations with in-video quizzes through good pedagogical design -- in Coursera and Udacity, most of the in-video quizzes require the content from the video, which is extremely frustrating for a number of reasons. With SY, in all instances I've seen, they're standalone -- you can figure out the question being asked from just what's on screen at the time the question comes up. I am wondering whether this would still be possible in other MOOCs, or specific to the level of content in SY. As a compare-and-contrast:

1. SY has simple questions (find a number on a number line, short equation, etc.). College-level courses typically have questions which are either much more complex (e.g. engineering design problems), or more contextualized (e.g. answer a question about a complex legal scenario). For quizzes limited to the size of the video, there strong temptation to put requisite context in the video itself.
   
2. In SY, the concepts themselves are short enough that a sequence on a given topic is typically just a few minutes. Most MOOCs are modular, but not quite this modular -- for complex topics, the minimum useful time a student might spend in a stand-alone sequence might be 15 minutes or sometimes even an hour. In SY, I would guess this is typically around 5-10 minutes (e.g. 10 elements at 30 seconds each). 

I'm curious -- how do you manage accessibility? 

Piotr

[John Lee]

Hi Piotr,

Sorry, this got lost in my inbox! Responses inline.

On Fri, Mar 20, 2015 at 4:23 PM Piotr Mitros <pi...@mitros.org> wrote:

On Wednesday, March 18, 2015 at 4:19:00 PM UTC-4, John Lee wrote:

We tried taking this idea to the extreme, and found it actually worked well for us -- rather than even six 5-minute videos, we found that we could do some interesting things with twenty 30-second videos.

Thank you! This is excellent. I now have a new examplar for good sequence design. I've been trying to push a model based on very short elements with students continuously interacting since creating the platform. The examplar I used before was the Pythagoras sequence in a little course I created for edX-internal use. 

We haven't seen any complaints regarding users wanting to rewind any further than the 30-second-ish segment.

We've found that the threshold for users to actually complain about something is surprisingly high. Even with tens of thousands of users, we'd found a number of basic usability issues that users simply did not bring up; they tend to come up in user studies, or working with learners. With SchoolYourself, the overall experience is great, but I actually find the UX interface a little annoying, although not for reasons of quizzes overlaid on videos:

1. I can't skip ahead if I understand something. Especially as a self-paced learner in a MOOC, often coming in with some prerequisite knowledge, I'd like to be able to do just the quizzes in many cases. I'm often either just reviewing material I know, or learning gaps I don't.
2. The progress bar looks like a scrubber, but clicking on it or sliding it doesn't work. In contrast, the scrubber looks like the volume control in many video players (e.g. compare to Amazon Instant Video on iPad).
3. Embedded in edX, the close button is almost invisible on a large screen. It's associated with the full screen, rather than with the lightbox modal. It took me a minute or two to figure out how to close it on my 4k display.
4. I can't plan my time. There's no good way to see what/how much stuff there is to do, or the structure of the lesson, even coarsely.
5. Related to this, from a motivation standpoint, it's nice to be able to see progress.

Thanks for your feedback. Most of these issues are things we are either thinking about or already working on, but I don't want to hijack the thread, so I'll move this to a separate discussion.

 

If the sequences were fixed content, I'd much prefer a well-designed learning sequence style interface (although with autoadvance -- something we desperately need -- not with clicking a poorly designed "next" button as we currently have). Given adaptive content, as you mentioned, the problem of clearly showing and allowing navigation through a sequence is a good bit harder. 

SY does resolve some of the more common frustrations with in-video quizzes through good pedagogical design -- in Coursera and Udacity, most of the in-video quizzes require the content from the video, which is extremely frustrating for a number of reasons. With SY, in all instances I've seen, they're standalone -- you can figure out the question being asked from just what's on screen at the time the question comes up. I am wondering whether this would still be possible in other MOOCs, or specific to the level of content in SY. As a compare-and-contrast:

1. SY has simple questions (find a number on a number line, short equation, etc.). College-level courses typically have questions which are either much more complex (e.g. engineering design problems), or more contextualized (e.g. answer a question about a complex legal scenario). For quizzes limited to the size of the video, there strong temptation to put requisite context in the video itself.
   
2. In SY, the concepts themselves are short enough that a sequence on a given topic is typically just a few minutes. Most MOOCs are modular, but not quite this modular -- for complex topics, the minimum useful time a student might spend in a stand-alone sequence might be 15 minutes or sometimes even an hour. In SY, I would guess this is typically around 5-10 minutes (e.g. 10 elements at 30 seconds each). 

This is a fair point, and it leaves a lot in the hands of the content creator. However, even we ended up being a bit surprised at what could be done with super-short videos, and we even found we were able to apply the same methodology to lots of other topics, including SAT, med school prep courses, grammar, and even business school case studies. Given what I've seen, I definitely think it's possible to go this route even for more complex kinds of questions, although it would require effort from the course creator -- simply chopping up existing hour-long lecture videos into 30 second chunks probably wouldn't lead to a great experience, but rather the videos would have to be created with the modularity in mind.

Regarding the ultra-modularity: I'd like to offer one more recent anecdote from our current AlgebraX course: we just released a few lessons covering the topic of "slope". Instead of one long-ish 10-15 minute lesson, we ended up releasing it as 7 separate lessons, each about 3-5 minutes long. This ended up being useful for a couple reasons: the course staff (and the students themselves) can see with pretty high resolution where their problem areas are, and it helps a little bit with the navigation problem in the sense that if someone is going back to review something, they don't have to try to zip their way through a lesson just to get back to a particular section in the middle.

 

I'm curious -- how do you manage accessibility? 

This is a tough one to address, given the amount of interactivity we're trying to bring -- for example, a question we might ask is "draw a function that has 3 inflection points". To allow people who can't use a mouse to at least be able to proceed, our policy is to have a "i need a hint" button that a user can press on any of these pages which will initiate a playback of some audio (of the teacher explaining the problem) synced to some simulated mouse actions that get replayed. On any questions that require this kind of input, our policy is to make sure that the last hint will leave the interactive elements in the state that's needed to proceed to the next video. Admittedly, though, this area is still something that needs quite a bit more work.

 

Piotr

--
You received this message because you are subscribed to the Google Groups "General Open edX discussion" group.

To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/e8b9cb9f-7978-448c-ae6e-f376f42cc92b%40googlegroups.com.

[Piotr Mitros]

Perhaps we're miscommunicating. I am in 100% agreement about ultra-short video segments in all contexts. For use on a computer where the student has full focus (as opposed to e.g. mobile while commuting, where hands might not be available and focus is split), I like videos to be in the 30 second-5 minute range, usually towards the lower end of that. I certainly have never run into a context where I would advise using a video over perhaps 5-10 minutes long. I can back that up with both data analysis from edX viewing patterns and classic (if somewhat methodologically imperfect) literature on attention spans.

I am arguing that the overall sequence length (not individual video length) is often fundamentally much longer in many edX courses. e.g.:

1. Trying to break through a deep-rooted misconception in physics is often a time-intensive process. For examples of strategies here, see Chapter 7 of How People Learn. Most require an extended, focused "push"
   
2. Explain a complex concept in circuits typically involves a combination of theory and application. It's more effective to do this in one sitting, when both are in working memory. 
   
3. In a history class, you're often telling a story. You'd like to have as much or as little time to tell that story. 
   

And, likewise, the ability to adapt to students is somewhat weaker. Of these, adaptivity would likely be helpful for #1. In #2, I'm not quite sure how to apply it -- most questions are multiconcept. It is easy to provide students with resources to navigate to an appropriate remediation if they are stuck (e.g. through community q&a, RecommenderXBlock, or similar), but it's somewhat harder to just take them there. In the context of something like history, I'm very unclear as to what adaptivity would do. In most cases, the assessments cover a very sparse subset of what is taught. 

This translates to some of the UX differences. 

Piotr

[Ben Piscopo]

Hi Colin,

Wow - this conversation goes waaaaay back. Any new threads on "embedded problems in videos"?

Ben

On Friday, August 15, 2014 at 3:42:34 PM UTC-4, Colin Fredericks wrote:

Heh - just wrote that this week. :)

Check it out: 

https://edge.edx.org/courses/HarvardX/HXDEMO/2014/courseware/98e2362d5cfa4fee908ff8db31405795/6065c638713e4cbb92096c9663127d72/1

Hit "play" and you should see four different questions pop up in the first few minutes. There are controls underneath the video for resetting, skipping questions, and going back one question.

It would take some work to change things so that students would be unable to skip the questions, and much more work to make sure they answered correctly. You're welcome to use the code and change it to fit your needs.

[Colin Fredericks]

I can't remember any off the top of my head. There are some even older threads about it, and there's the original thread where Anton Stupak helped me figure out how to get at the controls in the first place, but nothing else comes to mind at the moment.

https://groups.google.com/d/msg/edx-code/246e6UYiKmw/hP5Vz3QaxzoJ

For those who stumble across this and want to implement pop-up questions during your own videos (or a lot of other things), here's my GitHub repo that collects a lot of my javascript stuff:

https://github.com/Colin-Fredericks/hx-js

Ben's going to be (hopefully) bringing this to a hackathon soon, so there might be some improvements in the next week or two! And there's the eventual dream of actually getting this into the platform, which was the plan after the previous hackathon.

--
You received this message because you are subscribed to a topic in the Google Groups "General Open edX discussion" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/b5072854-c1a4-4da1-84b5-84389d4cbd76%40googlegroups.com.

[Giulio Gratta]

There's also this! https://github.com/Stanford-Online/xblock-in-video-quiz

Example of it in use is here: https://lagunita.stanford.edu/courses/StanfordOnline/OpenEdX/Demo/courseware/Videos/90c25286036641efb04d140a3a563aea/

Giulio Gratta
- - - - - - - - - - - - - - - - - - - - - - - - -
B.S. Engineering (Product Design)
Stanford University, Class of 2010

--
You received this message because you are subscribed to the Google Groups "General Open edX discussion" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/CABmE_WKHa9AhuRq_CdYj_C_mZVsuAmLGYZPMwzgRdDD2K3RYjg%40mail.gmail.com.

[Colin Fredericks]

Oh yeah! I totally forgot about that, partly because I can't do anything with custom XBlocks, and partly because It's not on the usual list of xblocks (though that hasn't been updated since October).

To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/CAJeEHK6u4bYmZv%3D_8m8vz_FZt9J0fudBY3p_b4Bi0M8Y95AHdQ%40mail.gmail.com.