WeBWorK exercises and randomization

[Sean Fitzpatrick]

Mostly a question for Greg:

We've run into a bug with images in WeBWorK problems where the same image gets (incorrectly) used across multiple problems. Not the first time we've seen it.

Last time this came up, there was the discussion around whether we should specify a "seed" for each problem.

With seed specified: we can guarantee the numbers used in each question, thereby ensuring that questions are identical to how they appeared in APEX V4.

With no seed specified: some numbers are randomly generated, so they may not match what appeared in past versions.

If I recall correctly, we decided to make things random in the past, because Greg was worried about the HTML (which was supplying answers) matching print, and potentially allowing students to get answers to their homework questions.

With that in mind, should we just get rid of the seed specification? It fixes the image bug (I think) and makes it harder for students to get answers to homework questions.

The only downside is that we can't guarantee questions are identical to past versions of the book.

[gregory...@gmail.com]

I think that matching the book isn't a goal we need strive for with WeBWorK. I've probably advocated for that in the past, but upon further reflection/time, those who look to use WeBWorK aren't likely to care if the problems match exactly with the printed text. 

So unpairing the HTML and the text should solve several problems, right? No seed = random problems (no cheating) and no image bug. 

Let me know if I'm reading this wrong, but if eliminating the seed fixes things with the images, let's do it & the other consequences will be ok.

[Sean Fitzpatrick]

Thanks Greg.

Eliminating the seed fixes some problems, as far as I can tell. The HTML and PDF should still agree, since both will use the same file generated by the WeBWorK server.

It's mostly a case of APEX v. PreTeXt will not match APEX 4.0.

--
You received this message because you are subscribed to a topic in the Google Groups "APEX/Active Calculus MBX conversion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/apexactive-calculus-mbx-conversion/FPj_qlNddZU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to apexactive-calculus-mbx...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/apexactive-calculus-mbx-conversion/610cad8d-1f61-4611-abd2-6cedcca8501cn%40googlegroups.com.

[Alex Jordan]

Some things keep getting confused and I will try to clear them up as best I can.  I see evidence in this thread that the confusion is still there. Sorry this is so long, but it's complicated.

Let's say it's a few years from now, and APEX is fully sourced with PreTeXt, and it's on edition 6. There is a print copy you can buy and there is an HTML version you can read online. And there is an EPUB you can read digitally, offline. Imagine Exercise 2.7.23 is a problem that is a "webwork" element in the PreTeXt source. It is coded to permit randomization. It is the 512th "webwork" in the PTX source. No @seed is declared in the PTX source.

* In print, Exercise 2.7.23 is a particular random version. When you load the HTML to Exercise 2.7.23, or the EPUB to Exercise 2.7.23, it will be the *exact same* random version that the print has. This is by design. In all cases, the problem is rendered using seed 512. You would have to go to extra lengths if you want these to mismatch. Like you could build the PDF, then edit the source files to declare a @seed on that problem, then build the HTML. Or you could build the PDF, insert a "webwork" early in the book so now this exercise uses seed 513, and then build the HTML. But whatever you do, you would have to go to extra effort to make these things not match.

* The above is intentional design. Imagine your class of 30 has 15 who prefer print and 15 who prefer HTML. You assign paper homework that includes Exercise 2.7.23. This design decision means that you will get 30 submissions that are all the same version of that exercise written up on paper. This eases the life of the grader, who can process those papers in an assembly-line manner, not distinguishing this half from that half. And all without the instructor having to specify assigning only from the print version, not from the HTML (or vice versa).

* Having this "webwork"-sourced exercise appear in HTML is not the same thing as having this exercise be interactive in HTML. If you ask me, the normal thing to do is to have this exercise be /static/ in HTML. Instructors can assign it in a traditional paper write-up assignment. There is no access to having WeBWorK feed you the solution or verify that you have a correct answer.

* Imho, there is a place for randomizable interactive exercises for student development and practice: inline exercises. Not the divisional exercises being discussed here.

* If you use WeBWorK to assign a problem set that includes Exercise 2.7.23, now you have something that will be randomly different from student to student, which is good. But only when they are logged in to WeBWorK, and this is still not putting the interactive version into their HTML reading.

* If you put the HTML version of the book inside Runestone Academy, that is another scenario. In that scenario, now we want the HTML version to have interactive webwork exercises. We want Runestone to manage recording when a student completes the interactive version embedded in their HTML, which is in turn embedded in Runestone. We also want Runestone to recognize that so-and-so is logged in, and *based on who that user is*, we want a random version that differs from the print version.

* On top of all of this, do we want edition 6 Exercise 2.7.23 to say the same thing as edition 4 Exercise 2.7.23?  OK, it's natural for the number to migrate. Maybe it used to be #19, and now it is #23. But if the edition 4 version asked you to add 5+7, is it OK that the edition 6 version asks you to add 4+9?

This last item is the most important thing that we need Greg's input on. Once I know the answer to that, I can advise how to handle all of the rest.

You received this message because you are subscribed to the Google Groups "APEX/Active Calculus MBX conversion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to apexactive-calculus-mbx...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/apexactive-calculus-mbx-conversion/CAH%2BNcPYi9JpWFgXxoTCoGhGCba-%2BFvW%2BXyNUP2ERfd93G5nSEQ%40mail.gmail.com.

[Sean Fitzpatrick]

Thanks Alex.

I think, for the purposes of my current textbook builds (which I am starting tomorrow, and need to have done by week's end), I should proceed without specifying a seed.

If we later want consistency across editions we can add seeds.

It was maybe two years ago (when "make interactive" was not yet an option) that we discussed having inline exercises that were interactive, and divisional exercises that were static.

If I remember correctly, that was more of a departure from previous APEX editions than Greg wanted to make. (And we couldn't decide whether this meant moving existing problems, or authoring new ones, or turning some of the examples into exercises.)

Is there any way, at the moment, to suppress solutions once you make a WeBWorK exercises interactive?

To view this discussion on the web visit https://groups.google.com/d/msgid/apexactive-calculus-mbx-conversion/CA%2BR-jre%3D1u9aW1pjEZYok8%3Dm8SwTra_FjEHcMvYTKwwDwfnQ4w%40mail.gmail.com.

[Sean Fitzpatrick]

Greg hit reply instead of reply-list.
Since I also do that all the time, I'll re-post his message.

For my current publication run, I'm not setting seeds, because I want to avoid the WeBWorK bug that's hitting HTML images. In the future, I agree: we should set a version.

From Greg:

Thanks, Alex. Yes, I was confused, and this helps.

I agree with the logic that the HTML & epub & print editions all have the same exercise 2.7.23. Makes total sense.

To your last point, it seems that many problems in V5 will be different than those in V4. Same *types* of problems, though different. If feasible, my knee-jerk reaction is for consistency across future editions. One of my goals from the outset was to not be changing problems with each edition. Our motivation/reasoning for doing so is far different than those of a commercial publisher, but the end result is the same: teachers have to rewrite hw sets and/or their solutions.

At the same time: if new editions only come out every 4 years or more, then new problems isn't terrible. I can't bring myself to view this as a "feature", though I can defend that it isn't a "bug". 

I imagine that many problems won't be changing each edition, too. Rather, just the ones that are a bit more cookie-cutter, right?

Alex, what are your thoughts on this topic: is consistency of problems across editions a worthy goal? 

Thanks.

To view this discussion on the web visit https://groups.google.com/d/msgid/apexactive-calculus-mbx-conversion/CA%2BR-jre%3D1u9aW1pjEZYok8%3Dm8SwTra_FjEHcMvYTKwwDwfnQ4w%40mail.gmail.com.

[Alex Jordan]

With my own projects, I leave our explicit seed declaration on each exercise. Except for the occasional case where I force some seeds to ensure that one exercise is not coming out too similar to the previous one.

This means that the random versions can change a lot. If I insert a new webwork exercise, even at the very end of section 1.1, then every exercise after that in the book gets re-randomized for the next build. This does not bother me if it's between editions.

In the case of APEX, it's already true that the exercise numbers migrate from version to version. That is, it's already the case that you will occasionally be inserting an exercise early in the exercises collection, so that an exercise that used to be #21 is now #22. So it's already the case that an instructor's curated list of what homework they want to assign needs to be reviewed when there is an edition change. But currently, that review is easy ("I still see the same 2+2 exercise, it's just moved to #22 now.") And if things are rerandomized, that review will be more intensive because the instructor would have to think about the question's structure. So I don't know where to land.

If we make this change one time, then after that we could look into a script/transform that would hard code each seed. Meaning that if right now an exercise has an *implied* @seed="522", we could actually write that into source. So later when a new exercise is added at the end of section 1.1, it has no effect on other exercises randomizations. But I think this is a thing to do in between editions 5 and 6. Doing it now between 4 and 5 would require a tedious review to manually check that we have the right seeds in place to match the edition 4 versions.

Sorry; I think I'm being a little helpful, but not all the way helpful :)

To view this discussion on the web visit https://groups.google.com/d/msgid/apexactive-calculus-mbx-conversion/d872b615-1b25-3ff8-ac95-7e255a20cc17%40gmail.com.

[Sean Fitzpatrick]

No, this is helpful. I am taking the easy way out at the moment, but there is context:

- My publishing deadline is the end of this week. (For print on demand PDF, not HTML, but we want the two to match!)

- As far as I know, none of our instructors have curated lists of homework questions from the book. (For routine exercises, we'd rather use WeBWorK: the marker (sorry, that's "grader" in American) budget is very limited, so we save human grading for more significant assignment problems, and for tests.)

- I'm currently working on the versions that are localized for use here.

Once I'm clear of this deadline, I can put the seeds back in. But maybe this is something we put off until we're ready to declare an edition?

To view this discussion on the web visit https://groups.google.com/d/msgid/apexactive-calculus-mbx-conversion/CA%2BR-jrf9iup59J57y19K9e6ZpHDZV21CFus-jQ1NJF1e5nGB4A%40mail.gmail.com.

[Alex Jordan]

If it were me, I would leave the explicitly declared seed out entirely, which is what you (Sean) are doing. Then the only downside is randomizable problems changing from one edition to the next. Which, maybe we can mitigate in the future between editions 5 and 6.

This is all separate (but related to) a WW bug. The images just shouldn't be overwriting each other in the first place. It's back to the top of my list to explore this.

To view this discussion on the web visit https://groups.google.com/d/msgid/apexactive-calculus-mbx-conversion/640e96fc-372e-1e02-94f4-e39d8b9385e9%40gmail.com.

[gregory...@gmail.com]

Sean should make his deadline, so whatever path makes this happen in the simplest way should be taken.

And yes, we can be more careful/intentional between versions 5 & 6. It isn't too big a deal to re-map questions and/or write new answer keys, but I don't want to pretend it is nothing and change things willy-nilly.