create random exams with sagetex

[p a]

The covid goddess appeared upon me in spring and told me to create random exams for online examination so that the individual solutions could not be shared. I started with auto-multiple-choice, but then I noticed its basic computation capabilities, based on the latex package fpeval, are not enough to build questions on many topics.

I tried to call sagetex from auto-multiple-choice, and I did succeed, but it was akward and buggy, since both sagetex and auto-multiple-choice hack LaTeX in incompatible ways to bypass the limitations of the verbatim environment.
Eventually, I wrote my own system for generating exams, based on sagetex alone. It works like this:

• "pdflatex file.tex" compiles the latex file, but does not eval sage, as in sagetex, so it is compatible with latex editors. This file has some markup for the solution, and for multiple choice questions (optional).
• the standard sagetex sequence works as usual:
  • pdflatex file.tex
  • sage file.sagetex.sage
  • pdflatex file.tex
• the command "./process file.tex" is just the three above commands in succesion
• the command "./process file.tex all", however, reads all student data from "students.csv" and generates
  • the statements id1.pdf, id2.pdf, etcétera, in the "question" folder
  • the solutions, id1.pdf, id2.pdf, etcétera, in the "solution" folder
• If there are forms in the pdf, then "file.data" contains info about the questions and the correct solution for each question.
  

It can be used in two ways:

• A multiple choice exam for each student, which you can send by email or distribute using a "folder" activity in moodle, and collect using an "assign" activity in moodle. After downloading the filled exams from moodle, the script in the "grade" folder helps grade automatically. It only works if the students fill the pdf forms correctly, it doesn't work if they draw on top using whatever means.
• An open question exam.
  

For both of them a detailed solution is useful to the student. For open questions, it can also be interesting for the grader, since it makes easier to detect mistakes in the computations. In the examples that you can find in the adjoint zip file, you will not find that possibility, but I did it in the messy exam with auto-multiple-choice and sageexam, it could be done with this system and I'm sure you know well what I mean.

I'd be glad to have some feedback. It is important to work out the details, so that a system like this can be used by more people that know little sage and python. There are many possible improvements, like text field questions that are checked with sage symbolic capabilities, exporting to moodle questions, and many others. The real treat will come when we start to design questions where the number of vectors, the dimension of each of the subspaces and their intersections, and so on, are different each time, so that the same question can be used for sparring a few times.

But I'd like to get the basics right first. Maybe the R exams module is showing the righteous way? But it doesn't work in a latex editor...

Regards

[kcrisman]

One possible solution (though not necessarily using SageTeX) is to use PreTeXt (https://pretextbook.org) to create things that can be randomized, and then use different stylesheets to mod out by solutions or include them.  (For instance, WeBWorK problems can be included, as well as Sage content.)

I do like the idea of automating the creation of the different files for each student the way you've described.  I wonder if people who have used the grading functionality on CoCalc might also have some wisdom here.  Of course this could be used for other LMS like Moodle as you say, though how to "assign" them would probably depend on the LMS.

Anyway, there are definitely people trying to do similar things right now with several different toolsets, so I hope that some others have some good thoughts on this.

[William Stein]

On Mon, Oct 19, 2020 at 8:29 AM kcrisman <kcri...@gmail.com> wrote:
>
> One possible solution (though not necessarily using SageTeX) is to use PreTeXt (https://pretextbook.org) to create things that can be randomized, and then use different stylesheets to mod out by solutions or include them. (For instance, WeBWorK problems can be included, as well as Sage content.)
>
> I do like the idea of automating the creation of the different files for each student the way you've described. I wonder if people who have used the grading functionality on CoCalc might also have some wisdom here. Of course this could be used for other LMS like Moodle as you say, though how to "assign" them would probably depend on the LMS.
>

In CoCalc, people tend to create exactly one assignment but with a
random seed that is a function of the student (e.g., the project id or
student name or something). We don't have anything more
sophisticated than that right now.

> Anyway, there are definitely people trying to do similar things right now with several different toolsets, so I hope that some others have some good thoughts on this.
>

> --
> You received this message because you are subscribed to the Google Groups "sage-edu" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-edu+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-edu/89ab1b0b-2167-4e60-b26f-8f2b14184c34o%40googlegroups.com.



--
William (http://wstein.org)

[carey g. butler]

Thank you for this great exchange. Similar issues here with us. 

--

[p a]

Hello, Karl:

I've browsed this PreTeXt, and it's interesting, but right now it is rather short on features. I looked on the R exams package because it exports to different LMSs, and also supports written exams, so it can identify the options that the student wrote on physical paper. Also, I think the two main things that can be improved from the R exams package are:

• a system like sage that understands linear algebra and symbolic calculus.
• a system that looks like latex as much as possible, compatible with everybody's favorite editor, with forward and inverse search.
  

Using pretext I'd lose the second advantage, and I'd also lose all the wonderful work in sagetex.

Regards

[kcrisman]

In that case (which I think is a reasonable assessment), I think you could try to think of a good way to integrate this into SageTeX, whose repo is at https://github.com/sagemath/sagetex  

Given that people use a lot of different ways to work with SageTeX (TeXShop, custom scripts, command line one-by-one), it would be great if rather than adding a "./process file.tex all" you were able to add a new Python (3) script that could be included along with the already-existing ones.  You'd have to add it to https://github.com/sagemath/sagetex/blob/master/scripts.dtx as well as add documentation to the https://github.com/sagemath/sagetex/blob/master/sagetex.dtx file, probably under a new section like \section{Using \ST to create randomized output for exams} or perhaps something more general than that.

I think that would be a very useful contribution, especially if it were really well documented so that people who really just barely know how to use SageTex could still follow the instructions successfully.  Hope this gets you excited to contribute!

[p a]

Thanks a lot, Karl, will certainly do, some day, but I have the feeling that I still have to make a few tiny decisions which, together, will account for much of the overall experience for non-sage-expert users.

Ahem, I just noticed the file that I attached to my first email did not come through. Here is a link to the project in its current form:

https://drive.upm.es/index.php/s/gmnObJEoqCrarS4

The command

./process ALyGExtra200626Parte1.tex all

will compile statement and solution for each of the students in the list, where statements are pdf files with embedded formulaires for multiple choice questions, while

./process AMPExtra200627Parte1.tex all

and the other sample files are open question exams.

Regards

[p a]

El lunes, 19 de octubre de 2020 a las 17:34:26 UTC+2, William Stein escribió:

In CoCalc, people tend to create exactly one assignment but with a
random seed that is a function of the student (e.g., the project id or
student name or something). We don't have anything more
sophisticated than that right now.

I also use the student id as a random seed, which makes very explicit how to reproduce the output exactly. However, I'm guessing that the goal there is a cocalc assignment for some coding problem, while our main goal is examination for courses not related to coding. I'd be surprised if cocalc can help with that...

[p a]

That's a good target, Karl, but another target could be the sage local/bin folder.

I rewrote the "process" script in order to make it more similar to the sage-notebook syntax, both in usage and coding practice (https://drive.upm.es/index.php/s/DHh26fgL0RsFCv5).

I think a

  sage -exam -init

option could also be useful, that would copy a template file, plus the required auxiliary files, like sagetex.sty, and then print a useful message. It would be followed by "sage -exam template.tex", without further ado.That would simplify the startup, requiring only that sage is installed. I don't see how this could be done if possibly-maybe-sageexam is distributed within the sagetex folder.

Anyway, that's decision to be taken later, since I still need to polish the syntax, refactor, add a couple new features, etc. I also have a longer goal to keep a local database of exam questions. I'm using git locally and I think it would be a good idea at this moment to make a public repo and keep it as an independent project before it is ready to be included in sage, because:

• This is a small project, that depends on sage and sagetex but not viceversa, and sage version agnostic, so it will be more agile to add small improvements without upgrading and testing sage regularly.
• It will make it a lot easier to install to non-experts before the time when it is completely integrated into sage.
• It would also make it simpler, if this small project grows, to submit it to the journal of open source education, for instance.
  

Would love to hear your opinions on those.

Regards

[kcrisman]

On Thursday, October 22, 2020 at 10:34:46 AM UTC-4, p a wrote:

That's a good target, Karl, but another target could be the sage local/bin folder.

Personally I think this makes more sense as something w.r.t. SageTeX since it is clearly a TeX thing using Sage.  Obviously many implementations are possible, though.

However, at this point I think that your proposal is far enough along that I recommend you discuss it on sage-devel :-) because people there would definitely have good comments on it at this point, and it is moved beyond the specifics of sage-edu.  Looking forward to reading a post there!

[p a]

ok, I'll write there, but I've discussing with colleagues, we had a couple of good ideas and I'll be changing the syntax a lot these days. I'll write then. See you