logical connection between problems

[Helena Kilger]

I successfully created my first problem and put it in a problem set (using example tracing tutor). I would like to connect the problems logically in this way: if the learner made no mistakes in the last three problems, then show a different (more advanced) graphical representation in the next problem. Everything else would stay the same. I would simply adjust the visibility of graphics depending on this if-statement. Is this possible? Are there any variables to set and use for these kinds of if-statements across problems? Can I use skills for that?

If it is not possible to use these kinds of if-statements inside of the brd file and across problems, maybe I could create multiple problems, one for each graphical representation, and adapt which problem is seen next depending on how well the learner did before? This would be like an outer loop adaptivity. I saw the "Selection Algorithm" and "Selection Command" options on the tutorshop site, but found no tutorials/documentation for the options.

Are these or similar actions possible and is there any additional documentation about skills/variables/if-statements/problem sequence? 
Thanks in advance!
Helena

[Jonathan Sewall]

To have an adaptive outer loop, you could choose one of Tutorshop's built-in problem selection algorithms or you could implement your own. You specify the algorithm in the Selection Algorithm field on the problem set's Settings page, shown below. In all cases, Tutorshop invokes the algorithm when the student begins and when the student has completed each problem in the problem set: the algorithm chooses the next problem from this problem set to present to the student.

1. Two of Tutorshop's built-in problem selection algorithms use skills for adaptivity. In CTAT, you can associate a step with one or more skills by using the Hints/Skills tab in the Link Editor on the behavior graph link proper to that step. When a student attempts to answer any step associated with skills, CTAT uses the attempt evaluation (correct or incorrect or hint) to recalculate P-Known, the probability of mastery, for each skill associated with that step. Tutorshop considers a skill mastered when this probability exceeds 95%.
1. With the Mastery Learning selection algorithm, Tutorshop will choose a next problem that could advance the student farthest toward mastery of all the skills required by the problem set. For example, if the problem set requires skills A, B and C and the student's probabilities of mastery are 80%, 60% and 40% respectively, then this algorithm would choose a next problem that exercises all 3 skills, with the goal of giving the student the opportunity to advance farthest toward completion by performing well in this next problem.
2. With Mastery Learning - Easier First, Tutorshop will choose a next problem that addresses the student's skill or skills closest to mastery among the unmastered skills required by the problem set. Using the example above, this algorithm would choose a next problem that exercises only skill A, the skill that the student appears to know best already.
   
2. Two of Tutorshop's other Selection Algorithm settings let you invoke your own algorithm to choose the next problem to present to the student. In both cases, the API is documented on this Github page, TutorShop Interface to Custom Problem Selectors.
1. With External Selection, Tutorshop will invoke a Linux program (for example, a program written in Python) that you would supply and we would install on the Tutorshop server machine. The program is expected to read Tutorshop's input data from stdin and write its output data to stdout, both in the JSON format described on the Github page just cited. Set the Selection Command on the problem set's Settings page to the Linux shell command needed to execute your program.
2. With Remote Selection, Tutorshop will make an HTTP POST request with its JSON input data as the request payload and expect a response of the form described on the Github page. Set the Selection Command on the problem set's Settings page to the URL for this HTTP request. The URL may be hosted on any server, wherever you install your algorithm.

For all algorithms, if you set the Initial Sequence count on the problem set's Settings page to N, then all students will begin with the first N problems in the problem set. The problem selection algorithm will choose the N+1st and later problems.

For your specific case, I recommend you start by creating multiple problems with differing graphical representations and use an adaptive outer loop. Unless you already have an implementation of your algorithm, I suggest you try using skills with Tutorshop's Mastery Learning - Easier First algorithm. You might tag steps in your first 3 problems with the initial skills students need, then tag steps in problems with the more advanced graphical representations with these skills plus additional skills. With Tutorshop's default values for 4 skill parameters, mastery can be reached with about 3 straight correct answers. Note that mastery is recalculated only on a student's first attempt at a step: if a first answer is incorrect, e.g., skill probabilities will change accordingly, but later answers on that same step will not trigger further changes. You might consider adding the CTAT Hint Widget to your student interface if you wish to show hints and skills.

--
You received this message because you are subscribed to the Google Groups "CTAT Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ctat-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ctat-users/a1127d6f-d7db-4d1e-a98a-ab782a5424b1n%40googlegroups.com.

[Helena Kilger]

Hi Jonathan, 

thanks for providing this detailed information. I tried out the selection algorithms and have some follow up questions:

1. I was not able to successfully test any selection algorithm. What I did so far: I created multiple problems (based on your suggestions some with skill 1 tagged and some with skill 1 + skill 2 tagged) to try out the Mastery Learning - Easier First selection algorithm. Then I used the green arrow ("Run problem set") to try it out, but it always stayed in the order in which I put the problems in the problem set. I never got different results, independent of me choosing "Mastery Learning - Easier First", "Sequential Order", or "Random Order" - I always seem to get what I would expect with "Sequential Order". Is this because I test it in the problem set? Should I test the sequence somewhere else?
2. I am still not sure if my idea is possible to implement with using "Mastery Learning - Easier First". What I would want is to keep the general math problem the same and only adaptively change by which graphical representation the math problem is accompanied. As an example: A perfect student would get Problem 1 + easy representation, 2 + easy, 3 + easy, 4 + intermediate representation, 5 + intermediate ... A more struggling student would get Problem 1 + easy, 2 + easy, 3 + easy, 4 + easy, 5 + easy, 6 + intermediate, ... I understand that I cannot switch the visibility of representations based on the student's performance in the problem(s) before. After reading your message I first thought I could simulate this adaptivity by creating all possible combinations (all problems with all representations) and define the sequence in which the students move through the problems. But now I think that this is not possible with "Mastery Learning - Easier First", because if I give every problem with intermediate representations the same new skill, every student would still start with the same problem (e.g., 4 + intermediate representation, even if it should be 6 + intermediate representation). For my study it is important to keep the order of problems the same between this condition and another condition with the same problems, but the students can freely choose between representations. Could you verify to me that I understand correctly that this is not possible (at least with the built-in problem selection algorithms)? If that is not possible, I will stop playing around with the outer loop adaptivity and simply increase the difficulty of representations for all students the same non-adaptive way (e.g., 3 easy, 3 intermediate, ...). 

Thank you so much for all your help! And thanks for providing this great tool :D

Helena