Draft proposed standard for Parsons problem definition

[Cliff Shaffer]

Hello to members of the Parsons Problems SPLICE Working Group!

I have finally produced an initial draft document for a proposed SPLICE format standard for defining Parsons problems. I have tentatively named this Parsons Problem Input Format (PIF). PIF is built on the PEML standard (see https://cssplice.org/peml/index.html).

The PIF draft standard document is available at: https://docs.google.com/document/d/1ZzEgS4_2SyS88fhWVp0041KmfWFnXBKgMWmPEDI7chw/edit?usp=sharing.

You can see a number of examples in PIF format in a GitHub repository at: https://github.com/CSSPLICE/peml-feasibility-examples/tree/main/parsons

You can reach all of the resources from: https://cssplice.org/parsons/index.html

My request to you is that you take a look at the draft standard document (and the examples if you like), and then send me feedback about what you like and don't like. Feel free to post here, or reply to me directly. Things to consider:

* How reader-friendly is this? Can you read one of these examples, and understand the problem being specified?
* How author-friendly is this for someone trying to write a Parsons problem in the file format?

* How implementation-friendly is this? Ultimately, we need to implement either direct support in systems, or translators for converting this format to the various input formats used by systems.

* How complete is this? Does it have enough features to support a reasonable universe of problems? Are there features in existing implementations that this format does not support (yet)?

My plans are to:

* Continue refining the spec. I need your feedback for this!

* Beef up the examples list to help document the spec.

* Make sure that the examples actually work, as best I can given limited ways at the moment to validate the files.

* I hope to have a student start work soon on implementing support for this specification, and at least one or two translators into existing system input formats.

[Shaffer, Cliff]

Thanks for responding! I will take feedback anyway that works for you. 🙂 All of the methods you suggested seem fine. And I expect this process to be ongoing for a while, so if you can get to it in a few weeks, that should be fine. I would like for the group to have made significant progress by the time of SIGCSE (end of February), and I hope that several of us can get together to discuss final details then. By the way, there will be a SPLICE workshop on the Wednesday that is the day before the conference sessions start.

Yes, to start validating files requires implementation. I am starting this process by turning the executable exercises into Code Workout exercises (since we have support for PEML there). That will let me test about half of the spec for those exercises. Meanwhile we will be working with the Runestone instantiation of what was originally JSParsons, to either do the first translator or else a direct input of the PIF files. Whichever seems easiest when we get there. Unfortunately, it's one more thing that our team is stretched thin on, so I can't guarantee how fast we actually make progress. But that is the plan.

              -- Cliff

--

Dr. Cliff Shaffer                                               

Professor
Department of Computer Science              Phone: (540) 231-4354
Virginia Tech, Blacksburg, VA 24061          WWW: www.cs.vt.edu/~shaffer

---

From: Seth Poulsen <seth.p...@usu.edu>
Sent: Friday, October 4, 2024 4:32 PM
To: Shaffer, Cliff <sha...@vt.edu>; SPLICE Parson's Problems Working Group <splice-parsons-pro...@googlegroups.com>
Subject: Re: Draft proposed standard for Parsons problem definition

 

Hi Cliff, 

Thanks for keeping me in the loop about this and making sure DAG based grading gets represented in the spec. 

Do you have a time frame you are looking for feedback in? I would like to go through everything carefully, but I am very busy the next few weeks.

How would you prefer to receive suggested edits? Should we use the 'suggested edit' feature on Google docs, or just add comments? Open PRs on Github?

For validating the files, it seems like a reasonable thing to do would be implementing a basic converter to put them into the syntax of one of the actual systems that use them, be it Runestone or PrairieLearn or whatever.

Thanks!

-Seth

---

From: splice-parsons-pro...@googlegroups.com <splice-parsons-pro...@googlegroups.com> on behalf of Cliff Shaffer <sha...@vt.edu>
Sent: Friday, October 4, 2024 2:10 PM
To: SPLICE Parson's Problems Working Group <splice-parsons-pro...@googlegroups.com>
Subject: Draft proposed standard for Parsons problem definition

 

--
You received this message because you are subscribed to the Google Groups "SPLICE Parson's Problems Working Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to splice-parsons-problems-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/splice-parsons-problems-working-group/4d9e6a52-18a5-4a20-8347-3861c9f1fea1n%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Cliff Shaffer]

I have made some changes to the draft document. In particular, I have added more explanation about "fixed" blocks, and added examples to the repository for more complicated support of fixed blocks and multi-line blocks.

Feedback is appreciated! If you have some good problems that you would like to see rendered into PEML (especially if they would stress test the feature support), please send me links!

                    -- Cliff

[Cliff Shaffer]

Based on feedback, I have now added another method for specifying the DAG in order-graded problems. Instead of an explicit section at the end, it is done implicitly by adding "parent" fields to the blocks.

I think the explicit version at the end makes the DAG slightly easier to "see". But the implicit method is clearly no harder for the author to type (its exactly the same characters other than the overhead lines for the [assets.test.files] block), and it also has the additional nice feature of making the distractor blocks easier to recognize (especially in pickone groups).

Thanks to Barb and Seth for their suggestions so far!

                         -- Cliff