Exposing the Course Version number

[ji...@opencraft.com]

Hi,

This post is partly to record discussions going on in a planning document, and partly to spark more discussion on the topic.

Nimisha, do you have a list of pending API improvements where this feature request could be captured?

Feature Idea

Update the Courses API to add the current published version ID, which is available from the modulestore.

Example: course-v1:mit.eecs+6002x+branch@published+version@519665f6223ebd6980884f2b

Impetus

We're working on adding a table to the analytics data which stores the tree of blocks used in a course, as retrieved from the Course Blocks API.

Since this table may be relatively expensive to maintain, we'd like to be able to update it only when a course has changed, i.e. when a course or any of its sections is Published.

If we knew the current published version ID, then we could update the table only if the current published version has changed.

Discussions:

Captured for posterity.

From Braden MacDonald <bra...@opencraft.com>:

Every course in split modulestore has a version (internally), and any change to the structure of the course (or any content, too, I suppose) will change the version (which is just a mongoDB ObjectId). So "all you need to do" is store the result of the Course Blocks API query for each course along with the "version of the published branch" of that course. Then, you only need to re-fetch the block structure if the current version doesn't match the cached result version. 

A course's ID string/key can actually include this version information, though this form is rarely seen: "course-v1:mit.eecs+6002x+branch@published+version@519665f6223ebd6980884f2b" 

From what I recall about split's "versions" table, a MongoDB query to fetch a list of all courses and the version of each would be trivial and fast, though I don't know if there is currently any API which exposes it.

 
FWIW I implemented content libraries using that approach - the randomized content block checks the version of the library to know when to prompt the course author to update. If we used that approach it should be non through some API that's not directly related to the modulestore. 

From Gabe Mulley <ga...@edx.org>:

Yeah, I thought about leveraging that data. I'm not aware of any API that exposes it, however, and am very wary of reaching in behind the modulestore API (given how complex it is). 

I like the idea of using the version, though, to determine which courses need to be pulled again instead of time. Maybe we could leverage entity tags? Basically set the etag to the version identifier? Maybe it already does this? 

I'm not very familiar with conditional get semantics, but I think we could then use "If-None-Match" to short-circuit and return a 304 if the course hasn't changed. 

Thank you!

--

Jill

@OpenCraft

 

[Gabe Mulley]

Note that more discussion about this has been happening on this ticket:

https://openedx.atlassian.net/browse/TNL-4891

Thanks,

-Gabe

--
You received this message because you are subscribed to the Google Groups "General Open edX discussion" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/cedb8238-e452-467d-ab25-544c73e8a359%40googlegroups.com.

[Adam Palay]

Hi Jill!

You mention wanting to add a table "which stores the tree of blocks used in a course". Can you say a little more about this? I'm wondering because we've been working on a feature to visualize and query course structures in a graph-like-context by putting modulestore data into a graph database. The thing we're building is only an internal tool (not meant to be instructor facing), but I wonder if the work could overlap at all.

Adam

[Nimisha Asthagiri]

Hi Jill et al,

Thanks for starting this thread.  (I had been away on vacation and am catching up.)

For the mobile app, we've been considering using ETags to eliminate extra work in updating its local cache.  However, the use case there is a bit more complicated since we need to check for user-access state-changes in addition to course-state changes.

For your use case, however, since you only care about course-state changes, it is very plausible now for us to build what's required with minimal effort.  With the following recent PR, we were able to update the SplitModulestore and enable a BlockTransformer to store the course_version (guid) and subtree_edited_on (timestamp) fields with the cached course structure:

https://github.com/edx/edx-platform/pull/13082

So now, this data is available to the Course Blocks API.  The subtree_edited_on timestamp may be all that you need to embed in the ETag when non-user-specific blocks are requested (i.e., when all_blocks=true URL param is specified).

Nimisha

[Jill Vogel]

That's amazing timing, thank you Nimisha!

I'm not seeing the new subtree_edited_on timestamp when I call the updated course blocks API, however.  I'm using a URL like this in my devstack:

http://localhost:8000/api/courses/v1/blocks/?course_id=course-v1:edX%2BDemoX%2BDemo_Course&all_blocks=true&depth=all&block_types_filter=course,sequential,vertical&requested_fields=children

Have tried adding subtree_edited_on,course_version to the requested_fields parameter, but that changed nothing.

Example returned block:

        "block-v1:edX+DemoX+Demo_Course+type@course+block@course": {
            "display_name": "edX Demonstration Course",
            "student_view_url": "http://helga.local:8000/xblock/block-v1:edX+DemoX+Demo_Course+type@course+block@course",
            "id": "block-v1:edX+DemoX+Demo_Course+type@course+block@course",
            "lms_web_url": "http://helga.local:8000/courses/course-v1:edX+DemoX+Demo_Course/jump_to/block-v1:edX+DemoX+Demo_Course+type@course+block@course",
            "type": "course",
            "children": [
                "block-v1:edX+DemoX+Demo_Course+type@sequential+block@edx_introduction",
                "block-v1:edX+DemoX+Demo_Course+type@sequential+block@simulations",
                "block-v1:edX+DemoX+Demo_Course+type@sequential+block@graded_simulations",
                "block-v1:edX+DemoX+Demo_Course+type@sequential+block@175e76c4951144a29d46211361266e0e",
                "block-v1:edX+DemoX+Demo_Course+type@sequential+block@07bc32474380492cb34f76e5f9d9a135",
                "block-v1:edX+DemoX+Demo_Course+type@sequential+block@workflow",
                "block-v1:edX+DemoX+Demo_Course+type@sequential+block@19a30717eff543078a5d94ae9d6c18a5",
                "block-v1:edX+DemoX+Demo_Course+type@sequential+block@basic_questions",
                "block-v1:edX+DemoX+Demo_Course+type@sequential+block@48ecb924d7fe4b66a230137626bfa93e",
                "block-v1:edX+DemoX+Demo_Course+type@sequential+block@dbe8fc027bcb4fe9afb744d2e8415855",
                "block-v1:edX+DemoX+Demo_Course+type@sequential+block@6ab9c442501d472c8ed200e367b4edfa"
            ]
        },

Thanks for your help!

Cheers,

--

Jill

@OpenCraft

--
You received this message because you are subscribed to the Google Groups "General Open edX discussion" group.

To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/CACSOLjdJxTYRU1sRBxcOSAVahFV1NHfYdTbrBMn80Y7MhX6Y_A%40mail.gmail.com.

[Nimisha Asthagiri]

Hi Jill,

Sorry for not being clear.  We haven't done the plumbing to surface the subtree_edited_on field into the Course Blocks API - as it wasn't needed for our purposes.  The approach we have been taking with the Course Blocks API as of now (for better or worse) is to add supported fields only if needed.

However, the rest of the plumbing has already been completed.  So to complete the plumbing, all that's needed is to add subtree_edited_on into the SUPPORTED_FIELDS list at: https://github.com/edx/edx-platform/blob/master/lms/djangoapps/course_api/blocks/transformers/__init__.py#L36

Other than updating the API docs, that's all that is needed for what you need.

Nimisha

To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/CAMZ1AtJ4cahScc%3DkhMbsX9GF0v__G_mu0mfsx0wq6LZtTG9uBA%40mail.gmail.com.

[Jill Vogel]

Oh awesome, thank you Nimisha!

--

Jill

@OpenCraft