Associate exam with module_type = problem, in edxapp.courseware_studentmodule?

[Samuel Marks]

How do I get the exam name?

I've been doing my head in exporting from MongoDB modulestore.structures to import to MySQL so that I can JOIN across the fields; but I've been having difficulty acquiring the full hierarchy.

From my understanding MongoDB stores the course structure—including exam name—and MySQL stores the student_id, grade, max_grade and timing (how long each question took to answer).

In MongoDB modulestore.structures is a collection of documents like so:

• _id: ObjectId, original_version: ObjectId, blocks: Array<{}>
  
• [opening up and enumerating block fields within block objects within blocks array]:
• block_id, block_type, fields
• fields.format contains exam name
• The only fields.format with exam name have block_type = 'sequential'
• But not all have. E.g.: out of my 10883 sequential blocks I have 262 non null fields.format
  

The block_type is a single one of these options, which I believe to be in this hierarchy:

1. course
2. chapter
3. sequential
4. survey || video || problem
   

In MySQL edxapp.courseware_studentmodule the field is called module_type (whereas MongoDB calls it block_type).

My many attempts at solving this problem: https://gist.github.com/SamuelMarks/b284c5fc4c6699cdfd603f6b9065c513

Help would be most appreciated.

Thanks

[David Ormsbee]

Hi Samuel,

The most straightforward way to get the exam name is by using the modulestore interface in Python code. If you create a Django management command, you can do something like this:

from xmodule.modulestore.django import modulestore

from opaque_keys.edx.keys import UsageKey

key = UsageKey.from_string("block-v1:edX+DemoX+Demo_Course+type@problem+block@d2e35c1d294b4ba0b3b1048615605d2a")

ms = modulestore()

seq = ms.get_item(key)

seq.display_name

Please note that this has to run in the context of Django and the virtualenv you run edx-platform with, or it will fail because it's missing libraries and configuration.

If you need to get all the sequences in a course, you can do:

from xmodule.modulestore.django import modulestore

from opaque_keys.edx.keys import CourseKey, UsageKey

course_key = CourseKey.from_string("course-v1:edX+DemoX+Demo_Course")

ms = modulestore()

sequences = ms.get_items(course_key, qualifiers={'category': 'sequential'})

for seq in sequences:

    print seq.location, seq.display_name

That will print both the location (the module_id used in the courseware_studentmodule table) and the display name for all sequences. There are a lot of attributes on SequenceDescriptor objects, so you can inspect them to see what else might be useful to you.

There were also recent additions that put this data in a more easily digestible form in MySQL (introduced in Ficus, but more stable in Ginkgo). Documenting the migration process is on the todo-list for Hawthorn, but the data model is documented here:

    https://openedx.atlassian.net/wiki/spaces/AC/pages/125927634/Grades+Data+Model

This is part of the larger re-working of how grades are calculated and stored:

    https://openedx.atlassian.net/wiki/spaces/AC/pages/159458764/Grades+Architecture

Hope that helps a little. Take care.

Dave

--
You received this message because you are subscribed to the Google Groups "General Open edX discussion" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/5637ceae-dbe0-488b-b166-c414243a3078%40googlegroups.com.

[Samuel Marks]

Thanks, I've added a little command for this: edx-platform$ ./manage.py cms --info-for-course-id '<course_id>'

 https://github.com/SamuelMarks/edx-platform/tree/SamuelMarks/ginkgo

Getting this output though, will debug in the morning:

Traceback (most recent call last):
  File "./manage.py", line 132, in <module>
    main()
  File "./manage.py", line 115, in main
    dump_course_info(edx_args.info_for_course_id)
  File "/opt/openedx/edx-platform/hacks.py", line 25, in dump_course_info
    ms = modulestore()
  File "/opt/openedx/edx-platform/common/lib/xmodule/xmodule/modulestore/django.py", line 330, in modulestore
    settings.MODULESTORE['default']['ENGINE'],
  File "/opt/openedx/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 49, in __getattr__
    return getattr(self._wrapped, name)
  File "/opt/openedx/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 160, in __getattr__
    return getattr(self.default_settings, name)
AttributeError: 'module' object has no attribute 'MODULESTORE

(done on my Open edX install, from my Fabric script: https://github.com/offscale/offregister-openedx based off https://github.com/regisb/openedx-install )

[Samuel Marks]

Finally got around to solving this problem - the secret was the dump_course_structure function (from edx-platform).

Wrote some parsers and docs for it: https://github.com/SamuelMarks/openedx-modulestore-utils

Now finding number of minutes taken per problem per exam is as easy as:

SELECT
  T0.*, T1.exam, T1.display_name,
  TIMESTAMPDIFF(MINUTE, T0.created, T0.modified)  minutes_taken
FROM edxapp.courseware_studentmodule T0
JOIN parsed_ms T1
ON T0.module_id = T1.block_id
WHERE T1.exam IS NOT NULL AND T0.module_type = 'problem';

:D