IMS Content Package, IMS-CP, and Learning Object Metadata, LOM, Work

johnk

unread,

Aug 29, 2011, 7:57:26 PM8/29/11

to Sakai Nakamura, lysh...@gmail.com

Hi Foks particularly Yushan,

Zach has asked me to look into the state of our development to support
import and export of IMS Content Packages, aka IMS-CP. Based on the
pointers Carl gave me, it looks like Yushan has done a functional
implementation pushed to the iep/open-experiments repository.

It's appears to be realized in two new bundles, content-packaging and
lom (learning object metadata) as well as two additional files in the
files bundle, IMSCPFileHandler.java and ExportIMSCP.java.

Looks like there are also changes to devwidgets/entity/entity.html,
devwidgets/lhnavigation/javascript/lhnavigation.js and devwidgets/
lhnavigation/lhnavigation.html pushed to ieb/3akai-ux from liyushan/
3akai-ux

I have several questions about the work at this point.
1) How complete an implementation is it?
2) What remains to be done?
3) Yushan, would you like me to collaborate on this task (I am a
developer from University of California, Berkeley and have been
working on the core nakamura team for several months)
4) Out of curiosity, I notice you didn't use the existing Learning
Object Metadata (LOM) Java API -lom4j. Was that because of licensing
issues lom4j being GPL?
5) Should we migrate the existing code into the sakaiproject/nakamura
and sakaiproject/3akai-ux repositories for use in post v1 branches? If
the answer to 4 is yes, we will need to put in the standard licensing
header since, as far as I can tell, the header is currently not there.

Comments and recommendations are appreciated.

Reference URLs to specs and JIRA's:
https://confluence.sakaiproject.org/display/KERNDOC/KERN-517+IMS+CP+Import
https://jira.sakaiproject.org/browse/KERN-517

https://confluence.sakaiproject.org/display/KERNDOC/KERN-518+IMS+CP+Export
https://jira.sakaiproject.org/browse/KERN-518

IMS-CP & LOM spec docs
http://edutechwiki.unige.ch/en/IMS_Content_Packaging
http://www.imsglobal.org/content/packaging/

http://en.wikipedia.org/wiki/Learning_object_metadata

Yushan

unread,

Aug 30, 2011, 6:10:47 AM8/30/11

to Sakai Nakamura

Hi, John,
I implemented importing and exporting IMS CP in the last two months, I
would like to
explain the steps of importing and exporting ims cp files in the
program first:

a) Import cp files
1) User choose zip files to upload
2) Program would scan whether there is imsmanifest.xml file, if it
contains that file, it would
be changed to sakai-doc type.
3) And then program would parse the xml file as a tree structure, if
an item's resource
is text type or directory node, the item would be add to the tree
structure.
4) If the resource contains local link in its content, it would
changed to our system path, for
example, if there is "images/1.jpg" in the resource, this link would
be replaced by "/p/image_id"
5) UI would display the file as sakai docs, and user can also download
the origin zip file from
the top.

b) Export cp files
Exporting cp files means exporting sakai docs as IMS CP files. There
is a download button for
each sakai doc, if user click it, program would generate a
imsmanifest.xml file and package the
pages as a zip file. If there is images in the sakai doc, the images
would also be packed, the
other embed content would be ignored.

For implementing these functions, I made changes in nakamura and 3akai-
ux projects. And I have
push them into ieb/open-experiments and ieb/3akai-ux SAKIII3890
branch(SAKIII3890 problem was resolved).

In ieb/open-experiments, I implemented as you said, and in ieb/3akai-
ux project,
I modified devwidgets/entity.html, devwidgets/lhnavigation/
lhnavigation.html,
devwidgets/lhnavigation/javascript/lhnavigation.js, devwidgets/
newaddcontent/newaddcontent.html
devwidgets/newaddcontent/javascript/newaddcontent.js, dev/javascripts/
content_profile.js.

The changes in 3akai-ux project are mainly for supporting multi-level
navigation
for a tree structure, you can see the UI from
https://confluence.sakaiproject.org/display/KERNDOC/KERN-517+IMS+CP+Import

About your questions:
1) It has a full implementation for IEEE LOM in lom bundle and
standard implementation in content-packaging
bundle, it doesn't support ims cp extension classes(ipointer, variant,
lingualTitle) now.
2) I don't think there is much remaining work now, but I really think
we need more examples for testing, I only got two
Melete ims cp files from Ian, and I wrote test examples in lom and
content-packaging bundles, but I think we need more actual
examples.
3) Sure, glad to work with you:) I am an academic visitor in
Cambridge(I am a graduate student in Tsinghua University),
and I am going back this Thursday, but I think I would continue sakai
work as a part time job since John Norman ask me to
continue.
4) One reason is I didn't know there is LOM API... the other is cp
files from Melete are not strictly standard. For example,
langstring element in melete files is <langstring>content</
langstring>, and its standard binding is <langstring><string>content
</string><lang>en</lang></langstring>; source and value elements in
vocabulary element are CharacterString type in IEEE lom,
but in Melete files, they're langstring type. That also why I said we
need more actual test examples.
5) I think we should add headers as sakaiproject requires, and I can
do that.

Yushan

> Reference URLs to specs and JIRA's:https://confluence.sakaiproject.org/display/KERNDOC/KERN-517+IMS+CP+I...https://jira.sakaiproject.org/browse/KERN-517
>
> https://confluence.sakaiproject.org/display/KERNDOC/KERN-518+IMS+CP+E...https://jira.sakaiproject.org/browse/KERN-518

Message has been deleted

johnk

unread,

Aug 30, 2011, 7:20:21 PM8/30/11

to Sakai Nakamura

Yushan,

Thank you very much for the informative reply. I am sure I will have
more questions as I try to exercise the code.

As I am going to tackle integrating the server side code first, one
question would be - do you have any curl statements that would
exercise the server-side code without the need to bring in the UX code
immediately?

Regarding the license header - yes, Zach says we should ask you to add
the header so that your intentions for the use of the code are clear
before we pull it into the nakamura 1.x branch.

In terms of integrating your code, I intend to:
1) Cherry pick your commits in chronological order
2) Build a version with your bundles and modified "files" bundle and
try to exercise the code as discussed above.
3) Coordinate with Nicolaas the integration (cherry picking) of your
UX code
4) Issue a pull request(s) for Zach or Carl to accept into the
nakamura 1.x branch

In terms of further work, it would be great if you want to continue
contributing to the nakamura core/managed project. In that case, we
would probably change the Git workflow to issue pull requests in
sakaproject/nakamura and sakaiproject/3akai-ux rather than the ieb
repositories if that is fine by you, Ian and John Norman.

Sounds like there is not much more work to do so maybe my role will be
more cherry-picker and tester than coder. If more coding is required,
will be happy to collaborate with you on that.

Will get back to you as soon as I have something working (or not
working ;-) ).

John K

> for a tree structure, you can see the UI fromhttps://confluence.sakaiproject.org/display/KERNDOC/KERN-517+IMS+CP+I...

> >https://confluence.sakaiproject.org/display/KERNDOC/KERN-518+IMS+CP+E...
>
> > IMS-CP & LOM spec docshttp://edutechwiki.unige.ch/en/IMS_Content_Packaginghttp://www.imsglo...
>
> >http://en.wikipedia.org/wiki/Learning_object_metadata

johnk

unread,

Aug 30, 2011, 7:29:07 PM8/30/11

to Sakai Nakamura

Made a JIRA for this work:
https://jira.sakaiproject.org/browse/KERN-2207

Jon Gorrono

unread,

Aug 30, 2011, 11:19:48 PM8/30/11

to sakai-...@googlegroups.com

Hi Yushan,

WRT ... examples of content packages, maybe you've already tried this
but it might be helpful to try some of the packages under the Content
Examples for SCORM 2004 4th since it uses CP v1.1.4

http://www.adlnet.gov/capabilities/scorm/scorm-2004-4th

One valuable goal would be the ability to import without getting
flummoxed by SCORM extensions.

Moodle has a long list of content creator tools too... some of them
have also passed compliance tests at one point or another as well..
but after a quick scan of this list I can't say a tool that would be
both compliant and able to produce 1.1.4 CP files popped out at me:

http://docs.moodle.org/20/en/Creating_SCORM_Content#Tools_for_Authoring

And if you've not done so already, maybe you can hazard employing the
Reload Content Packaging Tool from adl.... it's also listed on at that
page @ the url for adlnet.gov above. It's not always clear what you
are doing with that tool sometimes, but it might prove useful to you.

> --
> You received this message because you are subscribed to the Google Groups "Sakai Nakamura" group.
> To post to this group, send email to sakai-...@googlegroups.com.
> To unsubscribe from this group, send email to sakai-kernel...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/sakai-kernel?hl=en.
>
>

--
Jon Gorrono
PGP Key: 0x5434509D -
http{pgp.mit.edu:11371/pks/lookup?search=0x5434509D&op=index}
GSWoT Introducer - {GSWoT:US75 5434509D Jon P. Gorrono <jpgorrono - gswot.org>}
http{sysdev.ucdavis.edu}

Yushan

unread,

Aug 31, 2011, 6:58:59 AM8/31/11

to Sakai Nakamura

Hi, John,
It uses FileHandler to handle imscp file, so I think you can try using
curl to post file to createfile servlet, or upload zip file in UI(may
need to modify newaddcontent.js first).
I will start to work next week, and I'll add headers in the files and
change git workflow as you said, if you get any questions during your
work or find any bugs, please contact me:)

Yushan

Yushan

unread,

Aug 31, 2011, 7:01:52 AM8/31/11

to Sakai Nakamura

Hi, Jon, thanks for your suggestion, I'll try them.

Yushan

> >https://confluence.sakaiproject.org/display/KERNDOC/KERN-517+IMS+CP+I...

> >>https://confluence.sakaiproject.org/display/KERNDOC/KERN-518+IMS+CP+E...
>
> >> IMS-CP & LOM spec docshttp://edutechwiki.unige.ch/en/IMS_Content_Packaginghttp://www.imsglo...

>
> >>http://en.wikipedia.org/wiki/Learning_object_metadata
>
> > --
> > You received this message because you are subscribed to the Google Groups "Sakai Nakamura" group.
> > To post to this group, send email to sakai-...@googlegroups.com.
> > To unsubscribe from this group, send email to sakai-kernel...@googlegroups.com.

> > For more options, visit this group athttp://groups.google.com/group/sakai-kernel?hl=en.

johnk

unread,

Sep 6, 2011, 8:27:37 PM9/6/11

to Sakai Nakamura

Hi Yushan,

I am having difficulty getting the IMS-CP imports to render in the
UI.

Using the Add Content-->Upload Content dialog, I can get an IMS-CP
item to upload and be processed by your code in IMSCPFileHandler.

I can then see the item in the explore Content list of content but
when I click on it, I do not get the hierarchical page structure. It
looks like I just see the first or top level page.

As your code says it is creating a course from the CP, I might expect
to see it the the Explore Courses list of courses but I do not. I
believe this may be because the code is not adding the property
category=courses so the groups_all search is not finding it.

I also thought it might appear in the list of sakai documents shown in
the "add a new area"-->"existing document" dialog in a Course world.
However, it does not appear in that list either, likely because is
does not have the _mimeType property set to x-sakai/document so the
all_all.json search for sakai documents doesn't find it. I notice
your code in the IMSCPFileHandler, at line 170, explicitly removes
that property. Why is that property being removed?

Can you give me any help on what work flow I should be using to render
an uploaded IMS-CP? Or any other assistance that might explain why I
can't get the package to do the full hierarchical rendering of itself
in the UI? What steps do you go through in the UI? Maybe I am
missing something obvious.

For the rendering, I assume that the key client side files and
lhnavigation.js and entity.html. Is this correct or should I be
looking elsewhere in addition?

Thanks in advance for your help.

John K.

On Aug 31, 4:01 am, Yushan <lysha...@gmail.com> wrote:
> Hi, Jon, thanks for your suggestion, I'll try them.
>
> Yushan
>
> On Aug 31, 4:19 am, Jon Gorrono <jpgorr...@ucdavis.edu> wrote:
>
>
>
>
>
>
>
> > Hi Yushan,
>
> > WRT ... examples of content packages, maybe you've already tried this
> > but it might be helpful to try some of the packages under the Content
> > Examples for SCORM 2004 4th since it uses CP v1.1.4
>
> >http://www.adlnet.gov/capabilities/scorm/scorm-2004-4th
>
> > One valuable goal would be the ability to import without getting
> > flummoxed by SCORM extensions.
>
> > Moodle has a long list of content creator tools too... some of them
> > have also passed compliance tests at one point or another as well..
> > but after a quick scan of this list I can't say a tool that would be
> > both compliant and able to produce 1.1.4 CP files popped out at me:
>
> >http://docs.moodle.org/20/en/Creating_SCORM_Content#Tools_for_Authoring
>
> > And if you've not done so already, maybe you can hazard employing the
> > Reload Content Packaging Tool from adl.... it's also listed on at that
> > page @ the url for adlnet.gov above. It's not always clear what you
> > are doing with that tool sometimes, but it might prove useful to you.
>
> > On Tue, Aug 30, 2011 at 3:10 AM, Yushan <lysha...@gmail.com> wrote:
> > > Hi, John,

> > > I implemented importing and exportingIMS CPin the last two months, I
> > > would like to
> > > explain the steps of importing and exportingims cpfiles in the

> > > program first:
>
> > > a) Import cp files
> > > 1) User choose zip files to upload
> > > 2) Program would scan whether there is imsmanifest.xml file, if it
> > > contains that file, it would
> > > be changed to sakai-doc type.
> > > 3) And then program would parse the xml file as a tree structure, if
> > > an item's resource
> > > is text type or directory node, the item would be add to the tree
> > > structure.
> > > 4) If the resource contains local link in its content, it would
> > > changed to our system path, for
> > > example, if there is "images/1.jpg" in the resource, this link would
> > > be replaced by "/p/image_id"
> > > 5) UI would display the file as sakai docs, and user can also download
> > > the origin zip file from
> > > the top.
>
> > > b) Export cp files

> > > Exporting cp files means exporting sakai docs asIMS CPfiles. There

> > > bundle, it doesn't supportims cpextension classes(ipointer, variant,

> > > lingualTitle) now.
> > > 2) I don't think there is much remaining work now, but I really think
> > > we need more examples for testing, I only got two

> > > Meleteims cpfiles from Ian, and I wrote test examples in lom and

> > > content-packaging bundles, but I think we need more actual
> > > examples.
> > > 3) Sure, glad to work with you:) I am an academic visitor in
> > > Cambridge(I am a graduate student in Tsinghua University),
> > > and I am going back this Thursday, but I think I would continue sakai
> > > work as a part time job since John Norman ask me to
> > > continue.
> > > 4) One reason is I didn't know there is LOM API... the other is cp
> > > files from Melete are not strictly standard. For example,
> > > langstring element in melete files is <langstring>content</
> > > langstring>, and its standard binding is <langstring><string>content
> > > </string><lang>en</lang></langstring>; source and value elements in
> > > vocabulary element are CharacterString type in IEEE lom,
> > > but in Melete files, they're langstring type. That also why I said we
> > > need more actual test examples.
> > > 5) I think we should add headers as sakaiproject requires, and I can
> > > do that.
>
> > > Yushan
>
> > > On Aug 30, 12:57 am, johnk <johnfki...@gmail.com> wrote:
> > >> Hi Foks particularly Yushan,
>
> > >> Zach has asked me to look into the state of our development to support

> > >> import and export of IMS Content Packages, akaIMS-CP. Based on the

> > >>IMS-CP& LOM spec docshttp://edutechwiki.unige.ch/en/IMS_Content_Packaginghttp://www.imsglo...

Yushan Li

unread,

Sep 7, 2011, 12:06:36 AM9/7/11

to sakai-...@googlegroups.com

Hi, John,

Are you using the latest code? It seems that your code is an old version, you can see the code from this link. The sakai document type is set at line 186. You can try the latest code.

Best,

Yushan

johnk

unread,

Sep 7, 2011, 7:36:49 PM9/7/11

to Sakai Nakamura

Thanks, Yushan. I didn't have the latest code even though I did
cherry-pick the latest commit, not sure why.

I did copy the latest code from your repo's file but haven't been able
to test it as we have been having build problems on the 1.x branch all
day.

Will let you know later on whether the latest code fixed my problem

John K

On Sep 6, 9:06 pm, Yushan Li <lysha...@gmail.com> wrote:
> Hi, John,

> Are you using the latest code? It seems that your code is an old version, you can see the code from this link (https://github.com/liyushan/open-experiments/blob/master/bundles/file...). The sakai document type is set at line 186. You can try the latest code.

> > On Aug 31, 4:01 am, Yushan <lysha...@gmail.com (http://gmail.com)> wrote:
> > > Hi, Jon, thanks for your suggestion, I'll try them.
>
> > > Yushan
>
> > > On Aug 31, 4:19 am, Jon Gorrono <jpgorr...@ucdavis.edu> wrote:
>
> > > > Hi Yushan,
>
> > > > WRT ... examples of content packages, maybe you've already tried this
> > > > but it might be helpful to try some of the packages under the Content
> > > > Examples for SCORM 2004 4th since it uses CP v1.1.4
>
> > > >http://www.adlnet.gov/capabilities/scorm/scorm-2004-4th
>
> > > > One valuable goal would be the ability to import without getting
> > > > flummoxed by SCORM extensions.
>
> > > > Moodle has a long list of content creator tools too... some of them
> > > > have also passed compliance tests at one point or another as well..
> > > > but after a quick scan of this list I can't say a tool that would be
> > > > both compliant and able to produce 1.1.4 CP files popped out at me:
>
> > > >http://docs.moodle.org/20/en/Creating_SCORM_Content#Tools_for_Authoring
>
> > > > And if you've not done so already, maybe you can hazard employing the
> > > > Reload Content Packaging Tool from adl.... it's also listed on at that
> > > > page @ the url for adlnet.gov above. It's not always clear what you
> > > > are doing with that tool sometimes, but it might prove useful to you.
>

> > > > > To post to this group, send email to sakai-...@googlegroups.com (mailto:sakai-...@googlegroups.com).

> > > > > To unsubscribe from this group, send email to sakai-kernel...@googlegroups.com
>

> ...
>
> read more »

johnk

unread,

Sep 8, 2011, 8:56:24 PM9/8/11

to Sakai Nakamura, nicolaas...@caret.cam.ac.uk

Hi Yushan & Nico,

Still not having any luck getting an imported IMS-CP package to render
in a course.

The steps I took were:
1) Import with "Add Content"
2) Select the zip
3) Upload
4) Go to a course site
5) Click "Add new area"
6) Select Existing Document
7) Select the imported CP
8) Enter "IMS-CP Section" into Placement Title
9) Click create

Result is I get an left hand navigation item IMS-CP Section with no
sub-items.

Is this the correct workflow process?

One possible problem is that the "structure0" property of the IMS-CP
sakai doc looks incomplete
An example - structure0: "{"_poolpath":"/p/iogusNasaa","_childCount":
0}"

whereas a structure0 for a left hand nav item that has sub items looks
like:

structure0: "{"week1":{"_ref":"id74394910","_order":0,"_title":"Week
1","main":{"_ref":"id74394910","_order":0,"_title":"Week 1"}},"week2":
{"_ref":"id74394911","_title":"Week 2","_order":1,"main":
{"_ref":"id74394911","_order":0,"_title":"Week 2"}}}"
sakai:pooled-content-file-name: "Syllabus"

where Week1 and Week2 are sub-items.

Nico, could you confirm for me that structure0 is the data used to
build the hierarchical menu in the left hand nav?

Looking at your code in IMSCPFileHandler, I see the getLeafItems()
method that is used to populate the structure0 property is largely
commented out. I tried commenting that code back in so leaf items are
returned but am still getting an incomplete structure0.

Do you think that this may be the root cause of the rendering problem?

Are you able to get imported packages rendering properly in your
hands?

If so, can you send me a screen shot so I know what to look for?

Do you have any other suggestions for work flow in importing the
packages and selecting them for display, e.g. is the workflow listed
above correct?

I believe I do have your latest client side code in lhnavigation.js
and lhnavigation.html (containing Trimpath template).

Thanks again for your help.

And by the way, if you had a chance to put in the license headers it
would be appreciated. I could then merge that into my cherry-picked
version of your code. Thanks,

John K

> > > > > > > Comments and recommendations are appreciated....
>
> read more »

Yushan Li

unread,

Sep 9, 2011, 4:19:42 PM9/9/11

to sakai-...@googlegroups.com

Sorry about the delay and import error. I just tried again, and the screen shot is shown as the attachment (Import Archive.zip from UI).

First, I think the import steps is right and structure0 is complete, the UI will parse structure0 and add childcount information in the structure.

I think the error may be because you didn't change dev/javascripts/content_profile.js. In UI, I modified devwidgets/entity/entity.html, devwidgets/lhnavigation/lhnavigation.html,

devwidgets/lhnavigation/javascript/lhnavigation.js, devwidgets/newaddcontent/newaddcontent.html

devwidgets/newaddcontent/javascript/newaddcontent.js, dev/javascripts/content_profile.js

Or can you push your code to your git? I'll download them and try to find the problem.

I just pushed imscp codes with license head, I was trying to fork from sakaiproject, but I couldn't build it successfully, I'll try to make forks(UI&backend) from sakaiproject.

Best,

Yushan

--
You received this message because you are subscribed to the Google Groups "Sakai Nakamura" group.

Screen shot 2011-09-10 at 03.52.26.png

Archive.zip

johnk

unread,

Sep 9, 2011, 7:21:18 PM9/9/11

to Sakai Nakamura

> > On Sep 7, 4:36 pm, johnk <johnfki...@gmail.com (http://gmail.com)> wrote:
> > > Thanks, Yushan. I didn't have the latest code even though I did
> > > cherry-pick the latest commit, not sure why.
>
> > > I did copy the latest code from your repo's file but haven't been able
> > > to test it as we have been having build problems on the 1.x branch all
> > > day.
>
> > > Will let you know later on whether the latest code fixed my problem
>
> > > John K
>

> ...
>
> read more »
>
> Screen shot 2011-09-10 at 03.52.26.png
> 236KViewDownload
>
> Archive.zip
> 768KViewDownload

Yushan Li

unread,

Sep 11, 2011, 1:45:53 PM9/11/11

to sakai-...@googlegroups.com

Hi John,

I just found reason that make it fail to show navigation tree, If I replace lhnavigation.html and lhnavigation.js with mine, it would work. I noticed that lhnavigation.js has been updated, and "orderItems" function was modified. I modified IMSCPFileHandler to fix calculating order error and has pushed (So don't have to change lhnavigation.js, just merge the newest code in nakumura.).

About lhnavigation.html, it starts to be different from line 98 with mine, the codes between line 98 to 124 seem have to delete, you can compare with mine codes, thanks.

Best,

Yushan

Yushan Li

unread,

Sep 11, 2011, 11:12:27 PM9/11/11

to sakai-...@googlegroups.com

Sorry, I forgot the new list, and have move the reply to the new list

Best,

Yushan

Reply all

Reply to author

Forward

IMS Content Package, IMS-CP, and Learning Object Metadata, LOM, Work - KERN-517 & KERN-518

johnk

Yushan

johnk

johnk

Jon Gorrono

Yushan

Yushan

johnk

Yushan Li

johnk

johnk

Yushan Li

johnk

Yushan Li

Yushan Li