Convention for Encoding Grade Level in LRMI

[Brandt Redd]

Hi All:

I'm proposing a convention for encoding US grade levels in LRMI. In general and in the spirit of Schema.org, I think that we can establish conventions for things like this without the need for a formal standards process. I'm asserting this one as a test case.

Background:
An early draft of LRMI had a property for “Grade Level.” It’s a useful feature because it’s valuable to indicate the grade level that educational content is targeted at. However, LRMI is an international standard and the K-12 Grade system is a US convention. Instead, the EducationalAlignment property was designated for handling grade levels (along with everything else it does). This makes sense because EducationalAlignment is used for alignment with any kind of taxonomy. The CCSS is a taxonomy; so are Lexiles and other text complexity measures; and so are the 13 levels from Kindergarten to 12th grade.

The trouble is that the LRMI TWG didn’t publish a recommendation on how to encode US grade levels into EducationalAlignment and the people developing tagging tools have in some cases used TypicalAgeRange as a proxy for grade level.

TypicalAgeRange is intended to capture subject interests and appropriateness, not levels of educational attainment. It’s a concept that publishers call “Interest Level” and is typically coded as an age. For example kindergarteners tend to be attracted to bright colors and simple concepts while you need to be in teens or twenties before political debates become interesting.

Proposal:
So, we need a convention for encoding grade level in LRMI. There is a lot of wiggle room and no one approach is necessarily better than another – what’s important is that everyone uses the same convention. Here’s an example convention based on input from the SLC, AEP and Agilix. (Agilix is building a tagging tool for the SLC that is being used by the AEP):

educationalAlignment:
    alignmentType: educationLevel
    educationalFramework: US K-12
    targetName: Grade 5
    targetUrl: http://usk12.org/Grade5

Under this convention, Kindergarten would have the name “Grade K” and the URL “http://usk12.org/GradeK”. The rest are simple to derive from the example above.

Custodianship:

You'll notice the "usk12.org" domain in the proposal. I grabbed that domain to ensure it's available. SETDA has agreed to be the custodian and I'll hand it off shortly (assuming this convention gains a modicum of traction).

Alternative:

After the SLC, AEP, Agilix and I came up with the above framework I learned that the Achievement Standards Network has already developed a similar taxonomy here. And it's encoded in SKOS format here.

Under the ASN framework the above example looks like this:

educationalAlignment:
    alignmentType: educationLevel
    educationalFramework: US K-12
    targetName: Grade 5
    targetUrl: http://purl.org/ASN/scheme/ASNEducationLevel/5

The URLs are, of course, different.
ASN doesn't have a value for educationalFramework so I retained the one from my original proposal. ASN uses "Kindergarten" instead of "Grade K" and they include "Pre-Kindergarten" where my proposal doesn't include that level. They also have values for postsecondary education dividing that into lower-division, upper-division and graduate levels.

Please comment!

-Brandt

[Steve Midgley]

Hey Brandt,

I like the idea of having some semi-custom (localized) definitions rooted someplace on the web as you describe.

My first piece of feedback is that if you go for solution #1, that you add some folders into your path for target URL to give us some ability to add more stuff to that domain. Eg:

http://usk12.org/EducationalLevel/Grade5

Or something like that - not sure what folders would be key just indicating what you're defining in front of the actual key seems helpful..

My feedback on deciding between 1 and 2, would be just "what's wrong with solution #2?" If ASN already has this stuff defined with a long lived URL, why wouldn't we all just use that?

Best,

Steve

[Simon Grant]

In each case, what would be the correct URL for the grade level framework as a whole? There may be several reasons why clarity about this might be helpful.

Thanks

Simon

--
Simon Grant
+44 7710031657
http://www.simongrant.org/home.html

[Brandt Redd]

To Steve's question: I included both options partly because I wanted to represent the work that went into #1 before I found out about #2 and also to see whether there's concern about the ASN branding embedded in the #2 URLs (do we prefer something simpler and generic).

Simon's question relates to Steve's comment that perhaps there should be an intermedia folder. Using Steve's proposal, the URL for the framework as a whole would be "http://usk12.org/EducationLevel". Using my original proposal it would have to be just the bare domain "http://usk12.org" which I think is a weakness.

The ASN framework offers some richness which reflects the effort they put into this:

http://purl.org/ASN/scheme/ASNEducationLevel - The whole scheme that Simon asked about.

And for intermediate levels (or collections of levels)

http://purl.org/ASN/scheme/ASNEducationLevel/GradesPreKto12

http://purl.org/ASN/scheme/ASNEducationLevel/Postsecondary

http://purl.org/ASN/scheme/ASNEducationLevel/VocationalTraining

http://purl.org/ASN/scheme/ASNEducationLevel/LifeLongLearning

http://purl.org/ASN/scheme/ASNEducationLevel/ElementarySchool

http://purl.org/ASN/scheme/ASNEducationLevel/HighSchool

http://purl.org/ASN/scheme/ASNEducationLevel/MiddleSchool

And the base levels

http://purl.org/ASN/scheme/ASNEducationLevel/PreK

http://purl.org/ASN/scheme/ASNEducationLevel/K

http://purl.org/ASN/scheme/ASNEducationLevel/1

http://purl.org/ASN/scheme/ASNEducationLevel/2

http://purl.org/ASN/scheme/ASNEducationLevel/3

http://purl.org/ASN/scheme/ASNEducationLevel/4

http://purl.org/ASN/scheme/ASNEducationLevel/5

http://purl.org/ASN/scheme/ASNEducationLevel/6

http://purl.org/ASN/scheme/ASNEducationLevel/7

http://purl.org/ASN/scheme/ASNEducationLevel/8

http://purl.org/ASN/scheme/ASNEducationLevel/9

http://purl.org/ASN/scheme/ASNEducationLevel/10

http://purl.org/ASN/scheme/ASNEducationLevel/11

http://purl.org/ASN/scheme/ASNEducationLevel/12

Perhaps Stuart can enlighten us whether they debated the merits of "PrimarySchool" and "SecondarySchool" vs. the elementary, middle and high school choices they made.

[Simon Grant]

Many thanks, Brandt, for clarifying this.

Interestingly, as far as I can see the ASN (rich work indeed) scheme URI is actually
http://purl.org/ASN/scheme/ASNEducationLevel/
(with a terminal '/') rather than
http://purl.org/ASN/scheme/ASNEducationLevel
and that strikes me as rather good, because the scheme URI concatenates nicely with a plausible plain term to give the level URI.

This might suggest that your alternative scheme URI might be
http://usk12.org/EducationLevel/
or if you prefer (as I don't see the difference as significant)
http://usk12.org/EducationalLevel/

Best wishes

Simon

[Joseph Chapman]

Here is a non-U.S. example of a machine readable education vocabulary in SKOS from Australia:

http://vocabulary.curriculum.edu.au/schoolLevel.html

Joseph

--
Joseph Chapman

[Jim Goodell]

Brandt, I like the idea and construct and as tweaked with input from Steve & Simon.

(CEDS v3 has additional options for p20levels, such as a proposed option for "grade 13" to cover dual enrolled students, but that is a different use case, not for aligning resources, not an issue. Early learning is more concerned with developmental levels and ages, and both can be covered with typicalAgeRange and EducationalAlignment to the EL developmental framework .) ...It would be good for the framework authority (eg ASN SETDA) to match CEDS definitions for each grade level option. I think ASN list is already based on same NCES Handbook option set from which the CeDS list draws....we would just need to verify exact wording.

[Stuart Sutton]

Hi, Brandt:

This educationLevel vocabulary for the U.S. used by the ASN was developed as part of a U.S. Department of Education project back in 1996 and has been used consistently since in a number of projects--e.g., The Gateway for 21st Century Skills, the ASN, and a slight variant used with the National Science Digital Library (NSDL) [1].  Sorry, Brandt, but I am afraid I don't recall the discussions at the level of specific terms.  I do recall that much of the development was done at Syracuse University during a awesome snow storm. 

The initial framing was done by a group of about 20 people led by the U.S. Department of Ed's ERIC Clearinghouse on Information and Technology and included a rich cross section of representatives from the Department, state education departments, digital repositories of learning resources, and a few K-12 master teachers.  The early drafts relied on both the expertise of the participants and a fairly rich subsequent scan of term usage in practice. That was followed with evaluation of the emerging vocabulary by teacher focus groups before usage of the vocabulary began in early 1997.

I would note that from the beginning of this particular educationLevel vocabulary, the goal was to develop unambiguous identifiers for use by machines (thus the URI).  The human-readable labels were intended to be just that, labels.  When the vocabulary was transitioned from vanilla RDF to SKOS, there was discussion around not privileging any particular label for individual concepts by only using altLabel.  I guess common sense won out.

Brandt, sorry I cannot directly address your question.

Stuart

[1]  http://nsdl.org/contribute/educationLevel

--
Stuart A. Sutton,
CEO and Managing Director, Dublin Core Metadata Initiative
Associate Professor Emeritus, The Information School
University of Washington

[Stuart Sutton]

Brandt, note that the elementary/middle/high is used elsewhere in identifying levels in U.S. K-12.  See [1] & [2].

Stuart

[1]  http://en.wikipedia.org/wiki/Educational_stages#United_States_of_America_and_Canada
[2]  http://en.wikipedia.org/wiki/Education_in_the_United_States#School_grades

On Tue, Nov 6, 2012 at 11:40 AM, Brandt Redd <bra...@redd.org> wrote:

[Mike Collett]

Hi all

I downloaded the ASN levels in skos format

 encoded in SKOS format here

and tried to look at it in my editor but there seem to be a few problems and it raises a few questions for me.

This may not be incorrect but the concept scheme and the top concept have the same uri http://purl.org/ASN/scheme/ASNEducationLevel/

Some uris seem to have errors which means the structure breaks, eg

<skos:narrower rdf:resource="http://purl.org/ASN/scheme/ASNEducationLevel/GradesPreKto12"/>

...

<skos:Concept rdf:about="http://purl.org/ASN/scheme/ASNEducationLevel/PreKto12">

and

<skos:narrower rdf:resource="http://purl.org/ASN/scheme/ASNEducationLevel/PreK"/>

..

<skos:Concept rdf:about="http://purl.org/ASN/scheme/ASNEducationLevel/Pre-K">

and

<skos:narrower rdf:resource="http://purl.org/ASN/scheme/ASNEducationLevel/VocationalTraining"/>

..

<skos:Concept rdf:about="http://purl.org/ASN/scheme/ASNEducationLevel/VocationalEducation">

Postsecondary has only one narrower concept (Higher Education). Does this make it redundant or should it also include narrower concepts of Professional education and Vocational education?

Both Life-long learning and Vocational education, and possibly Professional Education, seem to be more to do with the context of learning rather than the level at which the learning is intended to  take place.

Is the intention to allow use of all the concepts that have preLabels in targetUrls and targetNames? It might be better to only include the most granular ones, preferably only those that are mutually exclusive and deal with the, often very contextualised, relationships to other concepts elsewhere.

Presumably Middle school, for example, implies all of grades 6,7and 8 are covered by a resource. Perhaps better to just give the levels 6,7 and 8 to avoid confusion and possible contradictions, eg what does Middle school and 8 mean?

Cheers
Mike 7:-D
-----------
Mike Collett, Schemeta
+44 7798 728 747
------------
www.schemeta.com
email: mi...@schemeta.com
twitter: @schemeta
skype: mikecollett

people are the network

[Stuart Sutton]

Thanks, Mike.  We'll check these out.

Stuart

[Jim Goodell]

CEDS intentionally avoid labels like "elementary", "middle" except for limited use cases.  There is too much variability in the way schools are configured and in the way these terms are applied in practice. State education agencies and school districts do not agree with the National Science Digital Library (NSDL) definitions (or any definitions which limits an entire school level by what the low and high grades are). Better to stick with a list of grades than try to label groups of grades. 

As I think about this more... it is worth recognizing that grade levels are a poor proxy for developmental levels unless there is universal acceptance of common grade-level expectations.  Better to link resources to standards and let standards define common grade level expectations.  If by promoting the convention for grade levels to LRMI some publishers choose to tag using educationalAlignment for grade levels instead of to competency-based standards, that could be a step backward.

[Stuart Sutton]

On Tue, Nov 13, 2012 at 6:06 AM, Jim Goodell <james.dona...@gmail.com> wrote:

CEDS intentionally avoid labels like "elementary", "middle" except for limited use cases.  There is too much variability in the way schools are configured and in the way these terms are applied in practice. State education agencies and school districts do not agree with the National Science Digital Library (NSDL) definitions (or any definitions which limits an entire school level by what the low and high grades are). Better to stick with a list of grades than try to label groups of grades. 

Jim, absolutely no classification scheme will garner universal agreement that it is the "right" way to slice and dice up a particular corner of the universe.  I don't have to agree with Getty's world view of art and architecture to use concepts effectively from its Art & Architecture Thesaurus.    The ASN uses a thesaural structure for US education levels concept scheme and nevertheless assigns specific grade levels (and groups of specific grade URI) that are relevant to a specific ASN competency and does not assign URI of the aggregate levels.   Others might make a different decision of best practice.  The thesaural structure operates in the background should someone (some system) want to use that structure for other purposes (aggregating, structuring views etc.).  If we had to have universal agreement on how to slice and dice aspects of the world before creating such knowledge organization systems, we'd have no classification schemes, no thesauri, etc...  Ask librarians whether they "agree" with how the Dewey Decimal Classification sees the world or how the Library of Congress sees it in its classification scheme :-)
 

As I think about this more... it is worth recognizing that grade levels are a poor proxy for developmental levels unless there is universal acceptance of common grade-level expectations.  Better to link resources to standards and let standards define common grade level expectations.  If by promoting the convention for grade levels to LRMI some publishers choose to tag using educationalAlignment for grade levels instead of to competency-based standards, that could be a step backward.

Jim, I agree with you as to what is perhaps "better"; but, I think we simply cannot second guess every use case and have to leave the options open:

learning resource==>education level
leaning resource==>competency==>education level

Stuart

[Kelly Peet]

For what it's worth, +1 on the suggestion that the standard linkages define grade level expectations.

At Academic Benchmarks (AB) we have more than 2.8M standards collected from hundreds of authorities world wide.  With each authority curated into our collection, we endeavor to faithfully represent the grade/age level intended by the author, along with the local labels (e.g.,. for a time, Arizona referred to Kindergarten by the label "Readiness").  We do the same with subject area, strands, and several other vocabulary-controllable pieces of metadata.  In addition, we maintain maps between our internal codification to our clients (i.e. private codifications) as well as to the ever-growing litany of public representations of the same, such as those mentioned/proposed in this thread.

Our use cases include answering questions like "what is the min/max/median grade/age level in which this concept is taught at this cognitive level".  Or, "for this concept, what cognitive level and grade level is asserted locally for this authority (i.e. when should it be taught and to what degree?)"  And, ultimately, to emit from our collection such data points using the representation adopted by our clients.

Even on the arguably banal topic of grade and subject codification (which barely touches on the complexity of the conceptual matter mentioned in education standards), there is a lot of industry fracture and alternative representations.  To wit, on quick check, to date we have encountered more than 80 different permutations of grade ranges on non-course-based leveling (K-2, preK-2, K-3, 5-6, 6-8, 5-8, etc).  The number jumps an order of magnitude (or several) when considering permutations of high school courses, the interminable local labels, and alternate grade range associations between and among them.

Simply, +1 on Jim's recommendations.
kelly

[Douglas Levin]

Long-time lurker on the list, first time poster. By way of introduction, my name is Doug Levin and I the executive director of the State Educational Technology Directors Association. This puts me squarely in the U.S. K-12 arena for my perspective.

Jim, I concur re: CEDS approach to avoid ES, MS, HS labels as there is a great deal of variability in how those are defined across the U.S., to say nothing of beyond our borders.

With respect to your second point, I think that both federal education policy - which dictates annual grade-based testing for accountability purposes - and many content standards statements themselves (such as those contained in the Common Core) are explicitly based on a grade-level determination. I concur with your observation that grade-level is a poor proxy for developmental level, but I do not see harm in a grade-based tag and some significant downsides to omitting it from my perspective (even if imperfect).

On Tue, Nov 13, 2012 at 9:06 AM, Jim Goodell <james.dona...@gmail.com> wrote:

[Joshua Marks]

Kelly et al,

 

80 permutations of US K-12 grade levels says it all. I am going to re-join the baby here and say everyone on this topic is right. I do agree that artificial leveling hinders natural learning progression. In that I favor Jim’s proposal to use the framework to define the level of materials. However, Kelly is also correct that the primary use case is finding stuff at the level the end user is interested in, and the way they define it. So from a LRMI tagging perspective, Stewart is correct, we need both a leveling alignments and topical (Framework) alignments to make the search use case at the heart of LRMI work well. The latter (Standards frameworks) can, but does not necessarily include grade level expectations (GLEs to some). If the standards does then the GLE can and should be mapped to a specific grade level as a LRMI tag. But from a tagging perspective both should exist simply because having grade level be implied by the standard alignment makes the search by level only use case fail. So this make Doug right too (Welcome to the party Doug!)

 

Now for a best practice, even though Curriki uses grade ranges presently, it would be best when creating LRMI tags to innumerate each grade level that is aligned rather than a rage or a proxy for a range like “Upper Elementary”, “Middle School”, etc. But again, the dominant community of use is the one that defined these levels and their terms to reference them (Making Stewart right again).  

 

Harmony achieved.

 

Joshua Marks

CTO

Curriki: The Global Education and Learning Community

jma...@curriki.org

www.curriki.org

 

I welcome you to become a member of the Curriki community, to follow us on Twitter and to say hello on our blog, Facebook and LinkedIn communities.

[Jim Goodell]

Good mediation Joshua, and good points Stuart, Kelley, and Doug (glad to have you join the discussion). I'll concede that multiple options are best for LRMI.

...maybe it is a matter of communications to inform publishers that their resource will be more discoverable if they tag by enumerating each applicable grade level rather than (or in addition to) the proxy for a grade range such as "elementary", and even more discoverable if also tagged with standards, and even more discoverable if aligned with more granular micro standards. ...or maybe simple/concrete messaging: for best discoverability tag with both 1) the list of grade levels AND 2) alignment to all applicable learning standard items. Perhaps this messaging can be part of whatever is published as the convention for grade level alignment.

-jim

[Douglas Levin]

I would endorse that approach.

[Kelly Peet]

A slight tangent to the point, more of a question to the original thinkers of LRMI's design, regarding Jim's point #2 as it relates to scalability:

Presuming a piece of content being tagged is applicable to many authorities (for argument sake, let's stay with 50 states for now), it would seem necessary to cite 50 URIs to assert and meet the requirement of "all applicable learning standard items".  My mind runs in a variety of directions:

- what about content that applies to more than one learning standard per authority (or to a portion of a learning standard)?

- what is the ultimate size of the payload making such assertions?

- did i read earlier that hidden-only markup may not be considered by search engines, so all these assertions must appear on the page?

- i hear Common Core pointed as one possible antidote to 50 states, but wonder about subjects besides Math and Language Arts, as well as the numerous varieties of state-specific Common Core standards (possibly making the presumptive 50 more like 100).

I completely understand if this line of questioning is quelled on this thread.  Just thinking one step down the road how to support LRMI while also being keenly attuned to the volume and scalability issues and how to address them.  We have a couple million content items tagged tightly to hundreds of authorities, more than 400M assertions across 10 subject areas.  Looking for pearls of wisdom.
kelly

[Mike Collett]

Hi Kelly

And these often change when a curriculum changes, for example new concepts, deprecated items or new structures.

Also there are multilinguality requirements that mean that translations can come online frequently. 

A question is: where are mappings and such changes managed?

To put all this in the metadata is asking for trouble and, as you imply, gives rise to massive legacy issues.

Our experience is that you try to keep the metadata to a minimum (unique ids plus perhaps one preferred label) and deal with the mappings, translations, navigation structures and change control via a separate service.

Cheers
Mike 7:-D
-----------
Mike Collett, Schemeta
+44 7798 728 747
------------
www.schemeta.com
email: mi...@schemeta.com
twitter: @schemeta
skype: mikecollett

people are the network

[Kelly Peet]

Mike,
comments in line.

On Wed, Nov 14, 2012 at 11:04 AM, Mike Collett <mi...@schemeta.com> wrote:

Hi Kelly

And these often change when a curriculum changes, for example new concepts, deprecated items or new structures.

Also there are multilinguality requirements that mean that translations can come online frequently. 

good points. in addition to curriculum, academic standards change in various ways on a variety of schedules.
 

A question is: where are mappings and such changes managed?

To put all this in the metadata is asking for trouble and, as you imply, gives rise to massive legacy issues.

put to the industry, i suspect the general answer to your question is some brand of content management system (CMS).   specifically, for Academic Benchmarks, we interact with a variety of CMS platforms based on our client's specific choices, while internally keeping track of content by tagging to a multi-faceted conceptual framework (cognitive, concept, constraint, limit, etc), which is, in turn, mapped to standards, allowing us to escape the legacy issues that inevitably arise with onboarding fleeting metadata, as you suggest.  standards alignment is a byproduct of our technique, and i might suggest a bit of a red herring in the discussion at hand.

Our experience is that you try to keep the metadata to a minimum (unique ids plus perhaps one preferred label) and deal with the mappings, translations, navigation structures and change control via a separate service.

exactly our approach.  minimum viable description for the intended audience, robust, real-time resolution services.

thanks for chiming in,
kelly

[Brandt Redd]

To summarize the main branch (I think Kelly's tangent may need more exploration), I think we've reached a rough consensus as follows:

1. Grade level markup is valuable. We have a preference for standards type alignment but many of our search use cases will still use grade levels (at least until Competency-Based Education becomes the norm).

2. Searchability will be optimized if a resource is tagged with all of the relevant grade labels (e.g. Grade 2, Grade 3, Grade 4) and not with ranges (e.g. Grades 2-4). The same argument applies to defined aggregates like "Elementary School" not to mention their inconsistent definitions.

3. The core set of level tags will include the 13 grades from K to 12. There are use cases where PreK and Post12 might also be valuable.

4. The URL schema should have an intermediate level to provide for future additions to the domain. The ASN URL schema already has this (e.g. "http://purl.org/ASN/scheme/ASNEducationLevel/5") while the proposed usk12.org schema would be enhanced (e.g. "http://usk12.org/EducationalLevel/Grade5").

5. The basic application of LRMI markup from my original post holds except for some open questions.

First off, I'm interested in any objections to what seems to be consensus on these points.

Second, we need to address the following open questions:

Q1. What URLs should be used? Since the ASN URLs already exist, I propose using them. Jim mentioned that there is a grade level enumeration for CEDS. I'm curious what the values are and whether they have (or could have) a URL form.

Q2. What should be the textual (non-url) form for Kindergarten? It could be "Grade K" for consistency with the other grades or it could be the word "Kindergarten." This might be dictated by the answer to Q1. ASN uses "Kindergarten."

Q3. Should PreK be included and if so what should the textual form be? Candidates include "Pre-Kindergarten" and "PreK". ASN uses "Pre-Kindergarten".

-Brandt

[Monty Swiryn]

Brandt,

Good summary. This sounds like what I was trying to promote in earlier discussions... Glad to see that your version is getting traction.

Regarding your questions, esp. Q2 and Q3, an important issue to consider is that teachers may be using different "spellings" for a grade level when they search for products.
Ex 1: Grade K, Kindergarten
Ex 2: Pre-K, PreK, Pre-Kindergarten, PreKindergarten
Ex 3: Grade 1, First grade

So deciding on one "value" for the text of the grade level may not be enough as far as the search engines are concerned...  How will the search engines know to include "Grade 1" in a search for "First grade"?

best,
Monty

[Mike Collett]

Hi Brandt

In response to 

"First off, I'm interested in any objections to what seems to be consensus on these points."

I agree with 1, 2, 3. Not quite so sure about 4 and 5.

Not so much an objection more a comment.

There seems to be a mixing up of identification and resolution and putting lots of assumptions about syntax and semantics in the URL. Sometimes this can be useful sometimes it is constraining and detrimental.

For example, to re-use some level identifiers to increase interoperability,  authorities may build their own frameworks by reusing schema.org's, or some other set of, identifiers for levels. This maybe to classify differently, to add levels or granularity or to change or translate the preferred labels, such as in Stuart's SKOS.

It is useful to identify the framework, and hence that set of acceptable levels in context.

In your earlier example:

educationalAlignment:
    alignmentType: educationLevel
    educationalFramework: US K-12
    targetName: Grade 5
    targetUrl: http://purl.org/ASN/scheme/ASNEducationLevel/5

..it would be beneficial to have a uri for the framework rather than assume it from the target url. In fact it may be better to be able at least to specific this as a URI (eye) rather than URL (ell) even if it is expressed as http.

So if tagged in England this might then become something like

educationalAlignment:
    alignmentType: educationLevel

    educationalFrameworkUri: http::www.gov.uk/DfE/NationalCurriclulumEngland/Levels  (fictitious uri)
    educationalFrameworkName: School Levels in England 

    targetName: Level 6 (not really accurate mapping but there to show that a US grade 5 may neither be grade nor 5 universally)
    targetUr*i*: http://purl.org/ASN/scheme/ASNEducationLevel/5

Or if an organisation in the Nederlands wanted to re-use the schema.org ids:

    alignmentType: educationLevel

    educationalFrameworkUri: http://purl.edustandaard.nl/Leerniveauaanduidingen (fictitious uri)
    educationalFrameworkName: lang=nl Algemeen leerniveau - Leerniveauaanduidingen

    targetName: PO groep 6
    targetUr*i*: http://purl.org/ASN/scheme/ASNEducationLevel/5

Cheers

Mike 7:-D
-----------
Mike Collett, Schemeta
+44 7798 728 747
------------
www.schemeta.com
email: mi...@schemeta.com
twitter: @schemeta
skype: mikecollett

people are the network

[Stuart Sutton]

Mike, I am very uncomfortable with the approach you suggest.  Comments below.

On Fri, Nov 16, 2012 at 3:22 AM, Mike Collett <mi...@schemeta.com> wrote:

 

In fact it may be better to be able at least to specific this as a URI (eye) rather than URL (ell) even if it is expressed as http.
 

So if tagged in England this might then become something like

educationalAlignment:
    alignmentType: educationLevel

    educationalFrameworkUri: http::www.gov.uk/DfE/NationalCurriclulumEngland/Levels  (fictitious uri)
    educationalFrameworkName: School Levels in England 

    targetName: Level 6 (not really accurate mapping but there to show that a US grade 5 may neither be grade nor 5 universally)
    targetUr*i*: http://purl.org/ASN/scheme/ASNEducationLevel/5

Or if an organisation in the Nederlands wanted to re-use the schema.org ids:

    alignmentType: educationLevel

    educationalFrameworkUri: http://purl.edustandaard.nl/Leerniveauaanduidingen (fictitious uri)
    educationalFrameworkName: lang=nl Algemeen leerniveau - Leerniveauaanduidingen

    targetName: PO groep 6
    targetUr*i*: http://purl.org/ASN/scheme/ASNEducationLevel/5

Mike, as a member of the team that created the ASN education level scheme back in 1995-6 and helped shepherd it through to it's current SKOS/RDF form for use in linked data, I think I can speak for my colleagues in saying that this general use of the scheme to denote (from your examples) anything "5-ish" in terms of  global education level expressions is not one we could support.  Having such a globally useful scheme would certainly be nice as some kind of universal "glue"; but, the ASN scheme is not that scheme.

While I know we are talking about LRMI and not ASN, I'd note that as a framework for describing competencies, ASN assumes that education level vocabularies will be numerous (as many as there are national authorities and customary national uses), precise and expressive of national prerogatives.  The scheme being discussed here is exactly that, a semantic expression of U.S. circumstances. 

I would much rather see England and The Netherlands bite the bullet and define their own education level schemes identified by URI with concepts identified by URI and follow-on with useful mappings between/among the schemes or mappings of education level concepts in the many schemes to some logical, common intermediary vocabulary (perhaps typical age like we see at [1]).   That would certainly make for a very nice project for foundation/NSF/EU funding since such efforts are nation/international in scope and involve fundamental data infrastructure!

In addition to the U.S., Australia has already defined it's education levels [2]-[3] in machine-readable form (among other vocabularies necessary to its schools sector [4] for use in a linked data environment).  ASN-US has a pending action to formally express the mapping between the U.S./Australia schemes (and is looking at the utility of a more universal "intermediary" mapping in that process). 

Stuart

[1]  http://en.wikipedia.org/wiki/Educational_stages
[2]  http://vocabulary.curriculum.edu.au/schoolLevel/export/schoolLevel.rdf
[3]  http://vocabulary.curriculum.edu.au/schoolLevel.html
[4]  http://vocabulary.curriculum.edu.au/

[Paul Libbrecht]

Mike,

I find your approach very pragmatic and almost appropriate.

Depending on who would produce such an annotation and how, it might indeed be quite imprecise but that happens.

E.g. Curriki only has pairs of school-years as educational levels...

I'll note however that generality has the advantage of being both easy to input and (hence search) and easy to compare.

Stuart,

comparability is really at heart here.

While I agree a precise URI would be really really desirable, you have to accept that in many regions, the precise naming of a level is *too precise*.

To be precise to the curriculum, in i2geo.net, we have encoded as many concrete levels we know existed in several countries. And the result so fine that the levels of my region are not comparable with that 50km away.

So if someone finds a resource on i2geo.net for my region, he might be sure he's a friend ;-).

So while I agree it is desirable to have specificity, it must happen with methods of generalization and comparison. These methods should be able to generalize a class of my region to a class of other regions of Germany and maybe to any learner of the same age... 

Let me insist, however, that these are not mappings!

It is impossible to map, for example, the levels in France to those Germany... except in terms of ages of the students. That is... the curriculum standards are fully different.

paul

PS: more about this initiative to encode educational levels at: http://i2geo.net/About/GeoSkills

[Mike Collett]

Paul

.. nice to be almost appropriate

... if you please some of the people all the time the chances are you might upset everyone else most of the time.

Stuart

Thanks for the comments.

England and the Nederlands have had levels for a long time, as have most countries, states, territories and regions. I was trying (perhaps unclearly) to illustrate reuse rather than suggest that is the way to apply LRMI levels in UK or European contexts. Most likely they would tag with their own frameworks of which as you say there are many many - but users might prefer to increase interoperability by reusing some existing levels, for example at some stage decide to use European wide identifiers within a localised framework, or US identifiers in a state framework.

This thread is about how to apply US specific levels in LRMI. It is not up to me how the US community choose to do this.

When Brandt assumed "we've reached consensus" I was not sure which we is we ;-)  LRMI or US LRMI.

A concern is that these examples are seen by this list, and beyond, as best practice exemplars or recommendations on how to apply levels in LRMI. 

In my opinion it has some weaknesses as an exemplar and so I added an element of mild dissent to the emerging consensus.

1. It seems to suggest that the identifier or URL of a framework should or could be implicit in the targetURL

2. it does not give a URI or URL for the educationalFramework

The outcome of this approach makes it difficult to reuse level identifiers from one framework in another localised framework.

As for mapping ...

Cheers
Mike 7:-D
-----------
Mike Collett, Schemeta
+44 7798 728 747
------------
www.schemeta.com
email: mi...@schemeta.com
twitter: @schemeta
skype: mikecollett

people are the network

[Stuart Sutton]

On Fri, Nov 16, 2012 at 8:41 AM, Mike Collett <mi...@schemeta.com> wrote:

Stuart

Thanks for the comments.

England and the Nederlands have had levels for a long time, as have most countries, states, territories and regions. I was trying (perhaps unclearly) to illustrate reuse rather than suggest that is the way to apply LRMI levels in UK or European contexts.

Mike, I wasn't meaning to imply that most countries and their political subdivisions (no matter how granular) don't have recognized education-level frameworks (the Wikipedia pages I've been referencing demonstrate they do have such generally recognized frameworks).  A people can't really discuss their education systems formally or informally without some generally acceptable means of referencing education level (as the people see it).  But I am not aware of any European schemas being expressible in machine-readable form on the open web (i.e., RDF)...or at least I have not seen them...would love to see them if they exist.  My objection was in the "reuse" of the ASN URI in a non-U.S. jurisdiction given the narrow U.S.-centric semantics of the scheme. 

But, forgetting the U.S. context, it would be nice to see examples using different education level frameworks from Europe and elsewhere and, Mike, how things like your <educationalFramework[Uri]> and <targetUr[i]> play out.
 

Most likely they would tag with their own frameworks of which as you say there are many many - but users might prefer to increase interoperability by reusing some existing levels, for example at some stage decide to use European wide identifiers within a localised framework,

or US identifiers in a state framework.

Yes, to the extent such European-wide education level frameworks can function as the glue tying to national and localized education level frameworks, that would be very useful (given machine-readable meas for referencing such mappings) -- Local==>National==>Europe-wide.  Is there any work going on in this area--i.e., machine mapping local to European-wide frameworks? 
 

This thread is about how to apply US specific levels in LRMI. It is not up to me how the US community choose to do this.

When Brandt assumed "we've reached consensus" I was not sure which we is we ;-)  LRMI or US LRMI.

Ah, yes...sometimes the "we" can be confusing in LRMI discussions.  I'm am hoping when looking at general best practice that the "we" is global (the search engines to be served certainly are).  
 

 

A concern is that these examples are seen by this list, and beyond, as best practice exemplars or recommendations on how to apply levels in LRMI. 

In my opinion it has some weaknesses as an exemplar and so I added an element of mild dissent to the emerging consensus.

1. It seems to suggest that the identifier or URL of a framework should or could be implicit in the targetURL

Not a reliable assumption...  While Brand't example uses a literal with "educationalFramework: US K-12", scheme URI would fulfill this need.
 

2. it does not give a URI or URL for the educationalFramework

True, not in Brandt's example.  Also, there is some dissonance between Brandt's example at the start this thread and what we see in LRMI 1.0 at http://www.lrmi.net/the-specification/alignment-object where educationalFramework doesn't exist.  I have trouble keeping up fully with the discussion.
 

The outcome of this approach makes it difficult to reuse level identifiers from one framework in another localised framework.

As for mapping ...

Yes.
 

Stuart

[Joshua Marks]

Mike et al,

 

Oh so many issues live in this one thread. I will however stick to your one point about exemplars and best practices. Clearly Brandt is focused on the US-K-12 challenge and is using it as a example in implementation. His ‘we’ is the local “We” in terms of the specific vocabulary and look-up. His ‘we’ is the “we-all” we when considering implementation best practices.

 

------------ Snip -----------

 

A concern is that these examples are seen by this list, and beyond, as best practice exemplars or recommendations on how to apply levels in LRMI. 

In my opinion it has some weaknesses as an exemplar and so I added an element of mild dissent to the emerging consensus.

1. It seems to suggest that the identifier or URL of a framework should or could be implicit in the targetURL

2. it does not give a URI or URL for the educationalFramework

 

--------- End Snip --------

 

Your insertion of URI and reference to the framework itself is rational _IF_ LRMI’s focus were machine discovery of this more semantic or formally structured type of framework relationship. However, this is not the objective, and it is not clear that any of the search engines would utilize a referenced URI framework in their search implantation. The objective is an ‘easy’ to implement tagging approach that web masters hosting learning resources can all deploy en-mass to mutual benefit in the web discovery use case for learning resources. You have to start with the assumption that users are entering raw search terms. The value of the framework alignment is to associate the words used to describe the resource with the resource, as very frequently the content is not self-descriptive in a meaningful way for search. Now supporting a URL that can infer a URI stricture and a framework reference is not precluded, it is just not required or explicit in LRMI.

 

We discussed at some length in the Technical Working Group URIs vs. URLs for framework alignments, and it was concluded that the more formal and semantic tagging was not helpful to the web search and discovery use case and overly complicated the implementation for webmasters. It can not be dependent on the existence of such URI frameworks, which often do not exist. It however is VERY useful for the machine selection and automated assembly use cases others are interested in.   LRMI can still support this if you can trust the URL link provided and infer it as a URI associated with a specific framework. This is why, in part, we included (Or are including) the alignment authority.

[Jim Goodell]

I'm overdue in responding to Brandt's Q1 further up the thread about what is in CEDS.  

CEDS has enumerations of grades, used in various contexts, but no URL. So if a the convention will be a URL then ASN or some alternative wins.  

Of course we'd like the CEDS version of grades that can be aligned to a learning resource the same as what is used by LRMI.  For the core K - 12 grades CEDS "Description" and ASN's "atlLabel" seem to match. We have some other levels that are different, e.g. CEDS has Preschool, Prekindergarten, and Transitional Kindergarten as separate options. However, in CEDS we can say that for the use case of learning resource metadata "Transitional Kindergarten" is not used.

Right now CEDS has this list of grade levels for K12 enrollment:

Description		Option
Infant/toddler		IT
Preschool		PR
Prekindergarten		PK
Transitional Kindergarten		TK
Kindergarten		KG
First grade		01
Second grade		02
Third grade		03
Fourth grade		04
Fifth grade		05
Sixth grade		06
Seventh grade		07
Eighth grade		08
Ninth grade		09
Tenth grade		10
Eleventh grade		11
Twelfth grade		12
Grade 13		13
Postsecondary		PS
Ungraded		UG
Other		Other

We also have a list for 'last grade completed' used by data elements with a broader P20W context, from the NCES Handbooks:

Description		Option
No school completed		01043
Preschool		00788
Kindergarten		00805
First Grade		00790
Second Grade		00791
Third Grade		00792
Fourth Grade		00793
Fifth Grade		00794
Sixth Grade		00795
Seventh Grade		00796
Eighth Grade		00798
Ninth Grade		00799
Tenth Grade		00800
Eleventh Grade		00801
12th grade, no diploma		01809
High school diploma		01044
High school completers (e.g., certificate of attendance)		02408
High school equivalency (e.g., GED)		02409
Career and Technical Education certificate		00819
Some college but no degree		01049
Formal award, certificate or diploma (less than one year)		01047
Formal award, certificate or diploma (more than or equal to one year)		01048
Associate's degree (two years or more)		01050
Adult education certification, endorsement, or degree		73063
Bachelor's (Baccalaureate) degree		01051
Master's degree (e.g., M.A., M.S., M. Eng., M.Ed., M.S.W., M.B.A., M.L.S.)		01054
Specialist's degree (e.g., Ed.S.)		01055
Graduate certificate		01052
Doctoral (Doctor's) degree		01057
First-professional degree		01053
Post-professional degree		01056
Other		09999

(This list has issues for the use case because of things like "12th grade, no diploma")

[Brandt Redd]

It looks like my push for consensus has shaken a few things loose. First, I need to reply to Mike's queries and suggestions with some details about my intentions and also the intent of the LRMI Technical Working Group.

First, intentions from the LRMI Technical Working Group as I understand them (other TWG members, please object if I don't get this right):

• The targetURL field is intended to be used as a URI. We are following conventions found elsewhere in Schema.org in using the URL acronym. It's supposed to unambiguously identify the target concept and it SHOULD be possible to derive the framework from the URI. In other words, the framework and the element withing the framework should both be identified by the URI and URIs should not be reused between frameworks.

• With the exception of alignmentType, the other properties of AlignmentObject (targetName, targetDescription, educationalFramework) are denormalizations of data that is associated with the URL and theoretically could be retrieved from a database. This denormalization is necessary 1) because major search engines won't dereference URIs or URLs to get at the details and 2) because there isn't a convention for encoding or distributing the metadata even if the search engines would look it up.

• educationalFramework is a late addition to AlignmentObject -- it doesn't exist in the LRMI 1.0 specification but is documented in the draft submission to Schema.org. It was specifically added for instances where URLs/URIs have not been defined for a particular framework. For example, in the absence targetURL, targetName="Grade 5" is ambiguous. It could be U.S. Grade 5 or U.K. or the Netherlands or some other framework. Adding the name of the framework disambiguates just as adding a URL does. So, someone tagging content could use targetUrl or educationalFramework. Using both is redundant (though still good practice). So, using an URI for educationalFramework is unnecessary (since you SHOULD be able to derive the framework from targetUrl) and counterproductive (since the purpose of the field is for situations where a URI has not been defined).

• An early version of the LRMI draft had a simple text field for grade level. That was rejected because grade level definitions and conventions differ from country to country. For that reason, the LRMI TWG indicated that AlignmentObject should be used to align to national or local taxonomies of grade levels. That's all fair except that it's not the LRMI TWG's responsibility to define those taxonomies -- hence the push for a convention in the broader community.

Now for my own intentions:

• If I had it to do over, this thread would be titled "Convention for Encoding U.S. Grade Level in LRMI." I'm attempting to fill the gap left by the LRMI TWG by defining the grade leveling convention for the United States.

• While this is a push for a U.S. convention. It should serve as a model for good practice in defining conventions for other countries.

-Brandt

[Kelly Peet]

Jim, Brandt, et. al.,
As I look at the list of grades for CEDS, I think:  Great, now I can associate a CEDS grade with other existing grade code schemes already captured (NCES, ASN, etc).  The lack of a URL doesn't detract a bit.  If I encounter a CEDS-tagged resource, and want to know how it compares to other grade schemes, such a grade-to-grade association can offer a rough translation (close enough?).  The topic at hand raises some questions for me:

1)  I might suggest it would not be difficult for CEDS to create something parallel to Brandt's current proposal, along the lines of:

http://ceds.ed.gov/educationLevel/Grade1, etc.

or, not implicating CEDS, Academic Benchmarks could establish something like:

http://codes.academicbenchmarks.org//educationLevel/Grade1, etc.

Or, more expansively, perhaps any organization with a grading scheme could establish such a set or URLS, but wouldn't the issue then become sense making among all the various schemes?  And, does the presence of a URL alone make a scheme somehow more valuable?

The initial proposal was leading toward a new scheme hosted at usk12.org (yet another), and has evolved to picking an existing ASN scheme (pick one).   I am trying to unseat a growing concern in the back of my mind that neither approach ("yet another" nor "pick one") resolves the underlying use case here.

To wit, isn't the underlying use case _already_ "sense making among various schemes"?   This seems like it will be true for Subject and other codes, and eventually competency statements (i.e., there will probably never be ONLY ONE).  Even if the URLs are nothing more than unique identifiers for denormalized data, if the chosen scheme doesn't have a category appropriate to the resource being tagged, it seems like it could hold up LRMI tagging efforts, or at least force a different coding scheme to enter the scene.  I dare suggest this is how there are so many coding schemes in existence today (i.e., the ones in existence don't cover all the bases, or maybe are parts of a monolithic or somewhat intractable scheme (e.g. SIF), and it is not too hard to craft one for someone's local purpose and translate to the others).

2) From some brief experimentation, it seems that all of the following links resolve to the exact same document:
http://purl.org/ASN/scheme/ASNEducationLevel/
http://purl.org/ASN/scheme/ASNEducationLevel/1
http://purl.org/ASN/scheme/ASNEducationLevel/2
etc

What semantic difference is imparted when different grade links all resolve to the same master document?  The answer seems to lie in unpacking the document.  Or, perhaps the idea (for now) is that these URLs are not planning to be unpacked, and the question is moot.

At some point, unpacking could emerge.  Then, what if the user does not wish to resolve to RDF (with SKOS and DC components)?  As Brandt notes, there is no adopted best practice for this, so it remains a bit of the wild west.

3) I think the point Jim makes about "hoping CEDS codes are used by LRMI" buttresses my point that the underlying use case may not be in choosing one scheme over another (i.e. if CEDS codes are picked, who would be hoping it was theirs instead). To me, this suggests a viable solution lies in committing some energy to translating between and among numerous, defacto schemes.

At Academic Benchmarks, in connection with our Learning Registry discussions, we attempted just such a translation service.  NOTE: It is for Common Core Standards (not grades), but I think there are some strong parallels:

http://academicbenchmarks.com/standards-translator

We continue to track the evolving identifiers for various versions of the Common Core State Standards (CCSS).  On initial release, we established our own GUIDs for each standard.  A couple years later, the CCSS authors released GUIDs of their own, dot notation, etc.  The evolution of the CCSS identifiers strikes me as very parallel to the grade discussion taking place now.  You can't erase history or abolish all except one set, so one sane response is to create some brand of translation.

Am I completely missing the point? 
kelly

[Stuart Sutton]

On Tue, Nov 20, 2012 at 4:40 AM, Kelly Peet <ke...@academicbenchmarks.com> wrote:

1)  I might suggest it would not be difficult for CEDS to create something parallel to Brandt's current proposal, along the lines of:

http://ceds.ed.gov/educationLevel/Grade1, etc.

or, not implicating CEDS, Academic Benchmarks could establish something like:

http://codes.academicbenchmarks.org//educationLevel/Grade1, etc.

Or, more expansively, perhaps any organization with a grading scheme could establish such a set or URLS, but wouldn't the issue then become sense making among all the various schemes?  And, does the presence of a URL alone make a scheme somehow more valuable?

Kelly, I'd claim that the presence of a URI (identifier) is more valuable for machines than text. 
 

The initial proposal was leading toward a new scheme hosted at usk12.org (yet another), and has evolved to picking an existing ASN scheme (pick one).   I am trying to unseat a growing concern in the back of my mind that neither approach ("yet another" nor "pick one") resolves the underlying use case here.

To wit, isn't the underlying use case _already_ "sense making among various schemes"?   This seems like it will be true for Subject and other codes, and eventually competency statements (i.e., there will probably never be ONLY ONE). 

+1 (and this bears repeating over and over and over again).
 

Even if the URLs are nothing more than unique identifiers for denormalized data, if the chosen scheme doesn't have a category appropriate to the resource being tagged, it seems like it could hold up LRMI tagging efforts, or at least force a different coding scheme to enter the scene.  I dare suggest this is how there are so many coding schemes in existence today (i.e., the ones in existence don't cover all the bases, or maybe are parts of a monolithic or somewhat intractable scheme (e.g. SIF), and it is not too hard to craft one for someone's local purpose and translate to the others).

2) From some brief experimentation, it seems that all of the following links resolve to the exact same document:
http://purl.org/ASN/scheme/ASNEducationLevel/
http://purl.org/ASN/scheme/ASNEducationLevel/1
http://purl.org/ASN/scheme/ASNEducationLevel/2
etc

What semantic difference is imparted when different grade links all resolve to the same master document?  The answer seems to lie in unpacking the document.  Or, perhaps the idea (for now) is that these URLs are not planning to be unpacked, and the question is moot.
 

At some point, unpacking could emerge.  Then, what if the user does not wish to resolve to RDF (with SKOS and DC components)?  As Brandt notes, there is no adopted best practice for this, so it remains a bit of the wild west.

Kelly (and Brandt), I may be off target, but there are "adopted best practices" for handling this issue as RDF in the form of W3C "Best Practice Recipes for Publishing RDF Vocabularies" [1].   The ASN is committed to RDF and Linked Data in assisting implementers today prepare for tomorrow.  Unfortunately, the ASN education level vocabulary is a poor example of potential since ASN has not really dealt with it in a manner that supports "content negotiation" as described at [1] that would move past the behavior limitations you describe, Kelly (our bad). 

But, the ASN education levels vocabulary could (will?) be handled by ASN like the RDF representation of the ASN competency data in which "content negotiation" per [1] is fully enabled (see [2] at part "B") making what is returned amenable to multiple serializations/presentations (rdf/xmlL, turtle, ntriples, json) as well as human-readable html; and doing so at the level of individual URI.

While you'll have to use your imagination a bit see this in terms of the ASN education level vocabulary, here are some ASN URIs emulating content negotiation for a specific node in the competencies:

http://asn.jesandco.org/resources/S114341F (human-readable html)
http://asn.jesandco.org/resources/S114341F.rdf (rdf/xml)
http://asn.jesandco.org/resources/S114341F.ttl  (turtle)
http://asn.jesandco.org/resources/S114341F.json

[1]  http://www.w3.org/TR/swbp-vocab-pub/
[2]  http://asn.jesandco.org/content/asn-resolution-service-overview
 

3) I think the point Jim makes about "hoping CEDS codes are used by LRMI" buttresses my point that the underlying use case may not be in choosing one scheme over another (i.e. if CEDS codes are picked, who would be hoping it was theirs instead). To me, this suggests a viable solution lies in committing some energy to translating between and among numerous, defacto schemes.

I totally agree, Kelly.  My (perhaps naive) hope is that there might emerge a cooperative, public mapping effort amongst data providers in this regard since it's likely be key public data infrastructure in the U.S., Europe, Asia etc. etc.

At Academic Benchmarks, in connection with our Learning Registry discussions, we attempted just such a translation service.  NOTE: It is for Common Core Standards (not grades), but I think there are some strong parallels:

http://academicbenchmarks.com/standards-translator

Needless to say, I think that this is excellent, Kelly, and illustrative of what I would consider good practice.  ASN is also mapping to publicly available  identifiers (as your example illustrates when you click on "View at source"). 
 

  You can't erase history or abolish all except one set, so one sane response is to create some brand of translation.

:-) well said.
 

Stuart