Foreign language cross-reference

[Samuel Bradshaw]

As mentioned in JIRA issues LDSMUSIC-285, LDSMUSIC-283, and LDSMUSIC-194, it would be good to put in cross-reference data for hymns. I guess the coding for this has already been started. I have a database I've been compiling for several months that cross-references the hymns and I wouldn't mind it being used for the LDS Music App. Here's what I have so far:

http://singpraises.net/hymns.chart.all?sort=1&type=num

The above chart is for the 28 foreign-language 1985-based hymnbooks only. I have info for other books, including a few of the Children's Songbooks, on this page:

http://singpraises.net/hymns

[Justin Wagner]

Samuel,

That is awesome!

We have written the database backend to support lookup of hymns that
are translations of a given hymn.

That table will certainly help Hilton prepare the data on the server.

The last part to all of this is to implement the front end UI that
makes it useful. There is a JIRA issue for this (although I don't
recall the number of it). If there are any developers wanting to
contribute, this would be a good place to start.

I can write it, but I haven't had the time to do it yet.

Justin Wagner
Sent from my iPhone

On Sep 25, 2012, at 11:18 PM, Samuel Bradshaw

[Samuel Bradshaw]

As an update, I've been entering more hymnbooks into the cross-reference chart; I also fixed some of the background code that wasn't matching everything right and that was making it load really slowly:

http://singpraises.net/hymns.chart.all?sort=1&type=num&sh1=1&sh2=2&sh3=3&sh4=4&sh5=5&sh6=6

I currently have the following hymnbooks lined up, matching corresponding hymns:

– All 28 green hymnbooks (I still need data for 22 Ukrainian, 4 Indonesian, and 5 Icelandic hymns).

– 11 Children’s Songbooks.

– 8 "Hymns and Children's Songs" books (there are several more at the BYU library that I'll be adding soon).

– 9 "Hymns Made Easy" books.

– Miscellaneous other books

Hopefully this will help!

[Justin Wagner]

Thanks Samuel.

I have created a JIRA issue (https://tech.lds.org/jira/browse/LDSMUSIC-318) to help us better track this improvement in the app.  I know the beta version currently has cross references between English, Spanish, and Korean.  I'm not completely familiar with how we are loading the cross reference information into the Content Management System (I'm sure Hilton is however).

Assuming this is something that somebody besides Hilton could do, then we can make sub tasks for different languages and start getting these knocked out.

Thanks again for the help, it is truly appreciated.

Justin

[Hilton Campbell]

Here's how cross-references work on the data entry site. Anyone is welcome to help us fill them out.

If you go to the data entry site (http://mitchfreed.com/~ldsmusic/) and look at a song, you'll notice there is a "Translation ID" (just below the song #.

Songs that have the same "translation ID" are automatically cross-referenced in the app. For example, the English hymn "Nearer, My God, to Thee", hymn number 100, has a translation ID of "Hymns-EN-100". The same hymn in Spanish, "Más cerca, Dios, de Ti", hymn number 50, also has translation ID "Hymns-EN-100". So these two hymns can be cross -referenced in the app.

The "translation ID" is completely arbitrary. I automatically assigned all the English songs an ID of the form "Hymns-EN-<hymn number>" or "Childrens-EN-<song number>".

I believe all of the Spanish hymns and songs already have translation IDs. Notice that some of the Spanish songs do not appear in the English songbook. For example, "Estrella de luz", Spanish children's song number 146, does not appear in English. So I gave it the translation ID of "Childrens-ES-146". If that song appears in some other language besides Spanish, it should use this same translation ID.

Hopefully that helps, let me know if I can clarify anything. If any volunteers would like to migrate data from Samuel's excellent cross-reference chart, that would be great. There may be a way to automate this, but I haven't had time to look into it.

Thanks,

Hilton

NOTICE: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.

[Justin Wagner]

@Hilton Thanks for spelling this out.  

I have a suggestion for @Mitch Freed that will help reduce errors in the translation-ID entry:

Can we make the input box for translation id be able to select from a drop-down of already known translation-IDs, or accept manual text entry?  I think this is called a combo box with manual entry.  

This way user's can just select this value from the drop-down unless they need to define a new translation-ID, in which case they just type it in.

Justin

[Justin Wagner]

Okay I have created some JIRA issues to start getting the collections cross referenced.

There is one main task (https://tech.lds.org/jira/browse/LDSMUSIC-318) which has a sub-task for each collection that needs to be cross-referenced.

Here are the sub-tasks I have created thus far, feel free to assign them to yourself and work on them if you desire.  Bonus points for anybody who figures out how to automated the task! ;)

Hymns-PT - (I've taken this one) - https://tech.lds.org/jira/browse/LDSMUSIC-323

Hymns-FR - https://tech.lds.org/jira/browse/LDSMUSIC-324

Hymns-IT - https://tech.lds.org/jira/browse/LDSMUSIC-325

Hymns-DE - https://tech.lds.org/jira/browse/LDSMUSIC-326

Hymns-SM - https://tech.lds.org/jira/browse/LDSMUSIC-327

If you want to work on a collection I haven't added yet, PLEASE DO!  If you don't know how to add the sub-task to JIRA just email me what one you are working on and I'll update JIRA.

Justin

[Collins Carrington]

Hey Justin, I may need some help getting started, but I volunteer to take the Hymns-IT sub-task that you just created!  Thanks either way.

-Collins Carrington

---

Date: Thu, 10 Jan 2013 11:07:00 -0800
Subject: Re: [LDS Music for iOS] Foreign language cross-reference
From: jrw...@gmail.com
To: ldstech-lds-...@googlegroups.com

[Mitch Freed]

That should be easy enough to do. Not sure how soon I'll get to it though. I've got another project I'm trying to finish up this month.

- Mitch

Sent from my iPhone

[Shawn Dowler]

I just finished the German translation IDs, and I'm not sure that a pick list would have made things any faster. What would have really helped would have been a Save button at the top of the form so I didn't have to scroll down and back up each time.

Shawn Dowler
Information Designer
shawn....@gmail.com
http://shawndowler.com/blog
http://twitter.com/shawndowler

[Mitch Freed]

That one would be very easy to do :) I might even be able to do that one tonight.

- Mitch

Sent from my iPhone

[Justin Wagner]

Taking that one step further having a "Save and Next" button that saves and then retrieves the info for the next item would speed things up even further.

Also if you changed the confirmation message called in "finalize_save_song" from being an alert to just being bold red text that gets displayed that would be quicker (so we don't have to dismiss the alert)

[Mitch Freed]

Great feedback.

- Mitch

Sent from my iPhone

[Samuel Bradshaw]

How are we going to keep track of hymns not in the English hymnbook that are in multiple other hymnbooks? For example, "Come, Thou Fount of Every Blessing" is in German, Hungarian, Thai, and Kekchi – which language will be the definitive correct "Translation ID" when we're typing it in?

Also, these are the hymns which we don't have cross-reference info for:

Ukrainian, #7, 18, 26, 31, 35, 43, 47, 49, 54, 55, 58, 85, 92, 105, 108, 140, 148, 159, 174, 175, 176, 177.

Indonesian, #24, 73, 78, 145.

Icelandic, #48, 83, 84, 107, 115.

If anyone has these hymnbooks and could send me a scan of those hymn numbers, I can try to find their matches.

— Samuel Bradshaw

---------------------
Samuel Bradshaw

http://singpraises.net/

---------------------

[Justin Wagner]

Do we have any type of global index associated with our collections? (Hymns-EN, Childrens-EN, Hymns-PT, etc)

If so we could say that the definitive correct "Translation ID" is that of the collection with the lowest index.

If the indices don't exist we could create indices based on order of addition into the app.

[Samuel Bradshaw]

When we have a definitive system I can put a column into the SingPraises.net chart that has the Translation ID.

[Samuel Bradshaw]

I set up a simplified page for getting the Translation IDs, but it only shows IDs for hymns that are in English hymnbooks right now – just waiting to hear on how we prioritize the languages to form the Translation IDs...

http://singpraises.net/hymns.chart.ldsmusicapp?hbid=1

[Hilton Campbell]

You people are awesome.

I was planning on order of addition into the app for ordering translation IDs. But if you can generate them easily and include them on your website, Samuel, a different scheme might be more appropriate.

Another option would be to give EN first priority, and then go in alphabetical order of language code from there. The problem with that is that when we add a language we would need to revise the existing content to use new translation IDs in some cases.

Any other ideas? Votes?

On Jan 11, 2013, at 2:42 AM, Samuel Bradshaw <samuel.h...@gmail.com>

 wrote:

[Hilton Campbell]

I just updated the staging app with all the changes you've been making, including translation IDs. I have to say, its pretty cool being able to switch between languages like that. Thank you for making it happen.

[DMS]

This is one of the most exciting developments I have seen! Thank you for your efforts and desires to hasten this work!  

Elder Sorrakowski

Coming to you from my iPhone 

[Eurico Costa]

In my opinion, you should break the collection field into two fields: Collection (hymns, children), and add a language field. The language would be equivalent to a translation code and the combination of both plus song # would be sufficient to uniquely identify a song in a given language. Unless I'm missing something which wouldn't be surprising ;)

Eurico Costa

ejc...@gmail.com

(801) 915-6813

[Samuel Bradshaw]

The Translation ID is for grouping together the same hymn in different languages. For example, "Nearer, My God, to Thee" has the Translation ID Hymns-EN-100. "Más cerca, Dios, de Ti" is the same song in Spanish, so it has the same Translation ID (Hymns-EN-100). The Translation ID can be shared by more than one hymn to group them together (it isn't a unique identifier).

— Samuel Bradshaw

[Justin Wagner]

Eurico,

Here is the logic behind our current scheme (which I think is similar to what you are proposing):

A "hymn book" or "children's hymn book" is represented as a "collection" which has a unique identifier that consists of its type and language.  (Hymns-EN) and each "item" in a collection is uniquely identified as its collection ID and its number in that collection (Hymns-EN-45).

Therefore when linking items for translation cross-reference we point to the items unique id (also referred to as external_id in code).

So we do have separate "fields" for type, language and number.

Let me know if this answers your question.

Justin

[Samuel Bradshaw]

I'm glad I'm able to help!

For prioritizing Translation ID languages, you're right that alphabetization would be hard to keep up as new hymnbooks are added. I think it would be good to base it on the order the print versions were published, if we have an accurate list for that.

Otherwise, we could just start with the main languages used in the church worldwide, and then add the rest of the languages arbitrarily:

First: English, Spanish, Portuguese, French;

Next: German, Italian, Chinese, Japanese, Korean, Russian;

Then (it doesn't have to be in this order): Romanian, Danish, Dutch, German, Icelandic, Norwegian, Swedish, Finnish, Hungarian, Bulgarian, Ukrainian, Thai, Fijian, Indonesian, Samoan, Tagalog, Tahitian, Tongan, Kekchi, Cebuano (CS), Albanian (CS);

Finally: Any "Hymns and Children's Songs" languages and any new hymnbooks the church will publish.

[Samuel Bradshaw]

Or, we could just do English, then alphabetical with the other 27 hymnbook languages, then Cebuano and Albanian (for which a Children's Songbook exists but not a hymnbook), then anything else just added at the end. That would probably be easiest.

[Eurico Costa]

It does, thanks for the clarification.

Eurico Costa

[Justin Wagner]

I vote for Samuel's most recent proposal "do English, then alphabetical with the other 27 hymnbook languages, then Cebuano and Albanian (for which a Children's Songbook exists but not a hymnbook), then anything else just added at the end".

Whatever proposal we decide on (I'll let Hilton make the final call), we need to publish it somewhere for reference, like on our wiki.

[Justin Wagner]

I guess I should say that I vote for this proposal because I think it will have the fastest time to implement.

[Hilton Campbell]

It sounds like no one has any strong opinions, so lets go with this proposal.

I started a wiki page, but I think it could use some rewording. Also, the large group of languages do not have language codes and thus are not sorted by language code. If someone has a minute to make some edits that would be great.

https://tech.lds.org/wiki/Translation_ID_Language_Precedence_(LDS_Music_for_iOS)

On Jan 11, 2013, at 2:54 PM, Justin Wagner <jrw...@gmail.com>

[Shawn Dowler]

I just updated the German translation IDs to correspond to the precedence listed on the wiki. I updated the other languages that those hymns affected, too, so they should all have the same ID.

Shawn Dowler
Information Designer
shawn....@gmail.com
http://shawndowler.com/blog
http://twitter.com/shawndowler

[Samuel Bradshaw]

All* the Translation IDs are now listed here:

http://singpraises.net/hymns.chart.ldsmusicapp?hbid=1

This will make it easier to enter the correct ID into the data entry site.

— Samuel Bradshaw

*For all the hymns I have listed in my database.

[Shawn Dowler]

Based on the precedence on the wiki, I used Hymns-DA-49 for German hymn number 83, but http://singpraises.net/hymns.chart.ldsmusicapp?hbid=1 uses the English children's songbook for the translation ID. Do we need to include these songbooks in the precedence list? And which is "correct?" I realize that we are defining "correct" as we go :)

Thanks,

Shawn Dowler
Information Designer
shawn....@gmail.com
http://shawndowler.com/blog
http://twitter.com/shawndowler

[Samuel Bradshaw]

I'm not sure... We need an official opinion on this...

Sent from my iPhone

[Justin Wagner]

I think the confusion comes from the fact that the numbered lists on the wiki started over for each section. (i.e. languages first added to app, languages with hymns, languages without hymns)

I fixed the wiki to be more explicit about the order or precedence, including specifying that in a given language the hymnbook takes precedence over the children's songbook.

According to the wiki page (https://tech.lds.org/wiki/Translation_ID_Language_Precedence_(LDS_Music_for_iOS)) the English Hymnbook and Children's Hymns have first precendence.  

Therefore I would say that the translation id for German hymn 83 would be Childrens-EN-62 as stated on Samuel's site, and not Hymns-DA-49.

@Shawn, let me know if this doesn't clarify your question.

Justin

[Justin Wagner]

I have created some more tasks on JIRA so we can keep the cross-referencing work going.

Here are the tasks still waiting to be completed.  

Cross Reference the Hymns-CEB Collection - https://tech.lds.org/jira/browse/LDSMUSIC-329

Cross Reference the Hymns-DA  Collection - https://tech.lds.org/jira/browse/LDSMUSIC-330

Cross Reference the Hymns-FR  Collection - https://tech.lds.org/jira/browse/LDSMUSIC-324

Cross Reference the Hymns-NL  Collection - https://tech.lds.org/jira/browse/LDSMUSIC-331

Cross Reference the Hymns-RU  Collection - https://tech.lds.org/jira/browse/LDSMUSIC-332

Cross Reference the Hymns-TH  Collection - https://tech.lds.org/jira/browse/LDSMUSIC-333

Cross Reference the Hymns-TL  Collection - https://tech.lds.org/jira/browse/LDSMUSIC-334

Cross Reference the Hymns-TO  Collection - https://tech.lds.org/jira/browse/LDSMUSIC-335

Cross Reference the Hymns-TY  Collection - https://tech.lds.org/jira/browse/LDSMUSIC-336

Cross Reference the Hymns-ZH  Collection - https://tech.lds.org/jira/browse/LDSMUSIC-337

If you want to help but aren't sure what to do, feel free to contact me and I can help explain this task better.  Also there is very good detail listed here https://tech.lds.org/jira/browse/LDSMUSIC-318

Thanks for all the effort everybody has been putting in, i'm really excited with how this feature is progressing!

Justin

[Justin Wagner]

Guys,

Thanks for all your hard work!  I am really excited to see the progress we are making on the app and this feature.

We are going to be extending this feature (showing translations of the current song) to not only show translations of a given Hymn, but also related Hymns in the same language.

In order to make this work properly we need to make some modifications to our translation-id algorithm.

The biggest change is that whenever there are multiple versions of a song in the same language (i.e. Men's Choir version, Children's version, Women's Choir version) they need to all be listed with the same translation id.  This will allow us to show these items as related by detecting that they all have the same language and translation id.

I think in some of the collections we have populated there may be instances where these items are listed with different translation IDs, so there may be some collections that need to be updated accordingly.

I have updated the Wiki page with verbiage to reflect these changes and attached a PDF which attempts to show these relationships.

Let me know if I need to clarify anything further.

Keep up the good work!

Justin

[Samuel Bradshaw]

Will it still be able to switch to the same type of song – choir hymn to choir hymn, or children's song to children's song?

For example, if I'm looking at English Hymn #317, "Sweet Is the Work (Women)," and I use the menu to switch to Chinese, will it go to the Women's version, or to the regular version? Or, if I'm looking at French Children’s Songbook page 2 (I Am a Child of God), will it switch to the English Children's Songbook, or to English hymn #301? Or will it show both in the list and allow me to choose?

<Translation_ID_diagrams.pdf>

[Justin Wagner]

Samuel,

In this case the list of translations would show multiple entries for the languages with multiple versions.

Basically there will be three sections in the CounterpartViewController popup window

1 - Sheet / Word Options

2 - Songs that have the same language and translation ID of the current song

3 - Songs that have the same translation ID of the current song (but not the same language) (can have multiple entries per language)

Let me know if this is clear.

[Samuel Bradshaw]

OK. I updated the Translation ID reference page here:

http://singpraises.net/hymns.chart.ldsmusicapp?hbid=1

Could a find and replace be done to fix the updated Translation IDs in already-entered hymnbooks?

[Justin Wagner]

I think that possibly Mitch (I'll let him confirm if this is really possible) could craft an update statement to run on the backend database if you provide a list of ones to find/replace.  

If the list is fairly long then I think we should just fall back on our manual entry method.

[Mitch Freed]

Sure, I could do an update.

- Mitch