ISLANDORA CLAW - Multilingual Fedora indexing

[Zoltán Kanász-Nagy]

Dear CLAW developers!

How is the multilingual drupal content indexed into Fedora? When you see the Fedora record, how can we see the differences or the language qualifiers, for example this title is english, this title is german and so on.

Thank you for the answer!

Zoli.

[Rosie Le Faive]

Hi Zoltán,

There's no standard way for multilingual content to exist in Fedora (that I know of). But there are ways to do it.

In a previous instance I had to tackle this question, and chose to create separate datastreams for metadata in different languages. So the French metadata was in MODS-FR and the English was in MODS-EN. Another way to do this in Fedora would be to save parallel information in a single MODS file, using the @lang attribute on different strings. 

To display them, you could either:

- write a custom display module that uses XPath to pull the appropriate values out of your metadata, and display them with their language tags. (hard)

- or you could put them in different solr fields, and use Islandora Solr Metadata for your display (more exposed configuration, less custom code)

When I did this with my MODS-XX fields, I had also to configure GSearch to index these particular datastreams and send them into solr fields that were language coded (_en_ms, for example). I think I also configured Solr's fields to run different language-specific parsing on these fields, to break them into text and use stop words appropriate for each language (i.e. I used Solr's pre-configured languages). 

If you do the @lang attribute option instead, you may need to alter Gsearch (MODS-to-solr or slurp_all_mods_to_solr) to use that attribute in the resulting solr fields (or you might not... I haven't tested it!)

If you want to go a step further and show the English title if the user's language is English, and German if the user is looking at the site in German, then you can use Mark Jordan's Context module along with different solr metadata configurations. 

[Rosie Le Faive]

Hey I just realized you're probably talking about CLAW/Fedora 4.  Sorry!

So ... RDF allows for strings to be language-tagged. I assume this would "work" out of the box in Fedora.

As for mapping between Drupal's fields and Fedora languages - I'm not sure that it's been done yet, but hopefully we can harmonize Drupal's version of multilingual strings with the RDF that goes to Fedora. Good point, and I'll bring this up in the Metadata Interest Group (Next meeting's March 5 - you're welcome to join us if you can!) 

[Daniel Lamb]

Diego was pretty thorough when writing the serializer.  https://github.com/Islandora-CLAW/jsonld/blob/36f37cb2bae7467fd6d3eacabd9e4a368ea77a7f/src/Normalizer/FieldItemNormalizer.php#L131-L139

Looks like it already will pass along language tags.  That means it would end up in both your Fedora and Triplestore with the proper language property.

--
For more information about using this group, please read our Listserv Guidelines: http://islandora.ca/content/welcome-islandora-listserv
---
You received this message because you are subscribed to the Google Groups "islandora" group.
To unsubscribe from this group and stop receiving emails from it, send an email to islandora+...@googlegroups.com.
Visit this group at https://groups.google.com/group/islandora.
To view this discussion on the web visit https://groups.google.com/d/msgid/islandora/def18632-c45c-470d-b178-9a3c7d82432e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
~Danny Lamb
Tech Lead
Islandora Foundation
http://islandora.ca

[Zoltán Kanász-Nagy]

Dear Rosie and Danny!

Thank you for the help. Yes, my question is for the Claw version.

Danny, thank you showing me Diegos development. We installed the system with the claw-playbook installer in the beginning of January. When i tried to test multilingual indexing into Fedora, i did not see any specialization.

I will inspect the log files.

Zoli. 

2018-02-23 21:21 GMT+01:00 Daniel Lamb <dl...@islandora.ca>:

Diego was pretty thorough when writing the serializer.  https://github.com/Islandora-CLAW/jsonld/blob/36f37cb2bae7467fd6d3eacabd9e4a368ea77a7f/src/Normalizer/FieldItemNormalizer.php#L131-L139

Looks like it already will pass along language tags.  That means it would end up in both your Fedora and Triplestore with the proper language property.

On 2018-02-23 04:15 PM, Rosie Le Faive wrote:

Hey I just realized you're probably talking about CLAW/Fedora 4.  Sorry!

So ... RDF allows for strings to be language-tagged. I assume this would "work" out of the box in Fedora.

As for mapping between Drupal's fields and Fedora languages - I'm not sure that it's been done yet, but hopefully we can harmonize Drupal's version of multilingual strings with the RDF that goes to Fedora. Good point, and I'll bring this up in the Metadata Interest Group (Next meeting's March 5 - you're welcome to join us if you can!) 

--
For more information about using this group, please read our Listserv Guidelines: http://islandora.ca/content/welcome-islandora-listserv
---
You received this message because you are subscribed to the Google Groups "islandora" group.

To unsubscribe from this group and stop receiving emails from it, send an email to islandora+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/islandora.
To view this discussion on the web visit https://groups.google.com/d/msgid/islandora/def18632-c45c-470d-b178-9a3c7d82432e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
~Danny Lamb
Tech Lead
Islandora Foundation
http://islandora.ca

--

For more information about using this group, please read our Listserv Guidelines: http://islandora.ca/content/welcome-islandora-listserv
---

You received this message because you are subscribed to a topic in the Google Groups "islandora" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/islandora/5vWtk42F1YA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to islandora+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/islandora.

To view this discussion on the web visit https://groups.google.com/d/msgid/islandora/8f01407f-880d-f1e6-03f4-aa8e3ba6f4e0%40islandora.ca.

[natk...@gmail.com]

Hi Zoli,

There does seem to be some gaps in supporting indexing of multilingual content in Fedora and Triplestore.  I've created a ticket here https://github.com/Islandora-CLAW/CLAW/issues/808 to followup.  Thank you for bringing the issue forward. 

On Tuesday, February 27, 2018 at 8:23:02 AM UTC-5, Zoltán Kanász-Nagy wrote:

Dear Rosie and Danny!

Thank you for the help. Yes, my question is for the Claw version.

Danny, thank you showing me Diegos development. We installed the system with the claw-playbook installer in the beginning of January. When i tried to test multilingual indexing into Fedora, i did not see any specialization.

I will inspect the log files.

Zoli. 

2018-02-23 21:21 GMT+01:00 Daniel Lamb <dl...@islandora.ca>:

Diego was pretty thorough when writing the serializer.  https://github.com/Islandora-CLAW/jsonld/blob/36f37cb2bae7467fd6d3eacabd9e4a368ea77a7f/src/Normalizer/FieldItemNormalizer.php#L131-L139

Looks like it already will pass along language tags.  That means it would end up in both your Fedora and Triplestore with the proper language property.

On 2018-02-23 04:15 PM, Rosie Le Faive wrote:

Hey I just realized you're probably talking about CLAW/Fedora 4.  Sorry!

So ... RDF allows for strings to be language-tagged. I assume this would "work" out of the box in Fedora.

As for mapping between Drupal's fields and Fedora languages - I'm not sure that it's been done yet, but hopefully we can harmonize Drupal's version of multilingual strings with the RDF that goes to Fedora. Good point, and I'll bring this up in the Metadata Interest Group (Next meeting's March 5 - you're welcome to join us if you can!) 

--
For more information about using this group, please read our Listserv Guidelines: http://islandora.ca/content/welcome-islandora-listserv
---
You received this message because you are subscribed to the Google Groups "islandora" group.

To unsubscribe from this group and stop receiving emails from it, send an email to islandora+...@googlegroups.com.

Visit this group at https://groups.google.com/group/islandora.
To view this discussion on the web visit https://groups.google.com/d/msgid/islandora/def18632-c45c-470d-b178-9a3c7d82432e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
~Danny Lamb
Tech Lead
Islandora Foundation
http://islandora.ca

--
For more information about using this group, please read our Listserv Guidelines: http://islandora.ca/content/welcome-islandora-listserv
---
You received this message because you are subscribed to a topic in the Google Groups "islandora" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/islandora/5vWtk42F1YA/unsubscribe.

To unsubscribe from this group and all its topics, send an email to islandora+...@googlegroups.com.

Visit this group at https://groups.google.com/group/islandora.

[dp...@metro.org]

Hi Zoli and Nat


Pull request in place
👀 Islandora-CLAW/jsonld#25 please let me know if that helps:


Note: Not sure if this deals with the SPARQL update, I tested this on a clean Drupal 8.4 (not the full claw stack=, but at least JSON-LD will show now the translated language (and base language) for xsd:string based fields (by removing the @type and relying on implicit type casting, sorry I did not write the JSON-LD 1.1 spec). I also added some notes to the pull request worth your attention (like the fact that once you translate you get multiple dateCreated)


Best!


Diego Pino N
Metro.org

[Zoltán Kanász-Nagy]

Hello Nat!

Thank you, I am following this ticket:  https://github.com/Islandora-CLAW/CLAW/issues/808 

Kind regards