Own (dynamic) fields for each library
3 views
Skip to first unread message
Uwe Reh
Aug 9, 2021, 3:38:53 PM8/9/21
to solrmarc-tech
Dear experts,
maybe it's just a FAQ, but I can't find a solution.
*Background*
In our union catalog we have several libraries. Their holding
information is coded in the marc:924 field (German convention).
I.e:
- 924a item's local number
- 924b library's id (ISIL)
https://en.wikipedia.org/wiki/International_Standard_Identifier_for_Libraries_and_Related_Organizations
- 924g item's shelfmark
(see example below)
*Request*
We need to implement for each library a own search field, containing
only their own shelfmarks {shelfmark_lib-a, ..., shelfmark_lib-n}
Normaly (pure SOLR) I would define a dynamic field 'shelfmark_*' and
would add for each marc:924 the value of subfield 'g' to
shelfmark_<value of subfield 'b'>
*Question*
With solrmarc it is easy to write custom methods to populate named
fields. But this names have to be fixed.
I found no way to configure something like:
shelfmark_<?> = custom(de.foo.bar.myIndexer), dataPerLibrary
"What is the right way to define something like this?"
Most likely, this attempt might conflicts the 'normal' concept of
solrmarc.properties.
So my alternative question is:
"Is it possible to manipulate the SolrInputDocument from a subclass of
org.solrmarc.index.SolrIndexer?"
Regards
Uwe
#######
Example
###
<?xml version="1.0" encoding="UTF-8"?>
<collection xmlns="http://www.loc.gov/MARC21/slim">
<record>
<leader>00000cam a2200000 c 4500</leader>
...
<datafield tag="245" ind1="1" ind2="0">
<subfield code="a">Popular title</subfield>
<subfield code="c">Kurt Gödel</subfield>
</datafield>
...
<datafield tag="924" ind1="0" ind2=" ">
<subfield code="a">Local id in library A</subfield>
<subfield code="b">DE-A</subfield>
<subfield code="g">AAA</subfield>
</datafield>
...
<datafield tag="924" ind1="0" ind2=" ">
<subfield code="a">Local id in library B</subfield>
<subfield code="b">DE-B</subfield>
<subfield code="g">BBB</subfield>
</datafield>
</record>
Should result in:
'AAA' -> shelfmark_DE-A
'BBB' -> shelfmark_DE-B
####
EOF
maybe it's just a FAQ, but I can't find a solution.
*Background*
In our union catalog we have several libraries. Their holding
information is coded in the marc:924 field (German convention).
I.e:
- 924a item's local number
- 924b library's id (ISIL)
https://en.wikipedia.org/wiki/International_Standard_Identifier_for_Libraries_and_Related_Organizations
- 924g item's shelfmark
(see example below)
*Request*
We need to implement for each library a own search field, containing
only their own shelfmarks {shelfmark_lib-a, ..., shelfmark_lib-n}
Normaly (pure SOLR) I would define a dynamic field 'shelfmark_*' and
would add for each marc:924 the value of subfield 'g' to
shelfmark_<value of subfield 'b'>
*Question*
With solrmarc it is easy to write custom methods to populate named
fields. But this names have to be fixed.
I found no way to configure something like:
shelfmark_<?> = custom(de.foo.bar.myIndexer), dataPerLibrary
"What is the right way to define something like this?"
Most likely, this attempt might conflicts the 'normal' concept of
solrmarc.properties.
So my alternative question is:
"Is it possible to manipulate the SolrInputDocument from a subclass of
org.solrmarc.index.SolrIndexer?"
Regards
Uwe
#######
Example
###
<?xml version="1.0" encoding="UTF-8"?>
<collection xmlns="http://www.loc.gov/MARC21/slim">
<record>
<leader>00000cam a2200000 c 4500</leader>
...
<datafield tag="245" ind1="1" ind2="0">
<subfield code="a">Popular title</subfield>
<subfield code="c">Kurt Gödel</subfield>
</datafield>
...
<datafield tag="924" ind1="0" ind2=" ">
<subfield code="a">Local id in library A</subfield>
<subfield code="b">DE-A</subfield>
<subfield code="g">AAA</subfield>
</datafield>
...
<datafield tag="924" ind1="0" ind2=" ">
<subfield code="a">Local id in library B</subfield>
<subfield code="b">DE-B</subfield>
<subfield code="g">BBB</subfield>
</datafield>
</record>
Should result in:
'AAA' -> shelfmark_DE-A
'BBB' -> shelfmark_DE-B
####
EOF
Demian Katz
Aug 10, 2021, 8:00:17 AM8/10/21
to solrma...@googlegroups.com
Uwe,
I'm not aware of a way to dynamically determine the field name, though that doesn't mean it doesn't exist; hopefully Bob can weigh in on that. It is clearly a feature that could be useful under some circumstances.
In the meantime, one question: do you know all the possible values of 924b? I assume you do, and that this is a finite number. If no better solution presents itself, you could likely set up conditional rules for all of the libraries to create the desired effect -- i.e. just include lines for every possible library-specific field, and only populate them when 924b matches the relevant value (using the conditional modifiers of the existing specification language). That is obviously significantly less elegant than having a dynamic field name, but you could probably write a simple script to output the necessary SolrMarc configuration based on a list of library codes, and it would give you the results you desire.
- Demian
-----Original Message-----
From: solrma...@googlegroups.com <solrma...@googlegroups.com> On Behalf Of Uwe Reh
Sent: Monday, August 9, 2021 3:39 PM
To: solrmarc-tech <solrma...@googlegroups.com>
Subject: [EXTERNAL] [solrmarc-tech] Own (dynamic) fields for each library
Dear experts,
maybe it's just a FAQ, but I can't find a solution.
*Background*
In our union catalog we have several libraries. Their holding information is coded in the marc:924 field (German convention).
I.e:
- 924a item's local number
- 924b library's id (ISIL)
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FInternational_Standard_Identifier_for_Libraries_and_Related_Organizations&data=04%7C01%7Cdemian.katz%40villanova.edu%7Cf969173c4eea4e92b0f508d95b6d5248%7C765a8de5cf9444f09cafae5bf8cfa366%7C0%7C0%7C637641347363761110%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=yJX1c6d%2Bna%2Fh%2BFEfNcDnvq6ehrLLb9AdsonojTJKWdA%3D&reserved=0
--
You received this message because you are subscribed to the Google Groups "solrmarc-tech" group.
To unsubscribe from this group and stop receiving emails from it, send an email to solrmarc-tec...@googlegroups.com.
To view this discussion on the web visit https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fsolrmarc-tech%2F03c610b0-ae34-fb55-16ba-cea7d9d0131c%2540hebis.uni-frankfurt.de&data=04%7C01%7Cdemian.katz%40villanova.edu%7Cf969173c4eea4e92b0f508d95b6d5248%7C765a8de5cf9444f09cafae5bf8cfa366%7C0%7C0%7C637641347363761110%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5vGhtdtM0J6rVXXw0xvPwbZhNScS0LtiApAVJPUoCcE%3D&reserved=0.
I'm not aware of a way to dynamically determine the field name, though that doesn't mean it doesn't exist; hopefully Bob can weigh in on that. It is clearly a feature that could be useful under some circumstances.
In the meantime, one question: do you know all the possible values of 924b? I assume you do, and that this is a finite number. If no better solution presents itself, you could likely set up conditional rules for all of the libraries to create the desired effect -- i.e. just include lines for every possible library-specific field, and only populate them when 924b matches the relevant value (using the conditional modifiers of the existing specification language). That is obviously significantly less elegant than having a dynamic field name, but you could probably write a simple script to output the necessary SolrMarc configuration based on a list of library codes, and it would give you the results you desire.
- Demian
-----Original Message-----
From: solrma...@googlegroups.com <solrma...@googlegroups.com> On Behalf Of Uwe Reh
Sent: Monday, August 9, 2021 3:39 PM
To: solrmarc-tech <solrma...@googlegroups.com>
Subject: [EXTERNAL] [solrmarc-tech] Own (dynamic) fields for each library
Dear experts,
maybe it's just a FAQ, but I can't find a solution.
*Background*
In our union catalog we have several libraries. Their holding information is coded in the marc:924 field (German convention).
I.e:
- 924a item's local number
- 924b library's id (ISIL)
- 924g item's shelfmark
(see example below)
*Request*
We need to implement for each library a own search field, containing only their own shelfmarks {shelfmark_lib-a, ..., shelfmark_lib-n} Normaly (pure SOLR) I would define a dynamic field 'shelfmark_*' and would add for each marc:924 the value of subfield 'g' to shelfmark_<value of subfield 'b'>
*Question*
With solrmarc it is easy to write custom methods to populate named fields. But this names have to be fixed.
I found no way to configure something like:
shelfmark_<?> = custom(de.foo.bar.myIndexer), dataPerLibrary "What is the right way to define something like this?"
Most likely, this attempt might conflicts the 'normal' concept of solrmarc.properties.
So my alternative question is:
"Is it possible to manipulate the SolrInputDocument from a subclass of org.solrmarc.index.SolrIndexer?"
Regards
Uwe
#######
Example
###
<?xml version="1.0" encoding="UTF-8"?>
<collection xmlns="https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.loc.gov%2FMARC21%2Fslim&data=04%7C01%7Cdemian.katz%40villanova.edu%7Cf969173c4eea4e92b0f508d95b6d5248%7C765a8de5cf9444f09cafae5bf8cfa366%7C0%7C0%7C637641347363761110%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=vK6QnMP9j7z2n6KHTNVOmcPvE0sT9TTxg9CSv9qqQdA%3D&reserved=0">
(see example below)
*Request*
We need to implement for each library a own search field, containing only their own shelfmarks {shelfmark_lib-a, ..., shelfmark_lib-n} Normaly (pure SOLR) I would define a dynamic field 'shelfmark_*' and would add for each marc:924 the value of subfield 'g' to shelfmark_<value of subfield 'b'>
*Question*
With solrmarc it is easy to write custom methods to populate named fields. But this names have to be fixed.
I found no way to configure something like:
shelfmark_<?> = custom(de.foo.bar.myIndexer), dataPerLibrary "What is the right way to define something like this?"
Most likely, this attempt might conflicts the 'normal' concept of solrmarc.properties.
So my alternative question is:
"Is it possible to manipulate the SolrInputDocument from a subclass of org.solrmarc.index.SolrIndexer?"
Regards
Uwe
#######
Example
###
<?xml version="1.0" encoding="UTF-8"?>
You received this message because you are subscribed to the Google Groups "solrmarc-tech" group.
To unsubscribe from this group and stop receiving emails from it, send an email to solrmarc-tec...@googlegroups.com.
To view this discussion on the web visit https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fsolrmarc-tech%2F03c610b0-ae34-fb55-16ba-cea7d9d0131c%2540hebis.uni-frankfurt.de&data=04%7C01%7Cdemian.katz%40villanova.edu%7Cf969173c4eea4e92b0f508d95b6d5248%7C765a8de5cf9444f09cafae5bf8cfa366%7C0%7C0%7C637641347363761110%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5vGhtdtM0J6rVXXw0xvPwbZhNScS0LtiApAVJPUoCcE%3D&reserved=0.
Uwe Reh
Aug 10, 2021, 9:34:55 AM8/10/21
to solrma...@googlegroups.com
Am 10.08.21 um 14:00 schrieb Demian Katz:
> ... It is clearly a feature that could be useful under some circumstances.
Hi Demian
Thanks for your supporting vote.
In our own HDS-System (Solr, Vufind, own 'solrPICA') we have a lot of
dynamic field to simulate parent child relations e.g 'local numbers per
department', 'parts/elements of series', search suggestions per department.
> i.e. just include lines for every possible library-specific field, and only populate them when 924b ...
Sorry, my initial description was not clear at this point.
With other colleges of other institutions, I'm working on a 'virtual
national union catalog index' of German scientific libraries¹. So we
have to deal with approx. 200M documents and round about 5K different
libraries.
Even with a script, this can't be handled with 'normal' fields. The
index speed would become unusable slow and every second, a new library
might join one of our unions
To be clear, we do not plan to offer a GUI for each library, actuality
we do not offer any GUI at all. The index is meant as a free service for
own frontends. Therefor it is impossible to preselect the few 'n'
libraries that would really use their local search field.
Uwe
1) https://www.agv-gvi.de (only German)
> ... It is clearly a feature that could be useful under some circumstances.
Hi Demian
Thanks for your supporting vote.
In our own HDS-System (Solr, Vufind, own 'solrPICA') we have a lot of
dynamic field to simulate parent child relations e.g 'local numbers per
department', 'parts/elements of series', search suggestions per department.
> i.e. just include lines for every possible library-specific field, and only populate them when 924b ...
Sorry, my initial description was not clear at this point.
With other colleges of other institutions, I'm working on a 'virtual
national union catalog index' of German scientific libraries¹. So we
have to deal with approx. 200M documents and round about 5K different
libraries.
Even with a script, this can't be handled with 'normal' fields. The
index speed would become unusable slow and every second, a new library
might join one of our unions
To be clear, we do not plan to offer a GUI for each library, actuality
we do not offer any GUI at all. The index is meant as a free service for
own frontends. Therefor it is impossible to preselect the few 'n'
libraries that would really use their local search field.
Uwe
1) https://www.agv-gvi.de (only German)
Demian Katz
Aug 11, 2021, 9:09:29 AM8/11/21
to solrma...@googlegroups.com
Thanks, Uwe, that makes a lot of sense and definitely underscores the need for dynamic field names. I'm hoping Bob can weigh in on this topic, and I'll give him a little time to do so... but if he doesn't, please feel free to remind me in a few days, and I'll be happy to dig into the code a little deeper and see if I can come up with any ideas.
- Demian
-----Original Message-----
From: solrma...@googlegroups.com <solrma...@googlegroups.com> On Behalf Of Uwe Reh
1) https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.agv-gvi.de%2F&data=04%7C01%7Cdemian.katz%40villanova.edu%7Cccca4340bf834b93717d08d95c03a471%7C765a8de5cf9444f09cafae5bf8cfa366%7C0%7C0%7C637641992984038833%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=2W67WMNM%2BH6l6JTZMy8DmJSUp1ZtqdCtH9QYgDsG%2Bcg%3D&reserved=0 (only German)
--
You received this message because you are subscribed to the Google Groups "solrmarc-tech" group.
To unsubscribe from this group and stop receiving emails from it, send an email to solrmarc-tec...@googlegroups.com.
To view this discussion on the web visit https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fsolrmarc-tech%2Fa7d8631c-e0ce-cedf-a4ae-ac6eae1b849d%2540hebis.uni-frankfurt.de&data=04%7C01%7Cdemian.katz%40villanova.edu%7Cccca4340bf834b93717d08d95c03a471%7C765a8de5cf9444f09cafae5bf8cfa366%7C0%7C0%7C637641992984048787%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2FDdOFSO9TC9oE3l6qujAagK1c5Z%2F4yepOVcumzTB4DI%3D&reserved=0.
- Demian
-----Original Message-----
From: solrma...@googlegroups.com <solrma...@googlegroups.com> On Behalf Of Uwe Reh
--
You received this message because you are subscribed to the Google Groups "solrmarc-tech" group.
To unsubscribe from this group and stop receiving emails from it, send an email to solrmarc-tec...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages