This definitely sounds like a case of SolrMarc making assumptions that it really shouldn't; I don't think it's SolrMarc's job to validate the subfields, especially given that the MARC standard is constantly evolving.
If I had to guess, though, I'd speculate that this is actually not a problem specific to SolrMarc, but rather something related to the underlying Marc4j library. Can somebody with more Marc4j experience confirm or deny that?
Assuming this is related to Marc4j, is there a way to loosen the restrictions, perhaps through a new SolrMarc configuration option?
- Demian
________________________________
From: Hannah Ullrich [hannah.ullr...@ub.uni-freiburg.de]
Sent: Friday, September 28, 2012 7:46 AM
To: vufind-t...@lists.sourceforge.net
Subject: Re: [VuFind-Tech] SolrMarc 2.4 upgrade
Hallo Stefan,
we have the same problem with the 689 fields.
I get the following info in our marc_error field:
<arr name="marc_error">
<str>Minor Error : Subfield tag is an invalid uppercase character, changing it to lower case. --- [ 689 : D ]</str>
<str>Major Error : Subfield tag is an invalid character, using first character of field as subfield tag. --- [ 689 : A ]</str>
</arr>
using vufind 1.3
Hannah
Am 28.09.2012 12:26, schrieb Winkler, Stefan:
Hi Demian,
I have just upgraded both the VuFind 1.x trunk and the VuFind 2.x master branch to use SolrMarc 2.4, the latest release. (VuFind 1.x was previously on 2.3.1, while 2.x was previously using a custom-built version very close to 2.4). Hopefully this will make Monday's 2.0beta release just a little bit easier to manage thanks to the inclusion of a known SolrMarc version. It also allows me to close a couple of JIRA tickets related to fixes/improvements in 2.4.
I don't anticipate any problems related to this upgrade, but please let me know if you run into any indexing troubles using the latest-and-greatest code.
> This definitely sounds like a case of SolrMarc making assumptions that it
> really shouldn't; I don't think it's SolrMarc's job to validate the
> subfields, especially given that the MARC standard is constantly evolving.
> If I had to guess, though, I'd speculate that this is actually not a
> problem specific to SolrMarc, but rather something related to the
> underlying Marc4j library. Can somebody with more Marc4j experience
> confirm or deny that?
> Assuming this is related to Marc4j, is there a way to loosen the
> restrictions, perhaps through a new SolrMarc configuration option?
> I get the following info in our marc_error field:
> <arr name="marc_error">
> <str>Minor Error : Subfield tag is an invalid uppercase character,
> changing it to lower case. --- [ 689 : D ]</str>
> <str>Major Error : Subfield tag is an invalid character, using first
> character of field as subfield tag. --- [ 689 : A ]</str>
> </arr>
> I have just upgraded both the VuFind 1.x trunk and the VuFind 2.x master
> branch to use SolrMarc 2.4, the latest release. (VuFind 1.x was previously
> on 2.3.1, while 2.x was previously using a custom-built version very close
> to 2.4). Hopefully this will make Monday's 2.0beta release just a little
> bit easier to manage thanks to the inclusion of a known SolrMarc version.
> It also allows me to close a couple of JIRA tickets related to
> fixes/improvements in 2.4.
> I don't anticipate any problems related to this upgrade, but please let me
> know if you run into any indexing troubles using the latest-and-greatest
> code.
> thanks,
> Demian
> --------------------------------------------------------------------------- ---
> Got visibility?
> Most devs has no idea what their production app looks like.
> Find out how fast your code is with AppDynamics Lite.http://ad.doubleclick.net/clk;262219671;13503038;y?http://info.appdyn...
> --
> You received this message because you are subscribed to the Google Groups
> "solrmarc-tech" group.
> To post to this group, send email to solrmarc-tech@googlegroups.com.
> To unsubscribe from this group, send email to
> solrmarc-tech+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/solrmarc-tech?hl=en.
I am copying this message back to Hannah and Stefan in case they did not see your reply on the solrmarc-tech list.
Is SolrMarc meant to enforce the rules of MARC21? If so, should there be an option to relax them? It seems that there is value in being able to work with non-standard records as long as they are structurally correct.
- Demian
From: solrmarc-tech@googlegroups.com [mailto:solrmarc-tech@googlegroups.com] On Behalf Of Simon Spero
Sent: Saturday, September 29, 2012 11:48 AM
To: solrmarc-tech@googlegroups.com
Subject: Re: [solrmarc-tech] More on SolrMarc and uppercase subfields
Upper case letters are not allowed in MARC21 records (though they *are* allowed in generic marc.
I thought that the DNB had completed the conversion to MARC21?
Simon
On Fri, Sep 28, 2012 at 7:56 AM, Demian Katz <demian.k...@villanova.edu<mailto:demian.k...@villanova.edu>> wrote:
Another message I'm copying to solrmarc-tech...
This definitely sounds like a case of SolrMarc making assumptions that it really shouldn't; I don't think it's SolrMarc's job to validate the subfields, especially given that the MARC standard is constantly evolving.
If I had to guess, though, I'd speculate that this is actually not a problem specific to SolrMarc, but rather something related to the underlying Marc4j library. Can somebody with more Marc4j experience confirm or deny that?
Assuming this is related to Marc4j, is there a way to loosen the restrictions, perhaps through a new SolrMarc configuration option?
- Demian
________________________________
From: Hannah Ullrich [hannah.ullr...@ub.uni-freiburg.de<mailto:hannah.ullr...@ub.uni-freiburg.de>]
Sent: Friday, September 28, 2012 7:46 AM
To: vufind-t...@lists.sourceforge.net<mailto:vufind-t...@lists.sourceforge.net>
Subject: Re: [VuFind-Tech] SolrMarc 2.4 upgrade
Hallo Stefan,
we have the same problem with the 689 fields.
I get the following info in our marc_error field:
<arr name="marc_error">
<str>Minor Error : Subfield tag is an invalid uppercase character, changing it to lower case. --- [ 689 : D ]</str>
<str>Major Error : Subfield tag is an invalid character, using first character of field as subfield tag. --- [ 689 : A ]</str>
</arr>
using vufind 1.3
Hannah
Am 28.09.2012 12:26, schrieb Winkler, Stefan:
Hi Demian,
I have just upgraded both the VuFind 1.x trunk and the VuFind 2.x master branch to use SolrMarc 2.4, the latest release. (VuFind 1.x was previously on 2.3.1, while 2.x was previously using a custom-built version very close to 2.4). Hopefully this will make Monday's 2.0beta release just a little bit easier to manage thanks to the inclusion of a known SolrMarc version. It also allows me to close a couple of JIRA tickets related to fixes/improvements in 2.4.
I don't anticipate any problems related to this upgrade, but please let me know if you run into any indexing troubles using the latest-and-greatest code.
Tel: +49-761 / 203-3877<tel:%2B49-761%20%2F%20203-3877>
--
You received this message because you are subscribed to the Google Groups "solrmarc-tech" group.
To post to this group, send email to solrmarc-tech@googlegroups.com<mailto:solrmarc-tech@googlegroups.com>.
To unsubscribe from this group, send email to solrmarc-tech+unsubscribe@googlegroups.com<mailto:solrmarc-tech%2Bunsubscri be@googlegroups.com>.
For more options, visit this group at http://groups.google.com/group/solrmarc-tech?hl=en.
--
You received this message because you are subscribed to the Google Groups "solrmarc-tech" group.
To post to this group, send email to solrmarc-tech@googlegroups.com.
To unsubscribe from this group, send email to solrmarc-tech+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/solrmarc-tech?hl=en.
Marc4J has always been very Marc agnostic -- it doesn't even assume Marc21, but works fine with UNIMARC and other European MARC variants. Meaning it makes absolutely no assumptions about field/subfield semantics, or what subfields or legal, or anything like that.
With one important principled exception: Marc4J _does_ follow the actual MARC specification (not Marc21, the generic MARC spec they are all based on, I forget the number), to the letter. If the MARC spec says that subfields are not case sensitive, then it would not surprise me to see Marc4J implementing that as written in the spec.
> This definitely sounds like a case of SolrMarc making assumptions that
> it really shouldn't; I don't think it's SolrMarc's job to validate the
> subfields, especially given that the MARC standard is constantly evolving.
> If I had to guess, though, I'd speculate that this is actually not a
> problem specific to SolrMarc, but rather something related to the
> underlying Marc4j library. Can somebody with more Marc4j experience
> confirm or deny that?
> Assuming this is related to Marc4j, is there a way to loosen the
> restrictions, perhaps through a new SolrMarc configuration option?
> I get the following info in our marc_error field:
> <arr name="marc_error">
> <str>Minor Error : Subfield tag is an invalid uppercase character,
> changing it to lower case. --- [ 689 : D ]</str>
> <str>Major Error : Subfield tag is an invalid character, using first
> character of field as subfield tag. --- [ 689 : A ]</str>
> </arr>
>> I have just upgraded both the VuFind 1.x trunk and the VuFind 2.x
>> master branch to use SolrMarc 2.4, the latest release. (VuFind 1.x
>> was previously on 2.3.1, while 2.x was previously using a custom-built
>> version very close to 2.4). Hopefully this will make Monday's 2.0beta
>> release just a little bit easier to manage thanks to the inclusion of
>> a known SolrMarc version. It also allows me to close a couple of JIRA
>> tickets related to fixes/improvements in 2.4.
>> I don't anticipate any problems related to this upgrade, but please
>> let me know if you run into any indexing troubles using the
>> latest-and-greatest code.
> --
> You received this message because you are subscribed to the Google
> Groups "solrmarc-tech" group.
> To post to this group, send email to solrmarc-tech@googlegroups.com.
> To unsubscribe from this group, send email to
> solrmarc-tech+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/solrmarc-tech?hl=en.
It is something I added. I had come across several records in our collection that had erroneous upper case subfield tags, and then found documentation that seemed to state unequivocally that subfield tags must be numeric 0-9 or lowercase alphabetic a-z. I thought that this restriction was true for MARC in general, rather than a specific implementation of MARC.
I'm pretty sure that this change is in the MarcPermissiveStreamReader portion of Marc4j. Since it appears to be causing a problem I think that it should be able to enabled/disabled, perhaps via a property.
I'll try to put together a minor release either of marc4j or solrmarc with this fix as soon as possible. My current thinking is that is should be disabled by default, and be able to be enabled with a property. Perhaps this is true for some other pieces of the error detecting/correcting code in the Permissive reader.
> I am copying this message back to Hannah and Stefan in case they did > not see your reply on the solrmarc-tech list.
> Is SolrMarc meant to enforce the rules of MARC21? If so, should there > be an option to relax them? It seems that there is value in being > able to work with non-standard records as long as they are > structurally correct.
> - Demian
> *From:*solrmarc-tech@googlegroups.com > [mailto:solrmarc-tech@googlegroups.com] *On Behalf Of *Simon Spero
> *Sent:* Saturday, September 29, 2012 11:48 AM
> *To:* solrmarc-tech@googlegroups.com
> *Subject:* Re: [solrmarc-tech] More on SolrMarc and uppercase subfields
> Upper case letters are not allowed in MARC21 records (though they > *are* allowed in generic marc.
> I thought that the DNB had completed the conversion to MARC21?
> Simon
> On Fri, Sep 28, 2012 at 7:56 AM, Demian Katz > <demian.k...@villanova.edu <mailto:demian.k...@villanova.edu>> wrote:
> Another message I'm copying to solrmarc-tech...
> This definitely sounds like a case of SolrMarc making assumptions that > it really shouldn't; I don't think it's SolrMarc's job to validate the > subfields, especially given that the MARC standard is constantly evolving.
> If I had to guess, though, I'd speculate that this is actually not a > problem specific to SolrMarc, but rather something related to the > underlying Marc4j library. Can somebody with more Marc4j experience > confirm or deny that?
> Assuming this is related to Marc4j, is there a way to loosen the > restrictions, perhaps through a new SolrMarc configuration option?
> I get the following info in our marc_error field:
> <arr name="marc_error">
> <str>Minor Error : Subfield tag is an invalid uppercase character, > changing it to lower case. --- [ 689 : D ]</str>
> <str>Major Error : Subfield tag is an invalid character, using first > character of field as subfield tag. --- [ 689 : A ]</str>
> </arr>
> I have just upgraded both the VuFind 1.x trunk and the VuFind 2.x
> master branch to use SolrMarc 2.4, the latest release. (VuFind
> 1.x was previously on 2.3.1, while 2.x was previously using a
> custom-built version very close to 2.4). Hopefully this will make
> Monday's 2.0beta release just a little bit easier to manage thanks
> to the inclusion of a known SolrMarc version. It also allows me
> to close a couple of JIRA tickets related to fixes/improvements in
> 2.4.
> I don't anticipate any problems related to this upgrade, but
> please let me know if you run into any indexing troubles using the
> latest-and-greatest code.
> -- > You received this message because you are subscribed to the Google > Groups "solrmarc-tech" group.
> To post to this group, send email to solrmarc-tech@googlegroups.com > <mailto:solrmarc-tech@googlegroups.com>.
> To unsubscribe from this group, send email to > solrmarc-tech+unsubscribe@googlegroups.com > <mailto:solrmarc-tech%2Bunsubscribe@googlegroups.com>.
> For more options, visit this group at > http://groups.google.com/group/solrmarc-tech?hl=en.
> -- > You received this message because you are subscribed to the Google > Groups "solrmarc-tech" group.
> To post to this group, send email to solrmarc-tech@googlegroups.com.
> To unsubscribe from this group, send email to > solrmarc-tech+unsubscribe@googlegroups.com.
> For more options, visit this group at > http://groups.google.com/group/solrmarc-tech?hl=en.
> -- > You received this message because you are subscribed to the Google > Groups "solrmarc-tech" group.
> To post to this group, send email to solrmarc-tech@googlegroups.com.
> To unsubscribe from this group, send email to > solrmarc-tech+unsubscribe@googlegroups.com.
> For more options, visit this group at > http://groups.google.com/group/solrmarc-tech?hl=en.
From: solrmarc-tech@googlegroups.com [mailto:solrmarc-tech@googlegroups.com] On Behalf Of Robert Haschart
Sent: Monday, October 01, 2012 3:40 PM
To: solrmarc-tech@googlegroups.com
Subject: Re: [solrmarc-tech] More on SolrMarc and uppercase subfields
It is something I added. I had come across several records in our collection that had erroneous upper case subfield tags, and then found documentation that seemed to state unequivocally that subfield tags must be numeric 0-9 or lowercase alphabetic a-z. I thought that this restriction was true for MARC in general, rather than a specific implementation of MARC.
I'm pretty sure that this change is in the MarcPermissiveStreamReader portion of Marc4j. Since it appears to be causing a problem I think that it should be able to enabled/disabled, perhaps via a property. I'll try to put together a minor release either of marc4j or solrmarc with this fix as soon as possible. My current thinking is that is should be disabled by default, and be able to be enabled with a property. Perhaps this is true for some other pieces of the error detecting/correcting code in the Permissive reader.
-Bob Haschart
On 10/1/2012 9:28 AM, Demian Katz wrote:
I am copying this message back to Hannah and Stefan in case they did not see your reply on the solrmarc-tech list.
Is SolrMarc meant to enforce the rules of MARC21? If so, should there be an option to relax them? It seems that there is value in being able to work with non-standard records as long as they are structurally correct.
- Demian
From: solrmarc-tech@googlegroups.com<mailto:solrmarc-tech@googlegroups.com> [mailto:solrmarc-tech@googlegroups.com] On Behalf Of Simon Spero
Sent: Saturday, September 29, 2012 11:48 AM
To: solrmarc-tech@googlegroups.com<mailto:solrmarc-tech@googlegroups.com>
Subject: Re: [solrmarc-tech] More on SolrMarc and uppercase subfields
Upper case letters are not allowed in MARC21 records (though they *are* allowed in generic marc.
I thought that the DNB had completed the conversion to MARC21?
Simon
On Fri, Sep 28, 2012 at 7:56 AM, Demian Katz <demian.k...@villanova.edu<mailto:demian.k...@villanova.edu>> wrote:
Another message I'm copying to solrmarc-tech...
This definitely sounds like a case of SolrMarc making assumptions that it really shouldn't; I don't think it's SolrMarc's job to validate the subfields, especially given that the MARC standard is constantly evolving.
If I had to guess, though, I'd speculate that this is actually not a problem specific to SolrMarc, but rather something related to the underlying Marc4j library. Can somebody with more Marc4j experience confirm or deny that?
Assuming this is related to Marc4j, is there a way to loosen the restrictions, perhaps through a new SolrMarc configuration option?
- Demian
________________________________
From: Hannah Ullrich [hannah.ullr...@ub.uni-freiburg.de<mailto:hannah.ullr...@ub.uni-freiburg.de>]
Sent: Friday, September 28, 2012 7:46 AM
To: vufind-t...@lists.sourceforge.net<mailto:vufind-t...@lists.sourceforge.net>
Subject: Re: [VuFind-Tech] SolrMarc 2.4 upgrade
Hallo Stefan,
we have the same problem with the 689 fields.
I get the following info in our marc_error field:
<arr name="marc_error">
<str>Minor Error : Subfield tag is an invalid uppercase character, changing it to lower case. --- [ 689 : D ]</str>
<str>Major Error : Subfield tag is an invalid character, using first character of field as subfield tag. --- [ 689 : A ]</str>
</arr>
using vufind 1.3
Hannah
Am 28.09.2012 12:26, schrieb Winkler, Stefan:
Hi Demian,
I have just upgraded both the VuFind 1.x trunk and the VuFind 2.x master branch to use SolrMarc 2.4, the latest release. (VuFind 1.x was previously on 2.3.1, while 2.x was previously using a custom-built version very close to 2.4). Hopefully this will make Monday's 2.0beta release just a little bit easier to manage thanks to the inclusion of a known SolrMarc version. It also allows me to close a couple of JIRA tickets related to fixes/improvements in 2.4.
I don't anticipate any problems related to this upgrade, but please let me know if you run into any indexing troubles using the latest-and-greatest code.
Tel: +49-761 / 203-3877<tel:%2B49-761%20%2F%20203-3877>
--
You received this message because you are subscribed to the Google Groups "solrmarc-tech" group.
To post to this group, send email to solrmarc-tech@googlegroups.com<mailto:solrmarc-tech@googlegroups.com>.
To unsubscribe from this group, send email to solrmarc-tech+unsubscribe@googlegroups.com<mailto:solrmarc-tech%2Bunsubscri be@googlegroups.com>.
For more options, visit this group at http://groups.google.com/group/solrmarc-tech?hl=en.
--
You received this message because you are subscribed to the Google Groups "solrmarc-tech" group.
To post to this group, send email to solrmarc-tech@googlegroups.com<mailto:solrmarc-tech@googlegroups.com>.
To unsubscribe from this group, send email to solrmarc-tech+unsubscribe@googlegroups.com<mailto:solrmarc-tech+unsubscribe @googlegroups.com>.
For more options, visit this group at http://groups.google.com/group/solrmarc-tech?hl=en.
--
You received this message because you are subscribed to the Google Groups "solrmarc-tech" group.
To post to this group, send email to solrmarc-tech@googlegroups.com<mailto:solrmarc-tech@googlegroups.com>.
To unsubscribe from this group, send email to solrmarc-tech+unsubscribe@googlegroups.com<mailto:solrmarc-tech+unsubscribe @googlegroups.com>.
For more options, visit this group at http://groups.google.com/group/solrmarc-tech?hl=en.
--
You received this message because you are subscribed to the Google Groups "solrmarc-tech" group.
To post to this group, send email to solrmarc-tech@googlegroups.com.
To unsubscribe from this group, send email to solrmarc-tech+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/solrmarc-tech?hl=en.
