how to handle sorting of multivalued fields

2,443 views
Skip to first unread message

Jeff Minelli

unread,
May 22, 2012, 10:23:25 PM5/22/12
to blacklight-...@googlegroups.com
While working on converting our Hydra instance to use solr 3.5 (from 1.4.1), I ran across the following blurb in http://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_5_0/solr/CHANGES.txt
* In previous releases, sorting or evaluating function queries on
fields that were "multiValued" (either by explicit declaration in
schema.xml or by implict behavior because the "version" attribute on
the schema was less then 1.2) did not generally work, but it would
sometimes silently act as if it succeeded and order the docs
arbitrarily.  Solr will now fail on any attempt to sort, or apply a
function to, multi-valued fields
How does one handle the sort on multivalued fields scenario? Is there a work around in Blacklight?

Thanks,

-jeff

Naomi Dushay

unread,
May 22, 2012, 10:53:15 PM5/22/12
to blacklight-...@googlegroups.com
Jeff,

Sadly, the only "solutions" I'm aware of are:

1. only use Solr fields that are NOT multiValued as sort fields
2.  use a version of Solr older than 3.5
3.  write code for Solr that will allow sorting on multivalued fields

- Naomi

--
You received this message because you are subscribed to the Google Groups "Blacklight Development" group.
To view this discussion on the web visit https://groups.google.com/d/msg/blacklight-development/-/rV3zyRvg1zIJ.
To post to this group, send email to blacklight-...@googlegroups.com.
To unsubscribe from this group, send email to blacklight-develo...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/blacklight-development?hl=en.

Jonathan Rochkind

unread,
May 23, 2012, 11:08:37 AM5/23/12
to blacklight-...@googlegroups.com
There's no way to sort on multi-valued fields in Solr. In past versions
it "appeared" to work, but was unreliable and had unclear semantics.

How do people actually handle this? At Solr indexing time, have your
indexing routine _create_ a single-valued field that has the semantics
you want for sorting. Do you want to sort by the _first_ value in the
multi-valued field? Then just put that first value alone into a single
valued field.

This kind of 'de-normalization', where data is contained in multiple
fields in different forms, is a pretty standard pattern in Solr, don't
let it scare you.

In no version of solr past or present has it been possible for a single
record to show up _multiple times_ in a search result set, one time for
each value in a multi-valued field you're sorting by. That's never what
was happenign, even in older versions of Solr. A fundamental part of
lucene/solr architecture makes this not happen.

On 5/22/2012 10:53 PM, Naomi Dushay wrote:
> Jeff,
>
> Sadly, the only "solutions" I'm aware of are:
>
> 1. only use Solr fields that are NOT multiValued as sort fields
> 2. use a version of Solr older than 3.5
> 3. write code for Solr that will allow sorting on multivalued fields
>
> - Naomi
>
> On May 22, 2012, at 7:23 PM, Jeff Minelli wrote:
>
>> While working on converting our Hydra <http://hydraproject.org/>
>> instance to use solr 3.5 (from 1.4.1), I ran across the following
>> blurb in
>> http://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_5_0/solr/CHANGES.txt
>> * In previous releases, sorting or evaluating function queries on
>> fields that were"multiValued" (either by explicit declaration in
>> schema.xml or by implict behavior because the"version" attribute on
>> the schema was less then 1.2) did not generally work, but it would
>> sometimes silently act as if it succeeded and order the docs
>> arbitrarily. Solr will now fail on any attempt to sort, or apply a
>> function to, multi-valued fields
>> How does one handle the sort on multivalued fields scenario? Is there
>> a work around in Blacklight?
>>
>> Thanks,
>>
>> -jeff
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Blacklight Development" group.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msg/blacklight-development/-/rV3zyRvg1zIJ.
>> To post to this group, send email to
>> blacklight-...@googlegroups.com
>> <mailto:blacklight-...@googlegroups.com>.
>> To unsubscribe from this group, send email to
>> blacklight-develo...@googlegroups.com
>> <mailto:blacklight-develo...@googlegroups.com>.
>> For more options, visit this group at
>> http://groups.google.com/group/blacklight-development?hl=en.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Blacklight Development" group.

Jeff Minelli

unread,
May 23, 2012, 9:48:48 AM5/23/12
to blacklight-...@googlegroups.com
Thanks Naomi. I was afraid that was the answer.

-jeff
Reply all
Reply to author
Forward
0 new messages