Newbie with Solr and Blacklight configuration

[KADY]

Hello,

I am new to Solr and Blacklight both. I have installed a standalone version of Solr and have indexed data into it and am able to query through the Solr UI. I want to integrate Blacklight with this stand alone version. For this, what I did is, I started the standalone Solr and then in Blacklight, I configured catalog_controller.rb to use some fields from the indexed documents. Note that the indexed fields are differently named from the one's which come out of the box in Blacklight. When I bring up the Rails server, I see the app come up with the name of the facets and everything working; but when I query, nothing comes back. This is the query which is running in the backend on Solr.

1949186 [qtp15583371-16] INFO  org.apache.solr.core.SolrCore  – [collection1] webapp=/solr path=/select params={facet=true&qt=search&wt=ruby&rows=10} hits=0 status=0 QTime=1 

I'm attaching the controller file for your ref.

Any help is appreciated.

Thanks/

[Justin Coyne]

have you set the 'qf' parameter in your solrconfig.xml?  It's possible you didn't specify which fields to run the query against.

Blacklight's default solr does that here: https://github.com/projectblacklight/blacklight-jetty/blob/master/solr/blacklight-core/conf/solrconfig.xml#L89

You can also put that in your Blacklight configuration within the CatalogController like this:

https://github.com/curationexperts/tufts/blob/master/app/controllers/catalog_controller.rb#L24

-Justin

--
You received this message because you are subscribed to the Google Groups "Blacklight Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blacklight-develo...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Erik Hatcher]

You may also want to have a look at our lws_blacklight project, where we configured BL to work with a mostly-arbitrary (“id” for uniqueKey field still required) Solr.  We had to make several small tweaks in various files.  Check out the project here:https://github.com/LucidWorks/lws_blacklight

Erik

--
You received this message because you are subscribed to the Google Groups "Blacklight Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blacklight-develo...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

<catalog_controller.rb>

[KADY]

I tried your recommendation by modifying both solrconfig.xml and the catalog controller. I did not have any luck. Attaching both for your reference.

Thanks.

To unsubscribe from this group and stop receiving emails from it, send an email to blacklight-development+unsub...@googlegroups.com.

[Erik Hatcher]

Looks like you have :qt => ‘search’ in catalog_controller but no “search” request handler defined in your solrconfig.xml

Erik

To unsubscribe from this group and stop receiving emails from it, send an email to blacklight-develo...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

<catalog_controller.rb><solrconfig.xml>

[KADY]

HI Erik,

Tried adding the req handler in solrconfig.xml without much luck. Can you please take another look at the attached file?

Thanks,

Kaushik

Erik

<catalog_controller.rb><solrconfig.xml>

[Erik Hatcher]

Looks ok now.  But not sure what the issue is.  What is Blacklight logging when you start up fresh and make a query?   And what is Solr logging?

Erik

To unsubscribe from this group and stop receiving emails from it, send an email to blacklight-develo...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

<solrconfig.xml>

[KADY]

Attached are the log files;

<solrconfig.xml>

[Erik Hatcher]

Just took a quick look.  Analyzing two requests to Solr:

106481 [qtp8295685-16] INFO  org.apache.solr.core.SolrCore  – [collection1] webapp=/solr path=/select params={facet=true&qf=id+DOCNUM+APPLID+ASSTYPEID+SUBMID+ASSDATE+REVORGID+REVROLE+PRODUCTID+PRODUCTDRUGTYPECD+PRODDRUGNAME+PRODRLDDRUGNAME&qt=search&wt=ruby&rows=10} hits=0 status=0 QTime=7 

This one does not have a q(uery) parameter, so it’s the first hit to the page.  I’ve put :q=>’*:*’ in catalog_controller before to alleviate that situation, so that facets are shown on the first page with no query (if you have them configured; but doesn’t look like you do) 

But now this one:

137200 [qtp8295685-16] INFO  org.apache.solr.core.SolrCore  – [collection1] webapp=/solr path=/select params={facet=true&q=ZESTORETIC&spellcheck.q=ZESTORETIC&qf=id+DOCNUM+APPLID+ASSTYPEID+SUBMID+ASSDATE+REVORGID+REVROLE+PRODUCTID+PRODUCTDRUGTYPECD+PRODDRUGNAME+PRODRLDDRUGNAME&qt=search&wt=ruby&rows=10} hits=0 status=0 QTime=1 

BL sent a q parameter this time.  Now we’re down to debugging the actual query.   Try this request with your browser directly to Solr:

   /select?facet=true&q=ZESTORETIC&spellcheck.q=ZESTORETIC&qf=id+DOCNUM+APPLID+ASSTYPEID+SUBMID+ASSDATE+REVORGID+REVROLE+PRODUCTID+PRODUCTDRUGTYPECD+PRODDRUGNAME+PRODRLDDRUGNAME&qt=search&wt=ruby&rows=10&debug=true

Now look at the debug section and look at the parsed query.  That should shed some light on why things aren’t matching.  This can be a field type, analysis, or query parser issue.  I don’t see a defType specified, and unless that is in your “search” handler definition it’s using the “lucene” query parser and that does not use the qf field, but rather uses the default field specified in your schema.xml and that might not be a field that matches your particular query.

Tricky stuff to troubleshoot for someone new to both Solr and Blacklight, but chances are you are only a tweak or so away from having success, I just don’t know what that might be at the moment exactly.

Erik

To unsubscribe from this group and stop receiving emails from it, send an email to blacklight-develo...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

<solr.log><blacklight.log>

[Kaushik]

On Mon, Jan 6, 2014 at 3:56 PM, Erik Hatcher <erikh...@mac.com> wrote:

/select?facet=true&q=ZESTORETIC&spellcheck.q=ZESTORETIC&qf=id+DOCNUM+APPLID+ASSTYPEID+SUBMID+ASSDATE+REVORGID+REVROLE+PRODUCTID+PRODUCTDRUGTYPECD+PRODDRUGNAME+PRODRLDDRUGNAME&qt=search&wt=ruby&rows=10&debug=true

The first query was the result of executing a search whithout entering anything in the AllFields field. As you mentioned, adding &q=* to it, brings up documents on solr. Not sure why blacklight is appending it.

On the other hand, when I executed the 2nd query with the debug ON, I get the following. Not sure what is wrong.

{'responseHeader'=>{'status'=>0,'QTime'=>1,'params'=>{'facet'=>'true','q'=>'ZESTORETIC','spellcheck.q'=>'ZESTORETIC','qf'=>'id DOCNUM APPLID ASSTYPEID SUBMID ASSDATE REVORGID REVROLE PRODUCTID PRODUCTDRUGTYPECD PRODDRUGNAME PRODRLDDRUGNAME','debug'=>'true','wt'=>'ruby','qt'=>'search','rows'=>'10'}},'response'=>{'numFound'=>0,'start'=>0,'docs'=>[]},'facet_counts'=>{'facet_queries'=>{},'facet_fields'=>{},'facet_dates'=>{},'facet_ranges'=>{}},'debug'=>{'rawquerystring'=>'ZESTORETIC','querystring'=>'ZESTORETIC','parsedquery'=>'text:zestoretic','parsedquery_toString'=>'text:zestoretic','explain'=>{},'QParser'=>'LuceneQParser','timing'=>{'time'=>1.0,'prepare'=>{'time'=>1.0,'query'=>{'time'=>1.0},'facet'=>{'time'=>0.0},'mlt'=>{'time'=>0.0},'highlight'=>{'time'=>0.0},'stats'=>{'time'=>0.0},'debug'=>{'time'=>0.0}},'process'=>{'time'=>0.0,'query'=>{'time'=>0.0},'facet'=>{'time'=>0.0},'mlt'=>{'time'=>0.0},'highlight'=>{'time'=>0.0},'stats'=>{'time'=>0.0},'debug'=>{'time'=>0.0}}}}}

Thank you.

[Erik Hatcher]

Sorry for the "forward" but original bounced because of incorrect from address. 

On Jan 6, 2014, at 16:13, Erik Hatcher <erikh...@me.com> wrote:

It’s querying for “text:zestoretic”, using the lucene query parser. Apparently that term (zestoretic) isn’t in the field named “text”. Maybe you’re not even using a field named “text”(but it must be defined in your schema at least or you’d get an error).

Probably the simplest thing to do is add :defType => ‘edismax’ to your catalog_controller default solr parameters.  Try adding &defType=edismax to that Solr request below and see if that solves the problem?

Erik

[Kaushik]

HI Erik,

So when I add the config to the controller, I get some new error on blacklight.

NoMethodError in Catalog#index

Showing C:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/blacklight-4.5.0/app/views/catalog/_document.html.erb where line #5 raised:

undefined method `gsub' for 38:Fixnum

Extracted source (around line #5):

2 3 4 5 6 7 8

<div class="document <%= render_document_class document %>"> <%= render :partial => 'document_header', :locals => { :document => document, :document_counter => document_counter } %> <%= render_document_partial document, :thumbnail, :document_counter => document_counter %> <% # main container for doc partial view -%> <%= render_document_partial document, :index, :document_counter => document_counter %>

[KADY]

On Monday, January 6, 2014 5:25:00 PM UTC-5, KADY wrote:

HI Erik,

So when I add the config to the controller, I get some new error on blacklight.

NoMethodError in Catalog#index

Showing C:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/blacklight-4.5.0/app/views/catalog/_document.html.erb where line #5 raised:

undefined method `gsub' for 38:Fixnum

Extracted source (around line #5):

2 3 4 5 6 7 8

<div class="document <%= render_document_class document %>"> <%= render :partial => 'document_header', :locals => { :document => document, :document_counter => document_counter } %> <%= render_document_partial document, :thumbnail, :document_counter => document_counter %> <% # main container for doc partial view -%> <%= render_document_partial document, :index, :document_counter => document_counter %>

The query which gets sent to Solr is fine, it has the deftype=edismax added to the query and when running it seperately, it returns the results. Its only in blacklight that I have the problem now. Thanks for your help. 

On Mon, Jan 6, 2014 at 4:51 PM, Erik Hatcher <erikh...@mac.com> wrote:

Sorry for the "forward" but original bounced because of incorrect from address. 

On Jan 6, 2014, at 16:13, Erik Hatcher <erikh...@me.com> wrote:

It’s querying for “text:zestoretic”, using the lucene query parser. Apparently that term (zestoretic) isn’t in the field named “text”. Maybe you’re not even using a field named “text”(but it must be defined in your schema at least or you’d get an error).

Probably the simplest thing to do is add :defType => ‘edismax’ to your catalog_controller default solr parameters.  Try adding &defType=edismax to that Solr request below and see if that solves the problem?

Erik

On Jan 6, 2014, at 4:09 PM, Kaushik <kaush...@gmail.com> wrote:

On Mon, Jan 6, 2014 at 3:56 PM, Erik Hatcher <erikh...@mac.com> wrote:

/select?facet=true&q=ZESTORETIC&spellcheck.q=ZESTORETIC&qf=id+DOCNUM+APPLID+ASSTYPEID+SUBMID+ASSDATE+REVORGID+REVROLE+PRODUCTID+PRODUCTDRUGTYPECD+PRODDRUGNAME+PRODRLDDRUGNAME&qt=search&wt=ruby&rows=10&debug=true

The first query was the result of executing a search whithout entering anything in the AllFields field. As you mentioned, adding &q=* to it, brings up documents on solr. Not sure why blacklight is appending it.

On the other hand, when I executed the 2nd query with the debug ON, I get the following. Not sure what is wrong.

{'responseHeader'=>{'status'=>0,'QTime'=>1,'params'=>{'facet'=>'true','q'=>'ZESTORETIC','spellcheck.q'=>'ZESTORETIC','qf'=>'id DOCNUM APPLID ASSTYPEID SUBMID ASSDATE REVORGID REVROLE PRODUCTID PRODUCTDRUGTYPECD PRODDRUGNAME PRODRLDDRUGNAME','debug'=>'true','wt'=>'ruby','qt'=>'search','rows'=>'10'}},'response'=>{'numFound'=>0,'start'=>0,'docs'=>[]},'facet_counts'=>{'facet_queries'=>{},'facet_fields'=>{},'facet_dates'=>{},'facet_ranges'=>{}},'debug'=>{'rawquerystring'=>'ZESTORETIC','querystring'=>'ZESTORETIC','parsedquery'=>'text:zestoretic','parsedquery_toString'=>'text:zestoretic','explain'=>{},'QParser'=>'LuceneQParser','timing'=>{'time'=>1.0,'prepare'=>{'time'=>1.0,'query'=>{'time'=>1.0},'facet'=>{'time'=>0.0},'mlt'=>{'time'=>0.0},'highlight'=>{'time'=>0.0},'stats'=>{'time'=>0.0},'debug'=>{'time'=>0.0}},'process'=>{'time'=>0.0,'query'=>{'time'=>0.0},'facet'=>{'time'=>0.0},'mlt'=>{'time'=>0.0},'highlight'=>{'time'=>0.0},'stats'=>{'time'=>0.0},'debug'=>{'time'=>0.0}}}}}

Thank you.

--
You received this message because you are subscribed to the Google Groups "Blacklight Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blacklight-development+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "Blacklight Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blacklight-development+unsub...@googlegroups.com.

[Erik Hatcher]

I’ll have to let someone else comment on this.  Be sure it’s defType, not deftype, but that’s not the issue here I imagine.  Maybe since you have an arbitrary Solr index that you need to override the _document partial to customize it to your fields?  That’d be my first guess.  We’ve overridden it like this: <https://github.com/LucidWorks/lws_blacklight/blob/master/app/views/catalog/_document.html.erb>, but that might not be right for your schema.

Erik

To unsubscribe from this group and stop receiving emails from it, send an email to blacklight-develo...@googlegroups.com.