Creating Solr query string

[Borja López Gómez]

Hi all,

I've been using the 'traditional' way to create a Solr query:

solrResults = solrManager.Query(new SolrQuery("field1:tv"), new QueryOptions { { Fields = new  [] { ... }, FilterQueries = ..., Start = 0, Rows = 0 } });

In this case, everything except Q and FQ are specified in the QueryOptions parameter (fl, facet, fq, start, rows...).

Now, I have a specific requirement, which is opening a door in the public interface, in order to receive all the QueryOptions in Solr native language, so that this new parameter is going to be appended to the solr query.

To sum up, the Q and FQ parameters are calculated by my algorithms, but the rest of them (FL, FACET, ROWS, START,...) are received separately in an ONLY string parameter, which needs to be appended. So finally I have something like this:
string q = "field1:smart samsung tv"
string fq = "field2:25"
string otherParams = "&fl:docId&start=0&rows=50"

I'd like to know if there's a way to keep using the SolrNet framework to execute this kind of queries, without parsing 'otherParams' to get all the parameters one by one, and using directly what my client send us.

Regards,
Borja.

[Paige Cook]

You can use the ExtraParams parameter on the your SolrNet query to pass the value of your otherParams string and have it be appended directly to the querystring. Please see the SolrNet Additional Parameters documentation for more details.

--
You received this message because you are subscribed to the Google Groups "SolrNet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to solrnet+u...@googlegroups.com.
To post to this group, send email to sol...@googlegroups.com.
Visit this group at http://groups.google.com/group/solrnet.
For more options, visit https://groups.google.com/groups/opt_out.

[Borja López Gómez]

Thanks for your answer.

That's okay, but this solution needs to parse the input parameters, in order to create the dictionary<string, string> by splitting, first by '&' character (to separate the params), and then by '=' character (to separate the key-value for each param).

I was looking for something easier, e.g. just somehow appending the params (as a pure string) to my query, in order to try to avoid possible errors parsing the input.

Regards,
Borja.

[Borja López Gómez]

Moreover, the 'start' and 'rows' params are not taken into account when I put them into that dictionary :S

[Paige Cook]

Looks like I gave you some bad advice. I do not know of any parameter or mechanism within SolrNet to accomplish what you want. Sorry for any inconvenience.

[Mauricio Scheffer]

Hi Borja,

This is a terrible idea for a few reasons:

1. Your caller could easily pass some parameters that would DoS your application. E.g some complex fq and rows=10000000000000

2. Your caller could easily pass some parameters that would make your application fail. Instead, you should design to limit your available options to the features you provide, and no more.

3. You fail to encapsulate Solr. It leaks to your caller, which means that both your application and your caller have to know about Solr's API.

This will make both applications (yours and your caller's) harder to maintain, there will be more errors, etc.

Instead of doing this, work with the person who asked you for this and identify concrete needs and use cases.

Cheers

--

Mauricio

[Borja López Gómez]

Hi Mauricio,

thanks for your answer :)

I agree with most of your points, but the needs are quite special.

In this case, our function is just to provide the backend for the caller's services. So, if they somehow decide to hack our functionality, this will be directly affecting to their performance, so that this is definitely not interesting for the caller.

In adition to that, this concrete client is much more expert in Solr issues than us, so the problem of knowing Solr's APIs disappears.

In conclusion, as they are the guys who manage the presentation layer (in this case a web application), they wanted to take control on rows, start, faceting, ..., but we didn't want to request that parameters one by one, in order not to modify our APIs in the future, just in case more new params were needed.

Finally, if this is worthy for someone else, I've implemented the next solution, based on your own approach (It would be really worthy for me to listen to your opinions about it):

http://bugsquash.blogspot.com.es/2010/12/customizing-solrnet.html

1) Creating a CustomSolrConnection class which overrides the 'Get' function from the ISolrConnectionClass.
2) Calculating the 'q' param, the 'fq' params (if needed), and the rest of params (string parameter from my client e.g. [...&fl=docId&start=20&rows=10...])
3) Compose the final querystring (q=...fq=...[rest of params])
4) Send this final querystring to the CustomSolrConnection Class
5) Calling the Get method of CustomSolrConnection class, which does not take the parameters from the IEnumerable<KeyValuePair<string, string>> parameters. Instead of that, takes the final querystring previously received.
6) Returning directly the Solr response as string, which was one of my client needs.

And that's it. Moreover, we keep offering the other API function, which uses 100% SolrNet to perform all the Solr queries and get results, so that my client can use both functions (same results) depending of their needs.

Thanks for your help :)

Kind regards,
Borja.

[Mauricio Scheffer]

Hi Borja,

Special needs do not turn a bad idea into a good idea ;-)

If the client has more experience with Solr than you and wants more control, then why not give them direct access to Solr? It's still not a good idea, but it's simpler and you could even block them the update request handler, or fix some Solr parameters via configuration if you want to retain some control. Still, the problem of not encapsulating Solr does not disappear. There will be duplicate logic managing Solr and managing the "domain" represented by the Solr schema. This means a higher maintenance cost. DRY applies. 

Moreover, I don't think it's a good idea to have a custom SolrConnection if calling Get() on it ignores its parameters. Instead, simply build the parameters and then pass them as IEnumerable<KeyValuePair<string, string>> to Get()

Cheers

--

Mauricio

[Markus Engelhardt]

Hi Mauricio.

I understand what you mean, but I also understand the need of having the option to pass Solr specific query strings
to Solrnet Queries.
What about the following scenario:
If have a Solr access layer, which implements Solrnet to make life more easy.
The rest of my application does not know about it.
So I can use solrnet for mapping, connection handling and so on and it helps me a lot.

But we will have very complex queries, that we are worried at present, if we will be able to produce all of this needs
with the Solrnet query objects.

If I had the option, to create my own query syntax in my own query parser class and execute them with the help of
Solrnet and also get my results with the help of Solrnet, this would take away a lot of worries we have here.

Furthermore, as I am already in my Solr specific layer,  I do not see any issues if that layer knows how to build a Solr
specific query string. If I use Ado.Net I can also use SQL directly if I want, and there are even more options of DBMSs
than just the one Solr we can access with Solrnet.
As this is my layer in my application, I do also not see that many risks here, like DoS, etc.

So I think this would be a good feature IMHO.

Regards,

Markus

[Mauricio Scheffer]

Hi Markus, that isn't necessarily related to the original discussion. I was opposing to opening the door for *other applications or users* to enter Solr syntax directly, i.e. an architectural issue not strictly related to SolrNet.

But of course you can create complex queries not directly supported by the current SolrNet query objects. There are many alternatives to do this:

 * Use SolrQuery instead (any string you give it will be passed directly to Solr).

 * Create a new class to represent this query type, implementing ISelfSerializingQuery to do the serialization/conversion

 * Create a new class to represent this query type, adding a new ISolrQuerySerializer to the container to do the serialization/conversion

Or you can even drop down to using the SolrConnection object directly, passing Solr parameters and queries in a list of KeyValuePair<string, string>, then optionally mapping back the response using SolrNet's mapping classes. I blogged about this in http://bugsquash.blogspot.com/2010/03/low-level-solrnet.html

SolrNet was designed from the start to be modular like that.

Cheers

--

Mauricio

[Markus Engelhardt]

Thanks a lot Mauricio, that really helped me...

Im a little but surprised, that I can pass  a complex query string to the SolrQuery as I thought I tried

this and I thought it gave me an exception, butr then I'll look closer at this once more.

Regards,

Markus