Limiting Elasticsearch Query Result Fields

[Barış Gülmez]

Hello,

I am new to DotCMS. Absolutely a great product! 

I have 2 queries regarding ES. I have searched the documentation and the group for an answer but no luck. 

1) With native ES queries, we can limit the fields returned by ES using "_source" or "fields" parameters in the query. But apparently it does not work with /api/es/search. Sample:

    {

        "_source": ["Article.name", "Story.name"],

        "query": {

            "query_string" : {

                "query" : "(Article.name:stress OR Story.name:stress)"

            }

        }

    }

Result set still comes with all the available data and meta-data fields.

Is there any way to limit the fields returned? I need to limit the fields since I only need title field in the search results page on my frontend.

2) I've tried to use /api/es/raw instead but somehow it does not have full set of the fields in the result set, apparently only Id fields. For instance, the above query returns empty but if I change the query to a field agnostic one as below,

    {

        "query": {

            "query_string" : {

                "query" : "stress"

            }

        }

    }

It returns 4 documents (as expected) but with these fields:

        "max_score": 2.7379642,

        "hits": [

            {

                "_index": "cluster_e7bwe1f0f3.working_20223223233",

                "_type": "_doc",

                "_id": "f9255af77b0eecwe03f5dc2604ff8a89_1",

                "_score": 2.7379642,

                "_source": {

                    "inode": "125762d1-539b-4we6b-989a-edfwe2ef198c",

                    "identifier": "f9255xcf77b0ewew8d03f5dc2604ff8a89"

                }

            },

Which is confusing because it means the relevant fields with "stress" word have been indexed but somehow result set does not have "name" fields ["Article.name", "Story.name"] of these two content types. 

Could you please help me to understand what am I doing wrong?

Thank you.

[Will Ezell]

With dotCMS, we only store the identifier and the inode in elasticsearch, so you cannot request specific fields back from it.  If you need to return only a few fields, I'd suggest you use the graphQL endpoint - which also takes a query, e.g. +Article.name:stress

There is a graphql playground in the back of dotCMS where you can try different queries - or you can use the example one hooked up to our demo site here (scroll about 1/2 down)

https://www.dotcms.com/product/features/graphql/ 

--
http://dotcms.com - Open Source Java Content Management
---
You received this message because you are subscribed to the Google Groups "dotCMS User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dotcms+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dotcms/7c0d0b47-3a66-4be6-b498-da8ac85b180cn%40googlegroups.com.

--

382 NE 191st St #92150

Miami, Florida 33179-3899

Main: 305-900-2001 | Direct: 978.294.9429

wi...@dotcms.com | dotcms.com

[Barış Gülmez]

Thanks, I got it. I have already tried GraphQL. It works fine and we can go with it for now. But how about search  suggestions (did you mean) and type ahead (auto complete)? I have tried below query with raw endpoint but since ES does not have the content fields it won't work with "name" field. Any recommendations to implement suggestions (did you mean) or type ahead (auto complete) features for specific content types and their fields?

curl -H "Content-Type: application/json" -XPOST http://localhost:8080/api/es/raw -d ' { "suggest" : { "title-suggestions" : { "text" : "gs pric rollrcoater", "term" : { "size" : 3, "field" : "title" } } } } '

[Will Ezell]

This works on the demo site - try:

curl -H "Content-Type: application/json" -XPOST https://demo.dotcms.com/api/es/raw -d '{ "suggest" : { "title-suggestions" : { "text" : "frenh Polynesi erything", "term" : { "size" : 3, "field" : "title" } } } }'

To view this discussion on the web visit https://groups.google.com/d/msgid/dotcms/3d62816a-0a75-43c4-8a67-1c4a60c337f6n%40googlegroups.com.

[Will Ezell]

For type ahead completions - you can set up a esCustomMapping for the field you want to try to complete on - I added this to our demo.dotcms.com site's Blog Title field and ran a reindex.

You can then run this query against it:

## type ahead

curl -H "Content-Type: application/json" -XPOST https://demo.dotcms.com/api/es/raw -d '
   {

     "typeAhead": {
       "blog-suggest": {
         "prefix": "fre",        
         "completion": {        
             "field": "blog.title"  
         }
       }
     }
   }
'

 

[Barış Gülmez]

Suggestions works thank you!

I get this with the typeahead one:

Unknown key for a START_OBJECT in [typeAhead].

Ps. I did re-index as well.

[Will Ezell]

Check the demo site before it resets. Make sure your field has the esCustomMapping prop and that it is set as it is on the demo.

To view this discussion on the web visit https://groups.google.com/d/msgid/dotcms/a8a4c686-8d52-41ff-a4cf-c0a0358f5970n%40googlegroups.com.

[Barış Gülmez]

Ok it works now. But somehow afterwards my other queries relying this "name" field started to return only 1 result. When I remove the custom prop and re-index, it returns the full set as expected. Is there any way to keep its initial behavior as well as auto complete?

[Will Ezell]

What query failed?  Maybe try to multimap the field to elasticsearch and use one for searches and one for autocompletion?   So on the demo server, I added this as the esCustomMapping:

{"type": "text",
   "analyzer": "my_analyzer",
      "fields": {

         "completeme": {

         "type": "completion"

      }

   }
}     

And you can query it like this:

curl -H "Content-Type: application/json" -XPOST https://demo.dotcms.com/api/es/raw -d '
   {
     "suggest": {

       "blog-suggest": {
         "prefix": "fre",        
         "completion": {        

             "field": "blog.title.completeme"  
         }
       }
     }
   }
'

To view this discussion on the web visit https://groups.google.com/d/msgid/dotcms/394c4643-ad8d-4c1d-b4e3-0208dac582b6n%40googlegroups.com.

[Barış Gülmez]

Yes both search and autocomplete work perfectly now! Thank you for the great support!