Limiting Elasticsearch Query Result Fields

23 views
Skip to first unread message

Barış Gülmez

unread,
Nov 16, 2022, 9:02:12 AM11/16/22
to dotCMS User Group
Hello,

I am new to DotCMS. Absolutely a great product! 

I have 2 queries regarding ES. I have searched the documentation and the group for an answer but no luck. 

1) With native ES queries, we can limit the fields returned by ES using "_source" or "fields" parameters in the query. But apparently it does not work with /api/es/search. Sample:

    {
        "_source": ["Article.name", "Story.name"],
        "query": {
            "query_string" : {
                "query" : "(Article.name:stress OR Story.name:stress)"
            }
        }
    }

Result set still comes with all the available data and meta-data fields.

Is there any way to limit the fields returned? I need to limit the fields since I only need title field in the search results page on my frontend.

2) I've tried to use /api/es/raw instead but somehow it does not have full set of the fields in the result set, apparently only Id fields. For instance, the above query returns empty but if I change the query to a field agnostic one as below,

    {
        "query": {
            "query_string" : {
                "query" : "stress"
            }
        }
    }

It returns 4 documents (as expected) but with these fields:

        "max_score": 2.7379642,
        "hits": [
            {
                "_index": "cluster_e7bwe1f0f3.working_20223223233",
                "_type": "_doc",
                "_id": "f9255af77b0eecwe03f5dc2604ff8a89_1",
                "_score": 2.7379642,
                "_source": {
                    "inode": "125762d1-539b-4we6b-989a-edfwe2ef198c",
                    "identifier": "f9255xcf77b0ewew8d03f5dc2604ff8a89"
                }
            },

Which is confusing because it means the relevant fields with "stress" word have been indexed but somehow result set does not have "name" fields ["Article.name", "Story.name"] of these two content types. 

Could you please help me to understand what am I doing wrong?

Thank you.

Will Ezell

unread,
Nov 16, 2022, 9:39:37 AM11/16/22
to dot...@googlegroups.com
With dotCMS, we only store the identifier and the inode in elasticsearch, so you cannot request specific fields back from it.  If you need to return only a few fields, I'd suggest you use the graphQL endpoint - which also takes a query, e.g. +Article.name:stress

There is a graphql playground in the back of dotCMS where you can try different queries - or you can use the example one hooked up to our demo site here (scroll about 1/2 down)

--
http://dotcms.com - Open Source Java Content Management
---
You received this message because you are subscribed to the Google Groups "dotCMS User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dotcms+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dotcms/7c0d0b47-3a66-4be6-b498-da8ac85b180cn%40googlegroups.com.


--



382 NE 191st St #92150
Miami, Florida 33179-3899
Main: 
305-900-2001 | Direct: 978.294.9429

Barış Gülmez

unread,
Nov 16, 2022, 10:21:50 AM11/16/22
to dotCMS User Group
Thanks, I got it. I have already tried GraphQL. It works fine and we can go with it for now. But how about search  suggestions (did you mean) and type ahead (auto complete)? I have tried below query with raw endpoint but since ES does not have the content fields it won't work with "name" field. Any recommendations to implement suggestions (did you mean) or type ahead (auto complete) features for specific content types and their fields?

curl -H "Content-Type: application/json" -XPOST http://localhost:8080/api/es/raw -d ' { "suggest" : { "title-suggestions" : { "text" : "gs pric rollrcoater", "term" : { "size" : 3, "field" : "title" } } } } '

Will Ezell

unread,
Nov 16, 2022, 11:05:11 AM11/16/22
to dot...@googlegroups.com
This works on the demo site - try:

curl -H "Content-Type: application/json" -XPOST https://demo.dotcms.com/api/es/raw -d '{ "suggest" : { "title-suggestions" : { "text" : "frenh Polynesi erything", "term" : { "size" : 3, "field" : "title" } } } }'

Will Ezell

unread,
Nov 16, 2022, 11:15:27 AM11/16/22
to dot...@googlegroups.com
For type ahead completions - you can set up a esCustomMapping for the field you want to try to complete on - I added this to our demo.dotcms.com site's Blog Title field and ran a reindex.

Screen Shot 2022-11-16 at 11.13.34 AM.png
You can then run this query against it:

## type ahead

curl -H "Content-Type: application/json" -XPOST https://demo.dotcms.com/api/es/raw -d '
   {
     "typeAhead": {
       "blog-suggest": {
         "prefix": "fre",        
         "completion": {        
             "field": "blog.title"  
         }
       }
     }
   }
'



 

Barış Gülmez

unread,
Nov 16, 2022, 12:13:56 PM11/16/22
to dotCMS User Group
Suggestions works thank you!

I get this with the typeahead one:

Unknown key for a START_OBJECT in [typeAhead].

Ps. I did re-index as well.

Will Ezell

unread,
Nov 16, 2022, 12:23:56 PM11/16/22
to dot...@googlegroups.com
Check the demo site before it resets. Make sure your field has the esCustomMapping prop and that it is set as it is on the demo.

Barış Gülmez

unread,
Nov 16, 2022, 3:00:01 PM11/16/22
to dotCMS User Group
Ok it works now. But somehow afterwards my other queries relying this "name" field started to return only 1 result. When I remove the custom prop and re-index, it returns the full set as expected. Is there any way to keep its initial behavior as well as auto complete?

Will Ezell

unread,
Nov 16, 2022, 4:46:52 PM11/16/22
to dot...@googlegroups.com
What query failed?  Maybe try to multimap the field to elasticsearch and use one for searches and one for autocompletion?   So on the demo server, I added this as the esCustomMapping:
{"type": "text",
   "analyzer": "my_analyzer",
      "fields": {
         "completeme": {
         "type": "completion"
      }
   }
}     


And you can query it like this:

curl -H "Content-Type: application/json" -XPOST https://demo.dotcms.com/api/es/raw -d '
   {
     "suggest": {
       "blog-suggest": {
         "prefix": "fre",        
         "completion": {        
             "field": "blog.title.completeme"  
         }
       }
     }
   }
'






Barış Gülmez

unread,
Nov 17, 2022, 1:41:51 AM11/17/22
to dotCMS User Group
Yes both search and autocomplete work perfectly now! Thank you for the great support!
Reply all
Reply to author
Forward
0 new messages