Elasticsearch crushes when retrieving user's recommendations

[sashaya...@gmail.com]

Hello! I'm facing some strange problem with ES. After my test data is sent to pio ang UR engine is trained it crushes with this error:

[INFO] [URAlgorithm] Query received, user id: Some(sasha), item id: None
[INFO] [URAlgorithm] Misconfigured date information, either your engine.json date settings or your query's dateRange is incorrect.
Ingoring date information for this query.
[INFO] [URAlgorithm] Query:
{"size":20,"query":{"bool":{"should":[{"terms":{"buy":["prod1","prod19","prod30","prod16","prod4","prod24"]}},{"terms":{"view":["prod1","prod17","prod19","prod24","prod25","prod14","prod6","prod16","mobile","apple","android","asus","amazon-kindle","laptops","birthday","prod4"]}},{"terms":{"like":["prod24","prod19","prod16","prod1","android","apple","amazon-kindle"]}},{"terms":{"rate":[]}},{"terms":{"share":["prod19","prod24","prod16","apple","android","laptops","prod4"]}},{"terms":{"click":["prod1","prod18","prod19","prod6","prod24","prod21","mobile","apple","laptops","android","asus","pc","prod4","birthday"]}},{"terms":{"show":["prod30","android"]}},{"terms":{"comment":[]}},{"constant_score":{"filter":{"match_all":{}},"boost":0}}],"must":[],"must_not":{"ids":{"values":["prod24","prod4","prod16","prod30","prod19","prod1"],"boost":0}},"minimum_should_match":1}},"sort":[{"_score":{"order":"desc"}},{"popRank":{"unmapped_type":"double","order":"desc"}}]}


[ERROR] [ServerActor] Query:
{
  "user" : "sasha"
}


Stack Trace:
org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to execute phase [query], all shards failed; shardFailures {[0DVj5f1DSBGNPAltinMORQ][urindex_1485170227611][0]: QueryPhaseExecutionException[[urindex_1485170227611][0]: query[((buy:prod1 buy:prod19 buy:prod30 buy:prod16 buy:prod4 buy:prod24) (view:prod1 view:prod17 view:prod19 view:prod24 view:prod25 view:prod14 view:prod6 view:prod16 view:mobile view:apple view:android view:asus view:amazon-kindle view:laptops view:birthday view:prod4) (like:prod24 like:prod19 like:prod16 like:prod1 like:android like:apple like:amazon-kindle) () (share:prod19 share:prod24 share:prod16 share:apple share:android share:laptops share:prod4) (click:prod1 click:prod18 click:prod19 click:prod6 click:prod24 click:prod21 click:mobile click:apple click:laptops click:android click:asus click:pc click:prod4 click:birthday) (show:prod30 show:android) () ConstantScore(*:*)^0.0 -ConstantScore(_uid:items#prod1 _uid:items#prod16 _uid:items#prod19 _uid:items#prod24 _uid:items#prod30 _uid:items#prod4 _uid:products#prod1 _uid:products#prod16 _uid:products#prod19 _uid:products#prod24 _uid:products#prod30 _uid:products#prod4)^0.0)~1],from[0],size[20],sort[<score>,<custom:"popRank": org.elasticsearch.index.fielddata.fieldcomparator.FloatValuesComparatorSource@354cc1d8>!]: Query Failed [Failed to execute main query]]; nested: ElasticsearchException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: UncheckedExecutionException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: NumberFormatException[Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; }{[0DVj5f1DSBGNPAltinMORQ][urindex_1485170227611][1]: QueryPhaseExecutionException[[urindex_1485170227611][1]: query[((buy:prod1 buy:prod19 buy:prod30 buy:prod16 buy:prod4 buy:prod24) (view:prod1 view:prod17 view:prod19 view:prod24 view:prod25 view:prod14 view:prod6 view:prod16 view:mobile view:apple view:android view:asus view:amazon-kindle view:laptops view:birthday view:prod4) (like:prod24 like:prod19 like:prod16 like:prod1 like:android like:apple like:amazon-kindle) () (share:prod19 share:prod24 share:prod16 share:apple share:android share:laptops share:prod4) (click:prod1 click:prod18 click:prod19 click:prod6 click:prod24 click:prod21 click:mobile click:apple click:laptops click:android click:asus click:pc click:prod4 click:birthday) (show:prod30 show:android) () ConstantScore(*:*)^0.0 -ConstantScore(_uid:items#prod1 _uid:items#prod16 _uid:items#prod19 _uid:items#prod24 _uid:items#prod30 _uid:items#prod4 _uid:products#prod1 _uid:products#prod16 _uid:products#prod19 _uid:products#prod24 _uid:products#prod30 _uid:products#prod4)^0.0)~1],from[0],size[20],sort[<score>,<custom:"popRank": org.elasticsearch.index.fielddata.fieldcomparator.FloatValuesComparatorSource@384ea1e0>!]: Query Failed [Failed to execute main query]]; nested: ElasticsearchException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: UncheckedExecutionException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: NumberFormatException[Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; }{[0DVj5f1DSBGNPAltinMORQ][urindex_1485170227611][2]: QueryPhaseExecutionException[[urindex_1485170227611][2]: query[((buy:prod1 buy:prod19 buy:prod30 buy:prod16 buy:prod4 buy:prod24) (view:prod1 view:prod17 view:prod19 view:prod24 view:prod25 view:prod14 view:prod6 view:prod16 view:mobile view:apple view:android view:asus view:amazon-kindle view:laptops view:birthday view:prod4) (like:prod24 like:prod19 like:prod16 like:prod1 like:android like:apple like:amazon-kindle) () (share:prod19 share:prod24 share:prod16 share:apple share:android share:laptops share:prod4) (click:prod1 click:prod18 click:prod19 click:prod6 click:prod24 click:prod21 click:mobile click:apple click:laptops click:android click:asus click:pc click:prod4 click:birthday) (show:prod30 show:android) () ConstantScore(*:*)^0.0 -ConstantScore(_uid:items#prod1 _uid:items#prod16 _uid:items#prod19 _uid:items#prod24 _uid:items#prod30 _uid:items#prod4 _uid:products#prod1 _uid:products#prod16 _uid:products#prod19 _uid:products#prod24 _uid:products#prod30 _uid:products#prod4)^0.0)~1],from[0],size[20],sort[<score>,<custom:"popRank": org.elasticsearch.index.fielddata.fieldcomparator.FloatValuesComparatorSource@1a04175b>!]: Query Failed [Failed to execute main query]]; nested: ElasticsearchException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: UncheckedExecutionException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: NumberFormatException[Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; }{[0DVj5f1DSBGNPAltinMORQ][urindex_1485170227611][3]: QueryPhaseExecutionException[[urindex_1485170227611][3]: query[((buy:prod1 buy:prod19 buy:prod30 buy:prod16 buy:prod4 buy:prod24) (view:prod1 view:prod17 view:prod19 view:prod24 view:prod25 view:prod14 view:prod6 view:prod16 view:mobile view:apple view:android view:asus view:amazon-kindle view:laptops view:birthday view:prod4) (like:prod24 like:prod19 like:prod16 like:prod1 like:android like:apple like:amazon-kindle) () (share:prod19 share:prod24 share:prod16 share:apple share:android share:laptops share:prod4) (click:prod1 click:prod18 click:prod19 click:prod6 click:prod24 click:prod21 click:mobile click:apple click:laptops click:android click:asus click:pc click:prod4 click:birthday) (show:prod30 show:android) () ConstantScore(*:*)^0.0 -ConstantScore(_uid:items#prod1 _uid:items#prod16 _uid:items#prod19 _uid:items#prod24 _uid:items#prod30 _uid:items#prod4 _uid:products#prod1 _uid:products#prod16 _uid:products#prod19 _uid:products#prod24 _uid:products#prod30 _uid:products#prod4)^0.0)~1],from[0],size[20],sort[<score>,<custom:"popRank": org.elasticsearch.index.fielddata.fieldcomparator.FloatValuesComparatorSource@25e7339>!]: Query Failed [Failed to execute main query]]; nested: ElasticsearchException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: UncheckedExecutionException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: NumberFormatException[Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; }{[0DVj5f1DSBGNPAltinMORQ][urindex_1485170227611][4]: QueryPhaseExecutionException[[urindex_1485170227611][4]: query[((buy:prod1 buy:prod19 buy:prod30 buy:prod16 buy:prod4 buy:prod24) (view:prod1 view:prod17 view:prod19 view:prod24 view:prod25 view:prod14 view:prod6 view:prod16 view:mobile view:apple view:android view:asus view:amazon-kindle view:laptops view:birthday view:prod4) (like:prod24 like:prod19 like:prod16 like:prod1 like:android like:apple like:amazon-kindle) () (share:prod19 share:prod24 share:prod16 share:apple share:android share:laptops share:prod4) (click:prod1 click:prod18 click:prod19 click:prod6 click:prod24 click:prod21 click:mobile click:apple click:laptops click:android click:asus click:pc click:prod4 click:birthday) (show:prod30 show:android) () ConstantScore(*:*)^0.0 -ConstantScore(_uid:items#prod1 _uid:items#prod16 _uid:items#prod19 _uid:items#prod24 _uid:items#prod30 _uid:items#prod4 _uid:products#prod1 _uid:products#prod16 _uid:products#prod19 _uid:products#prod24 _uid:products#prod30 _uid:products#prod4)^0.0)~1],from[0],size[20],sort[<score>,<custom:"popRank": org.elasticsearch.index.fielddata.fieldcomparator.FloatValuesComparatorSource@48273dde>!]: Query Failed [Failed to execute main query]]; nested: ElasticsearchException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: UncheckedExecutionException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: NumberFormatException[Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; }
 at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:237)
 at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$1.onFailure(TransportSearchTypeAction.java:183)
 at org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:565)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 at java.lang.Thread.run(Thread.java:745)

engine.json and data in event server are here: https://drive.google.com/open?id=0BxYhPQY2AFqqcVFMY1VvdGJfNFk

Do you have any idea, what am I doing wrong? Thank you in advance for any help!

[Pat Ferrel]

It looks like you have a bunch of malformed data in Elasticsearch. Can you show us your event input to the EventServer?

--
You received this message because you are subscribed to the Google Groups "actionml-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to actionml-use...@googlegroups.com.
To post to this group, send email to action...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/actionml-user/b6af36b6-b621-433e-a054-34d43fc51472%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[sashaya...@gmail.com]

Here is link to engine.json and event data, loaded to event server. There is also a table with loaded model for better representation.

https://drive.google.com/drive/folders/0BxYhPQY2AFqqcVFMY1VvdGJfNFk?usp=sharing

[sashaya...@gmail.com]

Today I've tried to make recommendations on smaller set of data with just two events(buy and view) in engine.json. Here it is:

[{"eventId":"F5deKVqES9FxJJrihYrqMAAAAVnPuBFCoRTyf4OB0go","event":"$set","entityType":"product","entityId":"prod1","properties":{"uid":["prod1"]},"eventTime":"2017-01-24T09:04:23.874Z","creationTime":"2017-01-24T09:04:23.884Z"},{"eventId":"H37UPoNX8-sleZd7a-YIPQAAAVnPviPykm75xg9Atd0","event":"$set","entityType":"cart","entityId":"fb1bbd30-acbc-4d91-ab08-3172ce1a8122","properties":{},"eventTime":"2017-01-24T09:11:01.874Z","creationTime":"2017-01-24T09:11:01.877Z"},{"eventId":"H37UPoNX8-sleZd7a-YIPQAAAVnPviP7smJfZQQix3Y","event":"add_to_cart","entityType":"cart","entityId":"fb1bbd30-acbc-4d91-ab08-3172ce1a8122","targetEntityType":"product","targetEntityId":"prod1","properties":{},"eventTime":"2017-01-24T09:11:01.883Z","creationTime":"2017-01-24T09:11:01.886Z"},{"eventId":"H37UPoNX8-sleZd7a-YIPQAAAVnPviQ5glPHsUE1Kak","event":"add_to_cart","entityType":"cart","entityId":"fb1bbd30-acbc-4d91-ab08-3172ce1a8122","targetEntityType":"product","targetEntityId":"prod5","properties":{},"eventTime":"2017-01-24T09:11:01.945Z","creationTime":"2017-01-24T09:11:01.948Z"},{"eventId":"JIf_x3HySZDq9PPEpb53zgAAAVnPuBW_iKQ2gw_-hS8","event":"$set","entityType":"product","entityId":"prod5","properties":{"uid":["prod5"]},"eventTime":"2017-01-24T09:04:25.023Z","creationTime":"2017-01-24T09:04:25.027Z"},{"eventId":"NW1c890Kiyt1NH3WLMTTxAAAAVnPuBRjl6SXaEr535I","event":"$set","entityType":"product","entityId":"prod3","properties":{"uid":["prod3"]},"eventTime":"2017-01-24T09:04:24.675Z","creationTime":"2017-01-24T09:04:24.679Z"},{"eventId":"U8HQtFAmwi12CGYpF0PgtgAAAVnPuMUxt1KBxc8XPW8","event":"$set","entityType":"user","entityId":"alice","properties":{},"eventTime":"2017-01-24T09:05:09.937Z","creationTime":"2017-01-24T09:05:09.941Z"},{"eventId":"U8HQtFAmwi12CGYpF0PgtgAAAVnPvFD4sIS9kg1zRO0","event":"view","entityType":"user","entityId":"alice","targetEntityType":"product","targetEntityId":"prod2","properties":{},"eventTime":"2017-01-24T09:09:02.328Z","creationTime":"2017-01-24T09:09:02.334Z"},{"eventId":"U8HQtFAmwi12CGYpF0PgtgAAAVnPvFGrvhG_TraSGUE","event":"view","entityType":"user","entityId":"alice","targetEntityType":"product","targetEntityId":"prod4","properties":{},"eventTime":"2017-01-24T09:09:02.507Z","creationTime":"2017-01-24T09:09:02.511Z"},{"eventId":"U8HQtFAmwi12CGYpF0PgtgAAAVnPvFHggFOHUkPssjM","event":"view","entityType":"user","entityId":"alice","targetEntityType":"product","targetEntityId":"prod5","properties":{},"eventTime":"2017-01-24T09:09:02.560Z","creationTime":"2017-01-24T09:09:02.563Z"},{"eventId":"U8HQtFAmwi12CGYpF0PgtgAAAVnPvFLkjgMHBl8encc","event":"buy","entityType":"user","entityId":"alice","targetEntityType":"product","targetEntityId":"prod2","properties":{},"eventTime":"2017-01-24T09:09:02.820Z","creationTime":"2017-01-24T09:09:02.823Z"},{"eventId":"U8HQtFAmwi12CGYpF0PgtgAAAVnPvFMCrWXzOme-kWI","event":"buy","entityType":"user","entityId":"alice","targetEntityType":"product","targetEntityId":"prod3","properties":{},"eventTime":"2017-01-24T09:09:02.850Z","creationTime":"2017-01-24T09:09:02.853Z"},{"eventId":"U8HQtFAmwi12CGYpF0PgtgAAAVnPvFMghEzk4-hS3cE","event":"buy","entityType":"user","entityId":"alice","targetEntityType":"product","targetEntityId":"prod4","properties":{},"eventTime":"2017-01-24T09:09:02.880Z","creationTime":"2017-01-24T09:09:02.883Z"},{"eventId":"gkaa5Gb2QaY0M5Kjy5iYBAAAAVnPuBUSpB7Z9QhFNeg","event":"$set","entityType":"product","entityId":"prod4","properties":{"uid":["prod4"]},"eventTime":"2017-01-24T09:04:24.850Z","creationTime":"2017-01-24T09:04:24.853Z"},{"eventId":"ponPgaOtSZo2c1IhrAPtkgAAAVnPuBO3iIpIU5Y5ATw","event":"$set","entityType":"product","entityId":"prod2","properties":{"uid":["prod2"]},"eventTime":"2017-01-24T09:04:24.503Z","creationTime":"2017-01-24T09:04:24.507Z"},{"eventId":"uRxfHMznXKoPqED7Cay54gAAAVnPuVuOnLPr6VF-_LY","event":"$set","entityType":"cart","entityId":"5d1543b4-0be1-400c-8c45-9e0508d075f1","properties":{},"eventTime":"2017-01-24T09:05:48.430Z","creationTime":"2017-01-24T09:05:48.437Z"},{"eventId":"uRxfHMznXKoPqED7Cay54gAAAVnPuVughuSxtyPVdnI","event":"add_to_cart","entityType":"cart","entityId":"5d1543b4-0be1-400c-8c45-9e0508d075f1","targetEntityType":"product","targetEntityId":"prod1","properties":{},"eventTime":"2017-01-24T09:05:48.448Z","creationTime":"2017-01-24T09:05:48.453Z"},{"eventId":"uRxfHMznXKoPqED7Cay54gAAAVnPuVv4hc07-Fc8K7w","event":"add_to_cart","entityType":"cart","entityId":"5d1543b4-0be1-400c-8c45-9e0508d075f1","targetEntityType":"product","targetEntityId":"prod2","properties":{},"eventTime":"2017-01-24T09:05:48.536Z","creationTime":"2017-01-24T09:05:48.542Z"},{"eventId":"uRxfHMznXKoPqED7Cay54gAAAVnPuVw_uOrg4KaYnRU","event":"add_to_cart","entityType":"cart","entityId":"5d1543b4-0be1-400c-8c45-9e0508d075f1","targetEntityType":"product","targetEntityId":"prod3","properties":{},"eventTime":"2017-01-24T09:05:48.607Z","creationTime":"2017-01-24T09:05:48.610Z"},{"eventId":"z3c6w210xrwv_IsGmI1OuAAAAVnPuMVdnNXmXgAp_OA","event":"$set","entityType":"user","entityId":"eva","properties":{},"eventTime":"2017-01-24T09:05:09.981Z","creationTime":"2017-01-24T09:05:09.984Z"},{"eventId":"z3c6w210xrwv_IsGmI1OuAAAAVnPviL8s8d4S69WXq4","event":"view","entityType":"user","entityId":"eva","targetEntityType":"product","targetEntityId":"prod1","properties":{},"eventTime":"2017-01-24T09:11:01.628Z","creationTime":"2017-01-24T09:11:01.634Z"},{"eventId":"z3c6w210xrwv_IsGmI1OuAAAAVnPviNBgSS8Nb9lORw","event":"view","entityType":"user","entityId":"eva","targetEntityType":"product","targetEntityId":"prod3","properties":{},"eventTime":"2017-01-24T09:11:01.697Z","creationTime":"2017-01-24T09:11:01.701Z"},{"eventId":"z3c6w210xrwv_IsGmI1OuAAAAVnPviN8sRKKb2Ngb0g","event":"view","entityType":"user","entityId":"eva","targetEntityType":"product","targetEntityId":"prod4","properties":{},"eventTime":"2017-01-24T09:11:01.756Z","creationTime":"2017-01-24T09:11:01.760Z"},{"eventId":"z3c6w210xrwv_IsGmI1OuAAAAVnPviOzsw3Kq3mqcgo","event":"view","entityType":"user","entityId":"eva","targetEntityType":"product","targetEntityId":"prod5","properties":{},"eventTime":"2017-01-24T09:11:01.811Z","creationTime":"2017-01-24T09:11:01.815Z"},{"eventId":"z3c6w210xrwv_IsGmI1OuAAAAVnPviRxuRWrQW5U7wQ","event":"buy","entityType":"user","entityId":"eva","targetEntityType":"product","targetEntityId":"prod5","properties":{},"eventTime":"2017-01-24T09:11:02.001Z","creationTime":"2017-01-24T09:11:02.004Z"},{"eventId":"z3c6w210xrwv_IsGmI1OuAAAAVnPviSOo6AwyUwg3Ak","event":"buy","entityType":"user","entityId":"eva","targetEntityType":"product","targetEntityId":"prod1","properties":{},"eventTime":"2017-01-24T09:11:02.030Z","creationTime":"2017-01-24T09:11:02.033Z"},{"eventId":"2Js-g6TvG57nVVXxwW8b_AAAAVnPuMWFiB1JulOGRFw","event":"$set","entityType":"user","entityId":"sasha","properties":{},"eventTime":"2017-01-24T09:05:10.021Z","creationTime":"2017-01-24T09:05:10.025Z"},{"eventId":"2X5vPnF74q6IC2V6KEE1SAAAAVnPuMTzqlVVq67yqaA","event":"$set","entityType":"user","entityId":"bob","properties":{},"eventTime":"2017-01-24T09:05:09.875Z","creationTime":"2017-01-24T09:05:09.883Z"},{"eventId":"2X5vPnF74q6IC2V6KEE1SAAAAVnPuVnMj2Urwfftjfw","event":"view","entityType":"user","entityId":"bob","targetEntityType":"product","targetEntityId":"prod1","properties":{},"eventTime":"2017-01-24T09:05:47.980Z","creationTime":"2017-01-24T09:05:47.986Z"},{"eventId":"2X5vPnF74q6IC2V6KEE1SAAAAVnPuVoPvMtoCnrZ800","event":"view","entityType":"user","entityId":"bob","targetEntityType":"product","targetEntityId":"prod2","properties":{},"eventTime":"2017-01-24T09:05:48.047Z","creationTime":"2017-01-24T09:05:48.051Z"},{"eventId":"2X5vPnF74q6IC2V6KEE1SAAAAVnPuVpGs8UoON2tONE","event":"view","entityType":"user","entityId":"bob","targetEntityType":"product","targetEntityId":"prod3","properties":{},"eventTime":"2017-01-24T09:05:48.102Z","creationTime":"2017-01-24T09:05:48.106Z"},{"eventId":"2X5vPnF74q6IC2V6KEE1SAAAAVnPuVp3vuJbTL8ql6E","event":"view","entityType":"user","entityId":"bob","targetEntityType":"product","targetEntityId":"prod4","properties":{},"eventTime":"2017-01-24T09:05:48.151Z","creationTime":"2017-01-24T09:05:48.154Z"},{"eventId":"2X5vPnF74q6IC2V6KEE1SAAAAVnPuVx2hN7Ve-Uu9gk","event":"buy","entityType":"user","entityId":"bob","targetEntityType":"product","targetEntityId":"prod3","properties":{},"eventTime":"2017-01-24T09:05:48.662Z","creationTime":"2017-01-24T09:05:48.665Z"},{"eventId":"2X5vPnF74q6IC2V6KEE1SAAAAVnPuVyYrbXtCPFGaH8","event":"buy","entityType":"user","entityId":"bob","targetEntityType":"product","targetEntityId":"prod1","properties":{},"eventTime":"2017-01-24T09:05:48.696Z","creationTime":"2017-01-24T09:05:48.699Z"},{"eventId":"2X5vPnF74q6IC2V6KEE1SAAAAVnPuVy2hjHqzW0P_7I","event":"buy","entityType":"user","entityId":"bob","targetEntityType":"product","targetEntityId":"prod2","properties":{},"eventTime":"2017-01-24T09:05:48.726Z","creationTime":"2017-01-24T09:05:48.729Z"},{"eventId":"6wnRtG4Z7lv5HiuPGCIuPQAAAVnPvFIfgQ3v3mXsbHQ","event":"$set","entityType":"cart","entityId":"7bf90119-b68e-4259-8a38-88a6d6bdbb74","properties":{},"eventTime":"2017-01-24T09:09:02.623Z","creationTime":"2017-01-24T09:09:02.627Z"},{"eventId":"6wnRtG4Z7lv5HiuPGCIuPQAAAVnPvFIqvfLcuQAv188","event":"add_to_cart","entityType":"cart","entityId":"7bf90119-b68e-4259-8a38-88a6d6bdbb74","targetEntityType":"product","targetEntityId":"prod2","properties":{},"eventTime":"2017-01-24T09:09:02.634Z","creationTime":"2017-01-24T09:09:02.637Z"},{"eventId":"6wnRtG4Z7lv5HiuPGCIuPQAAAVnPvFJpjr8-tSDNdzA","event":"add_to_cart","entityType":"cart","entityId":"7bf90119-b68e-4259-8a38-88a6d6bdbb74","targetEntityType":"product","targetEntityId":"prod3","properties":{},"eventTime":"2017-01-24T09:09:02.697Z","creationTime":"2017-01-24T09:09:02.701Z"},{"eventId":"6wnRtG4Z7lv5HiuPGCIuPQAAAVnPvFKrqP0HlK7-CmY","event":"add_to_cart","entityType":"cart","entityId":"7bf90119-b68e-4259-8a38-88a6d6bdbb74","targetEntityType":"product","targetEntityId":"prod4","properties":{},"eventTime":"2017-01-24T09:09:02.763Z","creationTime":"2017-01-24T09:09:02.766Z"}]

I've tried to load data step by step for each user(for bob,  alive and then for eva in that order). The result is: when loaded for bob and alice it is expected to recommend prod1(with some score) and prod5(with 0 score), but insted I get just prod5.

[INFO] [URAlgorithm] Query received, user id: Some(alice), item id: None

[INFO] [URAlgorithm] Misconfigured date information, either your engine.json date settings or your query

's dateRange is incorrect.

Ingoring date information for this query.
[INFO] [URAlgorithm] Query:

{"size":20,"query":{"bool":{"should":[{"terms":{"buy":["prod2","prod3","prod4"]}},{"terms":{"view":["prod2","prod4","prod5"]}},{"constant_score":{"filter":{"match_all":{}},"boost":0}}],"must":[],"must_not":{"ids":{"values":["prod4","prod3","prod2"],"boost":0}},"minimum_should_match":1}},"sort":[{"_score":{"order":"desc"}},{"popRank":{"unmapped_type":"double","order":"desc"}}]}


[INFO] [esClient$] Results: 1 retrieved of a possible 1

It seems really strange that engine doesn't recommend prod1, since there is correlation with bob's history. After that I've loded events for eva, what led to ES failure. What am I doing wrong? 

понедельник, 23 января 2017 г., 19:30:25 UTC+3 пользователь pat написал:

[sashaya...@gmail.com]

Hi, Pat! We've figured it out: this happens because of eventTime property. If we specify it as a day in the past (for instance, 01/01/2015), engine works as expected. If event time is added automatically it crashes ES. Do you know, why this happens and how then we should send events to pio of current time?

понедельник, 23 января 2017 г., 19:30:25 UTC+3 пользователь pat написал:

It looks like you have a bunch of malformed data in Elasticsearch. Can you show us your event input to the EventServer?

[sashaya...@gmail.com]

We also figured out that for correct predictions without any ES failures there ought to be events with eventTime at least 3 days before creationTime.
If some events happened 3 days ago, and some happened later it works as well, but with amount of recommendations reduced. How should events be sent then? Why this 3 days are so essential? Thank you for any explanation!

понедельник, 23 января 2017 г., 19:30:25 UTC+3 пользователь pat написал:

It looks like you have a bunch of malformed data in Elasticsearch. Can you show us your event input to the EventServer?

[Pat Ferrel]

Don’t use the engine.json used by the integration-test, it has complex parameters meant for testing, not your use. Start from engine.json.minimum.

The internal Elasticsearch query still looks very odd, please supply an example of your input and engine.json

To view this discussion on the web visit https://groups.google.com/d/msgid/actionml-user/357ff31f-0285-4e98-a6cc-62495781b0c2%40googlegroups.com.

[sashaya...@gmail.com]

Thank you for you patience! Here is engine.json and events loded to EventServer:

[

• {

  • eventId: "F5deKVqES9FxJJrihYrqMAAAAVnV0ZAPnrjOaMFppMQ",

• • event: "$set",
  • entityType: "product",
  • entityId: "prod1",

• • properties: { },
  • eventTime: "2017-01-25T13:29:58.031Z",
  • creationTime: "2017-01-25T13:29:58.031Z"

  },
• {

  • eventId: "JIf_x3HySZDq9PPEpb53zgAAAVnV0ZDSniISv_E6GAY",

• • event: "$set",
  • entityType: "product",
  • entityId: "prod5",

• • properties: { },
  • eventTime: "2017-01-25T13:29:58.226Z",
  • creationTime: "2017-01-25T13:29:58.226Z"

  },
• {

  • eventId: "NW1c890Kiyt1NH3WLMTTxAAAAVnV0ZC1iosScdU6QW0",

• • event: "$set",
  • entityType: "product",
  • entityId: "prod3",

• • properties: { },
  • eventTime: "2017-01-25T13:29:58.197Z",
  • creationTime: "2017-01-25T13:29:58.197Z"

  },
• {

  • eventId: "U8HQtFAmwi12CGYpF0PgtgAAAVnV0aWFpr28ox6PLKs",

• • event: "$set",
  • entityType: "user",
  • entityId: "alice",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.525Z",
  • creationTime: "2017-01-25T13:30:03.525Z"

  },
• {

  • eventId: "U8HQtFAmwi12CGYpF0PgtgAAAVnV0aYLmVV9IDXrdPI",

• • event: "view",
  • entityType: "user",
  • entityId: "alice",
  • targetEntityType: "product",
  • targetEntityId: "prod2",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.659Z",
  • creationTime: "2017-01-25T13:30:03.659Z"

  },
• {

  • eventId: "U8HQtFAmwi12CGYpF0PgtgAAAVnV0aYYhAjnHWrlnrk",

• • event: "view",
  • entityType: "user",
  • entityId: "alice",
  • targetEntityType: "product",
  • targetEntityId: "prod4",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.672Z",
  • creationTime: "2017-01-25T13:30:03.672Z"

  },
• {

  • eventId: "U8HQtFAmwi12CGYpF0PgtgAAAVnV0aYkpJIncJphvIc",

• • event: "view",
  • entityType: "user",
  • entityId: "alice",
  • targetEntityType: "product",
  • targetEntityId: "prod5",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.684Z",
  • creationTime: "2017-01-25T13:30:03.684Z"

  },
• {

  • eventId: "U8HQtFAmwi12CGYpF0PgtgAAAVnV0aYwo6AVEe3g1w0",

• • event: "buy",
  • entityType: "user",
  • entityId: "alice",
  • targetEntityType: "product",
  • targetEntityId: "prod2",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.696Z",
  • creationTime: "2017-01-25T13:30:03.696Z"

  },
• {

  • eventId: "U8HQtFAmwi12CGYpF0PgtgAAAVnV0aY8ohezsOVdeEI",

• • event: "buy",
  • entityType: "user",
  • entityId: "alice",
  • targetEntityType: "product",
  • targetEntityId: "prod3",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.708Z",
  • creationTime: "2017-01-25T13:30:03.708Z"

  },
• {

  • eventId: "U8HQtFAmwi12CGYpF0PgtgAAAVnV0aZIj7Y5djJXP94",

• • event: "buy",
  • entityType: "user",
  • entityId: "alice",
  • targetEntityType: "product",
  • targetEntityId: "prod4",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.720Z",
  • creationTime: "2017-01-25T13:30:03.720Z"

  },
• {

  • eventId: "gkaa5Gb2QaY0M5Kjy5iYBAAAAVnV0ZDEhe1ptSF_zMU",

• • event: "$set",
  • entityType: "product",
  • entityId: "prod4",

• • properties: { },
  • eventTime: "2017-01-25T13:29:58.212Z",
  • creationTime: "2017-01-25T13:29:58.212Z"

  },
• {

  • eventId: "ponPgaOtSZo2c1IhrAPtkgAAAVnV0ZClo4XbZ-FnJKw",

• • event: "$set",
  • entityType: "product",
  • entityId: "prod2",

• • properties: { },
  • eventTime: "2017-01-25T13:29:58.181Z",
  • creationTime: "2017-01-25T13:29:58.181Z"

  },
• {

  • eventId: "z3c6w210xrwv_IsGmI1OuAAAAVnV0aWTuksXCD41xOU",

• • event: "$set",
  • entityType: "user",
  • entityId: "eva",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.539Z",
  • creationTime: "2017-01-25T13:30:03.539Z"

  },
• {

  • eventId: "z3c6w210xrwv_IsGmI1OuAAAAVnV0aZUlChtxlVuEMY",

• • event: "view",
  • entityType: "user",
  • entityId: "eva",
  • targetEntityType: "product",
  • targetEntityId: "prod1",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.732Z",
  • creationTime: "2017-01-25T13:30:03.732Z"

  },
• {

  • eventId: "z3c6w210xrwv_IsGmI1OuAAAAVnV0aZgsNEDgiE0vDE",

• • event: "view",
  • entityType: "user",
  • entityId: "eva",
  • targetEntityType: "product",
  • targetEntityId: "prod3",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.744Z",
  • creationTime: "2017-01-25T13:30:03.744Z"

  },
• {

  • eventId: "z3c6w210xrwv_IsGmI1OuAAAAVnV0aZrs24HmkI1rWI",

• • event: "view",
  • entityType: "user",
  • entityId: "eva",
  • targetEntityType: "product",
  • targetEntityId: "prod4",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.755Z",
  • creationTime: "2017-01-25T13:30:03.755Z"

  },
• {

  • eventId: "z3c6w210xrwv_IsGmI1OuAAAAVnV0aZ3udU2Ih5qmrY",

• • event: "view",
  • entityType: "user",
  • entityId: "eva",
  • targetEntityType: "product",
  • targetEntityId: "prod5",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.767Z",
  • creationTime: "2017-01-25T13:30:03.767Z"

  },
• {

  • eventId: "z3c6w210xrwv_IsGmI1OuAAAAVnV0aaDprFgUh51qao",

• • event: "buy",
  • entityType: "user",
  • entityId: "eva",
  • targetEntityType: "product",
  • targetEntityId: "prod1",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.779Z",
  • creationTime: "2017-01-25T13:30:03.779Z"

  },
• {

  • eventId: "z3c6w210xrwv_IsGmI1OuAAAAVnV0aaOrtUnm6NxMZU",

• • event: "buy",
  • entityType: "user",
  • entityId: "eva",
  • targetEntityType: "product",
  • targetEntityId: "prod5",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.790Z",
  • creationTime: "2017-01-25T13:30:03.790Z"

  },
• {

  • eventId: "2Js-g6TvG57nVVXxwW8b_AAAAVnV0aWioNwnvU-bhrI",

• • event: "$set",
  • entityType: "user",
  • entityId: "sasha",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.554Z",
  • creationTime: "2017-01-25T13:30:03.554Z"

  },
• {

  • eventId: "2Js-g6TvG57nVVXxwW8b_AAAAVnV0aaaoBz8eFZfAKM",

• • event: "view",
  • entityType: "user",

• • entityId: "sasha",

• • targetEntityType: "product",
  • targetEntityId: "prod2",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.802Z",
  • creationTime: "2017-01-25T13:30:03.802Z"

  },
• {

  • eventId: "2Js-g6TvG57nVVXxwW8b_AAAAVnV0aalmETmy7FnXkg",

• • event: "view",
  • entityType: "user",

• • entityId: "sasha",

• • targetEntityType: "product",
  • targetEntityId: "prod4",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.813Z",
  • creationTime: "2017-01-25T13:30:03.813Z"

  },
• {

  • eventId: "2Js-g6TvG57nVVXxwW8b_AAAAVnV0aaxlbPnARjOhnA",

• • event: "view",
  • entityType: "user",

• • entityId: "sasha",

• • targetEntityType: "product",
  • targetEntityId: "prod5",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.825Z",
  • creationTime: "2017-01-25T13:30:03.825Z"

  },
• {

  • eventId: "2Js-g6TvG57nVVXxwW8b_AAAAVnV0aa8g9CQl3yaC9M",

• • event: "buy",
  • entityType: "user",

• • entityId: "sasha",

• • targetEntityType: "product",
  • targetEntityId: "prod3",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.836Z",
  • creationTime: "2017-01-25T13:30:03.836Z"

  },
• {

  • eventId: "2Js-g6TvG57nVVXxwW8b_AAAAVnV0abIt5yFWI3oT0o",

• • event: "buy",
  • entityType: "user",

• • entityId: "sasha",

• • targetEntityType: "product",
  • targetEntityId: "prod4",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.848Z",
  • creationTime: "2017-01-25T13:30:03.848Z"

  },
• {

  • eventId: "2X5vPnF74q6IC2V6KEE1SAAAAVnV0aV1lIUKNvB95-g",

• • event: "$set",
  • entityType: "user",
  • entityId: "bob",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.509Z",
  • creationTime: "2017-01-25T13:30:03.509Z"

  },
• {

  • eventId: "2X5vPnF74q6IC2V6KEE1SAAAAVnV0aWwgjE8P_9Zfns",

• • event: "view",
  • entityType: "user",
  • entityId: "bob",
  • targetEntityType: "product",
  • targetEntityId: "prod1",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.568Z",
  • creationTime: "2017-01-25T13:30:03.568Z"

  },
• {

  • eventId: "2X5vPnF74q6IC2V6KEE1SAAAAVnV0aW-ujF_161xpJw",

• • event: "view",
  • entityType: "user",
  • entityId: "bob",
  • targetEntityType: "product",
  • targetEntityId: "prod2",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.582Z",
  • creationTime: "2017-01-25T13:30:03.582Z"

  },
• {

  • eventId: "2X5vPnF74q6IC2V6KEE1SAAAAVnV0aXMvDWFwHZ2FCs",

• • event: "view",
  • entityType: "user",
  • entityId: "bob",
  • targetEntityType: "product",
  • targetEntityId: "prod3",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.596Z",
  • creationTime: "2017-01-25T13:30:03.596Z"

  },
• {

  • eventId: "2X5vPnF74q6IC2V6KEE1SAAAAVnV0aXZpEuCDqhbZVE",

• • event: "view",
  • entityType: "user",
  • entityId: "bob",
  • targetEntityType: "product",
  • targetEntityId: "prod4",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.609Z",
  • creationTime: "2017-01-25T13:30:03.609Z"

  },
• {

  • eventId: "2X5vPnF74q6IC2V6KEE1SAAAAVnV0aXmlMH630Wfigg",

• • event: "buy",
  • entityType: "user",
  • entityId: "bob",
  • targetEntityType: "product",
  • targetEntityId: "prod1",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.622Z",
  • creationTime: "2017-01-25T13:30:03.622Z"

  },
• {

  • eventId: "2X5vPnF74q6IC2V6KEE1SAAAAVnV0aXyma7hVkIKg38",

• • event: "buy",
  • entityType: "user",
  • entityId: "bob",
  • targetEntityType: "product",
  • targetEntityId: "prod2",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.634Z",
  • creationTime: "2017-01-25T13:30:03.634Z"

  },
• {

  • eventId: "2X5vPnF74q6IC2V6KEE1SAAAAVnV0aX_o2VMeGfHdJA",

• • event: "buy",
  • entityType: "user",
  • entityId: "bob",
  • targetEntityType: "product",
  • targetEntityId: "prod3",
  • properties: { },

• • eventTime: "2017-01-25T13:30:03.647Z",
  • creationTime: "2017-01-25T13:30:03.647Z"

  }
• 
  
  ]

{
  "comment":" This config file uses default settings for all but the required values see README.md for docs",
  "id": "default",
  "description": "Default settings",
  "engineFactory": "org.template.RecommendationEngine",
  "datasource": {
    "params" : {
      "name": "sample-handmade-data.txt",
      "appName": "catalog",
      "itemEntityType" : "product",
      "eventNames": ["buy", "view"]
    }
  },
  "sparkConf": {
    "spark.serializer": "org.apache.spark.serializer.KryoSerializer",
    "spark.kryo.registrator": "org.apache.mahout.sparkbindings.io.MahoutKryoRegistrator",
    "spark.kryo.referenceTracking": "false",
    "spark.kryoserializer.buffer": "300m",
    "spark.executor.memory": "4g",
    "es.index.auto.create": "true"
  },
  "algorithms": [
    {
      "comment": "simplest setup where all values are default, popularity based backfill, must add eventsNames",
      "name": "ur",
      "params": {
        "appName": "catalog",
        "indexName": "urindex",
        "typeName": "products",
        "comment": "must have data for the first event or the model will not build, other events are optional",
        "eventNames": ["buy", "view"]
      }
    }
  ]
}

Build and training are done without any errors. But any query to UR fails with next error:

[INFO] [URAlgorithm] Query received, user id: Some(alice), item id: None

[INFO] [URAlgorithm] Misconfigured date information, either your engine.json date settings or your query

's dateRange is incorrect.

Ingoring date information for this query.
[INFO] [URAlgorithm] Query:

{"size":20,"query":{"bool":{"should":[{"terms":{"buy":["prod2","prod3","prod4"]}},{"terms":{"view":["prod2","prod4","prod5"]}},{"constant_score":{"filter":{"match_all":{}},"boost":0}}],"must":[],"must_not":{"ids":{"values":["prod4","prod3","prod2"],"boost":0}},"minimum_should_match":1}},"sort":[{"_score":{"order":"desc"}},{"popRank":{"unmapped_type":"double","order":"desc"}}]}


[ERROR] [ServerActor] Query:
{
  "user": "alice"
}


Stack Trace:
org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to execute phase [query], all shards failed; shardFailures {[QACMUHt9TXCwG-DxKU--oQ][urindex_1485351081586][0]: QueryPhaseExecutionException[[urindex_1485351081586][0]: query[((buy:prod2 buy:prod3 buy:prod4) (view:prod2 view:prod4 view:prod5) ConstantScore(*:*)^0.0 -ConstantScore(_uid:items#prod2 _uid:items#prod3 _uid:items#prod4 _uid:products#prod2 _uid:products#prod3 _uid:products#prod4)^0.0)~1],from[0],size[20],sort[<score>,<custom:"popRank": org.elasticsearch.index.fielddata.fieldcomparator.FloatValuesComparatorSource@73cee357>!]: Query Failed [Failed to execute main query]]; nested: ElasticsearchException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: UncheckedExecutionException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: NumberFormatException[Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; }{[QACMUHt9TXCwG-DxKU--oQ][urindex_1485351081586][1]: QueryPhaseExecutionException[[urindex_1485351081586][1]: query[((buy:prod2 buy:prod3 buy:prod4) (view:prod2 view:prod4 view:prod5) ConstantScore(*:*)^0.0 -ConstantScore(_uid:items#prod2 _uid:items#prod3 _uid:items#prod4 _uid:products#prod2 _uid:products#prod3 _uid:products#prod4)^0.0)~1],from[0],size[20],sort[<score>,<custom:"popRank": org.elasticsearch.index.fielddata.fieldcomparator.FloatValuesComparatorSource@2f150d5>!]: Query Failed [Failed to execute main query]]; nested: ElasticsearchException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: UncheckedExecutionException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: NumberFormatException[Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; }{[QACMUHt9TXCwG-DxKU--oQ][urindex_1485351081586][2]: QueryPhaseExecutionException[[urindex_1485351081586][2]: query[((buy:prod2 buy:prod3 buy:prod4) (view:prod2 view:prod4 view:prod5) ConstantScore(*:*)^0.0 -ConstantScore(_uid:items#prod2 _uid:items#prod3 _uid:items#prod4 _uid:products#prod2 _uid:products#prod3 _uid:products#prod4)^0.0)~1],from[0],size[20],sort[<score>,<custom:"popRank": org.elasticsearch.index.fielddata.fieldcomparator.FloatValuesComparatorSource@69328abd>!]: Query Failed [Failed to execute main query]]; nested: ElasticsearchException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: UncheckedExecutionException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: NumberFormatException[Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; }{[QACMUHt9TXCwG-DxKU--oQ][urindex_1485351081586][3]: QueryPhaseExecutionException[[urindex_1485351081586][3]: query[((buy:prod2 buy:prod3 buy:prod4) (view:prod2 view:prod4 view:prod5) ConstantScore(*:*)^0.0 -ConstantScore(_uid:items#prod2 _uid:items#prod3 _uid:items#prod4 _uid:products#prod2 _uid:products#prod3 _uid:products#prod4)^0.0)~1],from[0],size[20],sort[<score>,<custom:"popRank": org.elasticsearch.index.fielddata.fieldcomparator.FloatValuesComparatorSource@a8fdf16>!]: Query Failed [Failed to execute main query]]; nested: ElasticsearchException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: UncheckedExecutionException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: NumberFormatException[Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; }{[QACMUHt9TXCwG-DxKU--oQ][urindex_1485351081586][4]: QueryPhaseExecutionException[[urindex_1485351081586][4]: query[((buy:prod2 buy:prod3 buy:prod4) (view:prod2 view:prod4 view:prod5) ConstantScore(*:*)^0.0 -ConstantScore(_uid:items#prod2 _uid:items#prod3 _uid:items#prod4 _uid:products#prod2 _uid:products#prod3 _uid:products#prod4)^0.0)~1],from[0],size[20],sort[<score>,<custom:"popRank": org.elasticsearch.index.fielddata.fieldcomparator.FloatValuesComparatorSource@52f36d0b>!]: Query Failed [Failed to execute main query]]; nested: ElasticsearchException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: UncheckedExecutionException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: NumberFormatException[Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; }

 at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:237)
 at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$1.onFailure(TransportSearchTypeAction.java:183)
 at org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:565)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 at java.lang.Thread.run(Thread.java:745)

If events are loaded with eventTime that is earlier than 3 days before today then query doesn't fail and response is quite accurate. The question is: why it happens like this? What is the root cause of ES exception shown above?
Thank you for any given advice and clarification!

вторник, 24 января 2017 г., 21:00:43 UTC+3 пользователь pat написал:

[Pat Ferrel]

• This is mostly OK. However the $set of users is not needed and is not used. $set of products with no properties is not needed except for random ranking where there are no events. The UR detects users and items from the usage events in normal cases.

• 
  
• {

  • eventId: "2Js-g6TvG57nVVXxwW8b_AAAAVnV0aWioNwnvU-bhrI",
  • event: "$set",
  • entityType: "user",
  • entityId: "sasha",
  • properties: { },
  • eventTime: "2017-01-25T13:30:03.554Z",
  • creationTime: "2017-01-25T13:30:03.554Z"

  },

To view this discussion on the web visit https://groups.google.com/d/msgid/actionml-user/52106be0-038c-437a-b31b-f215cb07f0ee%40googlegroups.com.