short prefix returns fewer results than a long prefix for FT.SEARCH on a tag?

19 views
Skip to first unread message

Jiri Kuthan

unread,
Nov 8, 2023, 9:41:13 AM11/8/23
to redi...@googlegroups.com
Folks,

excuse my newbie question but I'm too puzzled by finding a query with
a short prefix yielding empty result while a query with a longer prefix
(i.e. more restrictive) returns several.

any help please?

-jiri



the expression used in our search returns nothing: "FT.SEARCH" "profileIdx" "@IP:{ plain*},
@domain:{ba86ac2a\\-7f13\\-44b3\\-ba77\\-4dc29e6f6305\\#customprofile} ,-@delete:{true}"

while I add few more characters to the IP term (I.e. I would expect fewer results . however with longer prefix match) results come: ` "FT.SEARCH" "profileIdx"
"@ip:{ plain\#s*}, @Domain:{ba86ac2a\-7f13\-44b3\-ba77\-4dc29e6f6305\#customprofile} ,-@delete:{true}"


index created like this: 1698707841.289145 [0 172.17.0.1:61538] "FT.CREATE" "profileIdx" "ON" "JSON" "SCHEMA" "$.IP" "AS" "IP" "TAG" "SORTABLE" "$.domain" "AS"
"domain" "TAG" "$.delete" "AS" "delete" "TAG"

see the FT.INFO output bellow:

127.0.0.1:6379> FT.INFO profileIdx
1) index_name
2) profileIdx
3) index_options
4) (empty array)
5) index_definition
6) 1) key_type
2) JSON
3) prefixes
4) 1)
5) default_score
6) "1"
7) attributes
8) 1) 1) identifier
2) $.IP
3) attribute
4) IP
5) type
6) TAG
7) SEPARATOR
8)
9) SORTABLE
10) UNF
2) 1) identifier
2) $.domain
3) attribute
4) domain
5) type
6) TAG
7) SEPARATOR
8)
3) 1) identifier
2) $.delete
3) attribute
4) delete
5) type
6) TAG
7) SEPARATOR
8)
9) num_docs
10) "634"
11) max_doc_id
12) "99492"
13) num_terms
14) "0"
15) num_records
16) "1950"
17) inverted_sz_mb
18) "0.0021162033081054688"
19) vector_index_sz_mb
20) "0"
21) total_inverted_index_blocks
22) "1092"
23) offset_vectors_sz_mb
24) "0"
25) doc_table_size_mb
26) "0.0940847396850586"
27) sortable_values_size_mb
28) "0.03782081604003906"
29) key_table_size_mb
30) "0.025527000427246094"
31) geoshapes_sz_mb
32) "0"
33) records_per_doc_avg
34) "3.075709819793701"
35) bytes_per_record_avg
36) "1.137948751449585"
37) offsets_per_term_avg
38) "0"
39) offset_bits_per_record_avg
40) "nan"
41) hash_indexing_failures
42) "1"
43) total_indexing_time
44) "811.805"
45) indexing
46) "0"
47) percent_indexed
48) "1"
49) number_of_uses
50) (integer) 8024
51) cleaning
52) (integer) 0
53) gc_stats
54) 1) bytes_collected
2) "200954"
3) total_ms_run
4) "2887"
5) total_cycles
6) "627"
7) average_cycle_time_ms
8) "4.604465709728868"
9) last_run_time_ms
10) "3"
11) gc_numeric_trees_missed
12) "0"
13) gc_blocks_denied
14) "0"
55) cursor_stats
56) 1) global_idle
2) (integer) 0
3) global_total
4) (integer) 0
5) index_capacity
6) (integer) 128
7) index_total
8) (integer) 0
57) dialect_stats
58) 1) dialect_1
2) (integer) 1
3) dialect_2
4) (integer) 0
5) dialect_3
6) (integer) 0
7) dialect_4
8) (integer) 0
127.0.0.1:6379> "FT.EXPLAIN" "profileIdx" "@IP:{ plain*}, @domain:{ba86ac2a\\-7f13\\-44b3\\-ba77\\-4dc29e6f6305\\#customprofile} ,-@delete:{true}"
"INTERSECT {\n TAG:@IP {\n PREFIX{plain*}\n }\n TAG:@domain {\n ba86ac2a-7f13-44b3-ba77-4dc29e6f6305#customprofile\n }\n NOT{\n TAG:@delete {\n
true\n }\n }\n}\n"
127.0.0.1:6379> "FT.EXPLAIN" "profileIdx" "@IP:{ plain\\#s*}, @domain:{ba86ac2a\\-7f13\\-44b3\\-ba77\\-4dc29e6f6305\\#customprofile} ,-@delete:{true}"
"INTERSECT {\n TAG:@IP {\n PREFIX{plain#s*}\n }\n TAG:@domain {\n ba86ac2a-7f13-44b3-ba77-4dc29e6f6305#customprofile\n }\n NOT{\n TAG:@delete {\n
true\n }\n }\n}\n"
results fro the longer term @IP:{ plain\\#s*}

127.0.0.1:6379> "FT.SEARCH" "profileIdx" "@IP:{ plain\\#s*}, @domain:{ba86ac2a\\-7f13\\-44b3\\-ba77\\-4dc29e6f6305\\#customprofile} ,-@delete:{true}"
1) (integer) 4
2) "plain#sip.request.sig#123#ba86ac2a-7f13-44b3-ba77-4dc29e6f6305#customprofile"
3) 1) "$"
2)
"{\"IP\":\"plain#sip.request.sig#123\",\"domain\":\"ba86ac2a-7f13-44b3-ba77-4dc29e6f6305#customprofile\",\"CKM_sig2ua.memory\":[\"PolycomVVX\"],\"CKM_sig2ua.created\":1699013191,\"CKM_sig2ua.lastModifiedTS\":1699013191,\"timestamp\":1699013191}"
4) "plain#sip.request.sig#263751205#ba86ac2a-7f13-44b3-ba77-4dc29e6f6305#customprofile"
5) 1) "$"
2)
"{\"IP\":\"plain#sip.request.sig#263751205\",\"domain\":\"ba86ac2a-7f13-44b3-ba77-4dc29e6f6305#customprofile\",\"CKR_CKR.created\":1699012790,\"CKM_CKM.memory\":[\"Linksys-SPA942\"],\"CKM_CKM.created\":1699012790,\"CKR_CKR.num.5.val\":1,\"CKR_CKR.num.5.ts\":1699012790,\"CKR_CKR.num.0.val\":1,\"CKR_CKR.num.0.ts\":1699229762,\"CKR_CKR.num.sum\":1,\"CKR_CKR.div.5.val\":1,\"CKR_CKR.div.5.ts\":1699012790,\"CKR_CKR.div.0.val\":1,\"CKR_CKR.div.0.ts\":1699229762,\"CKR_CKR.div.sum\":1,\"CKR_CKR.numcnt.5.val\":1,\"CKR_CKR.numcnt.5.ts\":1699012790,\"CKR_CKR.numcnt.0.val\":1,\"CKR_CKR.numcnt.0.ts\":1699229762,\"CKR_CKR.numcnt.sum\":1,\"CKR_CKR.divcnt.5.val\":1,\"CKR_CKR.divcnt.5.ts\":1699012790,\"CKR_CKR.divcnt.0.val\":1,\"CKR_CKR.divcnt.0.ts\":1699229762,\"CKR_CKR.divcnt.sum\":1,\"CKR_CKR.ratio\":100,\"CKR_CKR.lastModifiedTS\":1699233144,\"CKM_CKM.lastModifiedTS\":1699012790,\"timestamp\":1699233144,\"CKR_CKR.num.1.val\":0,\"CKR_CKR.num.1.ts\":1699229762,\"CKR_CKR.div.1.val\":0,\"CKR_CKR.div.1.ts\":1699229762,\"CKR_CKR.numcnt.1.val\":0,\"CKR_CKR.numcnt.1.ts\":1699229762,\"CKR_CKR.divcnt.1.val\":0,\"CKR_CKR.divcnt.1.ts\":1699229762,\"CKR_CKR.num.3.val\":1,\"CKR_CKR.num.3.ts\":1699233144,\"CKR_CKR.num.4.val\":0,\"CKR_CKR.num.4.ts\":1699233144,\"CKR_CKR.div.3.val\":1,\"CKR_CKR.div.3.ts\":1699233144,\"CKR_CKR.div.4.val\":0,\"CKR_CKR.div.4.ts\":1699233144,\"CKR_CKR.numcnt.3.val\":1,\"CKR_CKR.numcnt.3.ts\":1699233144,\"CKR_CKR.numcnt.4.val\":0,\"CKR_CKR.numcnt.4.ts\":1699233144,\"CKR_CKR.divcnt.3.val\":1,\"CKR_CKR.divcnt.3.ts\":1699233144,\"CKR_CKR.divcnt.4.val\":0,\"CKR_CKR.divcnt.4.ts\":1699233144}"
6) "plain#sip.request.sig#263751204#ba86ac2a-7f13-44b3-ba77-4dc29e6f6305#customprofile"
7) 1) "$"
2)
"{\"IP\":\"plain#sip.request.sig#263751204\",\"domain\":\"ba86ac2a-7f13-44b3-ba77-4dc29e6f6305#customprofile\",\"CKR_CKR.created\":1699012804,\"CKM_CKM.memory\":[\"friendly-scanner\",\"PBX\"],\"CKM_CKM.created\":1699012804,\"CKR_CKR.num.0.val\":1,\"CKR_CKR.num.0.ts\":1699012804,\"CKR_CKR.num.1.val\":1,\"CKR_CKR.num.1.ts\":1699229776,\"CKR_CKR.num.sum\":1,\"CKR_CKR.div.0.val\":1,\"CKR_CKR.div.0.ts\":1699012804,\"CKR_CKR.div.1.val\":1,\"CKR_CKR.div.1.ts\":1699229776,\"CKR_CKR.div.sum\":1,\"CKR_CKR.numcnt.0.val\":1,\"CKR_CKR.numcnt.0.ts\":1699012804,\"CKR_CKR.numcnt.1.val\":1,\"CKR_CKR.numcnt.1.ts\":1699229776,\"CKR_CKR.numcnt.sum\":1,\"CKR_CKR.divcnt.0.val\":1,\"CKR_CKR.divcnt.0.ts\":1699012804,\"CKR_CKR.divcnt.1.val\":1,\"CKR_CKR.divcnt.1.ts\":1699229776,\"CKR_CKR.divcnt.sum\":1,\"CKR_CKR.ratio\":100,\"CKR_CKR.lastModifiedTS\":1699233158,\"CKM_CKM.lastModifiedTS\":1699012838,\"timestamp\":1699233158,\"CKM_CKM.counterExceeded\":1,\"CKM_CKM.counterRaised\":1,\"CKM_CKM.lastExceededTS\":1699012838,\"CKM_CKM.alertKeyId\":\"cstm#sip.request.sig_-686206323\",\"CKM_CKM.lastRaisedTS\":1699012838,\"CKM_CKM.docid\":\"a1a86f50-7a40-11ee-b9f8-0d631ed9abc7\",\"CKM_CKM._prevCounterRaised\":0,\"CKR_CKR.num.2.val\":0,\"CKR_CKR.num.2.ts\":1699229776,\"CKR_CKR.div.2.val\":0,\"CKR_CKR.div.2.ts\":1699229776,\"CKR_CKR.numcnt.2.val\":0,\"CKR_CKR.numcnt.2.ts\":1699229776,\"CKR_CKR.divcnt.2.val\":0,\"CKR_CKR.divcnt.2.ts\":1699229776,\"CKR_CKR.num.4.val\":1,\"CKR_CKR.num.4.ts\":1699233158,\"CKR_CKR.num.5.val\":0,\"CKR_CKR.num.5.ts\":1699233158,\"CKR_CKR.div.4.val\":1,\"CKR_CKR.div.4.ts\":1699233158,\"CKR_CKR.div.5.val\":0,\"CKR_CKR.div.5.ts\":1699233158,\"CKR_CKR.numcnt.4.val\":1,\"CKR_CKR.numcnt.4.ts\":1699233158,\"CKR_CKR.numcnt.5.val\":0,\"CKR_CKR.numcnt.5.ts\":1699233158,\"CKR_CKR.divcnt.4.val\":1,\"CKR_CKR.divcnt.4.ts\":1699233158,\"CKR_CKR.divcnt.5.val\":0,\"CKR_CKR.divcnt.5.ts\":1699233158}"
8) "plain#sip.request.sig#264350217#ba86ac2a-7f13-44b3-ba77-4dc29e6f6305#customprofile"
9) 1) "$"
2)
"{\"IP\":\"plain#sip.request.sig#264350217\",\"domain\":\"ba86ac2a-7f13-44b3-ba77-4dc29e6f6305#customprofile\",\"CKR_CKR.created\":1699012741,\"CKM_CKM.memory\":[\"pplsip\"],\"CKM_CKM.created\":1699012741,\"CKR_CKR.num.5.val\":0,\"CKR_CKR.num.5.ts\":1699233173,\"CKR_CKR.num.0.val\":2,\"CKR_CKR.num.0.ts\":1699229748,\"CKR_CKR.num.sum\":3,\"CKR_CKR.div.5.val\":0,\"CKR_CKR.div.5.ts\":1699233173,\"CKR_CKR.div.0.val\":2,\"CKR_CKR.div.0.ts\":1699229748,\"CKR_CKR.div.sum\":3,\"CKR_CKR.numcnt.5.val\":0,\"CKR_CKR.numcnt.5.ts\":1699233173,\"CKR_CKR.numcnt.0.val\":2,\"CKR_CKR.numcnt.0.ts\":1699229748,\"CKR_CKR.numcnt.sum\":3,\"CKR_CKR.divcnt.5.val\":0,\"CKR_CKR.divcnt.5.ts\":1699233173,\"CKR_CKR.divcnt.0.val\":2,\"CKR_CKR.divcnt.0.ts\":1699229748,\"CKR_CKR.divcnt.sum\":3,\"CKR_CKR.ratio\":100,\"CKR_CKR.lastModifiedTS\":1699233173,\"CKM_CKM.lastModifiedTS\":1699012741,\"timestamp\":1699233173,\"CKR_CKR.counterExceeded\":3,\"CKR_CKR.counterRaised\":2,\"CKR_CKR.num.1.val\":1,\"CKR_CKR.num.1.ts\":1699229790,\"CKR_CKR.div.1.val\":1,\"CKR_CKR.div.1.ts\":1699229790,\"CKR_CKR.numcnt.1.val\":1,\"CKR_CKR.numcnt.1.ts\":1699229790,\"CKR_CKR.divcnt.1.val\":1,\"CKR_CKR.divcnt.1.ts\":1699229790,\"CKR_CKR.lastExceededTS\":1699233173,\"CKR_CKR.alertKeyId\":\"cstm#sip.request.sig_86750997\",\"CKR_CKR.lastRaisedTS\":1699229790,\"CKR_CKR.docid\":\"c39906b0-7c39-11ee-8b20-9dd9b3f10622\",\"CKR_CKR._prevCounterRaised\":1,\"CKM_CKM_ua.memory\":[\"pplsip\",\"altered-ua\",\"yet-again-altered-ua\"],\"CKM_CKM_ua.created\":1699228664,\"CKM_CKM_method.memory\":[\"INVITE\",\"REINVITE\"],\"CKM_CKM_method.created\":1699228664,\"CKM_CKM_ua.lastModifiedTS\":1699228714,\"CKM_CKM_method.lastModifiedTS\":1699228714,\"CKM_CKM_ua.counterExceeded\":1,\"CKM_CKM_ua.counterRaised\":1,\"CKM_CKM_method.counterExceeded\":1,\"CKM_CKM_method.counterRaised\":1,\"CKM_CKM_ua.lastExceededTS\":1699228714,\"CKM_CKM_ua.alertKeyId\":\"cstm#sip.request.sig_86750997\",\"CKM_CKM_ua.lastRaisedTS\":1699228714,\"CKM_CKM_ua.docid\":\"423d1950-7c37-11ee-a7e6-5b2bcbc09c8d\",\"CKM_CKM_ua._prevCounterRaised\":0,\"CKM_CKM_method.lastExceededTS\":1699228714,\"CKM_CKM_method.alertKeyId\":\"cstm#sip.request.sig_86750997\",\"CKM_CKM_method.lastRaisedTS\":1699228714,\"CKM_CKM_method.docid\":\"423ddca0-7c37-11ee-a7e6-5b2bcbc09c8d\",\"CKM_CKM_method._prevCounterRaised\":0,\"CKR_CKR.num.2.val\":0,\"CKR_CKR.num.2.ts\":1699229790,\"CKR_CKR.div.2.val\":0,\"CKR_CKR.div.2.ts\":1699229790,\"CKR_CKR.numcnt.2.val\":0,\"CKR_CKR.numcnt.2.ts\":1699229790,\"CKR_CKR.divcnt.2.val\":0,\"CKR_CKR.divcnt.2.ts\":1699229790,\"CKR_CKR.num.3.val\":2,\"CKR_CKR.num.3.ts\":1699233130,\"CKR_CKR.num.4.val\":1,\"CKR_CKR.num.4.ts\":1699233173,\"CKR_CKR.div.3.val\":2,\"CKR_CKR.div.3.ts\":1699233130,\"CKR_CKR.div.4.val\":1,\"CKR_CKR.div.4.ts\":1699233173,\"CKR_CKR.numcnt.3.val\":2,\"CKR_CKR.numcnt.3.ts\":1699233130,\"CKR_CKR.numcnt.4.val\":1,\"CKR_CKR.numcnt.4.ts\":1699233173,\"CKR_CKR.divcnt.3.val\":2,\"CKR_CKR.divcnt.3.ts\":1699233130,\"CKR_CKR.divcnt.4.val\":1,\"CKR_CKR.divcnt.4.ts\":1699233173}"
127.0.0.1:6379> "FT.SEARCH" "profileIdx" "@IP:{ plain*}, @domain:{ba86ac2a\\-7f13\\-44b3\\-ba77\\-4dc29e6f6305\\#customprofile} ,-@delete:{true}"
1) (integer) 0
127.0.0.1:6379>

no results for the short term plain*

127.0.0.1:6379> "FT.SEARCH" "profileIdx" "@IP:{ plain*}, @domain:{ba86ac2a\\-7f13\\-44b3\\-ba77\\-4dc29e6f6305\\#customprofile} ,-@delete:{true}"
1) (integer) 0
redis doc says in https://redis.io/docs/interact/search-and-query/advanced-concepts/tags/ that tags support prefix matching with the regular * character:
FT.SEARCH idx "@tags:{ hello\\ w* }"

--
Dipl. Ing. Jiri Kuthan | Executive Vice President
ji...@intuitivelabs.com | Mobile: +49 179 50 61 546
Address : Barkhausen Str 75, 14612 Falkensee, Germany
Intuitive Labs GmbH, HRB 34115 P bei Handelsregister Potsdam
Reply all
Reply to author
Forward
0 new messages