Bad performance when searching without filters

Renato Isidio

unread,

Sep 13, 2022, 9:12:41 AM9/13/22

to lokiproject

Hi there! We've been experiencing something odd doing a query in all of our loki base. Quering {environment="homolog"} for 30 days we noticed that only one querier pod is working on it. Even setting the split_queries_by_interval: 15m. Doing some filter on this query, we see all of the 14 querier's pods are working. It only splits queries if we have filter on it (e.g. {environment="homolog"} | field = 1 ). Could you help us? Thank you in advance!

Best regards

Renato Isidio

Cyril Tovena

unread,

Sep 14, 2022, 2:14:14 AM9/14/22

to Renato Isidio, lokiproject

Hey Renato,

It's actually on purpose because without a filter the index is enough to find your logs and you shouldn't need parallelism especially if you return 1000 log lines.

How slow is the query ?

O conteúdo desta mensagem é confidencial e destinado exclusivamente aos destinatários. Caso a receba por engano, favor destruí-la e notificar o remetente de imediato. O correio eletrônico não configura meio seguro para transmissão de dados e o remetente NÃO se responsabiliza por eventual erro, atraso, extravio, interceptação ou infecção por vírus. Cabe ao destinatário solicitar versão física sempre que necessário.

The content of this message is confidential and was intended solely to its recipient. In case this message is received by mistake, please destroy it and notify the sender immediately. Electronic mails are not a safe channel for data transmission and the sender accepts NO liability for eventual errors, delays, loss, interception or virus infection. When necessary, the receiver must request a hard-copy version.

--
You received this message because you are subscribed to the Google Groups "lokiproject" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lokiproject...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lokiproject/ca3e13ce-34d6-4563-89d4-013f699d6204n%40googlegroups.com.

Renato Isidio

unread,

Sep 14, 2022, 9:43:57 AM9/14/22

to Cyril Tovena, lokiproject

Hi Cyril,

thank you for your reply. It takes more than 5minutes and we get a gateway timeout error. To get the first 1000 lines does the querier get the entire range's chunks?

Thanks

Renato Isidio

To view this discussion on the web visit https://groups.google.com/d/msgid/lokiproject/CAFWbsJw_hAN2bE49N%2By_KVNBx%2BFWjeVpjYSqzm4yDvhfGpraiQ%40mail.gmail.com.

Cyril Tovena

unread,

Sep 15, 2022, 5:02:16 AM9/15/22

to Renato Isidio, lokiproject

No it doesn't need to process all chunks, but it does need to look at all indexes in the range which might be the issue.

Reply all

Reply to author

Forward