Server timeout error 408

[eafi...@nelios.com]

Hi all,

I have a pretty weird bug that is troubling me, if anyone has any clues, there will be more than welcome!

We have a local arangoDB and a live arangoDB setup with the same configuration. I am continuously receiving a 408 timeout error from my application but only in the live server and not the local (development) one. I isolated the HTTP request that was returning the error, the request seems to run an aql query in the arango database. When I execute the specific query from the arango interface it returns results every time without an error both in my local and live database. On the contrary, when I send the HTTP request from POSTMAN service I sometimes get the following error : "Client Exception 408 Got a timeout while waiting for the server's response", while other times it returns the results. This happens ONLY in the live setup while in the local one it returns the results every time I send the request.

After sending several requests I noticed that every time that I got an error, the response time was more than 3 sec, while when I saw results the response time was always less than 3 sec, so I assume that is to blame. I checked the apache configuration that has the default timeout time at 30 seconds and then moved on to arango. After some digging, the only thing that I could find related to 3 sec was the following:  CLIENT options: --server.connect-timeout <double> connect timeout in seconds (default: 3)  from https://docs.arangodb.com/3.1/Manual/GettingStarted/Arangosh.html. Let me point out again that this happens only to the live server while sending the request in the local server may take up to 14 seconds but always returns the data.

The troubling thing is that I have the same setup and same configuration of arango in both machines, yet the timeout seems to be a problem only in the live one, does anyone have any idea?

Moreover, I could not find any documentation as to how to change that time and since I should have to do tests on a live server I want to be sure of what I am changing.

Thank you all for your time.

All the best,

Afiontzi Eleni

[Wilfried Gösgens]

Hi,
HTTP 408 could be generated by a proxy server like squid if its in the row?

In general, 3 seconds seems to be pretty long for an AQL query.
Did you revalidate using EXLPAIN that its properly using indices etc.?

Can you elaborate a bit more on your setup? Single/Cluster? Do you have a (maybe transparent) proxy on your way out into the internet that could be proying arangodb?

Cheers,
Willi

[eafi...@nelios.com]

Hi again,

Thank you very much for your prompt reply. 

I admit I am quite new to the whole server configuration/ system administration and have not myself setup the servers, do you know how I can check for a proxy in a remote server? The arangoDB setup is Single as far as I know.

Regarding the query, I have not written that either but seems like this when choosing the EXPLAIN. I can see that there is surely one index used.

Query string:
 LET params = {
             data: {
                 pagePreview: @previewMode,
                 langCode: @langCode,
                 fallbackLangCode: @fallbackLangCode,
                 searchLocalized: @searchLocalized,
                 pageStatus: @pageStatus,
                 inSiteMap: @inSiteMap,
                 activeLanguageMismatch: @activeLanguageMismatch,
                 pageAttrFilters: @pageAttrFilters,
                 pageAttrSort: @pageAttrSort,
                 pageFieldFilters: @pageFieldFilters,
                 pageFieldSort: @pageFieldSort
             }
         }FOR v, dOr, p IN @minDepth..@maxDepth OUTBOUND @id pageTree
         LET d = CALL("cms3::getPageChildrenVisitor", v, dOr, p, params)FILTER d != null &&
 (d.pageStd_typeKey==@filter0 && (LIKE(d.pageField_model,@filter10, true) ||
 LIKE(d.pageField_title,@filter11, true) || LIKE(d.pageField_content,@filter12, true) ) )  SORT
 d.pageStd_depth DESC, d.pageStd_vOrder DESC LIMIT 0, 20 RETURN d
 


Execution plan:
 Id   NodeType          Est.   Comment
  1   SingletonNode        1   * ROOT
  3   TraversalNode      597     - FOR v  /* vertex */, dOr  /* edge */, p  /* paths */ IN 1..10  /* min..maxPathDepth */ OUTBOUND 'page/1' /* startnode */  pageTree
  4   CalculationNode    597     - LET d = CALL("cms3::getPageChildrenVisitor", v, dOr, p, { "data" : { "pagePreview" : 1, "langCode" : "el", "fallbackLangCode" : "en", "searchLocalized" : 0, "pageStatus" : 20, "inSiteMap" : 1, "activeLanguageMismatch" : 0, "pageAttrFilters" : [ ], "pageAttrSort" : [ ], "pageFieldFilters" : [ "model", "title", "content" ], "pageFieldSort" : [ ] } })   /* v8 expression */
  5   CalculationNode    597     - LET #7 = ((d != null) && ((d.`pageStd_typeKey` == "product") && ((LIKE(d.`pageField_model`, "%hea%", true) || LIKE(d.`pageField_title`, "%hea%", true)) || LIKE(d.`pageField_content`, "%hea%", true))))   /* simple expression */
  6   FilterNode         597     - FILTER #7
  7   CalculationNode    597     - LET #9 = d.`pageStd_depth`   /* attribute expression */
  8   CalculationNode    597     - LET #11 = d.`pageStd_vOrder`   /* attribute expression */
  9   SortNode           597     - SORT #9 DESC, #11 DESC
 10   LimitNode           20     - LIMIT 0, 20
 11   ReturnNode          20     - RETURN d


Indexes used:
 By   Type   Collection   Unique   Sparse   Selectivity   Fields               Ranges
  3   edge   pageTree     false    false        52.77 %   [ `_from`, `_to` ]   base OUTBOUND


Traversals on graphs:
 Id   Depth   Vertex collections   Edge collections   Options                                   Filter conditions
  3   1..10                        pageTree           uniqueVertices: none, uniqueEdges: path  


Optimization rules applied:
 Id   RuleName
  1   move-calculations-up
  2   move-filters-up
  3   remove-unnecessary-calculations
  4   move-calculations-up-2
  5   move-filters-up-2

 If you have any further insights on these I would be grateful to hear them.

Best,

Eleni