An issue with the CATS API

[Bilal Shafqat]

Hi CATS Support,

We are experiencing an issue with the CATS API, below are the details:

• To get the daily changes from the CATS, we make multiple POST requests daily to get the modified JOBS, CANDIDATES, and PIPELINES.
  
  
  • For Example: If it's 21st Nov 2023, we will make a request to get the changes of 20th Nov 2023. (In short, every day we request to get the changes of the previous day)
    
    Request: https://api.catsone.com/v3/pipelines/search?per_page=100&page=1
    
    Body:
    {

"field": "date_modified",

"filter": "between",

"value": {

"gte": "2023-11-20T00:00:00+00:00",

"lte": "2023-11-20T23:59:59+00:00"

}

}

• We noticed that for the above request, we are not getting all the pipelines that were modified on 2023-11-20. We are experiencing this not for the single pipeline but multiple pipelines and multiple jobs - not sure about the exact number.
  • API should have returned the below pipeline in the response (because it was modified on 2023-11-20):
    
    
  • But it is not returning this pipeline based on the date_modified filters (see response below):
    
    
• Note: (We have investigated it further, and have some findings which might help in the resolution):
  • As we are getting the changes/modifications in our database every day, we know when the pipeline was modified previously: It was modified on: 2023-11-13 13:11:12.000
  • So, we made another request, and this time with this 2023-11-13 13:11:12.000 date. And the pipeline was returned in the response. Interesting 😊. It looks like something is wrong with the data partitioning/mapping at the CATS end:
    • For modified_date: 2023-11-13T13:11:12+00:00, this pipeline was returned (see response below):
      
      
• Note: Latest modified date for the pipeline was 2023-11-20T22:39:47+00:00 but it was not returning the pipeline for this modified_date, instead we were getting the pipeline in the response for the older modified_date 2023-11-13T13:11:12+00:00

• Some other pipelines (IDs) which are also affected:
  • 306256730
  • 306233740

Can you please look into this and investigate at your end?

Thanks,
Bilal

[Bilal Shafqat]

Hi CATS Support,

Is there any update regarding this? Has the CATS team started looking into this issue? We have been waiting for a long time now.

Thanks,

Bilal

[Steve Gagnon]

Sorry about the delay. I checked and you are correct that the date modified filter wasn't working for some pipelines. I looked into it further and found our search index isn't updating date modified for pipelines in some cases. I re-indexed your site's pipelines which should fix the cases you saw previously. I'll also be looking into a fix for indexing future pipelines.