API: browse information objects by created/updated?

[Paul Collins]

Hi. We have Artefactual-hosted AtoM and I am trying to use the API to retrieve all the records (of a certain media type) that were created or modified since a given date. I am using the "browse information objects" API documented here:

https://www.accesstomemory.org/en/docs/2.7/dev-manual/api/browse-io/

I know that we can sort by lastUpdated, but how can I *filter* by that field?

Sample URL I have tried:

https://catalogue.millsarchive.org/api/informationobjects?onlyMedia=1&mediatypes=136&startDate=2021-04-20

The startDate/endDate fields can be used as filters, but these appear to filter based on the actual historical date associated with the record, and not the created/updated date in AtoM. For example, the following URL has a future date (20 April 2021, more than a month from now) and yet it returns 56 records. I would want a future date to return nothing, since nothing can have been created/updated in the future yet!

Thanks.

Paul C.

[Dan Gillean]

Hi Paul, 

I haven't tested this yet, but I have a suggestion for you to try. 

As I understand it, the browse API endpoints support all the same parameters that the search index can use. If you look in the Advanced search documentation, you will see a method listed in th Expert searching section on how to search by creation or modification dates: 

• https://www.accesstomemory.org/docs/latest/user-manual/access-content/advanced-search/#searching-for-recently-created-or-modified-descriptions
  

I would suggest trying to craft an API call using these same parameters. What I often do when testing out API searching is perform the search in the browser first, and then change the base URL to point to the API endpoint, so the parameters are the same and I've confirmed they work in the user interface with search results. 

If you try it out, let us know how it goes! 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/daf2be59-c784-4835-a722-6cfaddcfe6c9n%40googlegroups.com.

[Paul Collins]

Thanks Dan.

Our archivist Nathanael indicates to me that the created/updated dates cannot be specified in the Web interface -- they do not appear there -- so we cannot try crafting our query that way.

I note the ElasticSearch parameters that you linked to, "updatedAt" and "createdAt". However, these names do not seem to work in a query string, e.g. appending this has no effect at all:

&updatedAt=[2021-04-01 TO 2022-12-31]

I note that the examples you linked to use a colon and not an equals sign, e.g.

updatedAt:[* TO 2012-01-01]

so they do not seem to fit the usual ?a=1&b=2&c=3  query string pattern.

Could you please show an example of how to use the ElasticSearch updatedAt parameter directly in a URL, such as the URL shown in my original post in this thread?

Paul Collins.

[José Raddaoui]

Hi Paul,

You'll need to use the "query" parameter in the URL to include that value. For example, entering "updatedAt:[2021-01-01 TO *]" in the GUI search-box will show you descriptions updated since the start of the year. This translates to "query=updatedAt%3A%5B2021-01-01+TO+*%5D" in the URL, which you should be able to use in the API endpoint.

Best regards,

Radda.

[Paul Collins]

Hi José. Unfortunately your suggestion is not working here.

I choose a future date and build the string, e.g. updatedAt:[2021-04-01 TO *]

I then include that in the URL as follows:

https://catalogue.millsarchive.org/api/informationobjects?onlyMedia=1&mediatypes=136&query=updatedAt%3A%5B2021-04-01%20TO%20*%5D

Despite the future date, this still returns some results (9 results at present). In fact, the updatedAt portion of the URL appears to have no effect, since I get 9 results whether that portion is included in the URL or not.

Again, there should be no results when asking for data that was updated in the future.

What are we doing wrong, or is there an AtoM bug?

[José Raddaoui]

Hi Paul,

That's quite rare and I can't reproduce locally. If you have access to the AtoM database, could you check the actual "updated_at" values in there? After connecting to the server and selecting the AtoM database, the following query should show you those dates:

SELECT * FROM object WHERE class_name='QubitInformationObject' ORDER BY updated_at DESC;

If they look alright, could you try rebuilding the search index? From the AtoM folder:

php symfony search:populate

I hope that gives some light to the issue.

Best regards,

Radda.

[Paul Collins]

From further experimentation, it seems that the "query=..." parameter needed to be "sq0=..." instead. Using this sq0 parameter appears to give the results we expected.

If we do have further issues with this then I will try the SQL query you suggested.  Thanks!

[José Raddaoui]

Thanks for sharing what worked Paul!

I initially thought all parameters used in the GUI would work in the API, but it looks like there is some logic in the GUI to sync the search-box and the first boolean query that is not present in the API ...

https://www.accesstomemory.org/en/docs/2.6/dev-manual/api/browse-io/#api-browse-io

Best regards,

Radda.