limited support for chained searches

[Brian Walsh]

Is there a solution to this on the roadmap?

`The inner query must match no more than 100 results`

We have run into limitations with queries of the form:

```

# return all Observations for female patients with a disease related to Kindney Observation?subject:_has:Condition:body-site.text=Kidney&subject.gender=female

```

Which produce an error

```

--- issue: - code: value details: text: invalid_query diagnostics: 'chained search failed: inner query on resource type ''Patient'', search parameter ''gender'' returned too many results to expand (>100)' severity: error resourceType: OperationOutcome

```

Documented here: https://cloud.google.com/healthcare-api/docs/how-tos/fhir-advanced-search

Warning: The Cloud Healthcare API has limited support for chained searches. The inner query must match no more than 100 results or the search returns an error.

[Paul Church]

There are three ways to pursue this kind of query involving large scale joins across references.

The first is to export/stream the data to BigQuery and run it in SQL, see https://cloud.google.com/healthcare-api/docs/how-tos/fhir-export-bigquery and https://cloud.google.com/healthcare-api/docs/how-tos/fhir-bigquery-streaming.

The second is an API called "execute cohort", currently in private preview. It supports fhirpath cohort definitions that can span many connected resources, runs as a long-running operation, and materializes the results into files in a GCS bucket. If you are interested in getting access to this feature, we are open to more testers.

The limitations on FHIR chained search are inherent to how the indexing is organized and are not likely to change, but the third option is to rearrange the search and/or break it into multiple steps. In some cases it's possible to rearrange the direction of chains/includes in a way that avoids the high cardinality side of a join but your example involves a lot of conditions, patients, and observations so that's not going to be the case here. Off the top of my head it looks like this example needs a search on Condition, a search on Patient, merge those results together after paging through all of the matches, and then (assuming there are a lot of patients) potentially several searches on Observation?subject={a section of the patient ID list} to compile all of the results.

As a side note I don't think the body-site param is quite right, it would be body-site:text=Kidney (':' instead of '.') - I highly recommend turning on strict handling when developing queries: https://cloud.google.com/healthcare-api/docs/how-tos/fhir-search#search_parameter_handling

--
You received this message because you are subscribed to the Google Groups "GCP Healthcare Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gcp-healthcare-di...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/gcp-healthcare-discuss/ca20542c-44b4-4e10-9d67-7034b23198f2n%40googlegroups.com.

[Brian Walsh]

Paul,
Thanks very much for the timely reply.  Much appreciated.
Yes, we would be interested in reviewing the "execute cohort" API.
Until then, we will probably break the query into sub-components.
Thanks again