Sharding PostgresQL with HAPI FHIR

[Rico Aarntzen]

Hello all,

For improving scalability and performance we would like to shard our database across nodes. Ive searched the HAPI FHIR documentation but could only find information about partitioning.

So my question is: Is sharding possible with HAPI FHIR (7.6.0) and PostgresQL? 

And if so, how?

Thanks in advance for your answer(s).

Best regards,

Rico

[Arne Van Eycken]

I will follow this as we have the same question

--
You received this message because you are subscribed to the Google Groups "HAPI FHIR" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hapi-fhir+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/hapi-fhir/74db93c1-ae07-46ae-8b8c-443114bb79bbn%40googlegroups.com.

[James Agnew]

By sharding, are you referring to support for Postgresql Table Partitioning?

If so, I guess the answer is that it's not yet supported, but this is coming as a part of our Feb release which is out this month. We will be writing a bunch more about how to do this in the next few weeks, but the next release introduces a feature called Database Partitioning Mode which will enable support for native database partitioning:

https://hapifhir.io/hapi-fhir/docs/server_jpa_partitioning/db_partition_mode.html

Cheers,

James

To view this discussion visit https://groups.google.com/d/msgid/hapi-fhir/CALOgEiT0aHbf1YzkHDBcn%3D%2BiMcd6VQOctB4XBh45%3DbYUqZ1e-w%40mail.gmail.com.

[Rico Aarntzen]

Hi James,

Thank you for your answer. I was referring to sharding data and not partitioning. Like this: (https://wiki.postgresql.org/wiki/WIP_PostgreSQL_Sharding)
I see some PostgresQL extensions like Citus (https://docs.citusdata.com/en/stable/get_started/what_is_citus.html) I can try to use with HAPI FHIR.

Not sure if anyone else already tried this with success? Do you know if this works/doesn't work James?

Thanks in advance,

Rico

Op woensdag 5 februari 2025 om 16:28:10 UTC+1 schreef james...@gmail.com:

[Muhammad]

Hello Rico,

Did you find any working solution for Sharding? 

We have attempted CitusData migrated the Hapi_r4 tables but the Hapi_Fhir_JPS_Server is so strict that it is every time checking the schema and refuses to start. We have tried many application configurations, but non of them seems to work.

Any help or guidance is appreciated.

Best Regards,
M

[James Agnew]

Hi Muhammad,

Supporting Citus Sharding requires a bunch of schema tweaks and changes to specific query strings. Support is currently only available as an option with our commercial package (Smile CDR / OmniVera).

Cheers,

James

To view this discussion visit https://groups.google.com/d/msgid/hapi-fhir/7889df36-e438-486d-b3ac-65f64aba5618n%40googlegroups.com.

[Muhammad]

Hi James,

Thank you for the quick clarification regarding Citus Sharding.

Regarding the commercial package (Smile CDR / OmniVera), could you confirm if it supports on-premises deployments, or is it restricted to a specific cloud provider?

Regarding the technical side, we have already made significant progress on the migration:

• Modified the schema for hapi_r4.
• Distributed 22 tables and added sharding key columns.
• Converted the rest of the tables to Reference tables, and other remaining tables are kept as Local tables.
• Updated constraints and indexes across the board.

We pointed the Hapi_Fhir_Jpa_Server connection string to the Citus coordinator and disabled schema initialization. However, the server is still crashing.

Does the commercial package include a specific storage mapper or SQL dialect designed for Citus that handles these query string changes?

Looking forward to hearing from you.

Best Regards,

M

[James Agnew]

> Regarding the commercial package (Smile CDR / OmniVera), could you confirm if it supports on-premises deployments, or is it restricted to a specific cloud provider?

On-prem deployment is supported. It's intended to be a fairly straightforward migration path for most HAPI JPA users.

> Does the commercial package include a specific storage mapper or SQL dialect designed for Citus that handles these query string changes?

Yup that's exactly it, plus Citus is a part of our standard upgrade/release QA cycle these days.

Cheers,

James

To view this discussion visit https://groups.google.com/d/msgid/hapi-fhir/dc12578c-5fb6-4df9-9fd6-7a852738f52cn%40googlegroups.com.