Replacement of Nested Set Model with Closure Table Model

[Edgar Rodríguez Silva]

Hello;

We have completed a new development that replaces our current nested set model with a closure table model. This change significantly improves performance, particularly for high-volume files.

We would like to include this development in upcoming versions. Could someone please provide us with guidance or instructions on how to proceed with this process?

Thank you for your assistance.

[Sarah Mason]

Hi Edgar,

Thank you for getting in touch about sharing code with the AtoM project. Further to our emails, I wonder if you would like to share the design specification for the closure table model here so the community can review it and share their thoughts. As it is a major architectural change, getting input from other AtoM users will help drive this process.

Thank you for coming to the Community Development meeting today -- for anyone who couldn't attend the meeting minutes have an initial discussion of this.

I'd like to put a call out to the AtoM community to review the specification and provide any questions or feedback to this group. We will also be discussing this proposal at the next AtoM Contributors Gathering on Thursday, 18 June 2026. It would be great to hear from members of the community things such as: 

- how often you rebuild the nested set

- how many records you have and how long the rebuild takes

- any other thoughts and questions about the design analysis when shared.

Thank you again, Edgar, for sharing your development work with the project! I hope we can get a robust discussion about this. We will be discussing more internally at Artefactual and I will be sharing our questions and feedback here.

Best wishes,

Sarah Mason

Contributor Success Specialist

[Edgar Rodríguez Silva]

Hello,

We're pleased to share the design analysis. In the coming weeks, we'll have some performance data comparing Nested Sets and Closure Tables on a catalog of approximately 600,000 records that we're upgrading to version 2.10.

Best regards

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/aW9dDL1TWGc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-user...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/ica-atom-users/cd0860ce-4f28-449b-98a1-763e91c7ecd3n%40googlegroups.com.

--

Edgar Rodríguez Silva

Project Manager

---

edgar.r...@xercode.es

(+34) 666 82 15 88 - 881 97 55 76

Rúa Hedras nº4 - 2A | 15895 | Milladoiro A Coruña, Spain

www.xercode.es

[Edgar Rodríguez Silva]

Hi Sarah and the AtoM community,

Following up on our previous message, we are sharing the performance report comparing the Nested Set Model and the Closure Table Model. This analysis was conducted using a sample catalog of 600,000 records.

We hope this data provides valuable insights for the community's review and the upcoming Contributors Gathering.

Best regards,

[Daniel Lovegrove]

Hi Edgar,

The performance improvements of the closure table model look promising versus the current nested set hierarchy implementation. The most important takeaway in my mind here is the footnote on page 5: that nested set write operations scale with the total number of rows O(n), and closure table write operations scale with depth. I imagine the maximum depth for most archives is not very large; I would consider that to be a fixed O(1) cost which scales with total database size much better than the nested set.

I have recently encountered issues with the nested set re-build process taking a very long time for a large AtoM instance I maintain. I made a pull request that was accepted to improve the nested set rebuild process by about 10x (see here: https://github.com/artefactual/atom/pull/2273). This doesn't change the fact that the nested set re-build scales with the database's size, but it does make re-builds much quicker. Since this change will be available in 2.11, would you be willing to re-test once those changes are available in the newest release? I'm curious to see how much of an improvement the changes I've made make.

Regards,

-Daniel L.

[Edgar Rodríguez Silva]

Hi Daniel,

Great work, that seems like a significant improvement. What volume of records does the AtoM instance where you tested this have?

We will try to run some comparisons with the patch you shared. Although we are about to deploy version 2.10 with the closure table for a client, we might be able to backport your patch to 2.10 and compare the performance.

Best regards,

To view this discussion visit https://groups.google.com/d/msgid/ica-atom-users/a821f6e6-ddb7-4969-8f17-b12de9a70825n%40googlegroups.com.

[Daniel Lovegrove]

Hi Edgar,

The largest instance I manage holds 3.3 million records. It took about 20 minutes to rebuild the nested set before the performance improvements, and it now takes about 2 minutes.

Sounds great about testing it, looking forward to seeing the results.

Best,

-Daniel