Sphinx performance analysis and enhancement

[danwos]

Dear all,

I'm supporting teams in the automotive industry, which have Sphinx projects with >15.000 files. ~11.000 are autogenerated, the rest is handwritten.
The build time is often > 1.5h, which is too much for them.

In the past I already did some performance analysis for Sphinx extensions, which results in some scripts to setup different test environments and measure the build times.
https://github.com/useblocks/sphinxcontrib-needs/tree/master/performance

However, I think Sphinx itself maybe has some potential for performance improvements.

Does anybody have done some analysis already in the past and could share her/his outcome or ideas for improvements?

My first step would be to define and create a reproducible test setup for any kind of Sphinx project. Including some performance analysis features.

(The above linked Sphinx-Needs solution will be a good starting point)

As you see, just some ideas, nothing concrete yet.
So I'm thankful for any kind of help, link or opinion.

Cheers,
  Daniel

[Charles Bouchard-Légaré]

Hi there!

I am not here to help because I fear I don't have any useful information, but I certainly would like to encourage you: this sounds like a good idea that would be useful for many.  Maybe if you would spin a separate repository it would be simpler to bounce ideas in issues and contribute?

[Daniel Woste]

Thanks for sharing your thoughts.
And luckily this is already done:
https://sphinx-performance.readthedocs.io/en/latest/

I also have made some performance measurements and summarised them in a spreadsheet:
https://docs.google.com/spreadsheets/u/0/d/1vBxxGtU79IL7k92lhiaf17tJl6xs1FGb_OgjVNzgRG0/htmlview

And for sure, I'm open to any kind of contribution :)

[Charles Bouchard-Légaré]

Oh, that is really nice! The difference between themes and the non-linear scaling of HTML is quite surprising. Starred :)

I am busy with some personal project for some time, but I'll certainly monitor your findings.  Thank you for this