Builder with incremental build support

[danwos]

Dear Sphinx developers and experts,

for the extension Sphinx-Needs I have created a builder to get all information to a JSON file (builder code on github).

But builds using this builder are always performing a full-build.
And not as e.g. HTML an incremental build, when it gets called again.

Do you have any hints, on what is needed to support incremental build inside a builder?
Do I have to return something specific for get_outdated_docs()?

Thanks already for your help,
 Daniel

[Komiya Takeshi]

Hi,

For your case, Domain will help you. It collects metadata (in your
case, "needs" data) from each document at reading phase.
They will be stored to the disk (as a part of env data) to cache them
for incremental builds.
https://www.sphinx-doc.org/en/master/extdev/domainapi.html

To understand how the domains work, this tutorial is helpful. Please
read and try it.
https://www.sphinx-doc.org/en/master/development/tutorials/recipe.html

Thanks,
Takeshi KOMIYA

2022年2月11日(金) 22:15 danwos <daniel...@useblocks.com>:

> --
> You received this message because you are subscribed to the Google Groups "sphinx-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-dev+...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/sphinx-dev/738756b4-289c-4781-98a7-fb81a114acebn%40googlegroups.com.

[danwos]

Thanks for the fast reply and the hints.

But I think, they are not really needed, as everything works great except my own builder for incremental build.
Also the HTML builder works perfectly with my extensions and all my data gets pickled/stored, as it is already part of the sphinx-env.
(Or I have missunderstood your hint... )

So the question is, how does a builder affects the decission of sphinx to perform an incremental build or not?

It can't be the data itself, as this works well with the HTML builder.

I know that config changes or a switch of the builder forces sphinx to perform a full build.
But this kind of information gets not touched by my builder.

Any ideas?

Thanks a lot for the help,
  Daniel

[Komiya Takeshi]

Oh, sorry. I misunderstood your question.

>So the question is, how does a builder affects the decission of sphinx to perform an incremental build or not?

Basically, each builder only minds performing an incremental build or
not in the writing phase. It does not mind about the reading phase.
So a builder can't invoke a "full" build including the reading phase
(if my understanding is correct).
Could you check your build log to ensure full-build is really invoked?

2022年2月12日(土) 5:54 danwos <daniel...@useblocks.com>:

> To view this discussion on the web, visit https://groups.google.com/d/msgid/sphinx-dev/4a38576c-1757-4db1-afca-63bbb48d20can%40googlegroups.com.

[Daniel Woste]

Thanks a lot for explanation, it helps me to understand that problem
must be in my extension.

And indeed, I found a bug, which deletes the .doctree-folder in some
case.


Thank you,
Daniel



On Mon, 2022-02-14 at 00:27 +0900, Komiya Takeshi wrote: