Build older documentation

[Laura Morales]

I'm interested in building older versions of the documentation, but I'm having problems with this.
For example if I want to build v0.13.5, I do this:

$ pip install -r requirements.txt -r requirements-dev.txt -r requirements-doc.txt .[docs]
$ cd docs
$ make dirhtml

and I get this error

----------------------------------------

# Platform: linux; (Linux-6.1.0-17-amd64-x86_64-with-glibc2.36)
# Sphinx version: 7.3.7
# Python version: 3.11.7 (CPython)
# Docutils version: 0.21.2
# Jinja2 version: 3.1.4
# Pygments version: 2.18.0

# Last messages:
# api
#
#
# reading sources... [ 0%]
# api-structure
#
#
# reading sources... [ 0%]
# contingency_tables
#

# Loaded extensions:
# sphinx.ext.mathjax (7.3.7)
# alabaster (0.7.16)
# sphinxcontrib.applehelp (1.0.8)
# sphinxcontrib.devhelp (1.0.6)
# sphinxcontrib.htmlhelp (2.0.5)
# sphinxcontrib.serializinghtml (1.1.10)
# sphinxcontrib.qthelp (1.0.7)
# sphinx.ext.autodoc.preserve_defaults (7.3.7)
# sphinx.ext.autodoc.type_comment (7.3.7)
# sphinx.ext.autodoc.typehints (7.3.7)
# sphinx.ext.autodoc (7.3.7)
# sphinx.ext.autosummary (7.3.7)
# numpydoc (1.7.0)
# sphinx.ext.doctest (7.3.7)
# sphinx.ext.extlinks (7.3.7)
# sphinx.ext.intersphinx (7.3.7)
# sphinx.ext.todo (7.3.7)
# sphinx.ext.autosectionlabel (7.3.7)
# nbsphinx (0.9.4)
# sphinx.ext.viewcode (7.3.7)
# sphinx.ext.graphviz (7.3.7)
# sphinx.ext.inheritance_diagram (7.3.7)
# matplotlib.sphinxext.plot_directive (3.9.0)
# IPython.sphinxext.ipython_console_highlighting (unknown version)
# IPython.sphinxext.ipython_directive (unknown version)
# sphinx_material (0.0.36)

# Traceback:
Traceback (most recent call last):
File "/home/user/Desktop/venv/lib/python3.11/site-packages/sphinx/cmd/build.py", line 337, in build_main
app.build(args.force_all, args.filenames)
File "/home/user/Desktop/venv/lib/python3.11/site-packages/sphinx/application.py", line 351, in build
self.builder.build_update()
File "/home/user/Desktop/venv/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 293, in build_update
self.build(to_build,
File "/home/user/Desktop/venv/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 313, in build
updated_docnames = set(self.read())
^^^^^^^^^^^
File "/home/user/Desktop/venv/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 419, in read
self._read_serial(docnames)
File "/home/user/Desktop/venv/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 440, in _read_serial
self.read_doc(docname)
File "/home/user/Desktop/venv/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 497, in read_doc
publisher.publish()
File "/home/user/Desktop/venv/lib/python3.11/site-packages/docutils/core.py", line 234, in publish
self.document = self.reader.read(self.source, self.parser,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Desktop/venv/lib/python3.11/site-packages/sphinx/io.py", line 107, in read
self.parse()
File "/home/user/Desktop/venv/lib/python3.11/site-packages/docutils/readers/__init__.py", line 76, in parse
self.parser.parse(self.input, document)
File "/home/user/Desktop/venv/lib/python3.11/site-packages/sphinx/parsers.py", line 83, in parse
self.statemachine.run(inputlines, document, inliner=self.inliner)
File "/home/user/Desktop/venv/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 169, in run
results = StateMachineWS.run(self, input_lines, input_offset,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Desktop/venv/lib/python3.11/site-packages/docutils/statemachine.py", line 233, in run
context, next_state, result = self.check_line(
^^^^^^^^^^^^^^^^
File "/home/user/Desktop/venv/lib/python3.11/site-packages/docutils/statemachine.py", line 445, in check_line
return method(match, context, next_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Desktop/venv/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2790, in underline
self.section(title, source, style, lineno - 1, messages)
File "/home/user/Desktop/venv/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 325, in section
self.new_subsection(title, lineno, messages)
File "/home/user/Desktop/venv/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 391, in new_subsection
newabsoffset = self.nested_parse(
^^^^^^^^^^^^^^^^^^
File "/home/user/Desktop/venv/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 279, in nested_parse
state_machine.run(block, input_offset, memo=self.memo,
File "/home/user/Desktop/venv/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 195, in run
results = StateMachineWS.run(self, input_lines, input_offset)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Desktop/venv/lib/python3.11/site-packages/docutils/statemachine.py", line 233, in run
context, next_state, result = self.check_line(
^^^^^^^^^^^^^^^^
File "/home/user/Desktop/venv/lib/python3.11/site-packages/docutils/statemachine.py", line 445, in check_line
return method(match, context, next_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Desktop/venv/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2357, in explicit_markup
nodelist, blank_finish = self.explicit_construct(match)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Desktop/venv/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2369, in explicit_construct
return method(self, expmatch)
^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Desktop/venv/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2106, in directive
return self.run_directive(
^^^^^^^^^^^^^^^^^^^
File "/home/user/Desktop/venv/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2156, in run_directive
result = directive_instance.run()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Desktop/venv/lib/python3.11/site-packages/IPython/sphinxext/ipython_directive.py", line 1033, in run
rows, figure = self.shell.process_block(block)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Desktop/venv/lib/python3.11/site-packages/IPython/sphinxext/ipython_directive.py", line 732, in process_block
self.process_input(data, input_prompt, lineno)
File "/home/user/Desktop/venv/lib/python3.11/site-packages/IPython/sphinxext/ipython_directive.py", line 584, in process_input
raise RuntimeError(
RuntimeError: Unexpected exception in `/home/user/Desktop/statsmodels/docs/source/contingency_tables.rst` line 57

----------------------------------------

How can I compile the documentation?

[josef...@gmail.com]

I don't know the details

My guess is version compatibility

Changes in packages required for statsmodels or doc build will affect whether older versions of statsmodels will still work with it.

I guess the only reliable way will be to use a (virtual) environment that has the versions of requirements at the time of the release

Even when doc build succeeds at release times, there might still be doc problems, e.g. missing notebooks because they failed to build.

But those should not stop the doc build.

Josef

--
You received this message because you are subscribed to the Google Groups "pystatsmodels" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pystatsmodel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pystatsmodels/trinity-b2a1e174-6611-4a21-97e7-214e8b97a6b1-1717865670100%403c-app-mailcom-lxa14.

[Laura Morales]

Thank you. Pinning dependencies to older versions helped, a bit. But there are still errors and exception no matter how hard I try. In addition, during the build process it looks like it's downloading some datasets from the Internet, which are no longer there (error 404). So building older versions of the docs is basically impossible :/

I get errors like:

HTTP Error 404: Not Found
ValueError: Dataset Moore was not found.

[josef...@gmail.com]

Yes, that's a problem

There are datasets in the notebooks that disappear or change over time on the internet.

And rdataset had changing urls

https://github.com/statsmodels/statsmodels/issues/3554

Kevin recently added some datasets to https://github.com/statsmodels/smdatasets

to avoid broken links in the notebooks.

However, I don't see Moore dataset.

Where is the Moore dataset used?

I also don't find a recent issue for Moore

Those links would need to be fixed in older versions of the notebooks and docs..

Josef

--
You received this message because you are subscribed to the Google Groups "pystatsmodels" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pystatsmodel...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/pystatsmodels/trinity-c28c90b9-de44-439b-9e23-586e6cf33e41-1717932029550%403c-app-mailcom-lxa14.

[josef...@gmail.com]

On Sun, Jun 9, 2024 at 7:56 AM <josef...@gmail.com> wrote:

Yes, that's a problem

There are datasets in the notebooks that disappear or change over time on the internet.

And rdataset had changing urls

https://github.com/statsmodels/statsmodels/issues/3554

Kevin recently added some datasets to https://github.com/statsmodels/smdatasets

to avoid broken links in the notebooks.

However, I don't see Moore dataset.

Where is the Moore dataset used?

I also don't find a recent issue for Moore

The last change for Moore that I found is 6 years ago

https://github.com/statsmodels/statsmodels/commit/31ec838d1eab2ec559ab69d42151085bdf9f18e1

which was an rdataset url change

[Laura Morales]

The HTTP error and Moore Dataset error are specifically from version v0.9.0. All the dependencies seem to be fine, but it's missing this (and probably other) datasets, so it's not compiling.

When building version v0.13.5 I do not get dataset errors, and I also do not get dependencies errors after pinning them, but I get other exceptions such as "no module named statsmodel.iolib.foreign.StataReader" or something else about unpacking the wrong number of arguments.

By the way, when building the most recent version, v0.14.2, I get this error from docs/source/release/version0.6.rst:

AttributeError: 'Series' object has no attribute 'co2'

I assume this is an old file of an old version, so I'm not sure if this error is expected/fine or if it shouldn't be there. Is there a way to fix this error?

[josef...@gmail.com]

Is the code in the version0.x.rst files executed?

In our docs it looks like the examples are not executed

https://www.statsmodels.org/dev/release/version0.6.html#release-0-6-0

AFAIK, we never update the release doc `version` rst files.

So, examples in those are based on the code at the time of the release and do not necessarily work with the current version.

That is, I think the `version` files should be treated as historical text and not as code that might still work.

We have a pretty strict backwards compatibility for the code in statsmodels, but that does not apply to examples that use datasets.

Josef 

--
You received this message because you are subscribed to the Google Groups "pystatsmodels" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pystatsmodel...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/pystatsmodels/trinity-2801abaf-ce14-4dc9-860e-566676cbaf1c-1717938689894%403c-app-mailcom-lxa14.

[Laura Morales]

When building v0.14.2 I see several warnings but no exceptions other than the one for version06.rst.
However, the "examples/" folder is empty, there is only a file index.html.
I just compiled it with "make dirhtml". What am I missing? Why is the examples folder empty?

[josef...@gmail.com]

I really don't know the build details.

I have not built the docs locally in many years, and don't have good guesses for common build problems.

Before the sphinx doc build, we have a run of "\statsmodels\tools\nbgenerate.py" that runs the example notebooks.

Maybe that did not run or run correctly in your case.

Josef

--
You received this message because you are subscribed to the Google Groups "pystatsmodels" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pystatsmodel...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/pystatsmodels/trinity-91c7c7c7-f0cd-4b45-b844-1bf10fa0873e-1717958253424%403c-app-mailcom-lxa08.

[Laura Morales]

Is there a build script somewhere? I could look at it how it's done.
 
 
 

Sent: Sunday, June 09, 2024 at 10:11 PM
From: josef...@gmail.com
To: pystat...@googlegroups.com
Subject: Re: [pystatsmodels] Build older documentation

[josef...@gmail.com]

Makefile and make.bat in the doc folder, AFAIU

with some instructions in the readme file

...\statsmodels\docs\Makefile"

--
You received this message because you are subscribed to the Google Groups "pystatsmodels" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pystatsmodel...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/pystatsmodels/trinity-554f5080-33b6-4af5-9076-85dfb2a30ef1-1717965194665%403c-app-mailcom-lxa08.

[josef...@gmail.com]

and our doc build is automatic on github

for example for a recent PR

https://github.com/statsmodels/statsmodels/actions/runs/9405988810/job/25919090466?pr=9270

which shows some more doc build problems

including another notebook link that seems to be dead

http://staff.feweb.vu.nl/koopman/projects/ckbook/OxCodeAll.zip'