Is sphinx-build extremely dangerous? Or is it just me?

[Dan Harasty]

I'm new to Sphinx, but a seasoned Python programmer.  I'm working through the Sphinx tutorials, and I may have by sheer luck barely avoided a disaster.  Please: someone tell me if I'm being over dramatic, and calm this Sphinx-noob down.

I'm so new, I'm not even really sure which part of Sphinx calls which, and which is exhibiting the [what I consider] extremely dangerous behavior: sphinx-build? sphinx-apidoc? make?  So in my ignorance, I'll just attribute all to sphinx-build.

It seems that sphinx-build imports every Python file it finds in the directory it is pointed to.  But of course, it can't distinguish a true "module"

(reusable code that performs no side-effects until functions are invoked) from a "script" (code invoke to do something).  It can't do this because that distinction is purely in the developer's head, not in Python or Python files themselves.

Therein is -- what I consider -- the extreme danger.  In our system, we occasionally have maintenance scripts sitting in the directories with the modules.  Script that do minor stuff like, oh, delete important system logs, kick off long-running (multi-hour) table generation routines, or alter production tables, or even drop entire databases.  

Imagine my panic when running sphinx-build for the first time, and I realize by the output that EVERYTHING is being imported... which means everything is being executed.  Did I leave any scripts in a state where they are deleting important files or dropping databases?  Are any of those configured to execute against our production system???

Apparently, by sheer luck, no script was configured so as to produce an irrecoverable side effect, and I think my system escaped unscathed.

But it could easily have happened.

Once my blood pressure returned to normal and my panic subsided, I went back to the tutorial docs, looking for an explanation that ALL files would actually be imported/executed. I didn't see that.  I looked for a warning: "if any of your modules or scripts in the tree perform side effects, put all that code in an "if __name___=='__main__'" block, or put "::sphinx-ignore-this-file" as a comment somewhere in the file.".  I didn't see that.

Did I miss that?

Has anyone considered how dangerous it is to execute EVERY PYTHON FILE in a large directory without proper warning (from Sphinx) and thorough code review (by the developer)?

I guess I expected Sphinx would do its work by a simple lexical analysis of the Python files... and not actually import/execute them.  I admit: the docs (the tutorials that I've read so far) don't SAY that.  But they also don't say "all will be imported/executed"... which should be a VERY BIG caveat/warning in the tutorials.

I love how the Sphinx docs look, and I'd like to use the system for my project and my team... But I need to have a rock solid way to make sure that the documentation build doesn't start trashing my operational system by executing scripts that had no intention of running or even have anything useful to be documented in them.

[Luc Saffre]

Thanks for reporting this. To be more precise, it is `sphinx-apidoc
<http://sphinx-doc.org/invocation.html#invocation-of-sphinx-apidoc>`_
who scans the filesystem and finds every Python module, writing Sphinx
source files. You must then run sphinx-build with `sphinx.ext.autodoc
<http://sphinx-doc.org/ext/autodoc.html>`_ enabled to actually import
all those modules. I suggest to add a warning about this possible
pitfall on both documentation pages.

Luc

> --
> You received this message because you are subscribed to the Google
> Groups "sphinx-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to sphinx-users...@googlegroups.com
> <mailto:sphinx-users...@googlegroups.com>.
> To post to this group, send email to sphinx...@googlegroups.com
> <mailto:sphinx...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/sphinx-users.
> For more options, visit https://groups.google.com/groups/opt_out.

[Georg Brandl]

Am 04.03.2014 15:09, schrieb Dan Harasty:
> I'm new to Sphinx, but a seasoned Python programmer. I'm working through the
> Sphinx tutorials, and I may have by sheer luck barely avoided a disaster.
> Please: someone tell me if I'm being over dramatic, and calm this Sphinx-noob down.
>
> I'm so new, I'm not even really sure which part of Sphinx calls which, and which
> is exhibiting the [what I consider] extremely dangerous behavior: sphinx-build?
> sphinx-apidoc? make? So in my ignorance, I'll just attribute all to sphinx-build.

It is in fact autodoc, but you're right, for the user that does not matter.

The former is good practice, so I encourage it in any case.

> Did I miss that?
>
> Has anyone considered how dangerous it is to execute EVERY PYTHON FILE in a
> large directory without proper warning (from Sphinx) and thorough code review
> (by the developer)?
>
> I guess I expected Sphinx would do its work by a simple lexical analysis of the
> Python files... and not actually import/execute them. I admit: the docs (the
> tutorials that I've read so far) don't SAY that. But they also don't say "all
> will be imported/executed"... which should be a VERY BIG caveat/warning in the
> tutorials.

I agree, this needs to be documented better.

Please have a look at
https://bitbucket.org/birkenfeld/sphinx/commits/00e44121ac086e0728ced51300251300f9040498
which I just pushed.

> I love how the Sphinx docs look, and I'd like to use the system for my project
> and my team... But I need to have a rock solid way to make sure that the
> documentation build doesn't start trashing my operational system by executing
> scripts that had no intention of running or even have anything useful to be
> documented in them.

The best way to ensure that is -- and that is unrelated to Sphinx -- is the
"if __name__ == '__main__'" guard. It's only 3 lines and a bit of indentation.

cheers,
Georg

[Robert Gomulka]

Is there a possibility to extract docstrings without importing the modules?
Something like modulefinder does to get imported modules names?
I don't know, parsing some form of AST (like here: http://stackoverflow.com/questions/9085350/parsing-python-module-docstrings)

That would definitely reduce number of side-effects, which, I agree, usually come from poor design ...

Regards,
Robert

[杭媛]

Strongly approve! Is there any update on this problem?

在 2014年3月4日星期二 UTC+8下午10:09:21，Dan Harasty写道：

[Kevin Sheppard]

No update is needed.  Simply add

if __name__ == "__main__":

   <code>

to any runnable script or code you do not wish to execute on import. This is the standard Python method to project code from execution on import, and there is no reason for Sphinx to invent a novel, Sphinx-specific way to do this. 

[杭媛]

Thanks for your reply

Kevin Sheppard <kevin.k....@gmail.com> 于2020年8月6日周四 下午3:41写道：

--
You received this message because you are subscribed to the Google Groups "sphinx-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sphinx-users/6f3dcb0e-adfd-4ada-bf32-d20ad665bd2eo%40googlegroups.com.

[Daniel Scott]

--
You received this message because you are subscribed to the Google Groups "sphinx-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sphinx-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sphinx-users/351bd2a2-2229-41af-9404-f5f7439a6af6o%40googlegroups.com.