Release process for Stan: built manual and doxygen and how this relates to tagging.

[Daniel Lee]

Hi all,

I realize I might be the only person that knows this stuff. If anyone wants to volunteer to take over, please let me know and take it on. I don't think I'm the most ideal person for managing it, but we can continue this way if no one steps up. And I'm not trying to keep people out of the build process -- I just don't think most people are interested.

I've been trying to script things. I'm not very good at scripting in bash, but I've gotten it to where it's fairly easy to release. If you want to find them, I'm holding onto the scripts in a public repo that's under my user (syclik):
https://github.com/syclik/stan-scripts/tree/master/release-scripts

Here's how the Math library is tagged:

1. We (stan development team) decide the Math library is done and ready for release. This is at a particular commit hash on the develop branch.

2. Someone updates the release notes (RELEASE-NOTES.txt). Commits this on the develop branch.

3. I start running the release script.

  a. This creates a new branch called "release/v<version>" where we specify the version. This is branched from develop.

  b. As a commit on the branch, we update all the version numbers. For example, we change all instances of 2.7.1 to 2.8.0.

  c. As a commit on the branch, we build the doxygen documentation (html files) and commit them to the branch. That is, we run "make doxygen" and then "git add doc/api/html"

  d. A pull request is made with the last commit -- the one that adds the documentation.

  e. The pull request is merged. This updates the develop branch to the commit.
  f. This commit is tagged as v<version>.

  g. The master branch is moved to the tag.

  h. A new issue is created: "Removing v<version> documentation"

  i. A new branch is created from the current develop branch (same as the tag, same as master). On that branch, we remove the built documentation: "git rm -f doc" and this is commited.

  j. A pull request is created from this branch.

  k. The pull request is merged, which updates develop immediately to a state where there is no built documentation.

That's it for tagging Math. We then move onto tagging Stan, which looks almost identical, but with some shuffling of the Math library.

In Stan, on the develop branch, the math library lives at "lib/stan_math." Notice, there's no version number in the folder name. When Stan is tagged, we (this is really a loose use of the term "we" and it might just be me) thought it would be best to know what tagged version of the Math library is used. So for the tag, we move lib/stan_math to lib/stan_math_<version>. This happens right at 3c in the directions above. Then at 3h, we move the location of the library back to lib/stan_math. (We also update it to the current state of develop, which doesn't have the built documentation.)

In the tagged version of Stan, we have built documentation (the manual and the doxygen) sitting at "doc/" and the tagged math library at "lib/stan_math_<version>." We don't stay in this state for very long. In fact, we move back to no built doc and lib/stan_math within a few minutes.

CmdStan does the same as Stan by changing the location of Stan from "stan" to "stan_<version>."

Hopefully that makes sense. Most of it is automated. Most of this was what I think is reasonable, but could probably be done better. If anyone wants to take over, please let me know. If anything is done improperly or wrong, we can discuss and fix it.

Daniel

[Bob Carpenter]

The build process isn't the bottleneck for me. It's the complexity
of thorough testing and the submodule dependencies. I've had the
indexing done for a while, but I'm stalled in doing the several days of
work it's going to take for the final tests. And, of course, Rayleigh's
completely blocked on writing functions for Ben due to our testing
requirements and the combinatorics.

We talked about what to do with the doc before, and nobody else
had any good ideas that would work with GitHub's limited ability to
do tagged distributions. I suppose we could do this the other way around
and ignore Git's own distributions and do our own. Then we wouldn't be
beholden to their versioning and we could just create temporary PDFs and
HTML that don't ever need to get checked in.

- Bob

P.S. Is there any reason not to put any scripts that are needed into the
relevant repos rather than on your personal repo? Along with these
instructions, perhaps?

> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.