Doing a release

[Aaron Meurer]

Now that GSoC has wrapped up, we should start looking at doing a release. As a start, can people

- Mark issues and pull requests that need to be completed before the release with the "SymPy 1.1" milestone

- Update the release notes with changes that have been merged https://github.com/sympy/sympy/wiki/Release-Notes-for-1.1

Any help you can provide on the above two points (especially the release notes) would be greatly appreciated. 

Aaron Meurer

[Shekhar Prasad Rajak]

Hello,

I am trying to complete these PRs : 

1. https://github.com/sympy/sympy/pull/11234

2. https://github.com/sympy/sympy/pull/12011

--

Shekhar

[Aaron Meurer]

I put them on the release milestone.

Aaron Meurer

> --
> You received this message because you are subscribed to the Google Groups
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sympy+un...@googlegroups.com.
> To post to this group, send email to sy...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sympy.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sympy/b695dbe3-e6ed-4e57-9e96-6967ceb464c2%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Shekhar Prasad Rajak]

I think, it will be easy during the new release to see the changes in one file, if we add a file in SymPy repo and keep updating it whenever something added/modified (Before merging PR). Something like this : https://github.com/pandas-dev/pandas/tree/master/doc/source/whatsnew

--

Shekhar

On Monday, 26 September 2016 22:43:16 UTC+5:30, Aaron Meurer wrote:

[Aaron Meurer]

Do you know how pandas generates those files? Are they generated
programmatically or by hand? Usually having a single file for release
notes gets to be a nightmare because of the merge conflicts.

Aaron Meurer

> --
> You received this message because you are subscribed to the Google Groups
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sympy+un...@googlegroups.com.
> To post to this group, send email to sy...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sympy.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/sympy/a3f4a6c5-8a9c-4560-b81b-0841323a8267%40googlegroups.com.

[Jason Moore]

SciPy does it too: https://github.com/scipy/scipy/tree/master/doc/release and the notes seem quite comprehensive and well organized.

To avoid merge conflicts we could require a single file for each item in the notes to be added and then a script compiles the full note set from those.

Jason
moorepants.info
+01 530-601-9791

> email to sympy+unsubscribe@googlegroups.com.

> To post to this group, send email to sy...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sympy.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sympy/a3f4a6c5-8a9c-4560-b81b-0841323a8267%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "sympy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscribe@googlegroups.com.

To post to this group, send email to sy...@googlegroups.com.
Visit this group at https://groups.google.com/group/sympy.

To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAKgW%3D6KnzE2nuGdmovQqs9UtDdPoZb-DHuAp2ReukURE2u8NKw%40mail.gmail.com.

[Aaron Meurer]

So let's figure out how they do it. Does someone do it by hand, or do
they just deal with the conflicts somehow?

I'm +1 to doing this somehow, because the current process isn't
working. We either need a process wherein everyone updates the release
notes with their pull requests (has to be done at the same time or
people won't do it), or just deal with release notes that are
generated automatically (I believe this is what the IPython/Jupyter
guys do).

Aaron Meurer

>> > email to sympy+un...@googlegroups.com.

>> > To post to this group, send email to sy...@googlegroups.com.
>> > Visit this group at https://groups.google.com/group/sympy.
>> > To view this discussion on the web visit
>> >
>> > https://groups.google.com/d/msgid/sympy/a3f4a6c5-8a9c-4560-b81b-0841323a8267%40googlegroups.com.
>> >
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "sympy" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to sympy+un...@googlegroups.com.

>> To post to this group, send email to sy...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/sympy.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/sympy/CAKgW%3D6KnzE2nuGdmovQqs9UtDdPoZb-DHuAp2ReukURE2u8NKw%40mail.gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to the Google Groups
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to sympy+un...@googlegroups.com.

> To post to this group, send email to sy...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sympy.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/sympy/CAP7f1AiN4wzhTfXxg%3DBqMyNR1ghCu-Ri_HWEvkX3RkWKmphrww%40mail.gmail.com.

[Isuru Fernando]

Same topic came up on sage-devel today with a solution.

https://groups.google.com/forum/m/#!topic/sage-devel/81qsOpoOKMM

Isuru Fernando

>> > email to sympy+unsubscribe@googlegroups.com.

>> > To post to this group, send email to sy...@googlegroups.com.
>> > Visit this group at https://groups.google.com/group/sympy.
>> > To view this discussion on the web visit
>> >
>> > https://groups.google.com/d/msgid/sympy/a3f4a6c5-8a9c-4560-b81b-0841323a8267%40googlegroups.com.
>> >
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "sympy" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to sympy+unsubscribe@googlegroups.com.

>> To post to this group, send email to sy...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/sympy.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/sympy/CAKgW%3D6KnzE2nuGdmovQqs9UtDdPoZb-DHuAp2ReukURE2u8NKw%40mail.gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to sympy+unsubscribe@googlegroups.com.

> To post to this group, send email to sy...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sympy.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sympy/CAP7f1AiN4wzhTfXxg%3DBqMyNR1ghCu-Ri_HWEvkX3RkWKmphrww%40mail.gmail.com.
>
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "sympy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscribe@googlegroups.com.

To post to this group, send email to sy...@googlegroups.com.
Visit this group at https://groups.google.com/group/sympy.

To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAKgW%3D6LsMT1DwqSym%3DSt-tWSumfYP5tC41o6eXa6_jPVs2k7jA%40mail.gmail.com.

[Shekhar Prasad Rajak]

The Files(https://github.com/pandas-dev/pandas/tree/master/doc/source/whatsnew) are written by hand with the PR. Means whenever a bug is fixed PR author must add the testcase for the bug, and also update the file. Similarly whenever anything new features implemented. 

See this PR : https://github.com/pandas-dev/pandas/commit/0fe491db358204544814aa2371fed4ea947532ab#diff-52364fb643114f3349390ad6bcf24d8fR286 . This PR fix the issue : #15048   and same is updated in the file (whatsnew/v0.20.0.txt) by the PR author.

--

Shekhar

On Monday, 26 September 2016 22:43:16 UTC+5:30, Aaron Meurer wrote:

[Aaron Meurer]

So how do they deal with merge conflicts in the file? I don't want
people to constantly be having to fix their branches because the
release notes file has conflicts from another merge.

Aaron Meurer

> --
> You received this message because you are subscribed to the Google Groups
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to sympy+un...@googlegroups.com.

> To post to this group, send email to sy...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sympy.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/sympy/d1f76e72-c2bd-4801-ab09-f3d94b0f2785%40googlegroups.com.

[Shekhar Prasad Rajak]

So how do they deal with merge conflicts in the file? I don't want 
people to constantly be having to fix their branches because the 
release notes file has conflicts from another merge. 

I don't know whether there is a good way to deal with this problem(merge conflicts). I think " bug fixed " header(in the whatsnew file ) must have sub-headers (means module names as sub-header like calculus, core , integral, solvers , etc ), so contributors will need to update the respective sub headers only( in which issue/bug is related). This may decrease the number of conflicts. 

Also contributors must know how to handle the merge conflicts. I hope it will not be the problem.

--

Shekhar

On Monday, 26 September 2016 22:43:16 UTC+5:30, Aaron Meurer wrote:

[robert...@posteo.net]

On Thursday, January 12, 2017 at 6:45:00 PM UTC+1, Aaron Meurer wrote:

So how do they [Pandas] deal with merge conflicts in the file? I don't want

people to constantly be having to fix their branches because the
release notes file has conflicts from another merge.

Vladimir Kiselev lists three options in a blog post [0]:
a) Placeholders in the file.
b) Setting the merge strategy "merge=union" for the changelog file in .gitattributes.
c) Separate files per entry.

It seems that Pandas does nothing of those yet.
Interestingly, GitLab has tried a), then b), and is now at c). (See [0],[1],[2],[3].)

[0] https://medium.com/@nettsundere/on-reducing-changelog-merge-conflicts-1eb23552630b#.sgoovezbz

[1] "Use gitattribute merge=union to reduce CHANGELOG merge conflicts.", https://gitlab.com/gitlab-org/gitlab-ce/commit/4377ba1c360cf6f4d15e3b5ad2a7ed7bc41f795e
[2] "Remove gitattribute entry for CHANGELOG.md", https://gitlab.com/gitlab-org/gitlab-ce/commit/59189438388191c044c0697d5bfb571693d95978
[3] https://gitlab.com/gitlab-org/gitlab-ce/tree/master/changelogs

Maybe some of this works for you.

--
Robert

[robert...@posteo.net]

FYI, I found more details on how GitLab currently proceeds (and reasons
why) at
https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/changelog.md
.

(You could have a Python equivalent of their bin/changelog tool.)

[Ondřej Čertík]

Hi Robert,

Thanks for this tip. I like what GitLab is doing. The changelog script
can then be run on Travis automatically to ensure things are properly
formatted, and so on.

Ondrej

[Anthony Scopatz]

Hello All, 

FWIW, we have a release script for xonsh that handles many elements of a release (tagging, changelog, building and deploying docs, pypi conda-forge, etc. ): https://github.com/xonsh/xonsh/blob/master/release.xsh It requires only minimal, metadata changes to use on other projects. I have ported it to xo, lazyasd, amalgamate, regolith, etc.

This includes the changelog feature.  Basically, every branch get's its own file in the news/ dir. At release time, the script glues all of these together. You can read more here: http://xon.sh/devguide.html#changelog

I'd be in favor or breaking this out into a "releaser" project too, if other folks are interested.

Be Well

Anthony

--
You received this message because you are subscribed to the Google Groups "sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
To post to this group, send email to sy...@googlegroups.com.
Visit this group at https://groups.google.com/group/sympy.

To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CADDwiVAKrmo7-q%2BKK9BvMTQ_S78-E2v-OUAGHNHGRgbgaBP4sw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Asst. Prof. Anthony Scopatz
Nuclear Engineering Program
Mechanical Engineering Dept.
University of South Carolina
sco...@cec.sc.edu
Office: (803) 777-9334
Cell: (512) 827-8239
Office: Horizon 1, #011 (ground floor, not first)
Book a meeting with me at https://scopatz.youcanbook.me/
Open up an issue: https://github.com/scopatz/me/issues
Check my calendar

[Aaron Meurer]

That would be useful.

The current script we use is
https://github.com/sympy/sympy/blob/master/release/fabfile.py. It
doesn't handle changelog (except for the authors section), but it does
just about everything else. It runs everything in a Vagrant machine,
and it handles running the tests, (tagging is done manually, for
safety), creating the tarball, doing several tests against the tarball
(there is a whitelist and blacklist against git to make sure all
necessary files are present and no unnecessary files are), uploading
the pre-release to GitHub, uploading the final release to GitHub and
PyPI, updating the documentation on docs.sympy.org, updating the NEWS
entry on sympy.org, and generating the authors list for the changelog
(the authors stuff is also managed by
https://github.com/sympy/sympy/blob/master/bin/mailmap_update.py and
https://github.com/sympy/sympy/blob/master/bin/authors_update.py).

This system is going to need some maintenance. fabric does not support
Python 3, so we need to move to something else that does. The Vagrant
box we are using doesn't support Python 3.3, so we will need to get a
new one by the next release (or else change the scripts to use conda).
It may make more sense to be using Docker (which didn't exist when
this was written). Doing conda-forge updates and changelog updates
would be nice as well. The changelog is the biggest thing, as it's the
most time consuming and currently the least automated.

Aaron Meurer

> https://groups.google.com/d/msgid/sympy/CAPk-6T4fzKMAyfFh7-MO9zyv%3DkN645-ufuUELaosfib4wcNQOA%40mail.gmail.com.