Request for Comment: REP 132 - Changelogs in Package Source Instead of Wiki

[Mirza Shah]

Hi all,

One of the aspects of ROS that we have wanted to revise for a while is how changelogs are maintained. Presently changelogs are implemented as a wiki subpage for each package and isolated from the actual package source code.

REP 132 proposes implementing changelogs as a simple text file that is part of a package's source tree. Keeping changelogs with source code allows them to be versioned with code in the VCS, make them easier to find, can be injected into binary packages (e.g. deb, rpm), and will hopefully encourage package maintainers to actually maintain their change lists. Details of our initial proposal can be found in a draft of REP 132 at https://github.com/ros-infrastructure/rep/blob/master/rep-0132.rst.  

Please take a look and provide feedback/discussion here.

Cheers,

-Mirza

[Jonathan Bohren]

Cool! I was just thinking about this today... 

I propose we add release (or at least release initiation) dates to the standard format. ChangeList.txt format sounds like a good idea.

Maybe for referencing issues we could just reference revision numbers / revision hashes?

-j

--
You received this message because you are subscribed to the Google Groups "ROS Buildsystem Special Interest Group" group.
To post to this group, send email to ros-sig-b...@googlegroups.com.
To unsubscribe from this group, send email to ros-sig-buildsy...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msg/ros-sig-buildsystem/-/ML_1JsHLuF0J.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Jonathan Bohren

PhD Student

Dynamical Systems and Control Laboratory

Laboratory for Computational Sensing and Robotics

The Johns Hopkins University

(707) 520-4736

j...@jhu.edu

[Mirza Shah]

Hey Jonathan,

I didn't add a date or rev for each entry as I figured that was redundant info from the VCS (version number ==  tag number which in turn maps to hash+date) and didn't want users to have to worry about it. Do you feel there's a reason why it should be added in explicitly?

As for being able to reference issues within each entry inline the way we can on GitHub, we might be able to do some filtering to automatically turn hashes into links. To keep it simple for now, we thought that embedded URIs should be sufficient. Yes you'll have to write out the entire link, but when it's rendered you'll only see the label. Do you think there's a strong enough use case that we should address it now?

-Mirza

[Jonathan Bohren]

On Fri, Jan 25, 2013 at 7:14 PM, Mirza Shah <ms...@willowgarage.com> wrote:

I didn't add a date or rev for each entry as I figured that was redundant info from the VCS (version number ==  tag number which in turn maps to hash+date) and didn't want users to have to worry about it. Do you feel there's a reason why it should be added in explicitly?

I was thinking that while it's possible to get the date information through the tag log, it could take some time to pull that information out from a live repo (if the user doesn't have one checked out) or a repo-browsing website (if it exists). 

Also it means you can show date information on the wiki without having to parse all the tags from the repo on the wiki backend. But if that's already in the plan, we probably don't need it in there.

As for being able to reference issues within each entry inline the way we can on GitHub, we might be able to do some filtering to automatically turn hashes into links. To keep it simple for now, we thought that embedded URIs should be sufficient. Yes you'll have to write out the entire link, but when it's rendered you'll only see the label. Do you think there's a strong enough use case that we should address it now?

I just saw that you had mentioned it. Putting links in is probably good enough to start.

-j

[tkruse]

Hi Mirzah,

Not sure whether I just get the phrasing wrong. I believe the REP should say that any ChangeList.txt must be a valid rst doc (though not vice versa). The filename should end in .rst (also for it to be nicely displayed on github). And I think it might be nice to make the name look less like "CMakeLists.txt", else people will often open the wrong one. Also it might be nice to fall back to using README.rst looking for a changelog in there.

As far as i read the rst spec (http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html), it does not define single dashes underlining as "second-level headings". It gives priority in the order encountered (yet another reason not to try out writing a custom parser for a subset language). So rst does not allow us to insist on single dash underlining. It allows us though to give our users the freedom to use the section headings of their choice.

Then the format specification seems not clear enough for me. It seems that the deb/rpm requirements are so strict that we cannot possibly allow users to go wild with rst. So the REP should not state that we don't allow it because it is "overwhelming", but just because deb/rpm requirements only allow flat lists of statements.

So things currently not explicitly forbidden in the rep are:
- Many section levels.
- what other elements are not supported, but still allowed? I assume *emphasis* is not forbidden?
- nested bullet points
- 2nd level headings which are not version numbers
- version numbers out of order (do we want to impose descending numbers?)

Here is an alternative design. Users are free to put any valid rst into the overall changelog, but not into version sections. A version section is a sectionwith a title  that is a full three digit) version number, and ends with the end of document or the next section heading. Within a version section, only *flat* lists of bullets are allowed, at least one bullet point must be proivided, and each bullet may only contain text with only inline markup. Version sections must be in strict descending order, gaps allowed (at the discretion of the package maintainer). subsections in version sections are not allowed, but version sections may themselves be subsections.
Outside of version sections, users may put whatever they feel like, using whatever rst elements they want. It does not hurt us to allow this, since we only care about version sections for deb/rpm and such.

Whether certain inline markup should also forbidden by the REP I don't know. In any case the REP should reference the some online rst specification and refer to it to describe what is allowed in versino sections and what not.

also not sure whether 1.0.2-2 is also a valid version number for us.

So here is something that would be possible that way:

---------
Changelog
---------

TODO
====

1. fix #42
2. Cleanup code

0.2
---

0.2.12
^^^^^^

- fixed #41, *finally*

0.2.0
^^^^^

- got 0.2.0 out

0.1
---

0.1.0
+++++

- Initial release



i won't say it's pretty, but it's technically sound, I'd say.
For issues, we cannot rely anything is on github, and even stuff on github might have bugtrackers elsewhere for whatever reason.
Similarly version hashes are not too great in case somebody moves from one type of SCM to another.

[Jack O'Quin]

Looks good. I, too, was recently thinking how desirable this would be.

Is there any thought of writing a tool to convert existing wiki ChangeList pages to this format? 

Something like the <<Ticket(trac, ticket)>> macro would be worth building, I think.

--
 joq

[Mirza Shah]

Hey Thibault,

Thanks for the feedback, here's my comments:

* I agree changing "ChangeList.txt" to something else...maybe just "CHANGELOG"

* You are correct about the RST headings, I can reword that to say we only supported dashed headings. 

* I did not mean to imply that the change list was complete RST...rather i was trying to say that it uses a very limited subset of RST that would allow it to be parsed and rendered without change by an RST parser. We went with a limited subset so as to avoid incompatibilities with the .rpm/.deb format and any other applications where we need to extract information from the log. The idea is to keep the format/structure constrained initially until we can get changelogs integrated into bloom and the wiki -- as you noted the rpm/deb formats are sort of strict so we don't want to deal with issues such as exotic markup mucking them up. Once we're confident things work well, we can slowly loosen the constraints.

* I did not specify the limited subset tightly enough as you pointed out. I do not think there's a need for nested headings, nested bullets, text style (italics, emphasis, etc) but again we can slowly bring these in with time. I can clarify this in the REP.

* Your proposed alternative sounds reasonable though not as simple as I personally would like. I'd love to hear feedback from others on this approach and if there's strong enough consensus, we can go that route.

* The version hashes are something I think we might want to do later down the line if the majority of the community has moved to GitHub --- but I agree, we shouldn't tailor things specifically to GitHub.

-Mirza

--
You received this message because you are subscribed to the Google Groups "ROS Buildsystem Special Interest Group" group.
To post to this group, send email to ros-sig-b...@googlegroups.com.
To unsubscribe from this group, send email to ros-sig-buildsy...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msg/ros-sig-buildsystem/-/5ioT0MLoj9MJ.

[Daniel Stonier]

A big +1 for this.

It looks quite reasonable to me. I'm happy to start with a simple subset of rst as well. I can't see changelogs needing hefty formatting support, and starting simple can get it on board, compatible and robust faster. Adding extra support (if really desired) can always go in later.

--
Phone : +82-10-5400-3296 (010-5400-3296)
Home: http://snorriheim.dnsdojo.com/

Yujin R&D: http://rnd.yujinrobot.com/

[tkruse]

Hi Mirzah,

On Saturday, January 26, 2013 3:27:52 AM UTC+1, Mirza Shah wrote:

* You are correct about the RST headings, I can reword that to say we only supported dashed headings. 

This would mean we'd have to write our own parsers, because rst parser will not fail is users use other headings than underline single dashing.
Writing our own parsers should be avoided at all costs imho. They have to be maintained, and their compatibility to rst has to be granted. We lose more than we win.

* I did not mean to imply that the change list was complete RST...rather i was trying to say that it uses a very limited subset of RST.

"subset" has strong implications that would have to be verified and validated through unit testing. I think what you mean is that changelogs will use a rst-like syntax, with less features, requiring a home-made parser for us to maintain because rst parsers will accept much more. I might be nitpicking, but for a REP it is important to be clear from the beginning.

as you noted the rpm/deb formats are sort of strict so we don't want to deal with issues such as exotic markup mucking them up. Once we're confident things work well, we can slowly loosen the constraints.

How will we ever be able to loosen the constraints, given the rpm/deb formats will not loosen up? And why should anyone spend any ime in the future on loosening up our own parser language? I thought ROS is supposed to move towards more reliance on standard tools (and languages).
 

* Your proposed alternative sounds reasonable though not as simple as I personally would like. I'd love to hear feedback from others on this approach and if there's strong enough consensus, we can go that route.

So one strong motivation for it is to allow using the README.rst to contain the changelog.

On a related note, I was also thinking of how easy it would be to have the changelog included in the sphinx documentation as well. Such as when you have a doc folder with .rst files, can this easily be integrated with your format. And again, as soon as we deviate from rst, by only being rst-like with respect to things like escape sequences, comments, inline markup, etc., then a valid CHANGELOG may not be valid RST, and fail for sphinx. i see no value in that.

I attached a quickly hacked python script to show the start of how what I say can be done. Try it on any changelog, or e.g. on
https://github.com/vcstools/vcstools/blob/master/doc/changelog.rst

I believe checking the version sections for forbidden stuff that way is easier than writing an own parser. So my solution is not very complex to solve, and it reuses rst parsers, and gives users a lot of freedom.

[Brandon Alexander]

+1 for an extension that would render nicely in GitHub.

While I agree we shouldn't rely on GitHub, including a .rst (or even a .md extension) would allow GitHub and others to render the changelog better while still being viewable as plaintext too.

--

You received this message because you are subscribed to the Google Groups "ROS Buildsystem Special Interest Group" group.
To post to this group, send email to ros-sig-b...@googlegroups.com.
To unsubscribe from this group, send email to ros-sig-buildsy...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msg/ros-sig-buildsystem/-/3YiRkJYuA5QJ.

[Isaac Isao Saito]

+1 for chaging file name.

About change content (not about format/syntax), wouldn't it be nicer
to give just a little spec/hint what to write? I like this passage
from fedora that's cited from REP [1]:

<quote>
Changelog entries should provide a brief summary of the changes done
to the package between releases, including noting updating to a new
version, adding a patch, fixing other spec sections, note bugs fixed,
and CVE's if any. They must never simply contain an entire copy of the
source CHANGELOG entries. The intent is to give the user a hint as to
what changed in a package update without overwhelming them with the
technical details. Links to upstream changelogs can be entered for
those who want additional information. If you wish to "scramble" or
"obfuscate" your email address in the changelog, you may do so,
provided that it is still understandable by humans.
</quote>

# Please bear with the long quote.

In particular, this quote allows loose description (by saying "give
the user a hint"). This could be only me, but often times when I'm to
write commit logs, I wonder to what level I have to be specific and
verbose...I'd be relieved if the spec like above is there.

[1] http://fedoraproject.org/wiki/Packaging:Guidelines#Changelogs

--
Isaac

> To unsubscribe from this group and stop receiving emails from it, send an
> email to ros-sig-buildsy...@googlegroups.com.

> To post to this group, send email to ros-sig-b...@googlegroups.com.

[Mirza Shah]

Hey Thibault,

I disagree on the subset qualification...what we have specified IS a subset of RST. If we think in terms of formal languages, the grammar for our restricted subset generates strings that are always members of the the set of strings that the  RST grammar can generate. However, the set of strings that are RST can generate is a superset of our changelog language. Hence, the suggested changelog language is a subset of the RST langauge.

I think what is being missed is not the syntactic rules of the changelog, but rather one level up at the semantic level (yes I'm thinking in terms of programming languages/compilers). Our changelog grammar (syntax) is a subset of the RST grammar, but our language is more than that as it has semantic constraints that grammar does not cover. For example, we have a restriction that the headings must be version numbers and we want them to be sequential (I did not mention the latter in the spec, but that needs to be added in). It doesn't matter what route you go, you will HAVE to at some point parse some of the log yourself in order to verify its correctness.

The reason for the format is not primarily "an RST parser can easily parse it", but rather, it's easily parseable by a custom parser (again you can't get around this) if we want to do things like 1) extract information from it (I'm sure we'll need this for bloom) and 2) if we need to transform it to other formats such as the rpm/deb changelog style and 3) it still is very readable in raw form. The parser for our restricted language is quite simple and I'm sure any of us could write a verifier for the format in less than an hour.

If we make the format more complex than it is now and allow full-fledged RST or some other language, then I think we're in for a world of hurt when we need to verify its correctness and extract information out of it.

We can discuss this tomorrow morning at our weekly developer's meeting and get feedback from the group on which route we should go...I believe anybody who wants can join in for that. We can then revise the REP and do another round on this SIG list.

-Mirza

[tkruse]

On Monday, January 28, 2013 9:55:47 PM UTC+1, Mirza Shah wrote:

Hey Thibault,

I disagree on the subset qualification...what we have specified IS a subset of RST. If we think in terms of formal languages, the grammar for our restricted subset generates strings that are always members of the the set of strings that the  RST grammar can generate.

I think it does not. Example:

0.1.1
-----

* |link| test

This does not contradict the rules you gave, but raises an error in an rst parser.

I think what is being missed is not the syntactic rules of the changelog, but rather one level up at the semantic level (yes I'm thinking in terms of programming languages/compilers). Our changelog grammar (syntax) is a subset of the RST grammar, but our language is more than that as it has semantic constraints that grammar does not cover. For example, we have a restriction that the headings must be version numbers and we want them to be sequential (I did not mention the latter in the spec, but that needs to be added in). It doesn't matter what route you go, you will HAVE to at some point parse some of the log yourself in order to verify its correctness.

No, correctness can (and should) be checked AFTER parsing, not during parsing. In programming language terms, one could say we need no custom parser, just a custom interpreter. Just like to verify package.xml for catkin package, we do not reinvent an xml parser.

 

If we make the format more complex than it is now and allow full-fledged RST or some other language, then I think we're in for a world of hurt when we need to verify its correctness and extract information out of it.

I expanded my earlier code to do just that, it extracts deb-like formatted text from arbitrary rst, also checking version numbers are in descending order.
It yields errors if in version sections, users used something that's not allowed in the REP. It allows embedding the changelog within any other rst document, such as README.rst. The code is still not very pretty and may not be robust enough yet, but it already has all the functions needed.

Input:

Changelog for package foo
=========================

0.1
===

0.1.27
^^^^^^

- dash for bullet list

.. rst comment

0.1.26
^^^^^^

+ using + for bullet list, and reference  `Python <http://www.python.org/>`_
+ second element *emphasized*

0.1.4
=====

text without bullet list

Output of python parse_changelog.py CHANGELOG.rst:

[(u'0.1.27', [u'dash for bullet list', u'rst comment']),
 (u'0.1.26',
  [u'using + for bullet list, and reference  `Python <http://www.python.org/>`_',
   u'second element *emphasized*']),
 (u'0.1.4', [u'text without bullet list'])]

 
There is some hurt in the code when checking for docutil rst node types, but I'll take that any time over custom parsers.

[tkruse]

Added github repo for code here:
https://github.com/tkruse/changelog_rst

[Mirza Shah]

Hey Thibault,
Ah yes you are correct regarding the grammar, my apologies. Regarding parsing and verification, correctness is indeed usually checked after parsing though it is often convenient to do some of the higher level stuff while building the abstract syntax tree. I did not look at your code earlier (my bad sorry), I see now that you use docutils to generate the AST (ie parse) and then traversing it to verify it. Yes, I agree now that this is a better solution :)

~mirza

--

You received this message because you are subscribed to the Google Groups "ROS Buildsystem Special Interest Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-buildsy...@googlegroups.com.
To post to this group, send email to ros-sig-b...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msg/ros-sig-buildsystem/-/XR9Gku_M9pQJ.

[tkruse]

So the code I have on github differs also from the REP in more ways. I allow for changelog entries that do not have bullet points, e.g.

0.1.0
-----

initial version


I think we do not gain anything by forcing users to use bullet points for single entries. Another difference is that the code currently also accepts version number prefixes in titles


0.1.0 (codename cobra)
------------------------

Also I have a branch on github that will accept nested bullet points, a small problem with that is how to transform that for deb/rpm, currently I add a * for each level of indentation.

I think it helps to look at other non-ROS examples of changelogs to get a good feeling of what features users may want for a changelog.
Looking e.g. here:
https://github.com/TomFrost/Hydrogen/blob/master/CHANGELOG.md
https://github.com/twitter/bootstrap/wiki/Changelog  (note the v prefix in v1.0.1)
https://github.com/angular/angular.js/blob/master/CHANGELOG.md
https://github.com/cloudhead/less.js/blob/master/CHANGELOG.md
https://github.com/celery/django-celery/blob/master/Changelog

While we cannot support all of the format details, I believe the REP should be the least constraining possible.

Supporting links to github issues would be great, but I think it would be best if we could leave that to github itself. Meaning if the wiki page shows the changelog in a passive way, but allows quick access to the CHANGELOG.rst github, and that one has active links, that would be quite sufficient. I guess the same goes for any other repository webserver, trac, jira, etc.

cheers,
  Thibault

[Mirza Shah]

When you talk about github issue links, are you talking about hash tags automatically turning into links? Will GitHub automatically do that with RST documents? If so, then yeah that's more than sufficient imo.

-mas

[tkruse]

Actually, github does not, must have been wishful thinking on my part. But we could invent a rst flavour that does so and suggest it to github. Still for the REP, that means no change, I still don't know how to link issues nicely other than by full url.

[Jack O'Quin]

On Wed, Jan 30, 2013 at 6:06 AM, tkruse <tibo...@googlemail.com> wrote:

Actually, github does not, must have been wishful thinking on my part. But we could invent a rst flavour that does so and suggest it to github. Still for the REP, that means no change, I still don't know how to link issues nicely other than by full url.

+1

This seems like something that would be good for many github projects, more useful than a ROS-only solution. I expect they would be receptive to the idea, especially if we can provide a proof-of-concept patch.

--
 joq

[tkruse]

So I have updated the REP at https://github.com/ros-infrastructure/rep/pull/17
There are some decisions which could do with some voting, or at least a review, if someone has the time.

 The places that will be searched are:
1. <package_root>/CHANGELOG.rst
2. <package_root>/README.rst
3. <package_root>/doc/CHANGELOG.rst

Within version section we will allow
* paragraphs
* transitions
* blank lines
* hyperlinks
* comments
* bullet lists
* enumeration lists
* inline markup
* directives (may be invisible in processed changelog)

We forbid (raises error):

* Definition lists
* Field lists
* Option Lists
* Literal blocks
* Line blocks
* Block quotes
* Doctest blocks
* Tables
* Subsections

version sections are sections which *start* with a version number, meaning match for: ^[0-9]+\.[0-9_]+\.[0-9_]+

Not sure whether we want enumerated / nested lists, I added preliminary support (that could be improved still) such that in the prototype it looks like this.

Input.

1. Item 1 initial text.

  a) item1a
  b) item1a

2. Item 2

  1. Item 2
  2. Item 2

* level1 bullet

 * level2 bullet
 * level2 bullet2

  * level3 bullet
  * level3 bullet2

Output (e.g. within the deb/rpm):

  1 Item 1 initial text.
  _1 item1a
  _2 item1a
  1 Item 2
  _1 Item 2
  _2 Item 2
  _level2 bullet
  _level2 bullet2
  __level3 bullet
  __level3 bullet2
  _level2 bullet3

We can also forbid nested bullets or nested enum lists, but I like to give users freedom.

For the wiki, we still have several options, listed.
 * raw display is quickest for the users and easiest for us, maybe
 * rendered display is nicer to the eye, allows following embedded hyperlinks
 * link to the source location is a bit worse for the users (navigating separate sites, but may be least effort)

The REP does not need to decide this, but opinions would still be great.

[Tully Foote]

Catching up with this discussion it appears that we're approaching a consensus.  

Of Thibault's requests for comments, I agree with most of it.  Though I have reservations about reading the README.rst looking for changelog, it seem to conflict with GitHub's standard Readme display being effectively the homepage of the project.  

Does anyone else have anything else to suggest?

Tully

--
You received this message because you are subscribed to the Google Groups "ROS Buildsystem Special Interest Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-buildsy...@googlegroups.com.
To post to this group, send email to ros-sig-b...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msg/ros-sig-buildsystem/-/UnY-xJuGUYwJ.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Tully Foote
tfo...@willowgarage.com
(650) 475-2827

[tkruse]

On Tuesday, February 26, 2013 10:16:00 AM UTC+1, Tully Foote wrote:

Though I have reservations about reading the README.rst looking for changelog, it seem to conflict with GitHub's standard Readme display being effectively the homepage of the project.  

Can you expand on that? I mentioned this specifically because already on Github many projects use README.rst for their changelog, as you can see if you google "github README.rst changelog". So I see this as compliant with github, not conflicting.
And for projects will a small enough Changelog, I find it reasonable and convenient to get the Changelog on the githb "homepage". And as I put it later in the search preference, if the changelog grows too large it can be placed in a separate file.

[Daniel Stonier]

Some people do use their github README.rst for other things though. If we can add the changelog feature without constraining what they currently do (even if that group is only a minority), that's always got to be the best course.

--
You received this message because you are subscribed to the Google Groups "ROS Buildsystem Special Interest Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-buildsy...@googlegroups.com.
To post to this group, send email to ros-sig-b...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msg/ros-sig-buildsystem/-/bUiPY_XNhWAJ.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

[Tully Foote]

On Tue, Feb 26, 2013 at 11:31 PM, Daniel Stonier <d.st...@gmail.com> wrote:

On 26 February 2013 20:20, tkruse <tibo...@googlemail.com> wrote:

On Tuesday, February 26, 2013 10:16:00 AM UTC+1, Tully Foote wrote:

Though I have reservations about reading the README.rst looking for changelog, it seem to conflict with GitHub's standard Readme display being effectively the homepage of the project.  

Can you expand on that? I mentioned this specifically because already on Github many projects use README.rst for their changelog, as you can see if you google "github README.rst changelog". So I see this as compliant with github, not conflicting.
And for projects will a small enough Changelog, I find it reasonable and convenient to get the Changelog on the githb "homepage". And as I put it later in the search preference, if the changelog grows too large it can be placed in a separate file.

Trying this search I found that only 4 of the 10 had changelogs in the file and of those most had significant other content, which would not fall in the allowable formatting.  Including images and tables.  

Elements are changelog but whole file is not parseable

https://github.com/plone/plone.directives.tiles/blob/master/README.rst 

https://github.com/lericson/pylibmc/blob/master/README.rst

https://github.com/garbas/vim-snipmate/blob/master/README.rst

https://github.com/jkbr/httpie/blob/master/README.rst

No changelog:

https://github.com/varnish/vagent2/blob/master/README.rst

https://github.com/docopt/docopt.php/blob/master/README.rst

https://github.com/peterix/dfhack/blob/master/Readme.rst

https://github.com/hyde/hyde/blob/master/README.rst

https://github.com/github/markup

https://github.com/rioderaca/LaTeX-Track-Changes/blob/master/README.rst

However none of these could be automatically parsed into a changelog without applying some very strong heuristics as to what's to be excluded.  And if we're going to be setting up automated tooling defining these heuristics is going to be close to impossible.  

 

Some people do use their github README.rst for other things though. If we can add the changelog feature without constraining what they currently do (even if that group is only a minority), that's always got to be the best course.

A quick search for what's supposed to be in a readme suggests that the standard is a lot more than just a changelog:

https://www.google.com/search?q=what+to+write+in+a+readme+file  leads to http://stackoverflow.com/questions/2304863/how-to-write-a-good-readme and http://www.wikihow.com/Write-a-Read-Me 

With all the above said, I don't expect that we can reliably expect README.rst to be parseable using our strict rules.  And unless we expect to be able to parse the README.rst file in most cases, and have it actually contain changelog content, not random other information, I don't feel that it should be in the search path.  

Tully

[Daniel Stonier]

That's alot of effort Tully :)

Not sure if I expressed my thoughts previously very accurately, but they were in the same vein. I couldn't actually understand why we'd want to pressgang the use of a readme when they already have varied uses and there is a very clear alternative that is common practice, the changelog file.

--
You received this message because you are subscribed to the Google Groups "ROS Buildsystem Special Interest Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-buildsy...@googlegroups.com.
To post to this group, send email to ros-sig-b...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msg/ros-sig-buildsystem/-/bUiPY_XNhWAJ.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Phone : +82-10-5400-3296 (010-5400-3296)
Home: http://snorriheim.dnsdojo.com/

Yujin R&D: http://rnd.yujinrobot.com/

--
You received this message because you are subscribed to the Google Groups "ROS Buildsystem Special Interest Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-buildsy...@googlegroups.com.
To post to this group, send email to ros-sig-b...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "ROS Buildsystem Special Interest Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-buildsy...@googlegroups.com.
To post to this group, send email to ros-sig-b...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[tkruse]

Hi,

I am not sure where you both get the idea that the current REP tries to restrict the README.rst to contain nothing else than a changelog. The very opposite is true, the current REP tries to make it possible to use the README.rst the way projects use it on github, exactly like the projects Tully looked at (I actually looked at those projects on github first. I did not ask Tully to look at projects that I had not looked at myself before. That would have been silly. I sometimes do silly things, but give me some credit).

I now made a pull request trying to make the REP more clear, since you both seem to have gotten the wrong idea: https://github.com/ros-infrastructure/rep/pull/29

To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-buildsystem+unsub...@googlegroups.com.

To post to this group, send email to ros-sig-b...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msg/ros-sig-buildsystem/-/bUiPY_XNhWAJ.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Phone : +82-10-5400-3296 (010-5400-3296)
Home: http://snorriheim.dnsdojo.com/

Yujin R&D: http://rnd.yujinrobot.com/

--
You received this message because you are subscribed to the Google Groups "ROS Buildsystem Special Interest Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-buildsystem+unsub...@googlegroups.com.

To post to this group, send email to ros-sig-b...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "ROS Buildsystem Special Interest Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-buildsystem+unsub...@googlegroups.com.

To post to this group, send email to ros-sig-b...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Jack O'Quin]

As best I can summarize this long discussion:

 * There is agreement about Thibault's proposal that we use a full .rst parser.

 * The recommended file name is CHANGELOG.rst.

 * The question of whether to include README.rst in the bloom search path has not been resolved, yet.

Are there other open issues?
--
 joq

[tkruse]

That's all issues. And in the meeting, there was general doubts about whether supporting the github README.rst would do us good, whether this would be understandable, usable, manageable, what breakage could occcur and such. Though there was no actual reason this cannot work, I failed to persuade the others of this being a useful feature.
So instead, the REP will be changed to only use a file called CHANGELOG.rst in the project root, and this one to only contain changelog sections, meaning the restricted set of elements I already identified. We start with that as first iteration, and if extending this seems reasonable as we go along, we can still do that.

I am not sure whether Mirzah is available for this work at all, meaning changing the REP and later implementing the tool. I fear he is now taken by other assignments, so I'll do the REP change one of these days, and leave it to Tully to organize implementation.

[Jack O'Quin]

Sounds good, Thibault. Thanks for your efforts.

+1 this will be a very useful enhancement.

--
 joq

[Mirza Shah]

Hi Thibault,

That is correct, I have been reassigned to another group and will not be working on this REP anymore.

-Mirza

--

You received this message because you are subscribed to the Google Groups "ROS Buildsystem Special Interest Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-buildsy...@googlegroups.com.

To post to this group, send email to ros-sig-b...@googlegroups.com.

[tkruse]

I updated the REP, it's in the same pull request.

I small issue came up:

Can anyone say more about whether we will also support rosbuild stacks and packages that way?

[Jack O'Quin]

That seems desirable. We're going to have rosbuild stacks around for years.
--
joq

[Tully Foote]

For rosbuild I would propose to recommend future support of Changelog.rst in the root of the stack.  However, that won't be a priority to implement in the near future. 

--
You received this message because you are subscribed to the Google Groups "ROS Buildsystem Special Interest Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-buildsy...@googlegroups.com.

To post to this group, send email to ros-sig-b...@googlegroups.com.

[dof...@gmail.com]

All,

I wanted to raise again the topic of easily retrieving version dates. When inspecting a project's changelog for the first time, one of the first things I do is go through the dates of all previous releases to have an idea of its level of activity (high, low) and development style (major/minor release frequency, (non) periodic release cycles, etc). Without the possibility to see many version/date pairs at once -- in the same place, this becomes much harder, and oftentimes not worth the extra overhead. So, is there a direct mechanism to retrieve version/date information?. It is not clear to me from reading the current thread.

TIA,

Adolfo.

[tkruse]

Hi Aldolfo,

the idea is that a release system can combine the information from the Changelog with the information from a VCS, to put a date on a version. So if you installed the debian package, you'd get a changelog where version number and date are combined.
This is also somewhat more reliable than what people would put in. Also it is possible to put changes in the datelog additionally if a maintainer feels this is useful, but those would not be processed.

To get what you want to see, you can also of course just use a visualization of the version control system, to give you tags and dates.

cheers,
  Thibault

[Adolfo Rodriguez Tsouroukdissian]

On Friday, March 8, 2013 9:55:03 AM UTC+1, tkruse wrote:

Hi Aldolfo,

the idea is that a release system can combine the information from the Changelog with the information from a VCS, to put a date on a version. So if you installed the debian package, you'd get a changelog where version number and date are combined.

Thibault,

Binary package changelogs are indeed a good place to look at. In the ROS ecosystem I was more used to referring to the changelog file/wiki page, but your suggestion makes perfect sense. Thanks.

Adolfo.
 

[tkruse]

There is a small concern about the rest of the title of the section header. If the section header reads:

"0.1.26 (2013-03-01) Hydro; stable; codename Wayne"
or something, that does not prevent seeing this as a version section, but what should happen to the suffix
"(2013-03-01) Hydro; stable; codename Wayne"
with respect to the generated binary changelog. We do not need to specify that in the REP, but if want to recommend adding the date to the section header, then we should also think about what to do with the rest of the section header.

I see these options:
* Ignore everything else behind the version number
-- this may lose some information
* Use everything behind the version number as an entry of the changelog
-- this may duplicate dates, and in particular create conflicting dates
* Use everything behind the version number that is not in round brackets as an entry of the changelog
-- a bit messy, more custom parsing of the title, need to define syntax for brackets
* Forbid anything else than a version number and an [ISO] date in round brackets, ignore date
-- this is a bit restrictive

if we wanted the last option, the REP would need changing

Thoughts?

[tkruse]

I see Dirk took the initiative to just change the REP, making the ISO date part mandatory (though it will be ignored) and forbidding any other elements to a title of a version section. This was briefly announced on ros-users, I just mention it here for completeness sake.
That decisionis was just made effective in the REP without justifying it in the REP, nor was it justified here or in the ros-users post. So if anyone wants a justification, I guess they have to go ask for it nicely.

[Dirk Thomas]

I did not change the REP directly but created a pull request: https://github.com/ros-infrastructure/rep/pull/33
I announced that specific change on ros-users and asked for feedback.
So only after receiving the feedback this change is going to be merged.

Regarding the content of the changes:
The REP states that we need all information necessary to generate changelog files for i.e. Debian and other.
* second last bullet under "Rational"
* "deb" example under "Popular Package Changelog Formats"
I thought it was clear enough that we require that information and the current draft just lacks the date value per changelog entry.

If anything is unclear please ask.
Or provide feedback on the pull request content itself.

- Dirk

> --
> You received this message because you are subscribed to the Google Groups "ROS Buildsystem Special Interest Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-buildsy...@googlegroups.com.
> To post to this group, send email to ros-sig-b...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msg/ros-sig-buildsystem/-/0QY-tCKYS4wJ.

[tkruse]

A lot more discussion happened on the github pull request. Unless I am mistaken, that discussion will be deleted should the pull request ever be deleted, also this is hidden away a bit. So I copy over stuff from there to here.

The main problem is around extracting information from the VCS, in particular in the absence of VCS information, e.g. when releasing from a tarball.

There are smaller problems mentioned that the debian changelog format requires not just the calendar date, but also clock time, and a maintainer, whether and how to extract it from a VCS, and in particular how to extract this froma VCS for versions in the past.

http://www.debian.org/doc/debian-policy/ch-source.html

package (version) distribution(s); urgency=urgency
  [...]
  -- maintainer name <email address>[two spaces]  date
package (version) distribution(s); urgency=urgency
  [...]
  -- maintainer name <email address>[two spaces]  date

The date has the following format:
day-of-week, dd month yyyy hh:mm:ss +zzzz

There was an idea about catkin_prepare_release modifying the changelog file entering system date and time to the corresponding version section.

Below is the recent summary by Dirk that is the result of this weeks meeting, however I see no mention of the maintainer name extraction for past releases.

I think that before voting on the REP again, someone should first implement a prototype that generates a valid debian changelog with multiple entries for an existing catkin package with some CHANGELOG.rst.




https://github.com/ros-infrastructure/rep/pull/33#issuecomment-14889098

Based on the feedback we have discussed this topic in our todays meeting and came to the following conclusion:

Headlines with the following format are considered changelog entries:
A version number or a version number followed by a timestamp in parenthesis.

A version number is in the form x.y.z as specified in https://github.com/ros-infrastructure/rep/blob/master/rep-0127.rst#version. For REP 132 this is not open for discussion - if different formats should be allowed that should be brought up against REP 127.

The timestamp must contain a full date (YYYY-MM-DD) and might be followed by a time (hh:mm:ss where the seconds are optional) and optionally followed by a time offset (+/-xxxx). The timestamp must be parsable by Python dateutil.parser.parse.

It is not necessary to allow free form text in the headline since it can be easily placed in other locations, i.e. in its own section or as a part of the sections content.

The implementation will take each headline, separate the version number from the (optional) timestamp and tries to parse both. If that fails the section is not considered to be a changelog entry but free form text.

Some examples what that implies:
A headline 1.2.3 (forthcoming) can be placed in the file, but it is not considered a changelog entry. It appears in the Wiki and provides valuable information to the users. During the release process bloom will uses the API to determine if it contains a changelog entry for version 1.2.3. Since the implementation will not consider that headline a valid changelog entry bloom and warn about a missing changelog entry. The maintainer must either updated the headline to contain a timestamp or remove the annotation.

Allow versions without a timestamp has the following sideeffect:
If bloom finds a valid changelog entry without a timestamp it can just use now() to satisfy the needs of i.e.e Debian packages. But for the user reading the changelog file in i.e. the Wiki a version number without a timestamp is ambiguous. It could be that this is an upcoming version (where the maintainer did not add an annotation like (forthcoming)) or it could mean it is a released version (but the maintainer did not provided a timestamp).

[tkruse]

Also using SVN, I have no idea how we could easily get version data from the scm, it would probably be a mess.

[Jack O'Quin]

We should stick with what works with tarballs, which is mostly
CHANGELOG.rst and package.xml, I suppose.

On Thu, Mar 14, 2013 at 2:31 PM, tkruse <tibo...@googlemail.com> wrote:
> Also using SVN, I have no idea how we could easily get version data from the
> scm, it would probably be a mess.
>

> --
> You received this message because you are subscribed to the Google Groups
> "ROS Buildsystem Special Interest Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ros-sig-buildsy...@googlegroups.com.
> To post to this group, send email to ros-sig-b...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/ros-sig-buildsystem/-/J2c2NwrmyiAJ.

>
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
joq